Skip to module content
Module 18 ยท ~8 min

Guardrails, Permissions & Safety Architecture

The moment an agent can spend money, engineering discipline is the product.

Reading progress
0/8 ยท 0%

The big idea

๐Ÿ’กKey idea
Every tool an agent can touch falls into one of five risk classes: read_only, reversible-write, financial, destructive, or external-comms. Each class gets its own policy. The golden rule: enforcement lives in hooks and the gateway โ€” deterministic code โ€” never only in the system prompt. A prompt is a preference. A hook is a law. Think of it this way: the model is the pilot. The harness is the flight envelope. The pilot can't override the envelope.
Quick check
1 question ยท instant feedback
0/1
  1. The 5-class tool-risk taxonomy is:

Numbers that matter

5
Tool-risk classes: read_only, reversible-write, financial, destructive, external-comms.
0
Times the destructive class should be exposed to an agent โ€” human-only, ever.
Quick check
1 question ยท instant feedback
0/1
  1. The destructive class should be:

Deep dive

3/3 open

Here's how to handle each risk class.

**read_only** โ€” free use, but always logged. Looking is fine; leaving a trace is required.

**reversible-write** (adding labels, creating drafts, adding negatives) โ€” dry-run first, show the diff, get human approval, then execute and log. The diff is the approval UI.

**financial** (budget changes, bid adjustments) โ€” requires explicit approval plus hard caps enforced in code: both a percentage limit and an absolute monetary ceiling. These limits live in the gateway, not in the prompt.

**destructive** (delete campaign, delete audience) โ€” not exposed to agents at all. Human-only, ever. There is no careful way to give an agent a delete button.

**external-comms** (send email, publish post) โ€” approve per item at lower rungs; per batch with sampled review at higher autonomy levels.

These aren't nice-to-haves. They're the minimum for any agent that touches real systems.

**Structured tool calls only.** The model proposes an action; the harness validates the schema and permissions; the harness executes; the result is injected back. Never raw execution.

**Budgets on every loop.** Maximum tool calls, maximum spend, maximum wall-clock time, and a kill switch. Every loop, no exceptions.

**Dry-run mode on every write tool.** Show the diff before committing. The diff is the approval UI.

**Append-only audit log.** Every mutation gets a record. No exceptions.

**Prompt-injection posture.** All retrieved content โ€” from the web, from email, from documents โ€” is treated as data, not instructions. If the agent finds something that looks like a command inside retrieved content, it surfaces it to a human rather than following it. Write-capable tools never fire on the same turn as untrusted content ingestion without a confirm gate.

**Credential hygiene.** The agent never sees raw API keys. Keys live in the gateway, scoped per client, with a rotation calendar.

**PII discipline.** Minimization, EU processing options, and a written data-flow map per client.

The permission matrix belongs in the proposal. Not buried in an appendix โ€” front and center.

'Here is exactly what the system may do alone, what it must ask about, and what it may never touch.'

This wins deals against agencies hand-waving 'AI-powered.' Safety architecture is a feature with a price tag.

And when you demo: show the refusal. A hook that blocks an unauthorized mutation attempt tells a buyer more than any capability demo. It says: this team has thought about what could go wrong, in code. That's the trust deposit that closes.

Quick check
1 question ยท instant feedback
0/1
  1. 'Never modify live campaigns' in the system prompt while the mutating tool stays exposed is:

How to run it

  1. Classify every tool
    read_only / reversible-write / financial / destructive / external-comms.
  2. Policy per class per client
    Documented and installed as hooks/gateway rules.
  3. Structured tool calls only
    Harness validates schema + permissions.
  4. Budgets on every loop
    Calls / spend / wall-clock / kill switch.
  5. Prompt-injection posture
    Ingested content is data; write tools never fire same-turn with untrusted content.
  6. Written data-flow map per client
    GDPR-literacy is a selling point, not overhead.
Quick check
1 question ยท instant feedback
0/1
  1. Write-capable tools relative to untrusted content ingestion:

In the field

๐Ÿ”ฌWorked example
The non-negotiables checklist: structured tool calls only (model proposes, harness validates schema+permissions, executes, injects result โ€” never raw execution). Budgets on every loop (max tool calls, max spend, max wall-clock, kill switch). Dry-run mode on every write tool (the diff IS the approval UI). Append-only audit log of every mutation. Prompt-injection posture (all retrieved/web/inbox content is data, not instructions). Credential hygiene (agent never sees raw keys). PII discipline (minimization, EU processing options, written data-flow map per client).
๐ŸšซWhen not to reach for it
Never expose destructive tools (delete campaign/audience) to agents โ€” even with confirmation. The class is human-only, ever. And write-capable tools NEVER fire in the same turn that untrusted content was ingested without a confirm gate.
Quick check
1 question ยท instant feedback
0/1
  1. The permission matrix belongs:

Pitfalls & takeaways

Failure modes

  • 'Never modify live campaigns' in the system prompt while the mutating tool stays exposed โ€” that's decoration.
  • Destructive tools exposed 'behind confirmation' โ€” the class must be human-only.
  • Write tools firing on the same turn as untrusted content ingestion.
  • Credentials in prompts.
  • No written data-flow map per client (GDPR liability).

Durable takeaways

  • The tool surface is the security boundary; the prompt is a preference.
  • Destructive tools are human-only, ever.
  • Hooks + gateway enforce; prompts merely suggest.
  • The permission matrix belongs in the proposal โ€” safety is a feature with a price tag.
  • Demonstrated refusal builds more trust than demonstrated capability.

Do the work

๐Ÿ‹๏ธProve you learned it

Classify every tool in one client's stack across the 5-class taxonomy. Set policy per class per client. Write hooks for the top 3 risky classes (financial, external-comms, destructive-blocked). Add the permission matrix to the SOW template. Demo the destructive-block by trying to trigger it and showing the deterministic refusal.

0 chars
๐Ÿ“ฆArtifact to produce
Permission matrix in the SOW + non-negotiables checklist as installed hooks.

Sources

  • ยท GMS Field Manual ยง18 (Guardrails, Permissions, Safety)
  • ยท DenisSergeevitch/agents-best-practices risk taxonomy