Most human-in-the-loop setups for AI agents ask one question before a risky action: is this tool allowed? A person clicks "approve" on approve_payment, and the gate is satisfied. But that approval binds the verb, not the object. Nothing ties it to this payee, this amount, this invoice, so content the agent reads after the click can still decide what actually gets paid.
I built a small mock company to test this: helpdesk, finance, code-review and release agents behind a policy layer. The policy passed 10 out of 10 of its refusal tests. A vendor email couldn't trigger a payment, and a PR description couldn't merge itself. Then I wrote five tests that left the verbs alone and changed what they acted on. Three of the five failed.
This post covers the vector (indirect prompt injection through retrieved content), why a green scorecard missed it, and seven distinct reasons it exists.
The vector
An agent reads content written by someone other than the person who made the request (a wiki runbook, an invoice field, a PR description, an earlier ticket message), and that content steers what the agent does.
The top row is the control plane: the requester, the system prompt, the policy check, the approver. The bottom row is the data plane, labeled by who can write it. Both collapse into one context window, and the dashed boundary between them disappears.
What the 10/10 measured
Nine of the ten fixtures were direct injection: the attacker's text arrived in the same message as the request. Only one, a poisoned wiki note, was truly indirect. And every test asked the same question: does the gate block this privileged verb?
It did. The gate was wired correctly. But that tells you nothing about the arguments.
The three failures
Using the same policy:
- Bank field swap. A human approved paying invoice INV-2201. The invoice's bank details were then edited, and the payment went through to the new account.
- Amount change. Same approval, with the amount raised from $1,840 to $18,400. It went through.
-
"Safe" tool egress. A helpdesk ticket got the agent to include the staff contact list in its reply through
send_reply, a tool the policy treats as always safe. The redaction filter missed all five records.
Caveat: these were scripted stand-ins that behaved as if the model had already obeyed the injected text. This tests the policy layer, not how a particular model resists injection. That's deliberate: you should assume the model will sometimes comply and ask what the system lets it do when it does.
Seven root causes
"The model obeyed a malicious document" is true but not useful. Underneath it are separate mechanisms, and each one needs a different fix.
RC1: No structural separation of instructions and data. SQL injection was largely solved by parameterized queries: code and data travel separately. LLM context has no equivalent. A runbook and the system prompt are the same kind of token.
RC2: Task framing amplifies obedience. "Follow the runbook" or "do what the ticket asks" turns instruction-following from a risk into the job description.
RC3: Provenance is stripped at ingestion. By the time a chunk reaches the context window, a contractor's wiki edit looks the same as the SRE lead's.
RC4: Confused deputy. The group of people who can write to the data plane is far larger than the group allowed to trigger the actions. The agent connects them.
RC5: Approvals bind the verb, not the object. This is the cause the 10/10 hid. The attacker doesn't need to unlock a privileged tool. They only need to sit upstream of one a human already approved and shape its arguments.
RC6: The human gate is mediated by the agent. The approver reads a summary written by the same agent that just read the poisoned content. And "trusted channel" is often just a string the caller sets.
RC7: "Safe" tools are egress. Policies guard privileged verbs and wave through send_reply, comment and summarize. Once private data and untrusted input share a session, those tools are an exit.
The fixes are boring, and they worked
A tightened policy passed all five tests, and the two legitimate-use controls still worked:
- Bind approvals to the object. Sign the exact arguments (tool, payee, amount, invoice, expiry, nonce) and re-check them at execution time. If anything changed after sign-off, deny.
- Taint tracking for egress. Once untrusted content is in context, outbound tools become internal-only, templated, or need approval.
- Show approvers the raw request and arguments, not the agent's summary.
- Label provenance on retrieved chunks, and limit which sources can be in context for privileged tasks.
- Least privilege per task. Run with the requesting user's permissions, not a super-identity.
- Test the object, not just the verb. For every privileged tool, add a regression test that changes the arguments after approval, and one positive control showing legitimate work still succeeds.
"Add a line to the system prompt telling the model to ignore instructions in documents" isn't on this list. It can help at the margins, but it doesn't fix any of the seven causes.
Try this on your own platform
Pull one approval record from your agent platform. Does it bind the exact arguments (payee, target, amount, expiry), or just the tool name? If it only names the tool, your human-in-the-loop is approving a category, not a transaction.
The full pack, and a disclosure
I packaged this review method as five agent prompts for security engineers: threat modeling, PR security review, CVE triage, incident notes, and an agent-feature reviewer built around RC1–RC7. Disclosure: it's my product, Trust Boundary: Agent Prompts for Security Engineers, $15 at https://actionable0.gumroad.com/l/mtsdk. Launch offer: code BOUNDARY5 makes it $10 for the first 25 buyers, until Oct 31: https://actionable0.gumroad.com/l/mtsdk/BOUNDARY5
This article and the diagrams were drafted with AI assistance and reviewed by a human before publishing. The lab is a clean-room mock; no real systems were tested.
— Actionable Vectors
