Most agent demos have a happy path: the model picks a tool, the tool returns 200, the agent moves on. Production is different. In production, the tool call sometimes returns nothing at all.
That is where a correct decision turns into a wrong outcome.
The gap between "decided" and "done"
Here is an illustrative scenario (not a real incident, but a very ordinary one).
An agent handles a support ticket and decides the customer is owed a refund. It calls POST /refunds. The payments service processes the refund, writes it to its ledger, and starts sending the response. The network hiccups. The agent's HTTP client hits its timeout and raises an error.
From the agent's point of view, the call failed. From the payment service's point of view, it succeeded. The agent's loop sees an error, and the most natural thing for it to do is try again.
Now the customer has two refunds.
The model reasoned perfectly. It picked the right action with the right arguments. Nothing about better prompting, a bigger model, or a smarter planner fixes this, because the failure happened after the reasoning was over.
Failure mode 1: treating a timeout as a "no"
A timeout tells you one thing: you didn't get an answer. It does not tell you the action didn't happen. Agent frameworks that wrap tool errors into "the tool failed, decide what to do next" collapse three different situations into one:
- the request never reached the service
- the request reached it and was rejected
- the request reached it, succeeded, and the response was lost
Only the first two are safe to retry blindly.
Failure mode 2: no idempotency key
If the agent had sent an idempotency key, the second refund call would have been a no-op that returned the original result. The key has to be generated by the application before the first attempt and reused on every retry of that same intent:
intent = actions.record_intent(
kind="refund",
order_id=order.id,
amount=amount,
idempotency_key=uuid4(), # created once, stored with the intent
)
try:
result = payments.refund(
order_id=order.id,
amount=amount,
idempotency_key=intent.idempotency_key,
)
actions.mark_confirmed(intent.id, result)
except Timeout:
actions.mark_unknown(intent.id) # NOT "failed"
The important part is where the key lives. If the model generates it, or if it's regenerated per attempt, it protects nothing.
Failure mode 3: state that only exists in the context window
If the record of "I already tried to refund order 4812" lives only in the conversation history, it disappears on a crash, a context truncation, or a restart of the worker. The intent needs to be written to durable storage before execution, so that whatever process picks the work back up can see it.
Failure mode 4: no "outcome unknown" state
Most action models have two terminal states: success and failure. Reliable ones need at least:
-
submitted: we sent it, no answer yet -
confirmed: the downstream system acknowledged it -
failed: the downstream system explicitly rejected it -
unknown: we can't tell
unknown is the one that matters. It's the state that should block automatic retries and trigger reconciliation instead.
Failure mode 5: no reconciliation path
When an action lands in unknown, something has to go find out the truth: query the payment service by idempotency key or order ID, compare against the ledger, and then move the intent to confirmed or failed. That can be a background job, a scheduled check, or a human queue for high-value actions. What it can't be is the agent deciding "probably fine" or "probably failed" on its own.
The reframe
Agent tool use is really remote procedure calls with an LLM choosing which procedure to call. Everything we already know about RPCs over unreliable networks still applies: idempotency, durable intent logs, explicit uncertainty, reconciliation.
The clean split is:
- The model proposes an action.
- The application owns the execution state: the key, the intent record, the status, the retry policy.
"Try again" is a fine default for reads. For anything that changes state, it's only safe once you know the first attempt didn't land, or once the downstream system guarantees a retry can't double-apply.
Build vs. integrate
If you already run payment or order flows with idempotency keys and outbox/ledger patterns, you mostly need to route agent actions through that same machinery instead of letting the agent call APIs directly. If you don't, that's the piece to build (or adopt) first, before giving an agent write access to anything that moves money or data.
How are you handling the "we don't know if it happened" case in your agent workflows? Are you blocking retries, reconciling, or escalating to a human?
