Retries Are Not a Recovery Strategy

A retry answers a narrow question: might the same operation succeed if I attempt it again?

Recovery has a harder job. It must bring the original business operation to a known, valid outcome after something went wrong. Getting there may require another attempt, a status lookup, resuming from persisted state, or compensation. If the system cannot resolve the operation safely, it must hand it to a person.

This difference matters as soon as an AI workflow does more than return text. If it retrieves data, calls tools, writes state, or continues after the HTTP request ends, adding three retries around the workflow is not a recovery design. It is three more chances to spend money, repeat a side effect, or lose track of what already happened.

A retry repeats an attempt

Suppose a support feature performs this workflow:

load the ticket and approved policy
    -> generate a reply
    -> validate the reply
    -> save it as a draft

The policy read returns 503 Service Unavailable with an applicable Retry-After response, and the dependency contract classifies it as transient. No application business state changed, and the request still has time left. A delayed retry may be reasonable.

Now suppose the draft save times out after the request reached the database. The caller cannot tell whether the write committed. Repeating the complete workflow creates a new model response and may save a second draft. Retrying only the write is safe when the write is naturally idempotent, or when the boundary can recognize the retry as the same logical operation. Otherwise, the second attempt may create another draft.

Both failures may appear as a timeout or dependency exception in application code. They do not have the same effect.

Retry belongs to the first few rows. Recovery begins when the system must find out what happened or restore a valid business state.

A timeout does not mean the operation failed

When the caller's timeout expires, it tells the caller that it stopped waiting. It does not prove that the callee stopped working, rolled back, or never received the request. The remote operation may never have started, may have failed, may still be running, or may have completed while its response was lost.

Those states should not collapse into one Failed result. I find it useful to separate the operation status from its business outcome:

public enum OperationStatus
{
    Pending,
    Running,
    PartiallyCompleted,
    OutcomeUnknown,
    Terminal,
    NeedsReview
}

public enum OperationOutcome
{
    Succeeded,
    Rejected,
    Failed,
    Compensated
}

public sealed record OperationState(
    OperationStatus Status,
    OperationOutcome? Outcome);

This type is deliberately simplified. As written, it permits invalid combinations such as Running with Succeeded, or Terminal with no outcome. Production code should enforce those invariants through its type design or validated construction. OperationStatus is a top-level recovery status, not a complete per-step workflow model. A workflow can be partially complete while the outcome of its latest step is still unknown.

Failed is a terminal unsuccessful outcome supported by authoritative evidence. A payment decline, for example, is a completed request with a Rejected business outcome. OutcomeUnknown means the application cannot yet determine the terminal outcome. NeedsReview is not a business outcome at all. It transfers ownership from automatic recovery to a person.

That extra state is mildly inconvenient. Good. The uncertainty already exists in the system, whether the type admits it or not. Encoding it gives the API, UI, recovery worker, and support tooling something honest to work with.

For the draft save, the application can allocate an operation ID before it submits the write:

Guid operationId = Guid.NewGuid();

var command = new SaveDraftCommand(
    OperationId: operationId,
    TicketId: ticketId,
    Reply: reply);

await draftOperations.CreateAsync(
    operationId,
    command,
    cancellationToken);

try
{
    await draftStore.SaveAsync(command, cancellationToken);

    await draftOperations.MarkTerminalAsync(
        operationId,
        OperationOutcome.Succeeded,
        cancellationToken);
}
catch (DraftSaveOutcomeUnknownException)
{
    using var recoveryWriteCts = new CancellationTokenSource(
        TimeSpan.FromSeconds(2));

    await draftOperations.MarkOutcomeUnknownAsync(
        operationId,
        recoveryWriteCts.Token);

    throw;
}

The example uses DraftSaveOutcomeUnknownException for a specific conclusion from the storage adapter: the attempt crossed the submission boundary, but the adapter cannot determine whether the write committed. A plain OperationCanceledException is not enough. Cancellation may happen before anything was submitted, in which case the write outcome is known.

Production code must still distinguish caller cancellation from an attempt timeout. It also has to handle failures while recording the state transition and prevent an older attempt from overwriting a newer result. Create the durable identity before the ambiguous boundary, then keep it for lookup and any safe retry.

The persistence contract must enforce uniqueness for the operation ID and reject reuse with different command data. Merely logging a GUID does not make the write idempotent.

Retrying the workflow is usually the wrong scope

Broad retry policies are attractive because they are easy to add around an endpoint or orchestration method. They also repeat every successful step before the failure.

If the support workflow fails while saving the draft, a workflow-level retry may retrieve the same policy again, pay for another model call, produce a different reply, and run validation again before it reaches the uncertain write. The retry has changed the thing being recovered.

AI workflows make this especially awkward:

• Model calls are metered and can be slow.
• A repeated generation is not guaranteed to return the same output.
• Tool calls can hide writes behind an interface that looks like an ordinary function call.
• SDKs, HTTP clients, queues, and workflow code may each have their own retry behavior.

Agent loops add another failure mode. Consider an approved workflow that is allowed to charge a customer:

model selects ChargeCustomer(...)
    -> application validates and authorizes the tool call
    -> payment succeeds
    -> tool response is lost
    -> agent loop starts again
    -> model produces another ChargeCustomer(...) call

The second model run may use a new tool-call ID or produce slightly different arguments. Approval and argument validation do not tell the payment boundary that this is the same charge. The application needs a stable business operation ID that exists independently of the model output and survives another model turn. Otherwise, the retry has changed both the reasoning and the command being recovered.

That last point can turn a small setting into a large amount of traffic. Three attempts in one layer and three in the next can produce up to nine calls to the struggling dependency. Under load, the retries may prolong the incident they were supposed to smooth over.

Per-request limits may still allow too much retry traffic when many requests fail together. A dependency-level retry budget caps the aggregate retries generated by concurrent requests. Once the budget is spent, requests get no additional attempts. Their initial attempts are a separate decision. A circuit breaker or admission-control policy can make new requests fail fast.

Retry the smallest operation whose failure is known to be transient and whose repetition is safe. Do not restart a workflow simply because one of its steps threw an exception.

Recovery starts with durable operation state

An in-memory retry loop can help the current request, but it does not provide durable ownership of the operation. A process restart or deployment can destroy its state, and request-scoped work may end when the caller disconnects.

When the result must still be resolved later, the application needs a durable record of the logical operation. That record needs enough information to identify and continue the same work:

• a stable operation ID, plus any boundary-specific idempotency identities needed for safe replay
• the command or a durable reference to it
• a command fingerprint used to detect and reject the same key being reused for different work
• the current state and attempt identity
• the next eligible recovery time
• enough outcome information for status lookup and support

A recovery worker can then process unresolved operations without inventing a new logical request:

Pending
    -> submit with the original idempotency key

OutcomeUnknown
    -> query the authoritative status
    -> retry the original command only when the contract makes that safe

PartiallyCompleted
    -> resume the next incomplete step or run a domain-specific compensation

Still unresolved after the recovery deadline
    -> NeedsReview

Recovery does not always mean forcing the requested state change to succeed. It may confirm a payment decline and close the operation with a Rejected outcome. If an external reservation succeeded but a later step failed, recovery may cancel the reservation and record Compensated. If neither action is safe to automate, the operation moves to NeedsReview and a person owns the next decision.

Recovery should produce a terminal, explainable business outcome or transfer ownership to manual review. An unresolved operation should not simply disappear when the retry loop stops.

Idempotency protects one boundary

An idempotency key tells a boundary that repeated submissions refer to one logical command. The boundary must atomically claim or enforce uniqueness for that command, including when two requests arrive concurrently. It can prevent the second request from applying the side effect again and, once available, return or reference the recorded result.

While the first request is still running, there is no result to return. The contract may expose the current operation status, wait for completion, or report that the operation is already in progress. A separate check followed by an insert leaves a race in which both requests can execute.

That protection is local to the boundary that enforces it. A database uniqueness constraint does not deduplicate an email already sent by another service. A job ID does not protect a payment call unless the same identity reaches the payment provider and the provider honors it.

Each side-effect boundary needs its own guarantee. A transactional outbox can make the handoff from a business transaction to messaging durable, but the relay may publish a message more than once. The receiving boundary still needs an inbox, an idempotent consumer, a provider-supported idempotency key, or a domain-specific deduplication rule.

Idempotency also does not answer these questions:

• Should the operation still run after the caller has gone away?
• How long should the system keep trying?
• What happens when status lookup remains unavailable?
• Can completed steps be reversed?
• Who owns an operation that never reaches a terminal state?

Those are recovery decisions.

A retry policy still needs boundaries

Retries are useful when the failure is plausibly transient, repeating the attempt is safe, and another result would still arrive in time to matter.

For each retryable interaction, define:

1. Which outcomes are transient. Do not retry validation errors, authorization denials, or other requests that cannot succeed unchanged.
2. The scope of the attempt. Retry the failed read or model call, not an entire sequence of completed work.
3. The per-attempt timeout and total budget. A new attempt should not start when there is no useful time left.
4. The maximum attempts and delay. Respect Retry-After where available, and use backoff with jitter rather than synchronizing every instance.
5. The behavior after the attempts end. Return a specific application outcome, let the circuit-breaker policy update its failure state, degrade explicitly, or hand the operation to durable recovery.

Also inspect retry behavior already present in SDKs and infrastructure. Adding an application policy without accounting for lower layers makes attempt counts and latency hard to predict.

Record the attempt number, duration, outcome category, and operation ID in telemetry. A request that succeeds on its third attempt still represents two failed dependency calls, more latency, and more cost. Occasional transient failures are normal. A rising retry rate or an increasing share of requests that need several attempts may show that the dependency is degrading. Final success alone hides that change.

When retries are enough

A bounded retry policy is often enough for an idempotent read or another side-effect-free call when the failure is clearly transient. The work stays inside the current request, no state is uncertain, and exhausting the attempts can return an honest failure to the caller.

Examples include a throttled metadata lookup, a connection failure where the client can establish that the request was never submitted, or a model call that has no tools and whose repeated cost is acceptable inside the remaining request budget.

When you need a recovery path

Design recovery when an operation changes state, spans several independently committed steps, or must reach a terminal result after the current request ends. You also need it when a response can disappear after a side effect commits, or when an external action requires compensation or human review.

Recovery costs more to build. It needs persisted state, ownership, deadlines, concurrency control, telemetry, and support procedures. That cost is a reason to keep workflows small and side effects explicit. It is not a reason to call a retry loop recovery.

Practical takeaway

Pick one state-changing AI workflow and find every retry around it, including retries inside SDKs and infrastructure.

For each one, write down what is known after the failed attempt, whether state may have changed, which identity the next attempt uses, and what happens after attempts are exhausted. Returning an error is a valid outcome when no state changed and the application has no promise to continue the work.

If state may already have changed, the operation is partially complete, or the business contract requires a terminal result after the request ends, "return an error and hope the user tries again" is not enough. That workflow still needs a recovery path.

Use an explicit OutcomeUnknown path when the application loses the response after submitting a side effect and cannot determine whether it completed. Resolve that state through authoritative lookup, a safe retry with the original identity, compensation, or manual review.

Related reading

• What Happens When Your AI Feature Fails?
• Use idempotency keys for retryable writes
• When an AI Request Should Become a Background Job

Sources

• Microsoft Learn: Retry pattern
• Microsoft Learn: Best practices for transient fault handling
• Microsoft Learn: Compensating Transaction pattern
• Microsoft Learn: Retry Storm antipattern
• Microsoft Learn: Azure Cosmos DB design pattern: Transactional Outbox