Give an agent run a single monotonic deadline, then lease every per-attempt timeout and backoff sleep out of what is left of it.
An agent answering one question rarely makes one web call. It searches, fetches a few pages, extracts fields, and sometimes drives a browser to confirm something. Each of those steps usually arrives with its own configuration: a per-request timeout here, three retries there, an exponential backoff somewhere else.
Configured that way, the worst case is a sum nobody ever computed. Four steps at ten seconds each, times three attempts, is two minutes of patience before the backoff sleeps are counted, and no caller agreed to it. The user gave up long before. Depending on the provider, some of those abandoned attempts are still charged, and each of them still consumed a slot in whatever rate limit applies.
The fix is small and structural: decide the deadline once, at the top of the run, and make every step downstream derive its own limits from what remains.
A deadline is not a timeout
A timeout is a duration attached to one operation. A deadline is an instant shared by everything the run touches. That difference matters because durations do not compose. Two operations with five-second timeouts can take eleven seconds together, and neither one violated its contract.
An instant composes trivially. Every step asks the same question, how much time is left, and every step gets an answer that already accounts for what earlier steps spent.
Store that instant on a monotonic clock. Python's time.monotonic() returns a value whose reference point is undefined, so only differences between two readings are meaningful, and it cannot go backward when the system clock is adjusted. A wall-clock deadline can be moved by an NTP correction in the middle of a run, which is a rare failure that is very hard to reproduce.
The undefined reference point has one consequence worth planning for. A monotonic instant is meaningful only inside the process that read it. When the deadline crosses a process or a service boundary, send remaining milliseconds instead, and have the receiver convert it back into a local instant on arrival. Each side then keeps its own clock, and the only thing on the wire is a duration that both ends can interpret without agreeing on a reference point.
Rule one: the per-attempt timeout is a lease
Treat the configured per-attempt timeout as a ceiling, not a value. What a step actually receives is the smaller of that ceiling and what the budget has left:
granted = min(configured, remaining - floor)
The subtraction is the interesting part. If a call is allowed to consume the last microsecond, the run ends with a timeout and nothing to show for it. Reserving a floor leaves room to record the outcome, release resources, and return a partial answer.
Rule two: refuse calls that cannot finish
When the remaining budget drops below the floor, the correct action is to raise locally rather than to start a remote call. A call given far less budget than the endpoint usually needs to answer is very likely to time out, and a doomed attempt still occupies a connection and a slot in whatever rate limit applies. On metered APIs it may be charged as well, since some providers bill on the request rather than on a successful response.
Refusing early converts a guaranteed slow failure into an immediate one, which is what the caller wanted anyway.
Rule three: sleeping is spending
Backoff is a common place where budgets leak, because a sleep looks like inactivity rather than consumption. It is not: a retry sleep spends the same seconds a request would.
So bound the backoff twice. Cap its exponential growth at a maximum, and check the chosen sleep against the remaining budget before committing to it. If the sleep would consume what is left, stop retrying and surface the failure while there is still time to report it.
Use jitter on the sleep. The AWS Builders' Library discussion of timeouts, retries, and backoff argues for randomizing the wait so that clients which failed together do not retry together; full jitter picks uniformly between zero and the current cap. Without it, a provider recovering from an outage can be hit by a synchronized wave of the same clients that just failed.
Rule four: retry only what is safe to retry
A budget check answers whether there is time to retry. It does not answer whether retrying is correct. RFC 9110 defines which request methods are safe and which are idempotent, and idempotency is the property that makes an automatic retry harmless. A repeated GET on a search endpoint is usually safe to retry, because the method is defined as read-only, though the specification constrains the method and not the implementation behind it, and the second request still costs quota. A repeated POST that queues a browser job may run the job twice unless the API accepts a client-supplied idempotency key.
The same specification defines Retry-After, the field a server sends to say how long to wait: RFC 9110 describes it on 503 and on redirects, and servers commonly send it with 429 as well. When you get one, it should beat your local backoff calculation, and it should still be checked against the deadline. A Retry-After longer than the remaining budget is a signal to give up, not to wait.
An implementation
The following is a synthetic, self-contained example. The timings, failures, and outputs are fabricated to make the control flow observable, and they are not measurements of any real provider.
"""Synthetic deadline-budget demo. Fabricated timings, not provider measurements."""
import asyncio
import random
import time
BASE_BACKOFF_S = 0.05
MAX_BACKOFF_S = 0.40
FLOOR_S = 0.02 # below this, no remote attempt is worth starting
class DeadlineExceeded(Exception):
pass
class Transient(Exception):
pass
class Budget:
"""One monotonic deadline, shared by every step of a single agent run."""
def __init__(self, total_s: float) -> None:
self.expires_at = time.monotonic() + total_s
def remaining(self) -> float:
return self.expires_at - time.monotonic()
def lease(self, step: str, want_s: float) -> float:
"""Largest timeout worth granting, or a refusal if nothing useful is left."""
left = self.remaining()
if left <= FLOOR_S:
raise DeadlineExceeded(f"{step}: {left:.3f}s left, below floor")
return min(want_s, left - FLOOR_S)
async def call_step(budget, step, operation, per_attempt_s, max_attempts, rng, trace):
last_error = None
for attempt in range(1, max_attempts + 1):
timeout_s = budget.lease(step, per_attempt_s)
trace.append((step, attempt, round(timeout_s, 3)))
try:
return await asyncio.wait_for(operation(attempt), timeout_s)
except (asyncio.TimeoutError, Transient) as exc:
last_error = exc
if attempt == max_attempts:
break
cap = min(MAX_BACKOFF_S, BASE_BACKOFF_S * 2 ** (attempt - 1))
sleep_s = rng.uniform(0.0, cap) # full jitter
if sleep_s >= budget.remaining() - FLOOR_S:
raise DeadlineExceeded(f"{step}: backoff outlasts the deadline") from exc
await asyncio.sleep(sleep_s)
raise last_error
def quick(label: str):
async def operation(attempt: int) -> str:
await asyncio.sleep(0.01)
return f"{label}(attempt={attempt})"
return operation
def hangs_first(n: int):
async def operation(attempt: int) -> str:
if attempt <= n:
await asyncio.sleep(10.0) # cut short by the per-attempt timeout
await asyncio.sleep(0.01)
return f"body(attempt={attempt})"
return operation
async def run_plan(total_s: float, rng: random.Random):
budget = Budget(total_s)
trace: list = []
results: dict = {}
plan = [
("search", quick("results"), 0.25, 2),
("fetch", hangs_first(1), 0.10, 3),
("extract", quick("fields"), 0.25, 2),
]
for step, operation, per_attempt_s, max_attempts in plan:
results[step] = await call_step(
budget, step, operation, per_attempt_s, max_attempts, rng, trace
)
return results, trace, budget.remaining()
if __name__ == "__main__":
results, trace, left = asyncio.run(run_plan(1.0, random.Random(7)))
assert results["fetch"] == "body(attempt=2)"
assert [(s, a) for s, a, _ in trace] == [
("search", 1), ("fetch", 1), ("fetch", 2), ("extract", 1),
]
assert all(granted <= 0.25 + 1e-9 for _, _, granted in trace)
assert left > 0.0 # the whole plan finished inside the one deadline it was given
try:
asyncio.run(run_plan(0.01, random.Random(7)))
except DeadlineExceeded as exc:
print(f"refused before dialing out: {exc}")
print(f"plan complete, {left:.3f}s unused")
Three properties are worth reading off that code. expires_at is written once and never extended, so no step can grant itself more room by asking twice. Every timeout passed to asyncio.wait_for comes from lease, so the ceiling and the remaining budget are enforced in one place. And the backoff sleep is checked against the budget before it happens, so a retry cannot outlive the run it belongs to. The second run never reaches that check: its budget starts below the floor, so lease refuses before the first request is dialed.
Choosing the numbers
A budget makes limits explicit, which immediately raises the question of what the limits should be. Percentiles from your own traffic are the best source. Where you have none yet, published evaluations give a starting point: NativePort, for example, publishes run-dated latency and error-rate figures per web capability, alongside quality and a cost metric, so that a search timeout and a browser timeout are not chosen from the same number. Whatever the source, keep the run date attached, because provider behavior drifts.
Make it visible
Finally, record what the budget did. Log the granted timeout and the remaining budget on every attempt, not just the final outcome, so that a slow run can be explained without rerunning it.
OpenTelemetry's HTTP semantic conventions already cover part of this: retries and redirects produce more than one physical request for one logical call, and each resend carries http.request.resend_count. Add the budget fields beside it and a trace answers the question that matters during an incident, which is whether the deadline was too small or the provider too slow.
References
-
Coordinating Concurrent Tasks: Timeouts and time.monotonic, Python standard library documentation, on
asyncio.wait_forand on the monotonic clock whose reference point is undefined. - Timeouts, retries, and backoff with jitter, Amazon Builders' Library, on capping retries and randomizing waits so clients do not retry in unison.
-
RFC 9110: HTTP Semantics, IETF, on safe and idempotent methods and on the
Retry-Afterheader field. -
Semantic conventions for HTTP spans, OpenTelemetry, on representing resends with
http.request.resend_count.
Disclosure
We’re the NativePort team, whose public leaderboards are cited once above as one possible source of published latency figures. AI assisted with drafting this article. During preparation, the code example was run and every cited URL was checked; editorial approval of this exact copy remains separate. All timings, failures, and outputs shown here are synthetic and describe no real provider.
