Originally published on Loop & Retry — field notes on building LLM agents that survive production.
Search "claude pricing" and you'll land on a table: dollars per million input tokens, dollars per million output tokens, one row per model. That table is accurate and almost useless for predicting what you'll actually pay, because the invoice at the end of the month isn't tokens × rate — it's tokens × rate × three multipliers most people never measure. I've written about each multiplier separately on this blog. This post puts them on one page and runs the arithmetic together, because the interaction between them is where the real surprises live.
The three multipliers, in the order people usually discover them:
- Cache misses. Prompt caching can cut your input cost by 90% on the reused part of a request — or do nothing, silently, if your request structure breaks it.
- Quadratic context growth. A long-running agent's token bill grows with the square of its step count, not linearly, because every step re-sends the whole transcript so far.
- Prepaid parallel retries. Best-of-N sampling for latency multiplies your token spend by N unconditionally, whether or not you needed the extra attempts.
None of these show up as a line item. All three show up in the total.
The rate is not the bill
Anthropic and OpenAI both publish the same kind of table: a flat per-model, per-million-token rate, separately for input and output. That's the number people search for under "claude cost," "claude api pricing," or "how much does claude cost," and it's genuinely the wrong place to start optimizing, for a specific reason — it describes the price of one token, and your bill is a function of how many tokens you actually send, which is almost never what you'd naively estimate from (prompt length) × (number of calls).
I use illustrative Sonnet-class rates throughout this post — $3 per million input tokens, $15 per million output tokens, matching what I've used consistently in the cost-beyond-tokens breakdown — because the ratios here are the durable part, not the absolute numbers, and they'll survive the next price change. Swap in your own current rate card; the shape of the argument doesn't move.
Multiplier 1: the cache that silently misses
Prompt caching is the closest thing to a free lunch in this business: mark a stable prefix — system prompt, tool schemas, a big retrieved document — with a cache breakpoint, and repeat calls that share that exact prefix pay roughly a tenth the input price on it instead of full price. The catch is that "exact prefix" means byte-identical, and there's no error when it isn't. A stray timestamp ahead of the breakpoint, a tool list that serializes in a different order, a loop step that runs past the cache TTL — any of these silently puts you back to paying full price, and nothing in the response tells you unless you're logging cache_read_input_tokens and noticing it's zero.
This matters for a Claude bill specifically because Claude Code and most agent harnesses re-send a large, mostly-stable system prompt and tool schema block on every single turn. If that block is genuinely stable and actually hits the cache, it's cheap. If it silently misses — which is the common failure mode, not the rare one — you're paying close to full input price on a multi-thousand-token block, every single call, and the invoice gives you no way to tell that from a model that's just expensive.
Multiplier 2: the transcript that grows quadratically
Even with caching working perfectly on the stable prefix, the part of the request that caching can't help — the running transcript of tool calls and results — grows every step, and that growth compounds. Step 40 re-sends everything steps 1 through 39 produced. Sum that across a run and total input tokens scale with the square of the step count, not the count itself. A 40-step agent run costs roughly four times a 20-step run, not twice, and a demo that runs 8 steps hides this completely — it only bites once a run is long enough to matter, which is exactly when nobody's watching the per-call cost anymore.
Here's where the two multipliers actually meet: prompt caching is a discount on the stable part of the request, and the growing transcript is, by definition, not stable. Caching flattens the floor of your cost curve; it does nothing to the slope. That distinction sounds academic until you run the numbers together.
SYS = 4000 # stable system prompt + tool schemas
OUT = 300 # tokens emitted per step
RESULT = 500 # tool result appended to the transcript per step
IN_PRICE = 3.0 / 1e6
OUT_PRICE = 15.0 / 1e6
CACHE_WRITE_MULT = 1.25 # 5-min TTL write premium
CACHE_READ_MULT = 0.1 # cache hit discount
def naive_run_cost(N):
total_in = total_out = 0
for k in range(1, N + 1):
transcript = (k - 1) * (OUT + RESULT)
total_in += SYS + transcript
total_out += OUT
return total_in * IN_PRICE + total_out * OUT_PRICE
def cached_run_cost(N):
total_in_billable = total_out = 0
for k in range(1, N + 1):
transcript = (k - 1) * (OUT + RESULT)
stable = SYS * (CACHE_WRITE_MULT if k == 1 else CACHE_READ_MULT)
total_in_billable += stable + transcript
total_out += OUT
return total_in_billable * IN_PRICE + total_out * OUT_PRICE
for N in (10, 20, 40):
naive, cached = naive_run_cost(N), cached_run_cost(N)
print(f"N={N:3d} naive=${naive:.4f} cached=${cached:.4f} savings={100*(1-cached/naive):.1f}%")
N= 10 naive=$0.2730 cached=$0.1788 savings=34.5%
N= 20 naive=$0.7860 cached=$0.5838 savings=25.7%
N= 40 naive=$2.5320 cached=$2.1138 savings=16.5%
Caching saves more than a third of the bill at 10 steps and less than a sixth at 40 — the exact same caching setup, working exactly as designed, delivering a shrinking benefit as the run gets longer. That's not a caching failure. It's the transcript, the part caching never touched, becoming a bigger share of an ever-larger total. If you benchmarked your caching win on a short test run and assumed it holds at production run lengths, you've overestimated it, and the gap grows with exactly the runs you care most about.
Multiplier 3: the retries you pay for whether you need them or not
The third multiplier doesn't come from a mistake — it comes from a deliberate design choice that prepays for a benefit you may or may not be collecting. Best-of-N sampling — firing N attempts at once and keeping whichever finishes first or scores best — trades money for tail latency. It's a legitimate pattern; self-consistency voting and low-latency SLAs both depend on it. But it's priced by worst-case attempts, always N of them, not by the expected number a sequential retry loop would actually need.
N = 40
naive, cached = naive_run_cost(N), cached_run_cost(N)
for label, base in (("naive", naive), ("cached", cached)):
for n_attempts in (1, 3):
print(f"{label:7} best-of-{n_attempts}: ${base * n_attempts:.4f}")
naive best-of-1: $2.5320
naive best-of-3: $7.5960
cached best-of-1: $2.1138
cached best-of-3: $6.3414
Look at the scale of these two effects side by side. Caching bought back 16.5% at N=40. Wrapping the same run in best-of-3 costs 3× — and that 3× is applied after the caching discount, so it erases the entire saving and then some: cached best-of-3 ($6.34) is still nearly 2.5× the naive single-attempt cost ($2.53). Caching is a percent-level lever. Best-of-N is a multiple-level lever. If you're chasing the first while ignoring whether the second is even switched on for the right requests, you're optimizing in the wrong units.
The one that isn't tokens at all
All three multipliers above are still token-shaped. The most common way a Claude bill surprises people isn't token-shaped at all: tokens are frequently the smallest of six cost axes an agent actually spends across — latency held by a waiting human, orchestration and infrastructure, per-call tool fees, human review, idle capacity. In a workload where every task gets human sign-off, the token line can be under 2% of the true per-task cost, and no amount of prompt trimming or caching touches the other 98%. Before you spend an afternoon shaving your token count, sum the other axes on your actual workload — the invoice only ever itemizes the one that's cheapest to fix and often not the one that's biggest.
What I'd actually check on your bill this week
-
Log
cache_read_input_tokensandcache_creation_input_tokenson every call, not just when something looks expensive. A cache that's silently missing looks identical to a model that's just pricier, until you check the one field that tells them apart. - Plot per-step prefill tokens against step number on your longest-running agent. A flat line means you've tamed the quadratic; a rising line means every additional step is costing more than the one before it, and truncating tool results into digests is the highest-leverage fix.
- Find every place you run N attempts in parallel and check the N is deliberate, not a default someone copy-pasted from an example. Best-of-3 "for safety" on a workload with no latency requirement is a 3× tax bought for nothing.
- Total your non-token axes before touching your token spend. If human review or idle capacity dominates your per-task cost, optimizing the model bill is real work spent on the wrong number.
The rate card tells you what one token costs. It never tells you how many you're actually going to send, and that number is set by your caching hygiene, your run length, and your retry strategy — three things fully under your control, and none of them on the pricing page.
