A few weeks back I watched an agentic pipeline burn a month of budget in three days. The instinct everyone reached for was the obvious one: "the frontier model is too expensive, switch to a smaller one." That's the wrong diagnosis, and it's worth writing down why.
When we finally tagged every token to the (step, tool, model) tuple that spent it, the picture was uncomfortable. The expensive model wasn't the problem. The problem was that every step — the trivial classifier, the summary, the hard reasoning call — silently defaulted to it, and nobody could see which step was responsible. We'd been arguing about which model to buy when we should have been arguing about which step was allowed to call it.
Cost in an agent isn't a number. It's a tree.
In a single LLM call, cost is one line item. In an agentic loop it's a tree: each step calls a model, maybe a tool, maybe a retry, maybe a sub-step that calls again. The bill at the end is the sum over that whole tree.
If you only ever see the total, you are optimizing blind. "Make it cheaper" with only a total in front of you means guessing. And the guess is almost always "use a smaller model everywhere," which throws away quality on the steps that actually needed the frontier model while leaving the real waste untouched.
The silent default that inflates everything
Most agent frameworks let you set "the model" once, at the top. So a routing decision that should have been per-step gets frozen into a single global choice. A classification step that a small model handles fine? Frontier. A summarization of text the model just produced? Frontier. The one genuinely hard reasoning call? Also frontier — but now it's lost in the noise.
The waste here isn't that frontier models are pricey. It's that ~80% of those calls never needed them, and without per-step attribution you can't even prove which 80%. You're paying a premium on steps whose output quality wouldn't have moved.
Per-step attribution is boring infrastructure, not a feature
There's no clever trick. You log, for every call: step id, tool, model, input/output tokens, latency, retry count. That's it. Once that data exists, the waste shows up in minutes — usually a handful of steps dominate the bill, and they're rarely the ones you'd have guessed.
I've lost count of how many times the "expensive" step turned out to be a retry loop quietly re-running a hard call five times because the tool returned a shape the parser rejected, not because the model was wrong. That's not a model-cost problem. That's an un-instrumented step.
Failover is where the blind spots explode
Here's the part that actually blows budgets. A model errors, or times out, or returns garbage. The naive fix re-runs the whole trajectory — or at best the whole step — against the same model. Cost spikes exactly when you can least afford it, on the run that was already failing.
A planned failover routes that single step to a fallback at the step boundary: same tool contract, different model, the rest of the run continues. The gap between "retry the universe" and "swap the model for this one call" is most of the cost story nobody talks about, because you only see it when things break — and by then you're not measuring.
The lever lives at the routing layer, not in prompts
The point that took me too long to internalize: you cannot prompt your way to per-step cost governance. The layer that decides which model serves each step is the only place that can both measure the spend and act on it. Put the attribution and the failover decision anywhere else and you're asking a component that doesn't see the calls to fix calls it can't see.
This is also where the compliance constraint and the cost lever happen to land on the same component. In Southeast Asia, PDPA-aligned handling means prompts and outputs can't wander out of region — so the routing layer has to be SG-hosted (Tencent Cloud) and keep data resident. That same in-region gateway is exactly where per-step attribution naturally lives: the router is already on the path of every token, so measuring them costs nothing extra. A gateway that exposes 25+ models behind one OpenAI-compatible endpoint isn't just a convenience for failover — it's the only place the measurement and the swap can share state.
The question I keep coming back to
When your agent's bill doubles, can you point to the three steps that caused it — or do you just know the total went up? If you can't attribute per step, the fix was never a cheaper model. It was instrumenting the routing you already had.
