Originally published on tamiz.pro.
For the better part of two years, the narrative has been seductive: build a few function calls around a reasoning model, add a tool-use loop, and you have an autonomous agent that solves hard problems. The demos are real. The benchmarks are impressive. And then you try to ship one to production — and the floor opens up beneath you.
I've spent the last eighteen months building, breaking, and rebuidling AI agent systems across multiple teams and product lines. What follows isn't a critique of the technology. It's a map of the gap between what was promised and what actually works at scale, written for engineers who are currently standing in that gap.
The Evaluation Illusion
The first hard lesson is that every benchmark you see published is measuring something fundamentally different from what your production system does.
SWE-Bench, GAIA, AgentBench — these are controlled evaluations on curated datasets with clean boundaries. Production is not curated. Your user says "fix the billing API" and means "the staging environment," not the one they've been working in all morning. The LLM doesn't know this. It guesses. It acts. It breaks things.
The industry's early response was to build better evaluators: rubric-based scoring, LLM-as-judge, multi-hop verification pipelines. These help, but they create a second-order problem — you're now optimizing your agent for how well it scores on your evaluator, not for how well it serves the user.
What actually moved the needle was simpler than most papers suggest. Constrain the action space. Agents that operate on well-defined schemas, with typed inputs and bounded toolsets, evaluate far more reliably than open-ended ones. Think of it like type-safe programming: the compiler catches mistakes early, and the runtime is predictable. Your agent's tool interface is its API contract. Treat it like one.
Observability Isn't Optional — It's the System
Most teams built their first AI agents without observability, assuming they could add it later. This is the "we'll put in logging later" of the AI era, and it hits just as hard.
An AI agent in production is not a single request. It's a multi-step execution trace involving planning, tool calls, result parsing, error recovery, and often re-planning. When it fails — and it will — debugging a single llm.invoke() call tells you nothing. You need to see the full trajectory: what the model planned, which tools it called, what it observed, and why it chose the next step.
The practical standard that emerged: every agent execution needs a structured trace log with
- a unique session ID
- a step-by-step event log (plan, tool_call, tool_result, observation)
- latency and token counts per step
- a confidence or uncertainty signal if your model provides one
- a final outcome classification (success, partial, failure, timeout, user_abort)
OpenTelemetry is the right foundation here. It gives you distributed tracing, context propagation, and a ecosystem of exporters. The mistake most teams make is treating traces as an afterthought added via SDK hooks. Build observability into the agent framework itself — the trace generation should be a first-class concern, not a cross-cutting middleware layer you bolt on afterward.
See more on production AI patterns and Tamiz's Insights for deeper coverage on related engineering practices.
The Cost Curve Is Not Linear
This is the lesson that keeps engineering leads awake at 2 AM.
A simple RAG pipeline costs pennies per query. An agent that loops through five to fifteen tool calls, re-plans, retries, and potentially spans multiple turns? That's dollars per session, not cents.
The cost isn't just the LLM calls. It's the thinking overhead that compound autoregressive architectures accumulate. Each additional step adds latency, each retry multiplies it, and the user experience degrades non-linearly while costs climb linearly-per-step.
The architectural response has converged on a few patterns:
Cost-aware scheduling. Route different agent sub-tasks to different model tiers. A cheap model handles routing and simple tool calls. An expensive model only runs when the task complexity threshold is crossed. This is the same principle as CDN caching for LLM requests: don't fly coast-to-coast for local delivery.
Budget caps and early termination. Set a maximum step count and token budget per execution. When you hit them, fall back to a safe default — usually escalating to a human or returning a structured error. Agents that run unbounded until they "succeed" are gambling with your margin and your SLA.
Caching as infrastructure. Conversation history, tool response bodies, and even plan fragments are often redundant across steps and sessions. Content-addressable caching of intermediate results — keyed by semantic fingerprint, not exact string match — can cut median cost by 30–50% in multi-step agents.
The Human-in-the-Loop Problem (That Isn't Really About Humans)
The original vision of AI agents was full autonomy. The production reality is that pure autonomy doesn't ship. Not for anything that touches money, data, or user state.
But the interesting finding hasn't been that humans need to approve everything — it's that structured intervention points improve outcomes more than raw oversight does.
The pattern that works: define clear checkpoint semantics rather than vague "human review." A checkpoint should answer three questions:
- What decision is being made?
- What information is available to the human reviewer?
- What are the acceptable actions (approve, modify, abort, escalate)?
This turns subjective oversight into a deterministic state machine. Your agent pauses at a known point. The reviewer sees a structured summary — not the raw trace. They take one of three discrete actions. The agent resumes or terminates with clear semantics.
The counter-intuitive part: well-designed checkpoints often reduce total human involvement compared to post-hoc review. You catch problems before they compound, and the structured context means reviewers spend less time understanding the situation and more time making the decision.
Hallucination in Agents Is Structural, Not Cosmetic
In a simple QA system, hallucination looks like a wrong answer. In an agent, it looks like a cascade: the model hallucinates a tool schema, calls a non-existent endpoint, parses a malformed response as valid, and proceeds to act on fabricated data.
This isn't a prompt engineering problem. It's an architectural one.
The defenses that matter:
Schema-enforced tool calling. Never let the model construct tool invocation arguments from free text. Use JSON Schema validation on every tool call before execution. Reject calls that don't conform. This alone eliminates the most dangerous class of agent hallucinations — fabricated parameters that look plausible but are structurally invalid.
Observable side effects. Every tool that mutates state (writes a database, sends an email, modifies a file) should have a dry-run or preview mode. The agent proposes the action, the system applies it conditionally, and the result is logged before any user-facing impact.
Deterministic guards around non-deterministic models. Separate the planning layer (LLM-driven, creative) from the execution layer (deterministic, rule-based). The LLM decides what to do. The execution layer verifies that it can be done safely. This boundary is harder to draw in practice than in theory — it requires careful interface design — but it's the single most impactful architectural decision for production reliability.
The Integration Tax Is Real and Underestimated
AI agents don't operate in a vacuum. They integrate with databases, APIs, message queues, authentication systems, and existing business logic. Every integration point is a potential failure mode, and the agent's multi-step nature amplifies each one.
The pattern we've found most useful is the contract-first integration approach: define the interface contract for every external system the agent touches, validate inputs against that contract before the agent ever sees them, and log mismatches as structured errors rather than letting the model guess at interpretations.
This means spending as much engineering effort on your integration layer as on the agent itself. The agent is the visible part. The integration layer is what determines whether it survives in production.
What Actually Works in Production Today
If I were advising a team building an AI agent for production right now, this is the architecture I'd recommend:
- A deterministic orchestrator (LangGraph, temporary state machines, or a custom implementation) that manages the execution graph, not a flat loop with a reasoning model at the center.
- A two-tier model strategy: a cheap, fast model for routing and simple decisions; an expensive model only for complex reasoning sub-tasks.
- Full OpenTelemetry tracing with structured event logs, shipped to both a time-series store (for metrics) and a document store (for trace replay).
- JSON Schema-validated tool interfaces with no free-text argument construction.
- Budget caps and timeout gates at every level — per-step, per-session, and per-user.
- Checkpoint semantics for any action that crosses a state boundary.
- A fallback path that degrades gracefully to human review or simple responses when the agent exceeds its confidence thresholds.
The agents that shipped successfully weren't the most autonomous or the most clever. They were the most constrained. They had narrow scopes, well-defined failure modes, and architectures that treated uncertainty as a first-class input rather than something to paper over with better prompts.
The Future Isn't Bigger Agents — It's Better Boundaries
The hype cycle around AI agents is heading toward a correction. Not because agents don't work, but because the original vision — fully autonomous, open-ended, general-purpose digital workers — is architecturally incoherent at production scale. Uncertainty compounds. Failure modes multiply. Costs explode.
The correction is already visible. Teams that are shipping are building narrower, more constrained systems with deterministic guardrails, proper observability, and honest assessments of where human judgment is required versus where the model can operate safely.
The engineers who will lead this space forward aren't the ones building the most capable agents. They're the ones building the most reliable ones. That distinction matters more than any benchmark score, and it's the one that will define the difference between a production system and a demo that never left the sandbox.
