Disclosure up front: I work on Radar, the open-source Kubernetes UI whose MCP server is one side of this benchmark. The method is published, so you can check the work.
Everyone is plugging AI agents into Kubernetes right now, and the default move is to give the agent a shell and let it run kubectl. It works. The agent gets there. But nobody seemed to have measured what that approach actually costs, so we built a harness and did.
The setup
- 52 fault-injection scenarios on a live EKS cluster: crashloops, misconfigurations, resource pressure, broken rollouts, and the messier indirect failures where the symptom is far from the cause.
- One model on both sides: Claude Sonnet 4.6.
- Two conditions. Condition A: the agent gets a shell and raw kubectl. Condition B: the same agent connects through Radar's MCP server.
- Same prompts, same success criteria, scored on whether the agent found the actual root cause.
The only variable is how the agent sees the cluster.
The results
Per-trial averages across all 52 scenarios:
| Metric (per trial) | kubectl | Radar MCP | Change |
|---|---|---|---|
| Tool calls | 45.8 | 11.1 | 76% fewer |
| Input tokens | 4.9M | 2.3M | 53% fewer |
| Output tokens | 3,040 | 1,039 | 66% fewer |
| Agent time | 334s | 169s | 49% faster |
| Pass rate | 77.6% | 80.8% | small increase (+3.2pp) |
| Diagnostic score | 0.765 | 0.862 | small increase (+0.097) |
The headline is the efficiency column. The accuracy rows are there to show the speedup wasn't bought with correctness: it nudged up rather than degrading.
Why the gap exists: it's the context, not the protocol
The tempting reading is "MCP beats shell access." That's not quite it, and it matters, because plenty of Kubernetes MCP servers are essentially a proxy to kubectl: the same raw output over a newer transport. Wrap kubectl in MCP tools and I'd expect numbers close to the kubectl column.
Watch a kubectl-driven agent work and the cost is obvious. It runs get pods, then describe, then logs, then get events, then describe on the owner, and after every call it re-derives the same structure from text: which replicaset belongs to which deployment, which service fronts which pods, what changed before things went bad. That structure never persists between calls. The agent pays to rebuild it, in tokens and in round trips, over and over.
Radar's MCP server hands the agent that structure as ready context. Two pieces do most of the work:
- The resource graph. Ownership chains, service routing, what's connected to what. One tool call returns the neighborhood of a failing resource instead of five calls' worth of raw output to correlate.
- The change timeline. What was deployed, scaled, restarted, or rolled back, and when. Most diagnoses are some form of "what changed?", and giving the agent the answer as data beats making it reconstruct the sequence from events and log timestamps.
That's the whole trick: better data in a better format. 45.8 tool calls collapse to 11.1 because most of those calls existed to rebuild context the server can simply provide.
Where the gap was widest
Not on the easy faults. A pod stuck in CrashLoopBackOff with an obvious error in its logs gets diagnosed either way, and kubectl does fine.
The gap opened on indirect, cross-cutting failures: the ones where the failing thing isn't the broken thing, and the trail crosses resource boundaries. Those are exactly the cases where a human relies on knowing the topology and the recent history, and the agent behaves the same way. With the graph and the timeline in context it followed the chain; with raw output it burned calls wandering.
Honest caveats
- The accuracy gain is small. +3.2 points of pass rate is a nudge, not a leap. If you want this benchmark to prove agents get dramatically smarter with MCP, it doesn't. It proves they get dramatically cheaper and faster at the same quality.
- One model. We ran Claude Sonnet 4.6. The mechanism (ready structure is cheaper to reason over than raw text) shouldn't be model-specific, but we haven't measured others yet. If you run it on GPT, Gemini, or a local model, I'd genuinely like to see the numbers.
- 52 scenarios. Big enough to see the pattern, small enough to inspect by hand. We'll grow it, and if there's a fault class you think is missing, tell me.
Try it or reproduce it
The full method is in the benchmark write-up this post is adapted from, and the background on how the MCP server builds live cluster context is here: MCP for Kubernetes: live cluster context for AI tools.
Radar is a single Go binary, Apache-2.0, no account, nothing installed in the cluster. It reads your kubeconfig and respects your RBAC:
curl -fsSL https://get.radarhq.io | sh && radar
Repo: github.com/skyhook-io/radar. The MCP server is in the box; point your own agent at it and see if your numbers look like ours.
