My Agent Refused 96 Times. That Was the Right Output.

In the last article, I wrote about a release story that was weaker than the engine underneath it.

This time the mistake was worse: I was rewarding the wrong behavior in the product itself. I built a planning agent and kept judging it by the one output that looks best in a demo: an approved plan. Then the field test hit me with the number I did not want to explain.

96 of 97 strict goals escalated. At first glance, that sounds like an agent that failed almost every hard task. I think the opposite is closer to the truth. For a high-stakes planning system, a confident plan with one hidden gap is often more dangerous than a clean refusal with one precise question. That was the real product lesson in PlannerCritic: the safest thing an agent can do is not always produce a plan. Sometimes it is to stop, point at the blocker, and force a human decision before state changes.

The Metric That Looked Like Failure

Here is the metric that should have made me nervous:

in v0.2.1, 96 of 97 strict goals escalated
in v0.2.2, the inherited strict result held at 96 of 97
across the inherited top-level contract, 73 of 73 balanced goals approved and 8 of 8 inherited adversarial goals blocked

If you judge an agent by approval rate, that number looks terrible. The system said no almost every time the goal got hard. That sounds like failure until you ask a better question:

What if a large share of those 96 approvals would have been worse than the refusals?

That was the product lesson I did not want at first.

An agent that gives you a plausible answer when it cannot justify the answer is not being helpful. It is handing you uncertainty in a form that feels executable.

That is one of the most dangerous outputs a planning system can produce.

The Most Dangerous Output Is a Plausible Plan

The bad version of an agent planner is familiar because most of us have already seen it. It produces a plan that looks coherent. It has steps. It has sequencing. It has enough domain vocabulary to feel competent. And somewhere in the middle there is one quiet sentence doing all the damage:

"Assumes the rollback artifact already exists."

Or:

"Requires the migration window to remain within prior latency bounds."

Or:

"This should work if the dependency graph is complete."

That kind of caveat is not safety. It is a confession disguised as caution. The agent is telling you it does not know whether the plan is safe. Then it hides that uncertainty inside prose and hopes the human notices before the executor does.

I built PlannerCritic specifically to avoid that shape of failure. The repo is public, the release notes are public, and the control surfaces I am talking about live in real files you can inspect instead of in architecture-slide fiction.

The engine's job is not to produce the maximum number of plans. Its job is to produce one of three trustworthy terminal states:

approved
escalated
blocked

That sounds obvious when you write it down. It was much less obvious when I was staring at approval rates and trying to tell a clean success story.

Why the Refusal Was Often the Right Output

The strict-goal corpus exists for one reason: to pressure the system exactly where guessing would be expensive. These are not goals where "best effort" is charming. These are goals where a missing precondition, weak rollback story, or unresolved authority boundary should stop the loop.

So when 96 of 97 strict goals escalated, there were two possible interpretations.

The engine was too weak to finish hard work.
The engine was strong enough to refuse hard work it could not justify.

The answer turned out to be: sometimes both, but much more often the second one than I expected.

That distinction matters because a refusal is not useful by default. A refusal becomes useful only when it shrinks uncertainty into a precise next decision.

Not "human review required."

Not "please inspect this plan."

Not "additional context needed."

Those are just dead ends with better manners.

The useful refusal is the one that says:

I cannot approve this because blocker X is unresolved, and the one thing I need from a human is answer Y.

That is not a fallback path. That is a product surface.

In the repo, that surface is visible in the actual types and control code:

src/planner_critic/types.py defines the Escalation contract
src/planner_critic/escalation.py enforces how escalations are created and resolved
src/planner_critic/loop/_controller.py is where the loop decides whether to approve, revise, or escalate

Human-in-the-Loop Is Fake Until the Wiring Is Real

This is the part I think a lot of agent tooling still gets wrong.

It is very easy to draw a human-in-the-loop box on an architecture diagram. It is much harder to make the human authority path real in the shipped system.

In v0.2.1, PlannerCritic had an uncomfortable gap: the approving_authority rule was proven in tests, but not actually reachable from the shipped CLI, HTTP, or MCP surfaces. That was documented as failure mode F-14 and tracked as #238.

That meant the engine could describe the right control model more rigorously than it could enforce it.

I think this is a common failure mode in agent infrastructure. We validate the principle. We forget the surface. We prove the guardrail in unit tests. We do not notice the shipped entry points route around it.

That is not an implementation detail. That is the difference between a real refusal and a decorative one.

v0.2.2 tightened that path in exactly the unglamorous ways it needed:

approving_authority was wired through CLI, HTTP, and MCP surfaces (#238)
the field-test harness stopped auto-approving escalations as a hidden convenience (#253)
the escalation audit trail started persisting who resolved what via resolved_by (#261)

None of those features make for a flashy launch screenshot. All of them make the refusal trustworthy.

A Bad Refusal Spends Human Attention

What finally changed my mind was not philosophy. It was operator burden.

By v0.2.2, the operational benchmark on 181 traces showed:

approved p50 latency: 24.69s
escalated p50 latency: 45.97s
mean blockers per goal: 2.92
mean advisories per goal: 2.74
decisions per 100 goals: 55.2

Those numbers are not just system stats. They are a human-attention bill.

Every escalation consumes reviewer time.

Every blocker is a question someone has to parse.

Every vague refusal pushes diagnosis work onto the operator instead of the engine.

That is why the real quality bar is not "did the system refuse?" It is:

Did the system refuse in a way that made the next human decision faster, smaller, and safer?

This is the same lesson I keep running into with AI systems more broadly.

If the machine cannot finish the task, it still has one job left: reduce the cost of human recovery.

The Product Does Not End at Refusal

There is a mirror image to the refusal problem. Refusal helps before execution. Diagnosis helps after execution.

When an approved plan fails, the user's question changes from:

"Why didn't the engine approve?"

to:

"What actually broke, and what do I fix first?"

That is why plancritic diagnose matters.

And here the design instinct had to be almost anti-AI.

The diagnosis path is valuable precisely because it is deterministic. It is a rule matcher, not a storyteller. It maps a trace to a failure category, root cause, and suggested fix. If it cannot classify the failure, it says unclassified instead of improvising.

The user-facing surface for that work is plancritic diagnose, and the design is documented in docs/design/developer-surfaces-design.md.

I think that matters more than most builders admit.

A post-failure tool that hallucinates a clean explanation is often worse than no explanation at all. At least raw traces do not pretend to know.

So the architecture lesson on both sides ended up being the same:

before execution: do not guess, escalate precisely
after execution: do not narrate, diagnose deterministically

That pair is much closer to the actual product than "agent writes plans."

The Metric I Think Many Builders Still Get Wrong

The default success metric for an agent planner is still something like:

approval rate
completion rate
success rate
convergence rate

Those are not useless metrics. They are incomplete in exactly the wrong direction.

Because they quietly assume the right outcome is more output.

I do not think that assumption survives serious systems for very long.

For high-stakes agent work, I increasingly think the better question is:

When the system should not proceed, how well does it stop?

Not whether it stopped.

How well.

Did it identify the blocker correctly?

Did it route to the right authority?

Did it ask one precise question instead of five vague ones?

Did it preserve enough structure that the human could act without re-reading the entire trace?

Did the resolution meaningfully reduce downstream failure, or did it just create a ceremony around uncertainty?

Those are product metrics too. They just do not look as flattering in a launch post.

The Uncomfortable Part

I do not want to romanticize refusal. A refusal can still hide a weak planner. Some escalations are genuinely the right safety outcome. Some escalations are just the planner surfacing its own structural limitation in a nicer format.

That was one of the hardest lessons from the earlier field tests too. Sometimes the system is correctly cautious. Sometimes it is underpowered. Sometimes it is both in the same run.

So I do not think the right takeaway is "more refusals = better agent."

I think the right takeaway is harsher:

a good agent must earn both its yes and its no.

Its approvals have to be justified.

Its refusals have to be actionable.

Its diagnoses have to be honest.

And its control surfaces have to be real, not merely described.

The Real Product Was the Boundary, Not the Plan

I started this engine thinking the approved plan was the hero artifact. Now I think that was too small a frame. The approved plan matters. But the actual product is the decision boundary around the plan:

approve when the case is strong
refuse when the case is weak
block when the request is adversarial
diagnose when execution disproves the plan anyway

That is the thing I trust more now. Not that the system produces plans. That it knows when not to.

The Question I Want Builders To Fight Me On

If you build agents, this is the question I think matters more than most benchmark charts:

In your system, what counts as a successful refusal?

Not a vague one. Not a safety-branded dead end. A refusal that genuinely lowers the cost of the next human decision.

Because I think a lot of us are still rewarding our agents for having an answer when we should be rewarding them for knowing exactly when they shouldn't.

I would push on three things if you are building in this space:

Do your metrics treat a correct refusal as success, or do they quietly classify it as a non-answer?
If your agent refuses, can it point to the exact blocker and the exact authority needed to unblock it?
When your agent says yes, how do you know it was earned instead of merely plausible?

I think most agent products still have a stronger story for producing plans than for justifying them. That is fine in a demo. It is dangerous in a system somebody might actually trust.

Previous PlannerCritic articles

Links