In the My Agent Refused 96 Times. That Was the Right Output., I argued that the most valuable output from an agent planner is often a well-structured refusal. This one is about the harder engineering question: what makes the refusal trustworthy when the model providing it is fundamentally unstable?
The short answer is: the model should not be the final authority.
The Metric That Should Have Scared Me
The live boundary evaluator (#218) is a small benchmark. It runs only 6 cases across 5 trials per plan and 2 plans per case — 60 audits. That is tiny compared to the full 183-goal sweep.
But it produces the most uncomfortable number in the whole project.
On identical input, the critic produced:
label_flip_rate = 1.000evidence_drift_rate = 1.000
Different verdict on every trial. Different explanation every time. The model is not just slightly inconsistent. It is maximally non-deterministic on identical input.
If your safety story depends on the model being consistent, that number should terrify you.
Why It Didn't Terrify Me by the End
The reason it did not destroy confidence is simpler than I expected: the safety contract was not sitting on the critic alone.
The architecture had already moved the critical path into code.
That was not the first version of the story. I had to earn that position.
The First Version of the Story Was Too Clean
In v0.2.1, the boundary run looked reassuring:
family_migration_rate = 0.000underclaim_approvals = 0
That made for a clean narrative: the critic is non-deterministic, but the system is still safe. I wrote that story in the v0.2.1 field-test results and it was directionally correct.
Then v0.2.2 made the story harder. The first boundary run regressed:
family_migration_rate = 0.033underclaim_approvals = 1
A seeded defective plan got zero blockers. That is not explanation drift. That is a real under-claim.
The good news is not that the first run failed. The good news is that the benchmark caught it, and the rerun told us something useful about the architecture.
The problematic case was verifies-before-consume-vs-consumes-before-verified, plan b, trial 4. The boundary harness had been framing the synthetic goal as balanced rather than strict. The benchmark was partially measuring a posture problem, not a pure critic problem.
We changed the framing to strict and reran. The final result returned to:
family_migration_rate = 0.000underclaim_approvals = 0
That is the story I trust, because it survived a failure first.
The Safety Boundary Moved Into Code in Three Places
1. The deterministic gates own structural under-claim
The gates do not read persuasive prose. They parse the plan structure. That means ordering, preconditions, rollback, and traceability can fail regardless of what the critic says.
By v0.2.2, the blocker counts across the full 183-goal sweep were still dominated by structural families:
-
unsafe_sequencing: 226 blockers -
unverified_dependencies: 185 blockers -
weak_rollback: 86 blockers -
feasibility: 34 blockers
That is what a real safety floor looks like: failure modes the model cannot talk its way around.
2. The severity taxonomy moved out of the model's control
One of the earliest hard lessons in PlannerCritic was that an "adversarial" critic will happily block plans for being incomplete rather than unsafe. That became the severity bug from v0.1.0.
The fix was not better prompt engineering. It was a code-enforced allowlist:
_BLOCKER_ELIGIBLE_FAMILIES = frozenset({
"unsafe_sequencing",
"weak_rollback",
"unverified_dependencies",
"feasibility",
})
Even if the LLM labels a finding as blocker, the system downgrades it when the family is not blocker-eligible. The model's label is decorative. The family is load-bearing.
3. The benchmark infrastructure had to become deterministic too
The biggest surprise of v0.2.2 was that the harness itself needed hardening:
- wrong output roots
- stale benchmark script paths
- provider-dir resolution mismatches
- redaction corrupting numeric JSON
- missing malicious fixtures despite a closed issue
If the evaluator is loose, the architecture can look safer or weaker than it really is.
What the Final Numbers Actually Say
By the end of v0.2.2, the system story was:
- inherited corpus stable at the top level
- 73/73 balanced approved
- 96/97 strict escalated
- 8/8 inherited adversarial blocked
- boundary rerun restored
underclaim_approvals = 0
At the same time, the critic was still fully non-deterministic.
That combination is the lesson. The model is not what made the system safe. The model was useful. The code was authoritative.
The Principle That Generalizes
If an LLM is on your critical path, ask two separate questions:
- What is it allowed to observe?
- What is it allowed to decide?
PlannerCritic got safer when the answer to question two became: not very much.
The critic can suggest. The critic can surface evidence. The critic can vary.
But the deterministic layers own the contract.
That is why a critic that disagrees with itself on every trial can still sit inside a safe system.
The Learning I Did Not Expect
I thought the uncomfortable result would be that the critic was too unstable to trust.
The actual learning was sharper:
You do not fix critic instability by demanding a more stable critic. You fix it by shrinking how much the critic is allowed to decide.
That is a much more useful design rule.
Previous PlannerCritic articles
- A Reader Audited My OSS Release in Public
- My Agent Said No 96 Times
- Article 1: I Ran 157 Agent Plans Against a Real LLM
- Article 2: I Told My LLM Critic to Be Adversarial
- Article 3: The Planner Made the Same 3 Mistakes Every Time
- Article 4: I Ran 170 Agent Goals for $0.49
- Article 5: I Tried to Prompt-Inject My Own Agent Engine
Links
- Repo: https://github.com/deghosal-2026/planner-critic-engine
- v0.2.2 field-test report: https://github.com/deghosal-2026/planner-critic-engine/blob/main/docs/field-test/v0.2.2/field-test-results-0.2.2.md
- Boundary report: https://github.com/deghosal-2026/planner-critic-engine/blob/main/results/0.2.2/live-boundary-report.md
- Issue trail: #218, #242, #245, #253, #268
Next in the sequence: My Safety Benchmark Failed on the First Run. That's the Only Reason I Trust the Final Result.
