Two weeks ago I counted 204 guards in my repos and found that 89 % had never been shown they can fail. I fixed that for a batch of them. Then tonight I filed a data-loss report about 1,000 files that were never lost — and nobody, including me, had a number to check it against.
Last time I wrote here, the number was 204 and 11 %: of the automated checks in my repositories that draw a conclusion, only about one in nine could prove it was capable of failing. @dev_michael's "AI didn't make me a worse coder, it made me a worse reviewer" is what started that count.
So I did the obvious follow-up work: I made a batch of guards prove they can fail. That fix is real and I'll describe it below, because it costs an afternoon and it works.
It also wasn't enough, and I found out the embarrassing way.
Part one: the fix that works
Every reviewer — a hand-written check, an LLM judge, a second agent grading the first — gets one known-bad case wired through the live path. Not a unit test beside the pipeline. The same entry point real work uses.
Our benchmark harness runs three gates per case, and a case that misses any of them does not run at all:
- Unsolved state → must go RED. The check must reject the broken repo.
- Solved state → must go GREEN. It must accept the fix.
- Known-bad → must go RED again. We re-plant the original mistake deliberately; if the check still passes, it never understood the bug.
Gate three is the one that earns its keep. It has caught real breakage — including, twice, in guards written that same week to catch exactly this class of problem.
This week I pushed it further, because a harness is only as good as the cases in it, and inventing cases is where everyone runs out of imagination. @shreyasht killed his own token-optimization project after discovering his best result — 97 % savings — came from the run that did no work at all: the agent asked a clarifying question, stopped, and the dashboard crowned it. A metric with no way to punish doing nothing eventually rewards it. His fix is one sentence: measure per solved task, never per task.
The imagination problem has a cheap answer, and it was sitting in our database: the incident history is the test suite. We took ten real recorded failures — the kind with a "what went wrong" field written by someone annoyed at the time — and had a model punch each one into a runnable check. Ten lessons in, ten runnable cases out, thirty gates, all thirty passed. Zero discarded.
One thing that surprised me: the first attempt failed the internals filter, and it failed because the example we handed the model contained a product name in a comment. The model copied it faithfully. The filter caught it. A guard I had written to be paranoid about human sloppiness caught a machine being obedient instead.
@wrobeltomasz described this same discipline independently in a comment thread while this draft sat unpublished: identify the checks, then run them "in simulation mode to confirm that they can indeed respond to invalid input" — moving verification "from statistics in the README to actual system resilience." Two people who have never met, same gate, same reason. That usually means the pattern is real and not a personal quirk.
Part two: the half I had not fixed
Here is what all of the above proves: on the day I wired it, that reviewer could say no.
It says nothing whatsoever about today.
@mk023 said the correcting sentence in a thread under my last post, and gave me permission to quote it here:
"Don't just test that the guard can fail — test that it is still guarding the thing you think it is guarding."
That completes the model. A green check can be lying in three orthogonal ways: falsifiability — can it go red at all; liveness — can it go red against today's system; and aim — is the boundary it defends still the one that matters. Part one only bought me the first. Marco's sentence is the other two, and the rest of this post is me finding out the hard way that he was right.
@james_anderson_h, in a thread about AI workspaces that ship with "guardrails — approval gates, an audit trail, a second agent that reviews the first," put the gap better than my question had: most tools hand you the audit log, not proof the veto still fires.
"A checker nobody has ever seen fail is indistinguishable from a checker that approves everything. The two produce identical logs right up until the day the rubber-stamp waves through the thing that hurts you."
My answer to him was a small thing I now think is the most useful idea in this post: the veto heartbeat.
Surface one date. "Last time this reviewer refused something: 2 days ago." First-class, visible, right next to the uptime number. The known-bad case runs on a schedule; if the last-refusal date ages past that schedule, staleness itself is the alarm. No log-diving, no trusting a vendor's dashboard, no archaeology. One date anyone can read in one glance — and the absence of fresh red finally looks like what it is.
James named the principle underneath it better than I had:
"Silence and health look identical unless you deliberately build a state for 'not verified lately'."
That is the whole bug in one line. Most systems model pass and fail and nothing else. A veto that quietly died therefore reads exactly like a veto that simply had nothing to refuse. The heartbeat is that missing third state, worn on the outside where an auditor can see it without your permission.
Where I was the punchline, tonight
I was going to publish the section above as the ending. Then I spent an evening proving the point against myself, and the story is better than the theory.
I went to check on a long-running harvest job spread across two machines. I connected to one, looked for the working directory, found nothing, looked for the process, found nothing. So I reported it: the data is gone, roughly a thousand collected items lost, and I filed a ticket saying so.
I had connected to the wrong machine. The numbering I used to pick it does not mean what I assumed it means — a fact that is written down in my own project instructions, in bold, with an example. I had read it before. I did it anyway.
Nothing was lost. The harvest was sitting exactly where it should be, and a second job was quietly running on it at that very moment.
Now the part that matters, because "I made a dumb mistake" is not an article. Why did the false report survive as long as it did? Because there was nothing to contradict it. No dashboard, no counter, no file with three numbers in it. To check my claim you had to log into two machines and count files by hand — which is precisely why nobody had done it in the two days before, either.
And when I finally did count, the numbers exposed something nobody had noticed: the two machines had been working the same list from opposite ends, had met in the middle days ago, and had since re-collected 332 items both already had. Not broken. Not alarming. Just quietly wasteful, in a system with no visible number to be quietly wasteful in front of.
The two failures have exactly one cause
A green check nobody can question and a red alarm nobody can question are the same bug wearing different clothes:
- A guard that has silently stopped being able to fail produces unfalsifiable green.
- A monitor with no visible state produces unfalsifiable red — my false report, which survived only because contradicting it required work.
Both come from the same missing thing: a number that a stranger can read without your cooperation. The veto heartbeat is that number for a reviewer. Three counters in a file is that number for a background job. Neither is clever. Both are absent almost everywhere, including — until this week — in a codebase belonging to someone who writes about this for a living.
@bert_sk_shim_cb93b1 hit the same class from the opposite side while this was being written, and named the part I had missed. His login check queried an endpoint scoped to a different auth method, so it returned "no username" no matter what — always negative instead of always green. Then the sentence that reframed this whole post for me:
"A red result tends to get remediated rather than investigated, so I had someone go log in again, which was unnecessary, and if the timing had been a bit different I'd have credited that as the fix and kept the broken check."
That is the asymmetry. Green invites complacency, but red invites action — and action feels like resolution. A broken always-red check gets a workaround performed in front of it, and the workaround gets credited. He very nearly filed an unnecessary login as the fix and kept the instrument that lied to him. Always-negative hides better than always-green, and it is the direction almost nobody watches.
His fix generalizes further than either of our stories: separate presence from value. The operational version I have landed on since is one line — print what you read before you print what you concluded. "No username at /whoami (auth scheme B)" is a bug report. "Not logged in" is a rumour with a status code. My false alarm would have died in thirty seconds if my own check had said which machine answered instead of only what I concluded about it.
@pm25coder shipped the same move at a different layer while I was writing this: their harness now appends every silent-safety-net event to an append-only file that survives restarts and log rotation, so "did the safety net ever misfire?" became a lookup instead of archaeology. Evidence that outlives the process that produced it. That is the whole idea, and it is worth stealing.
And John Green has been running the harshest version of this discipline on model evaluations: he reran his exam five times and watched his published winner evaporate — the behaviour that decided the ranking reproduced zero times out of five. His line stays with me: "The exam caught its author again — not in the answer key, not in the grader — in how confidently I read a single run."
Copy that
Paste this next to any pipeline that claims to have guardrails:
REVIEWER CHECKLIST — "exists" is not "works"
[ ] One known-bad case wired through the LIVE path (not a side test)
[ ] It runs on a schedule, not once at setup
[ ] Its last RED has a visible timestamp ("last refusal: …")
[ ] Timestamp older than the schedule => alarm, automatically
[ ] The metric it guards punishes doing nothing (per SOLVED unit)
[ ] Someone has seen every terminal state fire at least once
MONITOR CHECKLIST — an alarm needs a number too
[ ] Three counters, readable without logging into anything
[ ] The identity of the thing measured is IN the output
(which machine, which provider, which run — not inferred)
[ ] Silence has its own state, distinct from "healthy"
[ ] Every verdict prints the VALUE it read, not just the conclusion
[ ] Check the always-negative direction too: a check that can only
say no is as broken as one that can only say yes — and it hides
better, because red gets remediated instead of investigated
The 60-second version, if you do nothing else: open the oldest green check in your pipeline and ask when it last went red. If the answer needs a log search, you have found one. If you cannot answer it at all, you have found a bigger one.
What this does not prove
The three gates prove a check could fail on a case someone thought of. They say nothing about the cases nobody thought of, and harvesting your own incident history — which I recommend — has a bias baked in: those cases come from the same system that produced the failures, so they may be systematically easier than reality. If your harvested cases pass at a noticeably higher rate than your invented ones, that is a finding, not a win. Report the two separately.
The heartbeat has a failure mode too: a scheduled known-bad case can become a ritual that always passes, and then the fresh date is reassuring rather than informative. The honest mitigation is rotation — change the known-bad case periodically — and I am not going to pretend I have a clean answer for how often.
And the counters are not free. Three numbers per job is a small amount of code and a permanent amount of maintenance.
The question I'll leave you with
Find the oldest green checkmark in your pipeline — the linter gate, the LLM judge, the approval step that has passed everything for months.
When did it last say no? Do you know? Can you find out without asking the person who built it?
If the answer is "I would have to check the logs," you do not have a guardrail. You have a rubber stamp with excellent posture.
I would genuinely like to hear two things in the comments: the age of the oldest never-red green in your system, and — the one I now care about more — the last time an alarm in your system turned out to be wrong, and how long it took anyone to prove it.
Mine took an evening, and only because I finally counted.
Quoted with thanks: @mk023, whose one sentence is the whole of part two, @shreyasht's token-stack post-mortem, John Green's AI-verification series, @james_anderson_h's workspace thread, and @wrobeltomasz, @pm25coder and @bert_sk_shim_cb93b1 in the comments — Marco's and James' lines with their explicit blessing. Bert's always-negative direction is the half of this post I would not have found on my own.
I build cachly — memory for AI coding assistants, over MCP. ChatGPT and Claude remember your conversations. cachly remembers your system: the bug you fixed, why you chose Postgres, the deploy step that always breaks — and which earlier decision it contradicts. Every assistant you use reads the same memory, and every lesson carries the name of whoever learned it — so nobody has to learn it twice.
The incident-history-as-test-suite trick in this post runs directly on that memory: the lessons were already there, written at the moment someone was annoyed enough to record them.
Free tier, hosted in the EU: cachly.dev
