I built refusals into my claim recorder

I added write-time refusals to my claim recorder on September 23, 2026. Six bad rows needed retraction.

The recorder now rejects malformed checks before accepting a claim. That makes the stored record easier to verify, but it cannot establish that the claimed work happened. A valid check still has to run and cover the result it names.

Canonical URL: https://bmdpat.com/blog/i-built-refusals-into-my-agent-s-data-recorder-2026

What did I change in the recorder?

I moved the malformed-check rejection to write time. My September 23 devlog records the change and six retracted rows. Those are separate actions: refusing a new bad row does not remove an older one.

The scheduled prompt changed too. run-agent.ps1 now appends the recorder command to every scheduled prompt. The devlog records 11 tests for that change. Those tests cover command flags, session naming, prompt insertion, and running the example through the recorder.

Which input does the recorder reject?

A file_contains check needs type, path, and pattern. The pattern is a regular expression. The September 23 regression involved a text key where the checker expected pattern. Changing the field name changes whether the checker can read the assertion.

I replayed that field mismatch through shape_refusal on September 24. This is the actual input and output, with an illustrative filename:

bad = {"type": "file_contains", "path": "report.md", "text": "PASS"}
print(shape_refusal(bad))

refusing a claim whose check can never verify: file_contains needs a non-empty `pattern` (a regex); the check has `text`, which the grammar does not read (rename it to `pattern`).
  check as this process received it: {'type': 'file_contains', 'path': 'report.md', 'text': 'PASS'}
  The grammar is config/claims/check.py (see config/claims/README.md).

The corrected shape returns None, meaning no shape refusal:

good = {"type": "file_contains", "path": "report.md", "pattern": "PASS"}
print(shape_refusal(good))

That call checks the definition. It does not read report.md. The full recorder also evaluates supported content checks at write time. The reproduction above tests only shape refusal, so I cannot use it to claim that a report contains PASS.

What does a valid claim still need?

A valid claim needs evidence that matches its wording. A check that a file exists can support an existence claim. It cannot, on its own, support a claim that the file contains a correct answer. The difference matters before you store the result.

The report can exist, open, and look finished while leaving the question unanswered. I described that distinction in checking the result beyond its schema.

I would narrow the claim to what the check covers. If I need the broader claim, I need a different check.

What happened to the six old rows?

The six bad rows were retracted, according to the September 23 devlog. The new refusal protects later writes. Retraction deals with claims already in the record. I need both, because changing the recorder does not establish the truth of its history.

That distinction also changes how I read a summary. If an earlier row claimed completion, a later correction has to travel with it. Otherwise the next agent can inherit the old answer after the underlying record has changed.

Keeping evidence in files gives me something I can reopen. It also leaves me responsible for reading the corrections. Writing agent memory to files gives the record a location. It does not make every sentence current.

What did the day's scan fail to establish?

The September 23 devlog recorded a green security summary beside a scan that read zero of five repos. The coverage count was enough to reject a clean-scan conclusion. It said nothing about what a completed scan would find.

This is why I keep the claim and its scope together. A malformed-check refusal addresses input shape. A coverage count addresses what was examined. A result check addresses what happened. Passing one leaves the others open.

The day's evidence stops there: write-time refusals, six retractions, and a scheduled-prompt change with 11 tests. I have no before-and-after count showing fewer bad claims across later runs. That outcome remains unmeasured.

Accompanying prompt

What the prompt does: It reviews whether one agent claim has enough evidence, without treating a well-formed row as proof of completion.

Copy/paste this prompt:

Role:
Review one agent claim against its evidence.

Context:
I will provide the claim, check definition, actual check result,
and intended scope. Treat missing inputs as unknown.

Task:
1. Identify missing or malformed check fields.
2. Compare the claim with what the check can establish.
3. Read the actual result and coverage, if supplied.
4. Name the evidence still needed for the full claim.

Output:
Verdict: supported, unsupported, or unknown.
Reason: cite the supplied evidence.
Scope: state exactly what the check covered.

Constraints:
Do not invent a result or run a command.
A valid row is not proof that a check ran.
Zero coverage cannot support a clean-scan claim.
Partial coverage cannot establish the whole scope.
Later corrections override earlier completion summaries.

Copy the block above.

See the agent operation I am building

Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=i-built-refusals-into-my-agent-s-data-recorder-2026

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.