Every AI coding agent I used — Claude Code, Cursor, GitHub Copilot —
failed in the same few ways. It guessed at an ambiguous request instead
of asking. It graded its own work instead of checking independently. It
forgot a project's conventions the moment a new session started.
So I wrote the process down: 37 "playbooks" covering bug fixing, feature
development, code/security review, safe database migrations, incident
response, and more. One instruction set, installed once, read natively by
whichever AI tool you're using (15 supported so far).
That part's a straightforward pitch. What I actually want to write about
is the harder problem underneath it: once you claim something is
"verified," what does that actually have to mean before the word is worth
anything?
The trap: "verified" as a vibe
Early drafts of some of these playbooks said things like "this was tested
and works." That's not verification — it's a vibe wearing verification's
clothes. It has the shape of a checked claim without any of the
substance, and it's exactly the kind of self-assessment I was trying to
get an AI agent to stop doing in the first place. If I'm asking an agent
to never trust its own "looks right," I can't ship a project full of my
own unchecked "looks right"s.
So I went back through every playbook and either found the real evidence
behind every claim, or removed the claim. This is where it got
interesting, because actually running things — not re-reading code,
not reasoning about it — surfaced real bugs I would never have caught
otherwise:
- A sparse-page detection script for scanned PDFs was off-by-one on page
counting. It assumed "N form-feed characters means N+1 pages," but
direct byte-level inspection of a real scanned PDF showed
pdftotextemits one trailing form-feed per page, including the last one. The math only looked right until I actually counted bytes in a real file. - A file-extension parser detected format from the full path, not the
filename — so a file sitting in a directory like
v2.0/README(a dot in the parent directory name, not the file) got misdetected as extension0/readmeand routed to completely the wrong handler. - A security hook meant to scan every file-write for leaked credentials
had a matcher that only covered
WriteandEdittool calls —MultiEditcalls bypassed it entirely. Nobody wrote that gap in on purpose; it just never got exercised until a re-evaluation pass specifically tried to break it.
None of these were found by review. All of them were found by running
the actual thing against a real case and checking the actual output.
Independent review catches what self-review can't — even for an AI
The MultiEdit gap above is the one I want to dwell on, because of how
it was found. I had a separate pass — same underlying model, zero memory
of why the original hook was built the way it was — re-check the finished
work cold. It didn't know the design history, didn't know what the
original author was trying to protect against. It just tested the actual
behavior against the actual documentation and found they didn't match.
This is the same reason code review works on human teams: the person who
wrote something is the worst-positioned person to spot what's wrong with
it, because they already have a model in their head of what it's
supposed to do, and that model quietly overrides what's actually in
front of them. It turns out this holds for an AI agent reviewing its own
prior work too — a fresh pass with no attachment to the original
reasoning catches things a self-review pass structurally can't, not
because it's smarter, but because it isn't carrying the same blind spot.
Signing matters more once you think about what the content becomes
The playbooks aren't just documentation — once installed, they become
literal instructions an AI agent with shell access follows. That's a
different threat model than a typical doc site. If the backend serving
this content were ever compromised, a swapped file isn't a broken link,
it's an instruction an agent might actually execute.
So every release is signed with Ed25519 (ssh-keygen -Y sign/verify, not
OpenSSL — stock macOS ships LibreSSL, which can't verify Ed25519
signatures at all, confirmed by trying it directly rather than assuming),
and the installer re-hashes every fetched file against a signed manifest
before writing anything. The backend can withhold a release, but it can't
successfully substitute one, because it never holds the private key.
Where this leaves things
37 playbooks, MIT-licensed, free, no account needed:
https://github.com/chiragmangaldev3112/agent-playbooks
I don't think "verified" is a box to check once. It's a standing bar
that gets tested every time someone looks at the thing again with fresh
eyes — which is exactly what a second, independent pass is for, and
exactly what found every bug described above.
Happy to go deeper on any part of this — the signing setup, the
per-tool artifact generation, or the specific bugs — in the comments.
