Your AI Doesn't Know the Difference Between Knowing and Guessing
I asked a local model what seven times eight was.
Not in chat. Through a tool-calling setup I wired myself, where the model reaches for tools on its own.
It reached for a tool called "Math." That tool does not exist. It made it up, answered with full confidence, and did the same thing again on a second machine running identical config.
In plain chat, the same model answered 56 without a problem.
Two Failures, Not One
The obvious one is that it couldn't do tool-calling. Math and tool-calling are separate skills, trained separately, and this model had one and not the other.
The failure underneath is the one that matters.
It didn't know it was guessing.
It produced a mechanism it had no basis for and delivered it in the same voice it uses for things it actually knows. No hedge, no flag, no difference in tone between the real part and the invented part.
That's not an AI problem. That's the oldest reporting failure there is, running inside a model instead of inside a document.
I Already Had a Name for It
I built the Cybersecurity Intelligence Framework around three categories, because reports don't get rejected for wrong findings. They get rejected for claims sitting in the wrong category.
Proven. You can point at what supports it. A log line, a response body, a screenshot.
Inferred. Concluded from evidence, not directly stated. Say consistent with, not proves.
Assumed. No source yet. Name it, don't bury it.
✗ "The attacker accessed the database."
no source. stated as fact. category error.
✓ "An authenticated session accessed the database at 02:14,
consistent with the observed pattern."
same evidence. defensible.
Same evidence. Completely different standing.
The model did exactly what the top line does. Took something with no source and stated it as fact. The difference is it can't be taught to catch itself, so the catch has to happen outside it.
Which gives you two jobs. Find where it starts guessing. Then read what it hands back like a report you have to defend.
Job One: Find the Ceiling
Run it twice before you depend on it.
Level one: the simplest version of the task, answer you already know.
Level two: same task, wrapped in the structure you actually plan to use.
# Level 1 - simple, known answer
ask(model, "7 x 8") # -> "56" holds
# Level 2 - same model, wrapped in tool-calling
run_agent(model, "7 x 8") # -> calls tool "Math" (invented)
# -> confident answer, false mechanism
Reliability at level one tells you nothing about level two. If it holds at one and breaks at two, that gap is the ceiling.
Build under it.
Security already works this way. Burp Suite exists because nobody trusts what a browser reports at face value. An AI that invents a tool and a server that returns a clean response it should have rejected are the same failure in different clothes.
Job Two: Read the Output Like a Report
Everything it gives back goes through the same three categories.
Which parts can you point at a source for. Which parts are reasonable but unverified. Which parts came from nowhere and got stated anyway.
It won't mark them for you. One voice for all three.
What I Did With It
I run two environments, one for hunting and one for building, carrying the same local AI layer so a tool written for one runs on the other unchanged.
The tool that hallucinated got built, tested, and removed.
I didn't remove the AI. Direct chat and single-file generation held up fine. Multi-step autonomy didn't. So the automation runs the rote weight, capturing findings, drafting from the ledger, ranking scan output, and stops at every point where being wrong costs something. Those points are mine.
The ceiling test is what told me where to draw that line.
What's in the Toolkit
Twelve chapters, three parts.
Part I is how you think before touching anything. Treating the model as an opponent instead of an assistant. A six-step loop that takes a raw thought through capture, external testing, and defense before it earns a place anywhere. The ceiling test. A test for whether you needed to build a tool at all or whether a conversation with the right constraints already does the job. Learning a new team by what people actually own instead of what their title says. Compressing a concept to one sentence with no technical words. Then the three categories in full.
Part II is what to use and where to go. Real tool names by role: recon, intercept, pipeline, monitoring, and where the AI layer sits among them. A five-step certification path, ordered, prices and links included. Five prompts you paste your own situation into.
Part III is where each of these got tested and what it cost when it failed.
The recon workflow the loadout compresses is in the Bug Bounty Starter Kit. The reasoning system behind the three categories is in the CIF. This one sits across both.
Free preview covers the posture and the ceiling test. Enough to run on something today.
www.tagzauthor.com/l/ai-cyber-preview
Support TagzAuthor: ko-fi.com/tagzauthor
My author page: Amazon Bookstore
