Originally published at aideazz.xyz — cross-posted here with canonical link.
A field note from the AIdeazz AI Lab — a real incident on a live production system, written up from the logs. August 23, 2026.
A publishing bot ran out of credit, fell to a weaker model, and spent ten weeks describing a database the system has never used.
What it looked like from outside
A routine check before a technical interview found a published article describing this system's use of Redis distributed locks for LangGraph checkpointing — SETNX and DEL calls, a managed Redis cache, 5-15ms lock overhead, roughly 50ms per checkpoint operation, and Pydantic models for schema migration. None of it exists. The pipeline it described uses SQLite and a TypedDict. The article had been live under the founder's name since publication, and it was not the only one: eleven near-identical articles on the same topic had been published on a weekly cadence since mid-June, seven of them carrying the invented database, one of them mentioning it seventeen times.
What was actually happening
The daily publisher is generative — a scheduled job that asks a model to write an article from a topic brief, then publishes the result automatically. It was never given a source of truth to write from, so the model wrote from general knowledge about the topic rather than from this system. That was survivable while the strongest model answered. It stopped being survivable when the Anthropic balance emptied and the provider chain fell through to a cheaper fallback, which filled every gap it could not verify with the most statistically ordinary answer available. Checkpointing articles usually involve Redis, so the article involved Redis. Nothing failed. The scheduler fired on time, the model returned well-formed prose of the expected length, the publish step succeeded, and the pipeline reported a clean run every single time. The only signal that anything was wrong was the content itself, which no automated check was reading.
The fix
The generative daily publisher is disabled at the flag that gates its schedule, and the process restarted so the change is live rather than merely saved. What remains is the deterministic pipeline: articles assembled field by field from incidents in this journal, where every claim traces to something measured and there is no generation step for a model to fill. Nothing was deleted — the published articles stay up pending a separate remediation pass, because stopping the source matters more than tidying the output, and a cleanup that runs while the tap is open is wasted work.
How I know it worked
Verified from production, not from configuration. Zero Redis packages installed in the environment; the only occurrences of the string in the repository are entries in an applicant-tracking-vendor lookup table, where the Redis company is mapped to the recruiting tool it uses. The pipeline actually runs langgraph 1.0.6 with langgraph-checkpoint-sqlite 3.0.3 and an AsyncSqliteSaver. The schedule matches the evidence exactly — the publisher is set to 14:30 Panama, and the eleven articles carry publication timestamps between 19:30:15Z and 19:30:23Z. The running process named its own author in the log — Gemini returned 11,646 characters for the article body, immediately after the credit-exhaustion path was taken. After the fix, the same process logs "Daily blog: off" on startup. 117 articles are published in total; the duplicate clusters extend beyond this topic, the largest being ten variants of a single article.
The rule this earned
Redundancy protects availability, not truth. A fallback that returns text has satisfied every check the system knows how to run, and none of those checks ask whether the text is true — so a provider downgrade is an editorial event, not merely an operational one. Never let a model re-tell a fact it cannot inspect: assemble anything published under a human name from fields that were measured, and record which model wrote it, so that "who has been speaking for me since June" is a query rather than an excavation.
The named concepts behind it
Naming a failure mode is what makes it possible to recognise the same shape somewhere new, before it costs another weekend.
Fabrication under degradation
When the good model runs out, the fallback does not go quiet — it goes confident.
Every serious AI pipeline has a fallback chain, and the chain is right: when one provider fails, another answers, and the work continues. That is redundancy doing its job.
But redundancy protects availability, not truth. A retry that returns text has succeeded by every measure the system knows how to take. It ran, it returned, it was well-formed, it was the right length. Nothing in that check asks whether the text is true.
This is what makes the failure mode dangerous. A degraded model asked to write about a system it cannot inspect does not stop and say "I do not have this detail". It fills the gap with the most statistically ordinary answer — the thing that architecture usually uses. Asked about checkpointing, it reaches for Redis, because most checkpointing articles involve Redis. The output is fluent, technically plausible, internally consistent, and describes infrastructure that does not exist.
Compare it to a silent failure, which produces nothing and tells nobody. This produces something, and that something is worse, because it passes every automated check and every casual human read. Volume makes it worse still: a pipeline on a schedule does not fabricate once, it fabricates on a cadence, and each copy looks as reasonable as the last.
The defences are structural, not editorial:
- Never let a model re-tell a fact it cannot verify. Assemble published claims deterministically from fields that were measured. A template that interpolates a verified number cannot invent a different one.
- Name the writer in the artefact. If the output records which provider produced it, "everything since June was written by the fallback" is a query rather than an archaeology project.
- Treat a provider downgrade as an editorial event, not just an ops event. Credit exhaustion silently changes who is speaking in your name. That deserves an alert, not a log line.
- Cap the blast radius. Anything published automatically, under a real person's name, on a public surface, should require a verified source — or require a human before it goes out.
The reputational asymmetry is the part worth internalising. A crash costs you an afternoon. Published fabrication costs you the credibility of everything true you ever wrote next to it — and it is discovered by the reader, not by you.
Silent failure
The system did something reasonable, and told nobody.
The most expensive bug class there is, because the clock keeps running while everyone assumes things are fine.
A silent failure is not a crash. A crash is loud and gets fixed. A silent failure is a component making a defensible local decision -- drop this message, skip this record, return an empty string -- that nobody downstream is told about. From the outside, a system that is working perfectly and a system that is completely dead can produce the identical observation: nothing happened.
The defence is not "add more logging". It is to make the healthy state provable, so that "nothing happened" can be distinguished from "nothing was supposed to happen". Two things do that:
- Log the outcome, not the attempt. "sending notification" tells you nothing. "notification DELIVERED (id 4661)" versus "notification REJECTED 400" tells you everything.
- Run a canary. A synthetic transaction pushed through the real path on a schedule, which shouts when it does not come out the far end. Without one, you are relying on a customer to report your outage.
Verify from logs, not config
Configuration tells you what somebody intended. Logs tell you what happened.
A setting, an environment variable or a present API key is a statement of intent. It is evidence that somebody meant for a behaviour to occur. It is not evidence that the behaviour occurs.
The gap between the two is where the longest outages live, because reading the configuration feels like verification. It produces confident, wrong statements: the key is set, so the provider works; the schedule says every fifteen minutes, so it runs every fifteen minutes; the file was deployed, so the new code is running.
Each of those has a cheap, decisive check that costs seconds:
- Probe the dependency, do not read its credential. A key that exists proves nothing about the balance behind it.
- Grep for the action line, not the setup line. A startup banner proves the process started, not that it ever did its work.
- Compare timestamps after a deploy. If the running process is older than the file on disk, it is still executing the previous version from memory.
The rule this earns: never report a system's behaviour from its configuration. Grep the line that proves the behaviour happened, and quote it.
This note is one entry in a running wiki of production engineering lessons — every concept linked to the incident that taught it — at aideazz.xyz/ai-ops-wiki.html.
No customer data, credentials, hostnames or internal record identifiers appear in these write-ups.
