This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.
A user uploaded a 106-page PDF and clicked "Ask this book".
The screen said "Preparing… 0/0".
It said that for an hour.
No errors. No failed requests. No exceptions in the logs. Just one graph I still think about: a CPU-only container, pinned at 390%. Almost exactly four cores, flat, like a ruler.
The strange part: I pay for GPT-4.1 exactly so this kind of work never runs on my CPU. And the OpenAI dashboard showed zero traffic. Something was doing the work. Just not the thing I configured.
What was supposed to happen
TextStack is a reading app with an "Ask this book" feature. It is RAG over the book you uploaded. For table-heavy PDFs, plain text extraction is useless, so each page goes through vision parsing: page image in, Markdown out. That job belongs to GPT-4.1.
The provider is picked by a routing config. Simplified:
"Ai": {
"DefaultProvider": "ollama",
"Routes": {
"pdf.parse": "openai-pdf"
}
}
If a task has a route, it goes to that provider. If it doesn't, it falls back to the default. The default is a local Ollama, which I use for small cheap tasks.
Keep that fallback in mind. It is the villain of this story.
Debugging by absence
Most debugging starts with an error. This bug gave me none. The strongest clue was a thing that was missing.
Here is what I could see:
- The indexing job was running. Not failed, not stuck in a queue. Running.
- Zero requests to OpenAI. Not slow requests, not errors. Zero.
- The Ollama container was burning four cores. On a box with no GPU.
So the vision parsing was happening. On the local model. On CPU. I measured it later: about 42 seconds per page. The book had 106 pages. That is roughly 74 minutes of parsing for one book, all of it awaited inline by the indexing worker. The whole RAG pipeline sat behind it. From the outside it looked frozen at "Preparing… 0/0". From the inside it was working very hard on the wrong hardware.
Nothing failed. That was the problem. A crash would have paged me in a minute. A fallback that "works" can burn for an hour.
The root cause hurt a little
A few days earlier I shipped a reliability fix. Indexing used to run chunking inside the HTTP request, and it could get permanently stuck if the request died. So I moved the whole thing into a background Worker. Good change. I would do it again.
But the API and the Worker are separate processes, each with its own appsettings.json. The API config had the route:
"Routes": { "pdf.parse": "openai-pdf" }
The Worker config did not.
So when the parsing moved from the API into the Worker, the router looked up pdf.parse, found nothing, and did what fallbacks do. It quietly handed a vision parsing job for a 106-page book to a local model on a CPU-only container.
The work moved. The config did not. My reliability fix caused the regression.
The fix
One config block in the Worker's appsettings.json, mirroring the API:
"Routes": { "pdf.parse": "openai-pdf" }
plus the matching Ai:Pdf settings block. No code change. The container dropped from 390% to idle, OpenAI traffic came back, the book indexed in minutes.
Five minutes to write. Much longer to find.
Why this bug hid so well
Three reasons, and I think they generalize:
A fallback doesn't fail. It succeeds with the wrong tool. Every health check was green. The job was making progress. Slowly, expensively, but progress. There was no moment where the system could say "this is wrong", because by its own rules nothing was.
Nobody alerts on silence. I had alerts for errors and for high latency on my API. I had nothing for "an expensive external provider we depend on received zero calls today". Absence of a signal is the hardest thing to notice, because there is nothing to point at.
Config lives per process, but my mental model didn't. In my head there was one routing table. In reality there were two files, and they agreed only by accident, until they didn't.
What I changed in how I work
Three rules I took from this:
- When work moves between processes, config moves with it. That is now a checklist item on any "move X to the Worker" change: diff the relevant config sections between the two processes before shipping.
-
Make routing loud. One log line per job that says which provider was resolved and why:
pdf.parse -> ollama (no route, DefaultProvider fallback). That line would have turned an hour of confusion into a 30-second fix. - For expensive work, prefer fail-fast over fallback. A missing route for a cheap task can fall back. A missing route for vision parsing of a whole book should throw. A silent fallback is a decision the system makes without telling you. If the choice of provider matters, it should not be made silently.
The uncomfortable lesson on top: the bug was introduced by a fix I was proud of. Reliability work changes where code runs, and "where" is exactly what config is about.
Where in your system could a fallback take over silently, and how long would it take you to notice? Mine took a 106-page PDF and a CPU graph. I would love to hear yours in the comments.
I build TextStack, an open-source reader for technical books, in .NET. This bug is from the RAG indexing pipeline behind its "Ask this book" feature. github.com/mrviduus/textstack
