Why we vendor 76 MB of OCR models instead of loading them from a CDN

Almost every browser-OCR tutorial starts the same way:

<script src="https://cdn.jsdelivr.net/npm/tesseract.js@5/dist/tesseract.min.js"></script>

One line, no build step, works immediately. We started there too. We don't any more, and the reason is worth writing down — partly because it isn't the reason people usually give, and partly because moving the assets in-house introduced a failure mode we didn't anticipate and didn't catch.

1. The setup: invoices are a bad thing to upload

We build a tool that reads invoices and receipts and turns them into spreadsheets. Invoices are the category of document people are least comfortable handing to a third party: bank details, addresses, client names, and prices you may be contractually forbidden from disclosing.

The obvious architecture is to POST the file to a server, run OCR there, return JSON. It is easier to build, easier to scale, and easier to make accurate. It also means the document is now on someone else's disk.

So we run recognition in the browser instead.

2. What "processed locally" actually means

I want to be precise, because this is the claim people over-promise.

Document content is processed locally in the browser and file bytes are never uploaded. There is no upload endpoint. There is no FormData anywhere in the conversion or extraction pipelines — zero occurrences, and that is a grep you can run against your own codebase in five seconds. There is no server-side OCR provider and no fallback to one: if browser recognition fails, it fails, and nothing is sent anywhere as a retry.

Metadata is transmitted. When a conversion starts, the server receives exactly six fields:

{
  tool,               // which tool was used
  originalFilename,
  inputMimeType,
  inputSize,
  pageCount,
  idempotencyKey,     // so a retry isn't billed twice
}

That is the whole schema — there is no binary field in it. The server uses it to enforce quota and keep a job record. It never receives a page image, a rasterised bitmap, or a line of extracted text.

That distinction matters. "Nothing ever leaves your device" would be false, and I would rather say the accurate thing than the impressive one. If you are evaluating any privacy-first document tool, that is the question to ask: not does it process locally, but what exactly does the server still receive.

3. Two readers, one output shape

Not every PDF needs OCR. A PDF exported from accounting software already contains a text layer with exact character positions — running OCR on it would be slower and less accurate than reading what is already there.

So the pipeline branches:

• PDF with a usable text layer → read it directly through PDF.js.
• PDF without one, or any image → rasterise the page and run Tesseract.

Both paths emit the same positioned-line structure, so everything downstream — layout analysis, field parser, spreadsheet writer — has no idea which reader produced the page. That one decision is what keeps the codebase from forking into two half-maintained pipelines.

4. Tesseract as WebAssembly

Recognition is tesseract.js, a WebAssembly build of the C++ engine, running in a worker. We ship several core variants — a plain build, simd, relaxedsimd, and lstm versions of each — and the browser loads the fastest one it can actually execute. That is 43.25 MB of WASM cores, most of which any given visitor never touches.

5. Why the models are vendored

Tesseract needs a trained model per language. The default is to fetch them from a public CDN at run time. Ours are copied into our own public/ directory by a setup script that runs on predev and prebuild: the worker and cores are copied out of node_modules, so they always match the installed tesseract.js rather than drifting against whatever the CDN currently serves, and the models are downloaded once at build time, never at run time.

6. The CDN incident that motivated it

Our tool reserves quota before processing and releases it if the run fails, so a user is not charged for a conversion that never completed. Recognition depended on assets fetched from a public CDN at run time.

The CDN had a bad day.

The worker fetch hung. Not a clean error — a hang. Conversions sat mid-flight with quota already reserved and no result coming back. The failure was in someone else's infrastructure, on a request our own server never saw, and it looked to users exactly like our product was broken. Functionally, it was.

Two things were wrong with that design, and only one was about uptime:

1. Availability. Our OCR was only as reliable as a third party we had no relationship with and no status page for.
2. Privacy. We tell people documents do not leave their browser. Fetching the engine from a third-party CDN means that third party sees a request, with a referrer, every time someone starts a conversion. No document content — but a request pattern that maps to "this person is about to OCR something." That undercuts the promise in spirit even though it does not break it in letter.

7. The bug that self-hosting introduced

Here is the part I did not expect to be writing.

While fact-checking this article I stopped trusting the feature list and checked which models production actually served. The language picker offered seven options. Three models answered.

GET /tesseract/lang/eng.traineddata.gz  200
GET /tesseract/lang/fra.traineddata.gz  200
GET /tesseract/lang/ara.traineddata.gz  200
GET /tesseract/lang/deu.traineddata.gz  404
GET /tesseract/lang/spa.traineddata.gz  404

The picker read its options from a constant in the OCR module. The setup script downloaded its models from a different constant in a build script. German and Spanish had been added to the first list and not the second — far enough back that they are already present in the earliest commit in the repository's history, so the mismatch predates anything git can date for me.

Because langPath points at our own origin, the request did not fall back to a CDN and quietly succeed. It 404'd, and the conversion died.

That is the tax nobody mentions when they tell you to self-host your assets. On a CDN, the set of languages you offer and the set that exist are the same set, maintained by someone else. The moment you vendor them, that is two lists, in two files, with nothing connecting them.

8. The fix

One line in the build script, plus the two models it then downloaded.

For clarity about what "seven languages" means architecturally: there are five base models — English, French, Arabic, German, Spanish — and two combined modes, English + Arabic and English + French, which load two models into a single recognition pass for bilingual documents. Seven picker options, five files.

Before the two missing models were added it was 61.26 MB. These are the standard 4.0.0 models — byte-identical to what tesseract.js fetches by default. Self-hosting changed where the bytes come from, not what recognition does. Accuracy is unchanged, which was a deliberate constraint: a vendoring change that also silently altered output would be miserable to debug later.

And 76 MB is not page weight. None of it is in the initial bundle. A model is fetched on demand, once, when someone actually runs recognition in that language, and then cached. Someone who only converts an image to PDF downloads none of it.

9. The test that makes it not happen again

A one-line fix for a bug that shipped is not a fix; it is the same bug with a longer fuse. The real change is a test that fails if the two lists ever diverge again, in either direction:

• a language offered but not vendored → a 404 in production;
• a model vendored but not offered → dead megabytes in every deploy.

It reads the model list out of the build script as source text rather than by executing it, so the check needs no network and no downloaded assets, and behaves identically on a laptop and on a cold clone. I ran it against the pre-fix state before trusting it, and confirmed it fails there — a regression test you have never seen fail is a decoration.

The fix is deployed. All five model URLs return 200 in production, and I re-ran every one of the seven picker options against the live site — five base languages and both combined modes — confirming each loads its model, initialises the engine, and returns the expected text. Verifying in production rather than in a local build was the whole point: the bug only ever existed in what was deployed.

If you take one thing from this piece: when you move assets in-house, add a check that every option in your UI resolves to a file that exists.

10. Exact privacy boundaries

11. What you give up

Browser-side OCR is a real trade, not a free win:

• It is slower. You are running an OCR engine in a tab, competing with everything else on the page, on hardware you do not control.
• Model size is your problem. Every language you add is several more megabytes you are choosing to host, and a decision you can no longer defer to a CDN.
• Accuracy varies with the input. A clean 300 DPI scan is fine. A dim phone photo at an angle is not, and no amount of client-side cleverness fixes bad pixels.
• Handwriting is not what this is for. Tesseract is built for printed text. We do not market it for handwritten notes, because it is not good at them.
• Document complexity has limits. Multi-column layouts, dense tables, and stamped or overlapping text degrade the positional analysis the field parser depends on.

Because accuracy varies with input quality, the honest interface reports its own uncertainty. Every extracted field carries a confidence score and a needsReview flag, and both are written into the exported spreadsheet rather than shown once and forgotten. A wrong total that announces itself is recoverable. A wrong total that looks confident is the failure mode that costs someone money.

12. Would I do it again

Yes — but the deciding factor was not privacy, it was control. Privacy is what we can now describe precisely instead of approximately. Control is what stopped a stranger's CDN outage from looking like our bug.

The honest addendum is that self-hosting moved a class of failure from outside our control to inside it. That is an improvement, but only because a failure inside your control is one you can write a test for — and we did not write that test until two languages had been quietly broken for longer than our git history goes back.

If you are processing documents people would hesitate to email you, the browser is a genuinely good place to do it, provided you are honest about the trade: slower, input-sensitive, and you own every megabyte you ship.

Written from the implementation of EasyInvoiceOCR, which reads invoices and receipts in the browser.