We ran a real invoice through our own converter and opened the export in Excel.
The date on the page read 11/02/2019. The cell said 1110212019. The total was 9.75. The cell said 975. And Invoice Total $204.75, the one line a bookkeeper actually needs, was not in the file at all. That row did not exist.
Nothing errored. No red toast, no console warning. The file opened cleanly in Excel, the columns were aligned, and every number looked like a number. Which is the worst possible failure mode for a financial document: the output looked exactly as confident as correct output does.
This was not a regression. It was the capability boundary of a pipeline we had built ourselves, and finding out where that boundary sits took an A/B harness, a real browser, and a list of failures we did not expect.
Why run OCR in the browser at all
The popular reason given is privacy, and it is true, but it is not the engineering reason.
The engineering reason is that a converter which never receives the file has no retention problem to solve. There is no bucket to expire, no support ticket that starts with "please delete my upload", no third-party processor to add to a privacy policy. When a bookkeeper photographs a bank statement and hands it to a page, the server side of that transaction is a static file host. It does not know the file existed.
The cost is that you inherit every problem a server would have solved for you: model delivery, memory limits on a phone, and the fact that the OCR engine which runs in a browser is not the same engine you tested in Node.
We wrote about that cost, and about why Tesseract hands you a bag of words instead of a table, in an earlier post. This one is about what happens when that pipeline meets a real invoice.
That last one is where the date turned into 1110212019.
Six failures, none of which raised an error
We rebuilt the invoice sample as a repeatable fixture: one image, one browser, two builds side by side on localhost. Old build on port 8800, new on 8799, same Edge, same machine. Everything below is from that comparison, not from memory.
The slash read as a one. 11/02/2019 came out as 1110212019. At the source resolution, a / in a 9pt font is one to two pixels wide. There is no character there for the model to classify. It reads the stroke as 1 and moves on.
The decimal point disappeared. 9.75 exported as 975. Same cause, different victim. A tenfold error in a tax line, with no marker that anything went wrong.
The largest text on the page vanished. Invoice Total $204.75, the single most important line on an invoice, was dropped entirely. This one only reproduced in the browser. Our Node probe found the line; the browser build, using the SIMD LSTM core, discarded the oversized text block during page segmentation.
That discrepancy is worth pausing on. If we had validated in Node, we would have shipped believing the total was safe.
A decorative bar became a word. A solid blue header bar was returned as the word ree at 66% confidence, bounding box 1282×55. Sixty-six percent is above plenty of thresholds, so confidence filtering alone let it through into the export.
One invoice became six sheets. The page was split by horizontal gaps 2.5× the word height, so every visual block (the header, the address, the line items, the total band) became its own tab.
A misread digit was written as a number. This is the one that actually frightens us. Export logic converted any digit-shaped string into a numeric cell. So 975, a misread 9.75, became a real number in a real column, ready to be summed.
What actually fixed it
We tried four levers separately, so we could see which one was carrying the weight.
| Variant | Words | Key fields correct (of 6) |
|---|---|---|
| Baseline (original image, single pass) | 94 | 2 |
| Upscaled 2× | 96 | 3 |
| Upscaled + auto-levels | 96 | 3 |
| Upscaled + auto-levels + sparse second pass | 97 | 3 |
Upscaling was the main lever. Long edge below 1600px gets an integer upscale capped at 3×. That alone recovered Horne, BILL TO, 9.75, 26/02/2019 and 50.00.
The contrast pass did nothing on this image, and we kept it anyway. The levels-stretch output was byte-identical to the upscaled version, because the invoice already had enough contrast. It fires only when the p1–p99 luma range is narrower than 180, which means washed-out scans and phone photos in poor light. The conditional design is what let us add it without risking the images that already worked.
The second recognition pass mattered only in the browser. A sparse-text pass (psm 11) alongside the default page segmentation is what recovered the total line, in the browser build. In Node it added one word, because Node's build had already found the line. Same code, different result, decided entirely by which WASM core was loaded.
Do not trust confidence alone
The ree case is the one we keep coming back to.
Confidence 66 is not low enough to reject by threshold; plenty of legitimately hard text sits there. But the bounding box was 1282 pixels wide and 55 tall, for three characters. No font renders at a width-per-character ratio of 7.77.
So the filter that kills it is geometric, not statistical:
// width-per-character over height must stay text-like
export const WORD_FILTER = Object.freeze({
maxCharAspect: 2.5,
});
Confidence tells you how sure the model is. Geometry tells you whether the thing could be a word at all. For decorative elements, only the second question has a reliable answer.
The rule we adopted for numbers
Every numeric cell now has to clear a confidence threshold before it is allowed to be a number. Below 75, it stays text.
This is deliberately annoying. A text cell shouts at the user: Excel flags it, SUM() skips it, the bookkeeper notices. A wrong number sits in the column looking correct and quietly poisons the total.
For a document whose entire purpose is arithmetic, a value that cannot be trusted should not be allowed to pretend it is a number. We would rather hand over an obviously broken cell than a plausible wrong one.
There is a matching rule on the other side: when the user manually edits a cell in the preview, we treat that as confirmation and re-evaluate it under the same logic. Otherwise a person who fixes a digit would still get text, because the original OCR confidence never changed.
What this pipeline still cannot do
Here is where the boundary actually sits.
It is geometric reconstruction, not semantic extraction. The engine knows that two words sit on the same horizontal band; it does not know that INVOICE DATE is a key and the value beside it is its value. Key-value pairs do not get aligned. Address lines and metadata lines interleave by y-coordinate.
Number formats are not preserved. 100.00 comes out as 100. For invoices, the cents column carries meaning, and we lose it at the export layer. The export-format side of this (xlsx versus csv versus txt) is in the earlier post; here it stays a limitation we have not solved.
Handwriting is not in scope. A signature comes back as something like Sut, flagged as a low-confidence cell. We keep it rather than silently deleting it, because a flagged cell is information and a deleted one is not.
Multi-page and field-level accuracy are not solved here. If a workflow needs the invoice number, the tax amount, and the vendor name to be correct every time across arbitrary layouts, on-device OCR is the wrong tool. That needs a server-side vision model, and it needs the file to leave the device, which is a product decision rather than a technical one. We have not made it.
The tradeoff we accepted
The borderless document path went from about one second to about three, because it now runs a second recognition pass. Documents with ruled tables are unchanged at one second: if the de-lining step finds a real grid, the sparse pass is skipped, since a ruled table reads cleanly without it.
Three seconds and a correct total, versus one second and 975.
The tool is at ocr2excel.com if you want to test it against an invoice of your own. Recognition runs locally with WebAssembly, there is no signup, and the batch limits are 10 files, 50 MB total, 10 pages per PDF.
If you are building something similar, the two things we would pass on: validate in the browser rather than in Node, because the WASM core you ship is not the engine you benchmarked. And make the numeric path fail loudly, because in financial documents the silent errors are the expensive ones.
