I built the PDF-to-Markdown API expecting layout to be a solved problem. It isn't.
The failure mode was subtle and brutal: academic papers with two-column layouts came back as interleaved garbage. Line one of column A, line one of column B, line two of column A... The Markdown looked fine at a glance — headings, paragraphs, lists all present — but the reading order was destroyed. I fed one into a RAG pipeline and the retriever started quoting sentences that spanned two unrelated columns. The chunks were technically valid; semantically they were nonsense.
My first fix was naive: split the page down the middle by x-coordinate. Worked great until a figure spanned both columns and the caption got shredded. Then a wide table. Then a single-column abstract followed by two-column body. The geometry was lying to me in a new way every time.
What actually worked was a hybrid: detect column boundaries from the whitespace gutters — the vertical runs of empty space between blocks — then fall back to single-column flow when no stable gutter exists. And always respect explicit reading-order hints when the PDF has them (tagged PDFs, which are rarer than they should be).
The lesson I keep relearning: layout is semantic. A PDF isn't text with coordinates; it's a visual argument about how ideas relate. When you flatten it to Markdown, you're not extracting text — you're reconstructing the author's intent. Every column, every sidebar, every pull-quote is a decision about what matters together.
If you're building document pipelines, test on papers with figures spanning columns, tables with merged cells, and multi-column footnotes. The pretty PDFs are easy. The messy ones are the spec.
I ended up shipping it as part of the PDF-to-Markdown converter: https://x402.freeq.one/tools/pdf_to_markdown.html
