I built a universal document-to-Markdown API that auto-detects file type and routes PDF, DOCX, PPTX, XLSX, EPUB, HTML or plain text to the right converter. The first version trusted the file extension completely. A user sent a .txt file that was actually a PDF — my API happily tried to parse binary as text and returned a wall of mojibake with a 200 status. Confidence: 100%. Correctness: 0%.
The fix was embarrassingly simple: sniff the first few bytes. ZIP magic (PK\x03\x04) means DOCX/PPTX/XLSX/EPUB — then unzip and check for word/, ppt/, xl/, mimetype entry. %PDF means PDF. < followed by common HTML tags is a coin flip between HTML and XML, so I check for <!DOCTYPE html case-insensitively. UTF-8 BOM or printable ASCII with newlines is text. MIME type from the Content-Type header is a hint, not a truth — browsers and upload forms lie constantly.
The lesson: file extensions are a social convention, not a contract. The only honest signal is the bytes themselves. My detector now runs before any routing logic, and the extension is only used as a tiebreaker when the magic bytes are ambiguous (rare, but it happens with some CSV files that start with plain text).
The universal endpoint is at https://x402.freeq.one/tools/document_to_markdown.html — send it any document URL or base64 bytes, and it figures out what it's looking at before it tries to read it. No more trusting the label on the jar.
