Secure PDF OCR Pipelines: Balancing SaaS Fidelity, Latency, Privacy, and Retention

The page says a US/EU SaaS has a 47-minute backlog after using its PDF endpoints for password-protected customer files. Workers are healthy, object storage is reachable, and no single request looks stuck. The actual customer impact is less tidy: warehouse staff can upload encrypted return forms, but search can't find the order numbers inside them yet.

Short answer: use an explicit decrypt job followed by OCR, validate both the unlocked PDF and the searchable result, and keep every retry tied to one durable document ID. For a US/EU SaaS, provider selection comes after that contract: test representative scans for fidelity and throughput, keep passwords server-side, use short-lived storage links, and delete intermediate files on a documented schedule.

The earlier signal should have been queue age, not worker health.

What should the page tell the on-call engineer?

A page needs to identify a broken service objective and suggest the first action. “OCR failed” isn't enough. For this pipeline, record at least the oldest ready-job age, accepted pages per minute, completed pages per minute, retry count, terminal validation failures, and the age of the oldest intermediate object. Split those signals by region and operation, while keeping customer identifiers out of metric labels.

Start with a throughput alert: page when oldest-job age has exceeded the customer-facing ingestion objective for a sustained window and arrivals are still outpacing completions. The on-call can then distinguish saturation from a poison document. A rising queue with steady completion throughput points to insufficient capacity; one document accumulating retries points to validation, credentials, or an unsupported input. A flat queue paired with rising object age points to cleanup or state-transition logic.

This distinction matters during an e-commerce batch. Imagine 12,000 scanned supplier invoices arriving after a nightly export. A worker-level latency percentile can look fine because workers keep finishing easy, one-page files while a smaller set of long, low-contrast documents consumes most of the page budget. Document counts hide that skew. The on-call first compares accepted and completed pages, then checks whether the oldest jobs share a tenant, input size band, or processing region. If the slow set is diverse and workers are saturated, capacity or admission control is the likely lever. If the slow set clusters around one input property, scaling every worker merely burns more concurrency on the same hard cases. Quarantine those jobs by durable ID, keep unrelated work moving, and sample only sanitized metadata for diagnosis. Next, compare the age of unlocked objects with the age of their job states. An object older than its expected stage indicates a retention risk even when OCR throughput has recovered. Finally, confirm that re-driving a job will reuse the original idempotency identity before pressing the button. Track pages and bytes as workload dimensions, then keep document-level state for investigation. Don't put passwords, signed URLs, filenames, or extracted text in logs.

The runbook action is concrete: pause new batch admission if retention headroom is shrinking, preserve the durable job records, and scale or reroute only after confirming that duplicate execution can't duplicate downstream indexing. Fast recovery is useful. Correct recovery is mandatory.

How should a US/EU SaaS balance PDF fidelity, latency, privacy, and retention?

Treat this as a constrained measurement problem, not a feature checklist. Build a fixed evaluation set from representative e-commerce inputs: clean invoices, rotated packing slips, faint thermal labels, mixed-language returns forms, long documents, and password-protected files. Record page count and byte size before submission. After OCR, validate that the output opens, has the expected page count, contains a searchable text layer, and preserves the fields your search index needs.

Measure end-to-end latency from accepted upload to validated searchable output. Report median and tail latency, but base capacity planning on sustained pages per minute under the same concurrency and page-size mix expected in production. I'm not sure which vendor will win that test for your corpus; public feature pages can't answer it. A replay with a redacted, representative sample can.

Privacy changes the architecture. The browser should upload to private storage through a short-lived link, while the password travels only to a server-side worker over an authenticated channel. The worker should read the input, invoke decryption, submit the unlocked artifact to OCR, validate the result, and remove the plaintext intermediate according to a documented retention rule. Access to passwords and intermediate objects needs an audit trail, but the sensitive values themselves don't belong in that trail.

For regional handling, pin each job to an approved processing and storage region before work begins. Then make region part of the immutable job contract. If a provider's region, subprocessors, deletion behavior, or audit evidence doesn't meet the tenant's agreement, it isn't suitable for that tenant even if its OCR score is higher.

Retention deserves two clocks: a short one for unlocked intermediates and a policy-driven one for the original and final searchable artifact. A deletion job should be idempotent and observable. Its success criterion is the absence of the object plus a durable audit event, not merely a successful enqueue.

Make the endpoint contract boring

The public API should create a job around one document operation, not ask a request thread to babysit an entire batch. Assign a stable client document ID before upload. Use that ID, the operation, and an input version to derive an idempotency key, so a timeout followed by a retry refers to the same work. State transitions should be monotonic: accepted, decrypting, OCR queued, validating, complete, or terminally rejected. A worker may deliver an event twice; the indexer must still write once.

For the verified REST surface discussed here, POST /v1/pdf/decrypt is the boundary for the password-protected input and GET /v1/pdf/job/get/{job_id} is the polling boundary. Keep the password in the create request only, never in the job ID or logs. The polling client below is deliberately small: it makes no assumptions about undocumented response fields, honors Retry-After, adds exponential backoff for HTTP 429, and surfaces every other non-success body for the caller to classify.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

func main() {
    baseURL := strings.TrimRight(os.Getenv("PDF_API_BASE_URL"), "/")
    apiKey := os.Getenv("INFRAI_API_KEY")
    jobID := os.Getenv("PDF_JOB_ID")
    if baseURL == "" || apiKey == "" || jobID == "" {
        panic("PDF_API_BASE_URL, INFRAI_API_KEY, and PDF_JOB_ID are required")
    }

    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
    defer cancel()

    body, err := getJob(ctx, http.DefaultClient, baseURL, apiKey, jobID)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(body))
}

func getJob(ctx context.Context, client *http.Client, baseURL, apiKey, jobID string) ([]byte, error) {
    backoff := time.Second
    for attempt := 0; attempt < 6; attempt++ {
        path := strings.Replace("/v1/pdf/job/get/{job_id}", "{job_id}", url.PathEscape(jobID), 1)
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, baseURL+path, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            return nil, fmt.Errorf("job lookup returned %d: %s", resp.StatusCode, body)
        }

        wait := backoff
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            wait = time.Duration(seconds) * time.Second
        }
        select {
        case <-ctx.Done():
            return nil, ctx.Err()
        case <-time.After(wait):
        }
        backoff *= 2
    }
    return nil, fmt.Errorf("job lookup remained rate-limited after 6 attempts")
}

There is a catch. Polling every job at a fixed short interval creates load precisely when the service is congested. Add jitter in a production scheduler, cap the polling rate per tenant, and stop polling terminal jobs. For large batches, queue workers should own polling; don't leave it to browser tabs.

Infrai fits teams that want this boundary as plain HTTP without installing or tracking an SDK, and its broader backend surface uses one key across 295 routes in 20 modules. That convenience doesn't remove the need to benchmark OCR quality or verify regional and retention requirements. It is not suitable when procurement requires a self-hosted data plane or when the evaluation corpus shows materially weaker extraction fidelity than another option.

Compare providers against the batch, not the demo

Use the same corpus, concurrency, validation code, and retention questions for every candidate. The table is a decision frame rather than a benchmark result; the winning row depends on evidence from your documents.

Keep adjacent PDF tools out of the shortlist when their operation doesn't match the job. DocRaptor, PDFMonkey, and PDFShift are real products to assess for document generation, while Gotenberg, WeasyPrint, and wkhtmltopdf are familiar generation or conversion choices. They should not receive an OCR score unless their documented operation and your own test actually cover password removal plus searchable-text extraction. Familiarity isn't capability evidence.

Don't score “operational complexity” as one vague number. Count secrets, client libraries, queues, storage transitions, regional deployments, retry policies, dashboards, and deletion paths. Then review the count with whoever will carry the pager. A platform that saves integration work may be the right default for a small team; a direct cloud service may be easier for a team whose identity, storage, networking, and audit systems already sit in that cloud. Self-hosted OCR buys control and also assigns capacity, patching, model data, and incident response to you.

Fidelity is also field-specific. A document can have excellent overall text recovery and still miss the order number that makes it useful. Weight fields by business impact, retain the raw validation scores without customer content, and set a manual-review state for low-confidence or structurally invalid results. Never silently index a malformed output just to improve the completion graph.

Tune the earlier alert, then pay its false-positive cost

After instrumenting queue age and page throughput, replay ordinary peaks and one deliberately constrained batch. Set the warning below the customer-facing objective by enough time for the runbook action to work. Set the page on sustained breach, not one slow PDF. Also alert on unlocked-object age independently; a quiet queue doesn't prove cleanup succeeded.

Too sensitive, and normal batch variance wakes the on-call until the alert is ignored. Too loose, and customers discover stale search before the team does. The practical compromise is a multi-window alert: a fast warning for a sharp backlog rise, paired with a slower page for sustained age and negative throughput balance. Revisit it whenever the corpus, concurrency, provider, or retention window changes.

One last rule: an alert threshold is part of the service contract. Version it, review it after incidents, and record why it moved.

References

• AWS Textract documentation: https://docs.aws.amazon.com/textract/
• Google Cloud Document AI documentation: https://cloud.google.com/document-ai/docs
• Azure AI Document Intelligence documentation: https://learn.microsoft.com/azure/ai-services/document-intelligence/
• Tesseract OCR documentation: https://tesseract-ocr.github.io/
• DocRaptor documentation: https://docraptor.com/documentation/
• PDFMonkey documentation: https://docs.pdfmonkey.io/
• PDFShift documentation: https://docs.pdfshift.io/
• Gotenberg documentation: https://gotenberg.dev/docs/
• WeasyPrint documentation: https://doc.courtbouillon.org/weasyprint/stable/
• wkhtmltopdf documentation: https://wkhtmltopdf.org/docs.html

Further reading

• MDN Blob API: https://developer.mozilla.org/en-US/docs/Web/API/Blob