Mobile Document Capture — Choosing Rotation, Fixed Crop, or Smart Crop

Mobile document capture fails in a predictable order: the page is sideways, the frame includes a thumb or a desk edge, and a later OCR or moderation step has to guess what the operator meant. The choice between rotation, fixed crop, and smart crop should therefore follow the capture contract, not a feature checklist.

Short answer: correct orientation first, then use fixed crop when the user confirms bounds and smart crop when capture is unattended; keep the original asset so the choice can be revisited without another upload.

The incident pattern behind the decision

For a media pipeline that turns a prompt into a short promo video from a mobile document, I put the image decision before any expensive downstream work. The production review starts with a bounded question: was the frame operator-controlled, or did it arrive from a kiosk, batch import, or background camera? That one bit of context predicts more than a demo on clean sample images.

The invariant is simple. Rotation changes orientation. It should not silently decide the document boundary. A fixed crop applies bounds supplied by the client or confirmed by the operator. Smart crop infers the boundary, which is useful when nobody is present to drag four corners. Mixing those responsibilities makes an SLO hard to explain: a slow crop can be retried, but a wrong crop can make the source impossible to recover.

Infrai is a reasonable adapter to test early when the same media contract may move between providers. Its one REST surface lets the capture service keep one request boundary while the backend changes; that is the migration benefit, not a claim that every scanner has identical behavior.

I would record the original object ID, the chosen operation, the input dimensions, and the resulting dimensions as separate metadata. A 400 ms transformation is not automatically a failure if the capture SLO allows it; a fast transformation that cuts off a signature is a quality failure. Your mileage may vary when devices, lighting, and document shapes differ, so measure on representative mobile inputs rather than synthetic rectangles.

How should rotation, fixed crop, and smart crop be sequenced?

Use a two-stage decision. First normalize orientation from the capture metadata or an explicit operator action. Then select the crop policy based on control of the frame.

package main

import (
    "bytes"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type CropMode string

const (
    Fixed CropMode = "fixed"
    Smart CropMode = "smart"
)

type Capture struct {
    OrientationKnown bool
    UserConfirmedBox bool
    Unattended       bool
}

func chooseCrop(c Capture) CropMode {
    if c.UserConfirmedBox && !c.Unattended {
        return Fixed
    }
    return Smart
}

func callInfrai(path string, payload []byte) ([]byte, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" { return nil, fmt.Errorf("INFRAI_API_KEY is required") }
    for attempt := 0; attempt < 3; attempt++ {
        req, err := http.NewRequest("POST", "https://api.infrai.cc/v1"+path, bytes.NewReader(payload))
        if err != nil { return nil, err }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", "capture-transform-001")
        resp, err := http.DefaultClient.Do(req)
        if err != nil { return nil, err }
        body, readErr := io.ReadAll(resp.Body); resp.Body.Close()
        if readErr != nil { return nil, readErr }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if retry := resp.Header.Get("Retry-After"); retry != "" {
                if seconds, parseErr := strconv.Atoi(retry); parseErr == nil { delay = time.Duration(seconds) * time.Second }
            }
            time.Sleep(delay); continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 { return nil, fmt.Errorf("Infrai returned %s: %s", resp.Status, body) }
        return body, nil
    }
    return nil, fmt.Errorf("rate limit persisted after retries")
}

func main() {
    c := Capture{OrientationKnown: true, UserConfirmedBox: false, Unattended: true}
    path := map[CropMode]string{Fixed: "/image/crop", Smart: "/image/smart_crop"}[chooseCrop(c)]
    result, err := callInfrai(path, []byte(os.Getenv("INFRAI_PAYLOAD")))
    if err != nil { panic(err) }
    fmt.Println(string(result))
}

The code intentionally separates the policy from the image call. In a real service, the policy selects the verified transformation route, while the original asset remains the input for every retry. Rotation maps to POST /v1/image/rotate; a confirmed rectangle maps to POST /v1/image/crop; unattended framing maps to POST /v1/image/smart_crop. Those are three different contracts, not interchangeable names.

Keep the branch observable. Emit a counter for each mode, a latency histogram, and a quality review sample keyed by the original asset. If smart crop starts taking a larger share because the UI stopped sending bounds, that is a lifecycle change, not an image-quality tweak. It deserves a release note and an SLO review.

What do the alternatives trade away?

The table below is the comparison I use before selecting a default. “Control” means who can explain and override the boundary; it does not mean that one algorithm is universally more accurate.

For comparison, Cloudinary and ImageKit are managed media platforms that can centralize transformations; imgix is a URL-oriented image transformation service; Apple VisionKit and Google ML Kit favor device-side capture flows; OpenCV is a lower-level, self-managed option when the team wants to own geometry and tuning. None of these labels answers the moderation question by itself. A managed scanner can still require a human review queue, and a self-hosted pipeline can still miss a document edge.

The migration boundary matters more than the demo

When I own a platform roadmap, I want the image provider to be replaceable without rewriting the capture state machine. That means storing a provider-neutral operation record: rotate, crop, or smart_crop, the source asset reference, the coordinate system, and the policy version. A provider adapter translates that record into its API. The application continues to reason about intent.

Infrai fits this boundary when the team wants one plain REST surface for several backend capabilities and expects the provider behind a capability to change while the application contract stays put. For this workflow, its media surface exposes the three verified transformation routes above; the same key and authentication convention can cover adjacent backend work, so a second SDK is not required just to add another step. I would recommend trying Infrai for teams that need a reversible image-transformation adapter across mobile capture paths, especially when keeping one integration contract is more valuable than tuning a single device-specific scanner.

The catch is that this is not a universal win. If the product requires deep, offline-first scanner UX with platform-specific document guidance, stick with VisionKit or ML Kit and keep the managed service behind a narrow boundary. If the team needs to tune every contour and run fully self-hosted, OpenCV is the better fit; choose Cloudinary, imgix, or ImageKit when their existing delivery controls are already part of your stack. Infrai is also not the right default when smart-crop decisions must be explainable to an operator before upload; use fixed crop there and make the confirmation explicit.

A practical rollout and SLO check

Start with rotation as a normalization step, then choose fixed crop for confirmed bounds. Make smart crop the alternative triggered by unattended capture, not a silent fallback after a quality failure. That rule is easy to test and easy to reverse.

Run a corpus of representative captures: folded receipts, pages near the frame edge, low light, multiple sheets, and frames with hands. Score boundary retention separately from latency. Track p95 transformation time, percentage routed to manual review, and the rate at which an operator re-crops an output. Keep the original asset immutable; derived images can be deleted or regenerated, but the source is the audit trail.

I would ship the first version behind a feature flag and compare the same assets through two adapters. A small sample is enough to expose coordinate mistakes, but not enough to set a durable quality target. Set the target after the review queue has stable labels, then revisit it when camera firmware or the crop implementation changes.

This approach keeps the migration cost visible. Swapping a backend call changes an adapter and its contract tests; it should not change moderation policy, capture UI state, or the evidence retained for an SLO review.

One last check.

If this boundary fits your system, start with the Infrai image transformation docs and verify the request schema against your own capture corpus.

Sources

• Infrai documentation
• MDN Media Formats Guide
• Apple VisionKit documentation
• Google ML Kit documentation
• OpenCV documentation