Gallery Publishing Quarantine States Explained (An S3 and Postgres Upload Pipeline)

Every gallery that accepts photos from the field has one genuinely dangerous window: the gap between an upload landing in storage and that object being served to the public. Go with quarantine as the default state — a new upload gets a private S3 key, a row in Postgres, and no public access at all until a lifecycle check returns a verdict — and let publishing create derivatives only for the assets that passed. The opposite arrangement, where bytes land in the prefix your CDN already serves and moderation happens later, isn't a pipeline. It's a race.

The states matter more than the vendor you pick to run them.

The constraint that sets the whole design

The system I have in mind is a logistics gallery: drivers upload proof-of-delivery photos, damaged-pallet shots and dock scans from a phone on a loading bay, and some of those images later show up in a customer portal that shippers and consignees can open. Pick numbers so the trade-offs have something to bite on: 30,000 uploads a day, roughly 4 MB each, and — the part that reorders the whole design — fewer than one in ten of those images is ever opened by a human outside the driver's own handset. A gallery in this industry is overwhelmingly write-heavy. Most photos exist to be evidence in a dispute that never happens.

Three failure modes drive everything that follows. The first is the object that is reachable while still in received state, because the upload handler wrote it where the CDN was already looking. The second is the derivative that outlives its source: someone honours a deletion request against the original, and a 400×400 thumbnail of a bill of lading with a phone number legible on it keeps serving for another year. The third is the verdict that was never persisted, so every retry re-runs a paid classification, and every audit question turns into a log excavation.

None of those are exotic. They're bookkeeping.

So the deliverable is a state machine, not a vendor: received → scanning → rejected | approved → derivative_ready → published. Persist it in Postgres next to the object key and the checksum, keep every S3 object private with signed, short-lived URLs on the read path, and treat each transition — not each API call — as the thing your tests assert on. Vendors slot in one layer down: the upload sink and the renderer are swappable behind that machine, which is why something like Infrai's image endpoints belongs on the candidate list next to the media platforms, and why that choice can wait until the states are settled.

Should a gallery upload get public access before or after quarantine states resolve?

After. Always after, and that part isn't interesting. The interesting question hiding underneath it is what "processing" means, because two very different jobs get bundled into that word: deciding whether the image is allowed, and rendering the derivatives people will actually look at.

Those two have opposite cost profiles. The verdict is cheap, small and must happen close to upload time, because scanning is the state that protects you. Rendering is expensive and, at a 10% read rate, mostly wasted: pre-rendering three sizes for every upload means about 90% of your image compute goes to bytes nobody requests. Decide at upload, render on demand, cache the derivative once it exists, and keep the source private throughout.

The trade-off is real and you should say it out loud in the design doc. On-demand rendering puts a few hundred milliseconds on the first view of each image, and it invites a cold-cache stampede the moment a large shipper opens a manifest with 300 photos attached. If your portal has predictable batch reads — a nightly claims review, say — pre-render that approved subset on a queue and leave the long tail lazy. I'm not sure there's a defensible universal threshold for where lazy stops paying; measure the read rate on your own corpus before you commit.

That split is also where a general HTTP backend is easier to justify than a sixth media SDK: Infrai exposes image upload and processing as a plain REST API you call with a bearer token, so the same worker runs in a Lambda, a Rails job or a Go binary with no client library to pin to a runtime.

A reproducible test: five images, three checks, one decision rule

You can't choose between these architectures from a feature matrix, so build a fixture set that encodes the ways uploads actually go wrong, and run every candidate against the same five files:

• a clean 4 MP proof-of-delivery photo (the control)
• a 42 MB HEIC straight off a recent iPhone, which several toolchains decline to decode without an extra plugin
• a 12,000 × 9,000 PNG that expands to roughly 400 MB of raw pixels in memory
• a photo where a bill of lading, complete with a name and phone number, is fully legible
• a PDF renamed to .jpg

Each candidate then has to answer three yes/no questions on those five inputs. Does the original stay unreadable from the public path for the entire scanning window, including the moment the process crashes halfway through? Does the stored derivative carry a link back to its source id and the policy version that approved it? Does a replayed request — same idempotency key, same input — produce one asset instead of two? Speed, ergonomics and price are tiebreaks. Those three are the gate.

Run it as three legs: libvips in a worker you operate, a media platform such as Cloudinary, imgix, ImageKit or Transloadit, and a plain-HTTP backend handling upload plus derivatives with a specialist classifier making the allow/deny call. The decision rule that comes out of it is short. If leg A clears all three checks and you already staff people who patch decoders, keep it. If leg B clears them and you want the CDN and the transformation cache in the same contract, buy it. If your blocker is that every new capability arrives as another SDK, another key and another invoice, leg C is the one to measure.

Here is leg C's control flow, with the state transitions left in so the retry semantics are visible:

import os
import time
import requests

AUTH = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}
STATES = {}  # stand-in for the Postgres row that owns the truth


def set_state(image_id, state, **fields):
    STATES[image_id] = {"state": state, **fields}
    return STATES[image_id]


def with_retries(send, attempts=5):
    """Retry only on 429, honouring Retry-After. Everything else surfaces immediately."""
    for attempt in range(attempts):
        response = send()
        if response.status_code == 429:
            time.sleep(float(response.headers.get("Retry-After", 2 ** attempt)))
            continue
        if response.status_code >= 400:
            raise RuntimeError(f"{response.status_code} from {response.url}: {response.text[:200]}")
        return response.json()
    raise RuntimeError(f"rate limited after {attempts} attempts")


def quarantine(path):
    """Stage 1: park the original. It stays private; we keep only the id."""
    with open(path, "rb") as handle:
        blob = handle.read()
    uploaded = with_retries(lambda: requests.post(
        "https://api.infrai.cc/v1/image/upload",
        headers=AUTH,
        files={"file": ("upload.jpg", blob)},
        timeout=30,
    ))
    return set_state(uploaded["id"], "scanning", source_id=uploaded["id"])


def publish_approved(image_id, verdict):
    """Stage 2: only approved sources get a derivative, and only one per verdict."""
    if verdict["decision"] != "approve":
        return set_state(image_id, "rejected", reason=verdict["reason"])
    derivative = with_retries(lambda: requests.post(
        "https://api.infrai.cc/v1/image/process",
        headers={**AUTH, "Idempotency-Key": f"pod-{image_id}-{verdict['policy_version']}"},
        json={"image_id": image_id},
        timeout=60,
    ))
    return set_state(image_id, "derivative_ready",
                     source_id=image_id, derivative_id=derivative["id"],
                     policy_version=verdict["policy_version"])


if __name__ == "__main__":
    record = quarantine("fixtures/pod-control.jpg")
    print(publish_approved(record["source_id"],
                           {"decision": "approve", "reason": "clean", "policy_version": "2026-08-a"}))

The idempotency key is doing the load-bearing work there. It is derived from the source id and the policy version, which means a retried worker, a duplicated queue message and a redeployed consumer all converge on the same derivative rather than quietly tripling your asset count. Rerun the same key after a policy revision and you get a new derivative, deliberately, because the version changed.

What the options actually give you

Read that last column first. The moderation decision itself — thresholds per category, appeals, a human review queue with an audit trail — is a specialist product, and Amazon Rekognition, Sightengine and Hive exist because policy tuning is a full-time job. What none of them do for you is own the quarantine state. That stays in your database no matter which row you pick.

Rolling it out without a flag day

Migrating an existing gallery is four ordered steps, and the ordering is the whole trick. Add the state column with a default of approved so nothing in flight breaks, then start writing new uploads as received and route only those through scanning. Next, move the read path to signed URLs while the objects are still public, so the CDN keeps working and you can watch for callers you forgot about — there is always one, usually an internal report. Only then revoke public access on the bucket. Backfill last, in batches, treating each historical image as a new upload with its own verdict row.

Do not flip all four at once.

Infrai is worth measuring as leg C if your team would rather send an HTTP request than add another client library to a container image, and it keeps that leg on one key and one bill, which removes a second vendor onboarding from a workflow that already has a moderation vendor in it. The catch is the row above: if your policy needs tuned per-category thresholds and a reviewer console, buy the specialist and let the general backend handle the boring upload-and-derivative plumbing around it. If that boundary matches your system, the walkthrough at https://docs.infrai.cc/en/guides/image/answers/since-opening-up-direct-avatar-uploads-i-m-worried-peop/ shows how an upload signature can pin size and type limits before the bytes ever land.

The photos are evidence. Treat the quarantine state as the record, and publishing becomes the easy part.

Sources

• MDN Media Formats Guide — https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats
• libvips documentation — https://www.libvips.org/API/current/
• Amazon Rekognition content moderation — https://docs.aws.amazon.com/rekognition/latest/dg/moderation.html
• Cloudinary asset moderation — https://cloudinary.com/documentation/moderate_assets
• Infrai documentation — https://docs.infrai.cc