Short answer: keep the original auction image private until its metadata passes a small, deterministic gate; publish derivatives only after moderation has a traceable input and an expiry policy.
That sounds like a media task. In a logistics auction, it is a records task with pixels attached. A pallet photo can carry a useful capture timestamp, a GPS coordinate that should never be public, or an orientation flag that a thumbnail worker quietly ignores. If the intake record is wrong, every public derivative repeats the mistake at a different size.
What the bill is actually made of
The visible storage bill is rarely the whole cost. The larger term is retention multiplied by copies: original uploads, quarantine copies, thumbnails, inspection crops, CDN objects, and failed jobs left behind for a retry. I once traced a “cheap” gallery to 11 objects for one 4 MB photograph. The image itself was innocent; our retry path kept each intermediate file for 30 days.
Start with a ledger, not a resize command. For each asset, record byte size, hash, dimensions, media type, metadata decision, derivative set, and deletion deadline. The hash lets a duplicate upload share a retention decision without trusting a filename.
| Class | Keep private | Make public | Retention decision |
|---|---|---|---|
| Original | Full bytes and complete metadata | Never | Until the auction record and dispute window close |
| Review copy | Stripped pixels for moderation | No | Delete after a short review window |
| Listing derivative | Orientation-correct pixels, scrubbed metadata | Yes | Match listing lifetime |
| Audit record | Hash, ruleset version, reviewer result | No | Keep as long as policy requires |
The change that moves the dominant term is usually deleting intermediate copies, not shaving a few kilobytes from a JPEG. Generate a review copy in memory or an isolated scratch area, publish one derivative family, then remove the scratch object when the job is acknowledged. The catch is operational: if you delete the only original before a buyer dispute is resolved, you lose evidence and may have to relist from a seller upload that no longer exists.
Keep it boring.
How should auction image intake validate metadata before public derivatives?
Treat metadata validation as a gate with explicit outcomes: accept, quarantine, or reject. “Accept” means the parser recognized the media type, dimensions are within policy, orientation is resolved, and sensitive fields are removed from the public representation. “Quarantine” means a human or a second parser must decide. “Reject” is for a file that violates a hard rule, such as an unsupported format or a decompression limit.
Here is a small policy function. It does not parse bytes; that belongs to a maintained image library. Keeping the policy pure makes it easy to test with fixtures from real camera phones and warehouse scanners.
from dataclasses import dataclass
@dataclass(frozen=True)
class Metadata:
media_type: str
width: int
height: int
orientation: int | None
has_gps: bool
def decide(meta: Metadata) -> str:
allowed = {"image/jpeg", "image/png", "image/webp"}
pixels = meta.width * meta.height
if meta.media_type not in allowed:
return "reject"
if pixels == 0 or pixels > 40_000_000:
return "quarantine"
if meta.orientation not in {None, 1, 3, 6, 8}:
return "quarantine"
if meta.has_gps:
return "accept_after_scrub"
return "accept"
The accept_after_scrub branch is deliberate. GPS is not a reason to discard a useful lot photo, but it is a reason to prove that the public derivative no longer contains location tags. Store the original in a restricted bucket, apply the scrub, and verify the output by reading it back. A filename ending in .jpg is not evidence of a JPEG; inspect the declared type and the decoded structure.
Where retention and moderation collide
Moderation coverage is a decision axis, not a checkbox. A tiny thumbnail can hide a hazardous label; a full-resolution original can expose a worker's face or a warehouse address. Route the original to a restricted review queue, and let the moderator see the dimensions and capture context without making that object cacheable by a public URL.
I keep two clocks: a publication clock and an evidence clock. The listing derivative expires with the auction listing. The evidence clock covers disputes, takedown requests, and compliance holds. They should be represented as timestamps in the asset ledger, because “delete after review” is not a reproducible instruction when a queue is paused over a weekend.
Short queues fail in boring ways. A worker may finish moderation after a derivative has already been cached, or a seller may replace an image while the old URL remains valid. Version the asset ID, include the ruleset version in the audit record, and send cache invalidation only after the new decision is committed. I've seen a 202 response treated as success even though the metadata scan had not run; the public thumbnail then outlived the quarantine record. The incident took an afternoon to reconstruct because the job table recorded only an auction ID and a final URL, while the object store had three files with the same basename. We had to compare byte hashes, inspect CDN headers, and ask a moderator which crop they had actually reviewed. Since then, each derivative event carries the source hash, policy version, decision timestamp, and a reason code. That extra row costs almost nothing beside another retained image, and it turns a guess into an auditable chain when a buyer disputes what was shown.
Testing the gate with hostile, ordinary files
Your fixture set needs more than a clean phone photo. Include a rotated image, an image with GPS, a truncated upload, an oversized pixel count, a valid file with the wrong extension, and a duplicate hash under two auction IDs. Assert both the decision and the retention event. A passing parser test with a missing delete event is still a retention bug.
Instrument counts by decision, media type, parser version, and derivative outcome. Alert on a rise in quarantine age, not just on request failures. I'm not sure a single age threshold fits every auction house; your mileage will vary with dispute rules, but the threshold should be written down and reviewed with legal and operations.
The least complex architecture wins here: one intake ledger, one restricted original, one scrubbed derivative pipeline, and a queue that can explain every public object. Add another processing stage only when it closes a measured moderation gap. Otherwise it becomes another copy to retain and another place for a stale URL to escape.
