Short answer: For classified ad photos, use a quarantined upload state machine that validates bytes and derivatives before publication, with digest keys and explicit lifecycle states to control cache cost.
Classified-ad photo systems should treat every upload as an untrusted payment-like event: accept bytes into quarantine, validate them before publication, and make each state transition idempotent. The deciding constraint is storage and cache cost, because a malformed or needlessly large derivative can multiply across replicas long before anyone notices.
The useful boundary is a small state machine, not a single “upload succeeded” boolean. It gives operators an auditable answer to a deceptively hard question: which exact bytes were checked, transformed, and served?
The invariant ledger behind an image upload
I model an image record with an immutable source digest, declared media type, decoded dimensions, and a lifecycle state. The source object is write-once; derivatives are replaceable artifacts tied to that digest. A listing may reference a derivative only after the validator has committed its result.
That ordering protects reconciliation. If a worker retries after a timeout, the digest and operation key lead it to the same result instead of creating a second resize tree. This is the same exactly-once mindset I use for ledger entries: at-least-once delivery is normal, while duplicate effects are not acceptable.
The failure boundary is explicit. Network interruption, an unsupported format, an excessive pixel count, and a policy rejection are different outcomes and deserve different retry behavior. A retryable transport error can return to received; a policy rejection must remain terminal and retain its reason for audit.
How should safe upload and lifecycle validation share one boundary?
The upload endpoint should do cheap checks synchronously, then enqueue a validation job. Cheap means authenticated listing ownership, byte-length limit, and a recognized container signature; it does not mean trusting a filename or a client-supplied Content-Type. Full decoding, orientation normalization, metadata stripping, and derivative generation happen in an isolated worker.
Here is the critical path in Go. The interfaces are deliberately generic so the same contract works with filesystem, object storage, or a self-hosted gateway.
package media
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
)
type State string
const (
Received State = "received"
Valid State = "valid"
Published State = "published"
Rejected State = "rejected"
)
type BlobStore interface {
PutIfAbsent(ctx context.Context, key string, data []byte) error
Get(ctx context.Context, key string) ([]byte, error)
}
type RecordStore interface {
Transition(ctx context.Context, digest string, from, to State, reason string) error
}
func Ingest(ctx context.Context, blobs BlobStore, records RecordStore, data []byte) (string, error) {
if len(data) == 0 || len(data) > 12*1024*1024 {
return "", errors.New("payload outside upload policy")
}
sum := sha256.Sum256(data)
digest := hex.EncodeToString(sum[:])
if err := blobs.PutIfAbsent(ctx, "quarantine/"+digest, data); err != nil {
return "", err
}
if err := records.Transition(ctx, digest, "", Received, "accepted"); err != nil {
return "", err
}
return digest, nil
}
The worker must decode from quarantine, never from a public cache key. It records the decoder and policy version with the digest, then writes derivatives under keys such as derived/{digest}/webp-1200. A publish transaction changes the listing pointer only after all required derivatives exist. Cache invalidation is then a consequence of a pointer change, rather than a best-effort side effect.
Choosing validation depth without wasting cache capacity
A strict pipeline can reject more than a marketplace needs. A useful policy table makes that trade explicit:
| Check | Protects | Cost | Result when it fails |
|---|---|---|---|
| Signature and byte limit | Parser exposure and oversized objects | Low, on ingress | Terminal rejection |
| Full decode and pixel limit | Decompression bombs and worker memory | Medium, isolated CPU | Terminal rejection |
| Orientation and color normalization | Consistent thumbnails | Medium, transform pass | Retry only on infrastructure error |
| Metadata removal | Location and device privacy | Low to medium | Policy rejection if mandatory |
| Derivative dimension cap | Cache and egress spend | Medium, proportional to pixels | Keep source, skip that derivative |
Do not confuse a valid image with a useful image. A 20-megapixel source can decode correctly and still be an unreasonable thumbnail input. Conversely, retaining the original may be necessary for a seller's later edit, so “reject derivative” and “reject upload” should be separate decisions.
I once traced a cache-cost spike to a perfectly valid panorama: every listing generated six near-identical widths, and a cache key included a changing query parameter. The validator had passed; lifecycle economics had not. The fix was a canonical width set and a digest-based key, not a looser format rule.
Small detail, large bill.
It failed.
Observability and reconciliation are part of correctness
Every transition should emit an event containing digest, listing identifier, operation key, policy version, decoder result, and byte counts before and after transformation. Keep the event append-only. A dashboard that shows only HTTP 201 responses cannot tell whether a derivative was published, evicted, or orphaned.
Reconciliation runs against two inventories: records that claim published, and objects that actually exist under the derivative prefix. Missing objects enqueue regeneration with the original digest; unreferenced objects age out after a retention window. This process must be idempotent, because it will overlap with ordinary workers during deploys.
Metrics should separate rejected bytes, accepted source bytes, derivative bytes, cache hit ratio by width, decode latency, and state-transition failures. Alert on monotonic growth of quarantined records and on a widening gap between published pointers and present objects. Those signals identify lifecycle drift before a buyer sees a broken photo.
The shortcut is to store the client payload directly at a public URL and let a CDN or browser discover whether it is an image. It has one legitimate use: a private, low-risk internal tool where users are trusted, images never leave an access-controlled network, and there is no derivative fan-out. It is unsuitable for public classified ads, where untrusted bytes, privacy metadata, and cache amplification meet.
A queue-only design has the opposite trade-off: excellent isolation, but slower feedback to sellers. If a marketplace needs an immediate preview, it can generate a bounded, disposable preview after signature checks while keeping publication behind full validation. That compromise preserves the safety boundary without pretending that preview success proves lifecycle readiness.
The decision rule is therefore concrete: use immutable digest keys, explicit states, isolated decoding, and a reconciliation loop; tune derivative widths and retention against measured cache behavior. Your mileage may vary on the exact limits, and I’m not sure a single policy fits every category, but the boundary itself should remain stable even as formats and storage providers change.
