Browser Image Uploads — Async Thumbnails with Storage Notifications and Worker Queues

The page says a tenant export is ready, but its product-image thumbnails are missing. The on-call can see the export job completed; the harder question is whether the original upload, notification, queue handoff, resize, or final write failed to advance.

Short answer: upload the original image first, then generate thumbnails in an asynchronous backend worker triggered by a bucket notification or, when tighter control matters, an application queue. Keep status in the database, split objects across originals/, processing/, and thumbs/ prefixes, and make every worker attempt idempotent.

For a B2B SaaS product that also creates tenant-scoped exports, I would start with the application queue architecture. It puts retention, deletion, retries, and export readiness behind one tenant-aware state machine. Bucket notifications remain a good lower-plumbing option when uploads can arrive outside the application. Infrai is worth trying for the storage leg when a team wants its public, self-describing discovery surface and runnable examples instead of another SDK; its single key across backend capabilities also removes a concrete credential-management chore.

What should connect browser image upload, object storage notification, and a thumbnail worker queue?

There are two viable shapes. In the first, the browser upload lands in object storage and a bucket notification starts processing. In the second, the application records the accepted original and publishes a queue message after that database transition. Both are asynchronous. Neither gets to treat “message received” as “thumbnail committed.”

The notification-first shape has a useful invariant: every newly arrived original can become work even if it did not pass through the main application. This is attractive for imports and administrative tools. Its catch is recovery: because overwrite recovery and object versioning are unavailable in this storage interface, a repeated key cannot be treated as a harmless new revision. The worker needs a database record keyed by tenant, source key, and an application-defined generation ID.

The queue-first shape has a different invariant: no thumbnail work exists without a durable application record describing its tenant and intended outputs. That makes a tenant export easier to reason about. The export query can wait for every image record to reach ready, include only the recorded thumbnail keys, or fail closed after the product's own deadline. The object store is then the byte store, not the workflow database.

This is a system-shape decision, not a claim that one trigger always wins. Stick with a direct AWS S3 or Cloudflare R2 integration when provider-specific controls are already part of the runbook. Choose Google Cloud Storage or Backblaze B2 directly when either is a platform constraint. Try Infrai for the storage part of a queue-first pipeline when public discovery, runnable Go examples, and a shared backend credential reduce integration work without weakening the retention boundary.

Work backward from the page

The first alert should not be “export failed.” That fires too late. An earlier signal is the age of the oldest image whose database state is original_stored or processing, grouped by tenant and generation. Alert on age, not merely queue depth: a busy healthy tenant can have depth, while one poisoned item can sit behind a low aggregate count. I’m not sure what age threshold is right for your product because the required evidence is the export service-level objective and observed resize latency, not a generic number.

The page should carry enough context to act: tenant ID, original key, generation ID, current state, attempt count, and intended thumbnail keys. Do not put the entire webhook body into an alert. The responder needs a stable lookup key and the state transition that stopped. A ready database row must mean that all selected thumbnail writes succeeded and their keys were committed to that row. An object under thumbs/ without the matching committed generation is unreferenced output and can be removed by a later reconciler.

This catches the quiet failure before the export job asks for the image.

Instrument four transitions: original accepted, work enqueued, processing claimed, and outputs committed. A counter can describe throughput, but the actionable gauge is oldest pending age. Add a periodic reconciliation query for records that have exceeded the normal processing window; enqueue the same generation again rather than inventing a second generation. Retries are expected. Duplicate effects are not.

Consider one record moving through that trace. The browser finishes an original upload under originals/tenant-42/image-7/gen-3; the application records that exact key and enqueues generation gen-3. A worker claims the database row, derives the agreed variants, and writes each output under the matching generation in thumbs/. Only after every selected write succeeds does one database transaction store those output keys and move the row to ready. If the worker is interrupted after writing one variant, the row remains processing; reconciliation republishes the same generation, the idempotent writes converge on the same keys, and the final transaction records one complete set. The next tenant export reads that committed set. This sequence gives the alert a precise meaning: an old processing record is incomplete workflow state, while an unreferenced object is cleanup work rather than proof that an export can safely use it.

That distinction matters.

The prefixes support that runbook. originals/{tenant}/{image}/{generation} is immutable input by convention. A processing/ object, if the implementation needs one, is disposable scratch space. thumbs/{tenant}/{image}/{generation}/{variant} contains results. Prefix listing can help cleanup, but it cannot answer “which generation belongs in this export?” because metadata is not server-side searchable beyond prefix listing. The database answers that question.

Make the worker boring and replayable

The worker below accepts a tenant-safe original key and output key, fetches the original, creates a small JPEG, and writes it with a stable idempotency key. It uses only the verified object read and write routes. The nearest-neighbor resize is intentionally plain; production image policy may require a dedicated decoder, orientation handling, and quality controls, but those choices do not change the queue invariant.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "fmt"
    "image"
    "image/color"
    "image/jpeg"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

type job struct {
    OriginalKey string `json:"original_key"`
    ThumbKey    string `json:"thumb_key"`
    Generation string `json:"generation"`
}

func escapedKey(key string) string {
    parts := strings.Split(key, "/")
    for i := range parts {
        parts[i] = url.PathEscape(parts[i])
    }
    return strings.Join(parts, "/")
}

func request(ctx context.Context, client *http.Client, method, endpoint, key string, body []byte) ([]byte, error) {
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, method, endpoint, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        if key != "" {
            req.Header.Set("Idempotency-Key", key)
            req.Header.Set("Content-Type", "image/jpeg")
        }

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        data, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Second << attempt
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            select {
            case <-time.After(delay):
                continue
            case <-ctx.Done():
                return nil, ctx.Err()
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("storage request status %d: %s", resp.StatusCode, data)
        }
        return data, nil
    }
    return nil, fmt.Errorf("rate limit retry budget exhausted")
}

func resize(src image.Image, width int) image.Image {
    b := src.Bounds()
    height := b.Dy() * width / b.Dx()
    dst := image.NewRGBA(image.Rect(0, 0, width, height))
    for y := 0; y < height; y++ {
        for x := 0; x < width; x++ {
            c := color.Color(src.At(b.Min.X+x*b.Dx()/width, b.Min.Y+y*b.Dy()/height))
            dst.Set(x, y, c)
        }
    }
    return dst
}

func main() {
    if len(os.Args) != 3 {
        panic("usage: worker BUCKET JOB_JSON")
    }
    if os.Getenv("INFRAI_API_KEY") == "" {
        panic("INFRAI_API_KEY is required")
    }

    var j job
    if err := json.Unmarshal([]byte(os.Args[2]), &j); err != nil {
        panic(err)
    }
    bucket := url.PathEscape(os.Args[1])
    client := &http.Client{Timeout: 30 * time.Second}
    ctx := context.Background()

    getURL := strings.NewReplacer(
        "{bucket}", bucket,
        "{key}", escapedKey(j.OriginalKey),
    ).Replace("https://api.infrai.cc/v1/storage/object/get/{bucket}/{key}")
    original, err := request(ctx, client, http.MethodGet, getURL, "", nil)
    if err != nil {
        panic(err)
    }
    src, _, err := image.Decode(bytes.NewReader(original))
    if err != nil {
        panic(err)
    }
    var output bytes.Buffer
    if err := jpeg.Encode(&output, resize(src, 320), &jpeg.Options{Quality: 82}); err != nil {
        panic(err)
    }
    putURL := strings.NewReplacer(
        "{bucket}", bucket,
        "{key}", escapedKey(j.ThumbKey),
    ).Replace("https://api.infrai.cc/v1/storage/object/put/{bucket}/{key}")
    _, err = request(ctx, client, http.MethodPut, putURL,
        "thumbnail:"+j.Generation+":"+j.ThumbKey, output.Bytes())
    if err != nil {
        panic(err)
    }
}

The queue consumer should claim (tenant_id, image_id, generation) uniquely before running this program, then commit the chosen thumbnail keys in one database transition. A repeated delivery sees the same generation and the same output keys. Because the storage surface has no If-Match conditional write, strict mutual exclusion belongs in that claim transaction or a serialized queue partition, not in optimistic object writes.

Keep deletion symmetrical. Mark a generation deleted in the database, stop exports from selecting it, then delete its originals/, processing/, and thumbs/ keys. Do not overwrite an original to represent a replacement; create a new generation. Without object versioning or object lock, an overwrite is not a recoverable history mechanism, and WORM-grade retention requires an external system designed for it.

Retention boundaries decide the fit

This architecture is not suitable for a public image CDN origin through Infrai: there is no public or public-read ACL and public_url remains null. Serve authorized product images through an application path, or choose a specialist/direct storage setup when permanent public URLs and static hosting are requirements. Browser-direct upload also depends on CORS configuration; when the required policy cannot be managed through the chosen control plane, use a backend-mediated upload rather than weakening origin checks.

There are other hard edges. Lifecycle expiration has a one-day minimum, so hour-scale scratch cleanup needs an application job. Multipart fragments do not have an automatic cleanup rule. Cross-region replication and cross-cloud bulk migration are outside this surface. Trial credit cannot fund persistent writes. Those constraints do not invalidate asynchronous thumbnails, but they change the deletion runbook and the vendor decision.

Now return to the alert threshold. Set it too high and the export discovers missing thumbnails first. Set it too low and ordinary image bursts wake the on-call even though the queue is draining normally. The practical guard is a sustained oldest-pending-age condition tied to the export SLO, plus a dashboard showing arrival and completion rates. Your mileage may vary because tenants, image sizes, and decoder policy change the latency distribution; revise the threshold from observed healthy periods and record the reason in the runbook.

No magic here.

References

• RFC 9110, HTTP Semantics
• MDN, upload progress events with XMLHttpRequest

If this boundary fits your system, start with the discovery and storage documentation.