Next.js API Route and Sharp: Node.js Private Image Thumbnails in Object Storage

Short answer: keep the original private, create a fixed set of Sharp derivatives behind a server-controlled Next.js API route, and make retention a versioned manifest rather than an expiration guess. A thumbnail is ready only when its expected object exists; an upload is deletable only when the original, derivatives, and database record have a recorded deletion state.

This matters for a media team storing training artifacts. A useful thumbnail makes a review queue pleasant, but it is not the source of truth. The original image and its retention decision are. If those two concerns are mixed into one upload request, a slow decode can damage the API SLO, and a later cleanup job can remove an object that still has a valid training reference.

The thumbnail is a retention record, not a convenience file

Treat the API route as an admission controller, not as an image workshop. It authenticates the uploader, checks the declared and detected media type, bounds the input size, assigns an upload ID, and records the retention class. It then stores the original under a deterministic private key. Sharp can produce named derivatives such as thumb-256.webp; the application stores each derivative separately and records the exact source key, transform version, and retention deadline.

Do not accept arbitrary width and height values from the browser. Fixed variants make CPU, memory, output count, and storage growth countable. They also make a retry boring: media/{upload-id}/thumb-256.webp remains the destination after a timeout, instead of every attempt leaving another object behind.

The route may perform the transform only when measured load leaves enough latency and memory headroom. Otherwise it should acknowledge the accepted original, enqueue the upload ID and manifest, and let a worker fetch the private source. The execution location changes; the correctness rule does not. A caller that receives 429 should honor Retry-After or use bounded exponential backoff, and the operation identity must survive that retry.

The following Go companion models the part that should be shared by the route and the worker. It does not pretend that storage providers have one universal SDK. The important contract is the manifest.

package main

import (
    "fmt"
    "os"
    "path"
    "regexp"
)

type derivative struct {
    name   string
    format string
}

var uploadIDPattern = regexp.MustCompile(`^[a-zA-Z0-9_-]+$`)

var required = []derivative{
    {name: "original", format: "bin"},
    {name: "thumb-256", format: "webp"},
}

func objectKey(uploadID string, item derivative) (string, error) {
    if !uploadIDPattern.MatchString(uploadID) {
        return "", fmt.Errorf("invalid upload ID")
    }
    return path.Join("media", uploadID, item.name+"."+item.format), nil
}

func ready(uploadID string, written map[string]bool) (bool, error) {
    for _, item := range required {
        key, err := objectKey(uploadID, item)
        if err != nil {
            return false, err
        }
        if !written[key] {
            return false, nil
        }
    }
    return true, nil
}

func main() {
    if len(os.Args) != 2 {
        panic("usage: go run main.go <upload-id>")
    }

    written := make(map[string]bool, len(required))
    for _, item := range required {
        key, err := objectKey(os.Args[1], item)
        if err != nil {
            panic(err)
        }
        written[key] = true
        fmt.Println(key)
    }

    isReady, err := ready(os.Args[1], written)
    if err != nil {
        panic(err)
    }
    fmt.Printf("ready=%t\n", isReady)
}

The real route supplies the original bytes to Sharp and writes the derivative bytes to the manifest's named destination. It should issue a private, short-lived access URL only after checking the requesting user's authorization. There is no permanent public image URL in this design. That is suitable for review material and private training inputs; it is not suitable for a public image host whose contract depends on anonymous, durable links.

Can a Next.js API route keep Sharp thumbnails private through a deletion deadline?

Retention belongs in application state because “delete after thirty days” is not enough to reproduce a decision. Store a policy version, the class selected at ingest, the source and derivative keys, and the effective deletion timestamp. For a media training artifact, the policy might say that an original and all generated derivatives share one deadline, while a legal hold changes the state to held without changing the bytes. The exact periods are business policy, not a storage default to invent in code.

Deletion should be a state transition with an auditable work item. Mark the record deletion-requested, stop issuing new access URLs, delete every key in the manifest, verify absence through the storage control plane, and then mark the record deleted. If a worker is retried, it uses the same upload ID and key list. A missing object during a repeat deletion is a converged result, not a reason to create a new destination.

The awkward case is a late derivative, and it deserves a concrete sequence rather than a reassuring sentence about eventual consistency. Imagine an upload accepted at 09:00 with a policy deadline recorded for 09:30. The original is present, but the worker has spent too long decoding a large source and has not yet written thumb-256.webp. At 09:30, the retention job claims the upload and changes its state to deletion-requested. At 09:31, the worker wakes up with valid derivative bytes and a perfectly good network connection. If it writes without checking state, the system has recreated data after its deletion decision; if the cleanup job deletes only the original key, the late derivative becomes an untracked object. The worker must therefore reload the current state immediately before each write, and the deletion job must own a stable manifest while it removes every expected key. If the state is deletion-requested or deleted, discard the derivative and record that outcome. If the deletion job is already running, let the upload ID serialize the decision in the database or queue. This is where a simple manifest prevents an expensive class of leaks: cleanup can enumerate the expected keys without trying to infer them from filenames, and an audit record can explain why bytes were discarded instead of silently losing the trail.

Keep it private.

I put accepted, original-stored, derivatives-stored, ready, deletion-requested, and deleted on the state diagram. Six states. The useful invariant is that ready means the complete manifest exists, never merely that the last callback returned success. If a policy change needs a new transform, create a new manifest version and preserve the old one until its own retention decision is satisfied.

Storage boundaries that survive a retention audit

The decision is about control-plane obligations, not a logo. A managed object store can remove a large amount of replication and durability work, while a self-hosted system may offer controls or locality that a shared abstraction does not. The platform team's on-call budget and the retention audit are both part of the capacity plan.

The catch is that a common API cannot make a missing storage primitive appear. If compliance requires immutable retention, storage-level conditional updates, independent cross-region replication, or a provider-specific browser upload flow, keep that requirement at the direct control plane or choose an abstraction that explicitly supports it. A single key and billing relationship can be operationally attractive, but it is not a substitute for a retention proof.

For this thumbnail pipeline, the boundary is acceptable when uploads are server-mediated, the application owns authorization, variants are fixed, and the manifest is the source of truth. It is not suitable when the product promises public URLs, arbitrary on-demand transformations, or an archive whose deletion and legal-hold behavior depends on controls outside the application. Stick with the direct provider or a self-hosted design when those controls are non-negotiable.

Verification and rollback for the deletion ledger

Test the invariants with a representative image distribution, not one friendly JPEG. Verify rejected media types and oversized inputs at the API boundary; verify that duplicate delivery converges on one upload ID; verify that a presigned access URL is issued only after authorization; and verify that ready stays false while any required derivative is absent. The test should include a worker finishing after a deletion request, because that ordering is more revealing than a green upload demo.

Watch the route's tail latency against its SLO, process memory, resize concurrency, queue age, incomplete manifests, deletion age, and the count of objects past their policy deadline. I would page on a growing deletion age before paging on a transient transform retry: the former is a retention risk, while the latter may be ordinary work. Three seconds is not a universal SLO, and I'm not claiming it is; your mileage may vary. Set the threshold from the runtime, traffic shape, image dimensions, and error budget you actually operate.

Rollback is easier when keys and state names do not change. During a move from synchronous processing to a worker, stop dispatching new work, replay existing IDs through the last verified processor, and keep the same manifest and deletion rules. Do not “clean up” by deleting the whole prefix: that can erase a newer policy version or an item placed under legal hold. Reconcile by upload ID, record each attempted key, and make the final deletion decision from current policy state.

The operational rule is small enough to remember: derive privately, publish readiness only from the manifest, and delete by the same identity that created the objects. That keeps thumbnails useful without letting them become an untracked second archive.

References

• AWS S3 documentation, Presigned URLs: https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
• Google Cloud Storage documentation: https://cloud.google.com/storage/docs