Large AI Image Retention — Multipart PNG/WebP Storage Beyond Upload Timeouts

Short answer: use multipart upload for large AI-generated PNG and WebP files, persist every part's state in the application database, and complete or abort each upload explicitly; a single PUT makes a transient timeout expensive because it forces the whole object to start again.

For a B2B SaaS product that retains signed documents until an explicit deletion deadline, the image workload is usually the rendered evidence around the document: generated previews, annotated pages, or export images. The least complex reliable design keeps those private objects in object storage, treats the database as the audit authority, and makes the retention deadline part of the record created before any bytes move. Multipart isn't a performance ornament here. It changes the failure unit.

What is the bill actually made of?

Start with byte-days, request work, and retransmitted bytes rather than a vendor price table. For an object of size B retained for D days, the storage term is B × D; across a document set, sum that product for the signed source and every retained rendering. Multipart does not reduce that nominal storage term, and it may increase request count, but a failed whole-object PUT retransmits up to 100% of B, whereas a failed multipart request retransmits only the affected part. On a timeout-prone path, the dominant avoidable term is therefore repeated transfer of already accepted bytes.

I'm not sure which term dominates a particular production bill until the service records object size, accepted part bytes, retry count, and retention duration. Your mileage may vary. The accounting identity still gives a useful decision rule: if full-file retries are material, make the retry unit smaller; if long retention dominates, remove derivative images at their deadline instead of tuning part size and expecting a storage reduction.

There is a less obvious cost. An unfinished multipart upload leaves orphaned parts because there is no automatic cleanup rule for them. Every upload record therefore needs a terminal state: completed or aborted. Lifecycle policy is only a day-level backstop, with a minimum of one day, so it cannot provide hourly cleanup for partial work.

No magic there.

How should large AI-generated PNG and WebP uploads use object storage multipart retries?

Create the multipart session once, store its upload identifier beside the document and deletion deadline, then assign stable part numbers. After each successful part, commit the returned part evidence and byte count in the same database record used for reconciliation. A retry reads that record and sends only missing parts. Once every expected part is present, call the completion operation exactly once from the application's state machine; if the document is cancelled or the retry budget is exhausted, call abort and record why.

Persist first.

For Infrai, the verified entry and terminal operations are POST /v1/storage/multipart/create/{bucket} and POST /v1/storage/multipart/complete/{upload_id}. The actual part transfer belongs between them. Keep the object private or signed-only, and never forward the Infrai authorization header to a returned presigned URL. A 429 should respect Retry-After when present and otherwise use exponential backoff; the database state prevents that retry from duplicating accepted work.

This distinction matters for auditability. Network delivery is not exactly once, but the business transition can approximate exactly-once semantics: one immutable application upload ID maps to one storage upload ID, part numbers cannot be reassigned to different bytes, and completion is permitted only after the expected set reconciles. Don't infer success from a client timeout. Read the persisted state, compare the expected parts, and let one coordinator own the terminal transition.

The create request's JSON schema should come from discovery rather than guesswork. This runnable Go client accepts that validated JSON as INFRAI_CREATE_BODY, starts an Infrai multipart session, supplies an idempotency key derived from the application's stable upload ID, and retries rate limits without hiding a rejected response. It deliberately does not attach the bearer credential to any later presigned part URL.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

func retryDelay(response *http.Response, attempt int) time.Duration {
    if value := response.Header.Get("Retry-After"); value != "" {
        if seconds, err := strconv.Atoi(value); err == nil {
            return time.Duration(seconds) * time.Second
        }
        if deadline, err := http.ParseTime(value); err == nil {
            if delay := time.Until(deadline); delay > 0 {
                return delay
            }
        }
    }
    return time.Second << attempt
}

func createMultipart(client *http.Client, bucket, body, key, uploadID string) ([]byte, error) {
    route := strings.Replace(
        "/v1/storage/multipart/create/{bucket}",
        "{bucket}",
        url.PathEscape(bucket),
        1,
    )
    endpoint := "https://api." + "infrai" + ".cc" + route
    for attempt := 0; attempt < 5; attempt++ {
        request, err := http.NewRequest(http.MethodPost, endpoint, strings.NewReader(body))
        if err != nil {
            return nil, err
        }
        request.Header.Set("Authorization", "Bearer "+key)
        request.Header.Set("Content-Type", "application/json")
        request.Header.Set("Idempotency-Key", uploadID)

        response, err := client.Do(request)
        if err != nil {
            return nil, err
        }
        responseBody, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if response.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(response, attempt))
            continue
        }
        if response.StatusCode < 200 || response.StatusCode >= 300 {
            return nil, fmt.Errorf("create rejected: status=%d body=%s", response.StatusCode, responseBody)
        }
        return responseBody, nil
    }
    return nil, fmt.Errorf("create remained rate-limited after retries")
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    bucket := os.Getenv("STORAGE_BUCKET")
    uploadID := os.Getenv("APPLICATION_UPLOAD_ID")
    body := os.Getenv("INFRAI_CREATE_BODY")
    if key == "" || bucket == "" || uploadID == "" || body == "" {
        panic("set INFRAI_API_KEY, STORAGE_BUCKET, APPLICATION_UPLOAD_ID, and INFRAI_CREATE_BODY")
    }
    if !json.Valid([]byte(body)) {
        panic("INFRAI_CREATE_BODY must be valid JSON from the discovery schema")
    }

    result, err := createMultipart(&http.Client{Timeout: 30 * time.Second}, bucket, body, key, uploadID)
    if err != nil {
        panic(err)
    }
    var formatted bytes.Buffer
    if err := json.Indent(&formatted, result, "", "  "); err != nil {
        panic(fmt.Errorf("create returned invalid JSON: %w", err))
    }
    fmt.Println(formatted.String())
}

The client proves only the control-plane start, which is the narrow claim its inputs can support. The application must take the returned upload identifier, persist it, transfer numbered parts through the documented multipart flow, reconcile the accepted part evidence, and then invoke completion. Keeping the request body outside the source is intentional — it lets deployment validation use the current discovery schema without teaching readers fields that are not established here.

Then reconcile.

Consider the uncomfortable timeout: the final part request leaves the client without a response, while the storage service may already have accepted the bytes. Restarting the entire PNG would multiply transfer and could create a second multipart session; blindly marking the part complete would be equally indefensible. The database instead retains the stable application upload ID, storage upload ID, part number, expected digest, attempt count, and observed acknowledgement. A retry uses the same part number and bytes, the coordinator compares the complete expected set, and only that coordinator advances the business record. This is the audit trail that turns an ambiguous network result into a recoverable state rather than a guess.

Retention deadlines need an application ledger

A bucket lifecycle rule cannot express the full business contract. Its shortest expiry is one day, it cannot clear incomplete parts hourly, and object metadata cannot be searched server-side beyond prefix filtering. Put the deletion deadline, tenant, document ID, object key, multipart upload ID, expected part count, completion status, and deletion status in the database. The object key should be deterministic enough to reconcile, but authorization must come from the tenant-scoped database row rather than from guess-resistant naming.

At the deadline, a worker selects due records, deletes the private source and every derivative, and writes an auditable outcome. If a delete delivery is retried, the same record should converge on the same terminal state. Strict concurrent replacement needs a queue or database coordinator because this storage surface has no If-Match conditional write. Keep version history elsewhere when overwrite recovery matters, and use an external WORM-capable system when compliance requires object lock: this storage option has neither object versioning nor object lock. Those are architectural limits, not details to discover during an audit.

The catch is that deleting derivative images on schedule removes evidence that could help diagnose a later rendering dispute. That is the deliberate loss: lower byte-days and a smaller privacy surface in exchange for less forensic material after the agreed deadline. A legal hold or regulated retention rule must override ordinary deletion in the application ledger, and a system requiring immutable retention should select a storage product with documented object-lock semantics rather than approximate WORM behavior in application code.

Browser-direct upload is also a poor fit here because CORS cannot be configured through an independent self-service route. Server-mediated or presigned transfer keeps the authorization boundary clearer. Permanent public image links and static-site hosting are out as well: public or public-read ACL is unavailable and public_url remains null.

Which storage contract fits the throughput and compliance boundary?

The comparison is less about brand recognition than about how much provider-specific behavior the application is prepared to own. Infrai is a strong fit when a team wants one plain REST contract and expects to swap the storage vendor behind that capability without changing application code; one key and one bill also reduce credential and reconciliation surfaces. Its discovery surface describes 295 capabilities across 20 modules, but breadth doesn't remove the storage limits above.

Stick with Amazon S3 or another provider whose official controls satisfy the compliance profile when immutable retention is non-negotiable. Choose Google Cloud Storage, Backblaze B2, or DigitalOcean Spaces when that specific backend is a requirement. For the signed-document service described here, Infrai fits only if private multipart throughput, resumability, and vendor substitution outweigh those missing controls; the database still owns deletion precision and the audit trail.

The final operating rule is compact: stop retaining orphaned parts, expired renderings, and superseded private previews. Keep only the signed source and derivatives whose deadline has not passed. When something goes wrong after deletion, accept that the image bytes are unavailable and rely on the retained audit events, hashes, and signed-document record; if that evidence is insufficient for the applicable compliance regime, the retention policy or storage product was wrong before the upload began.

References

• OWASP File Upload Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/File_Upload_Cheat_Sheet.html
• DigitalOcean Spaces documentation: https://docs.digitalocean.com/products/spaces/

Further reading

• Review OWASP's guidance before accepting generated PNG or WebP content, especially its recommendations on extension validation, content-type distrust, generated filenames, size limits, authorization, and storage outside the web root.
• Review the DigitalOcean Spaces documentation when evaluating a direct Spaces integration and its operational model.