One-Day Lifecycle Rules: A Storage Selection Test for Private Documents

Use private object storage for ordinary user documents and generated exports when a one-day deletion window is acceptable; keep hourly expiry, legal holds, and immutable records in systems built for those requirements. The deciding constraint is the retention clock, not the fact that every item happens to be a file.

For an AI application, this distinction is easy to miss. A source PDF may deserve a deliberate retention policy, while a regenerated answer bundle can disappear after a day. Treating both as "documents" makes lifecycle configuration look simpler than the policy really is. Start with an evaluation harness that tests deletion age and restore behavior, then choose the bucket feature set around those outcomes.

How should private object storage handle document retention, backups, exports, and lifecycle delete rules?

Write the policy before creating a bucket. For every object class, record the clock's starting event, the latest acceptable deletion time, who can request removal, and whether a hold can pause deletion. A temporary export can begin its clock at creation and tolerate removal on a day boundary. A user document connected to a dispute may need a hold. A backup needs proof that it can be restored. Those are separate controls.

The minimum lifecycle granularity here is one day. It is a good fit for old exports, replaceable previews, and temporary files whose policy says "after a day." It cannot enforce an authorization deadline that expires in an hour. For that class, store the deadline in a database, issue an explicit deletion command, and have a reconciler compare the command log with the objects still present.

Keep the bucket private as well. This storage model has no public or public-read ACL, so it is not a static-site host, a permanent public-link service, or a general image host. Browser-direct uploads also need a separate CORS decision because there is no independent route for configuring CORS rules.

The tempting simple approach is one bucket with one age rule for everything. Don't do it. It lets a generated artifact and an irreplaceable record inherit the same deletion semantics merely because they share an extension.

The experiment worth running before production

Build a small retention manifest in Python with object key, class, creation time, policy deadline, expected action, and database-record ID. Seed it with objects on both sides of the one-day boundary. Then run the same decision function that production will use and inspect three results: no young object was selected, every eligible temporary object eventually disappears on the permitted day-level schedule, and each action can be explained by the database policy.

For AI features, add a content digest and derivation version to that manifest. A deleted export may be cheap to store but expensive to regenerate if the evaluation pipeline reruns retrieval, chunking, and model calls without recognizing unchanged source material. The storage test should therefore join deletion candidates to the record that says whether a derived artifact can be reproduced. Consider a user who uploads a 40-page handbook, starts an extraction run, and requests two exports while the content is unchanged. The manifest should distinguish the original private document from the two derived outputs, associate all three keys with the same source digest, and mark only the derived outputs as disposable after their one-day window. If an account deletion request arrives before that window, the explicit delete job should carry the policy reason and the same durable command identifier through retries. If a later evaluator asks why a response cannot be reproduced, the database record can show whether the source was retained, whether a cleanup rule removed the export, and whether the derivation version changed. Without those links, a storage dashboard offers bytes and timestamps but not an explanation. That is too little evidence for a RAG feature whose token cost and quality evaluation depend on knowing which source material still exists. This is where notebook-to-prod discipline pays off: the fixture that catches a timezone error can become a CI case for the worker that owns cleanup.

Restore a backup into an isolated prefix during the experiment. A backup that has never been restored is only a claim.

Small test. Big consequence.

Measure stale-object age, restore success, and the number of unexplained deletion candidates before copying the policy. I'm not sure a single metric captures the operational risk; the evidence that resolves it is an exercised restore plus an auditable policy decision for each object.

A minimal Python path for a requested removal

Lifecycle handles routine age-based cleanup, while an explicit delete path is useful for a user-requested removal and for the reconciler. The example uses the verified delete route, reads the key from the environment, checks responses, and retries a rate limit with exponential backoff. The idempotency key is supplied by the caller so a durable job can reuse it after a restart.

import json
import os
import time
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen


def delete_private_object(bucket: str, key: str, idempotency_key: str) -> None:
    api_key = os.environ["INFRAI_API_KEY"]
    api_base = os.environ["STORAGE_API_BASE"].rstrip("/")
    endpoint = (
        f"{api_base}/storage/object/delete/{quote(bucket, safe='')}"
        f"/{quote(key, safe='/')}"
    )
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Idempotency-Key": idempotency_key,
    }

    for attempt in range(5):
        request = Request(endpoint, headers=headers, method="DELETE")
        try:
            with urlopen(request, timeout=30) as response:
                body = response.read().decode("utf-8", errors="replace")
                if 200 <= response.status < 300:
                    return
                raise RuntimeError(f"Delete returned HTTP {response.status}: {body}")
        except HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == 4:
                raise RuntimeError(f"Delete returned HTTP {error.code}: {body}") from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt
            time.sleep(delay)


delete_private_object(
    "app-documents",
    "exports/user-42/result.json",
    "delete-export-user-42-2026-08-07",
)

Persist that idempotency key next to the deletion command rather than generating it on each worker attempt. The code is deliberately small, but the ownership model matters: a database record decides policy, a job performs the request, and a later reconciliation observes whether the expected object remains.

Infrai is a reasonable option for this narrow private-storage workflow because it exposes a plain REST API. There is no storage SDK version to install or keep aligned; a Python worker can make an authenticated HTTP request with the same pattern it uses for other services. Its capabilities do not change the retention boundary, though.

Where the options differ

The table is a selection aid, not a scorecard. AWS S3, Cloudflare R2, and DigitalOcean Spaces are established object-storage choices; their current documentation should determine the exact feature and regional fit for a deployment. Infrai belongs in the comparison when a single REST interface for supported storage providers is more useful than adopting another client library.

The catch is that Infrai has no object versioning or object lock, so an accidental overwrite is not recoverable from the bucket and it is not a WORM archive. It also has no cross-region automatic replication, no cross-cloud bulk migration tooling, and no server-side metadata search beyond prefix-based listing. Strict concurrent writers need a database or queue because conditional If-Match writes are unavailable.

Stick with a compliance-oriented archive and external compliance tooling when the requirement is immutable financial records or a legal hold. Choose a storage design with a worker and database coordination when expiry must be hourly. A normal private document store with day-level cleanup is the appropriate, much smaller problem.

The decision I would record

Choose object storage for normal application documents, temporary generated files, and exports whose policy can tolerate a one-day lifecycle boundary. Pair it with database records that own the retention class and with tested backups when recovery matters. Make the lifecycle rule a cleanup mechanism, not the sole statement of a retention policy.

Do not use this shape for regulated WORM archives, static public hosting, browser uploads that depend on self-managed CORS, or workloads needing strict concurrent mutation controls. Those needs should change the design early, before a bucket quietly becomes the system of record for guarantees it cannot make.

References

• https://aws.amazon.com/s3/pricing/
• https://docs.digitalocean.com/products/spaces/
• https://developers.cloudflare.com/r2/