A marketplace workload needs a spending boundary before its invoice arrives. Provider routing preferences are useful for expressing constraints on access and budget, not for chasing vendors. The least complex useful design admits only eligible destinations, reserves budget before dispatch, and records why each attempt was allowed. The dominant storage term to examine first is the per-attempt audit record: at an illustrative 1,000,000 attempts a day, a 2 KB record per attempt is about 2 GB of raw records daily, before indexes or replicas. These are capacity-planning inputs, not measurements of a deployed system.
Short answer: provider routing preferences order destinations after hard constraints on access, budget, and evidence have eliminated ineligible candidates. A preferred destination cannot override a workload spend cap. If a fallback cannot be explained from a saved policy version and decision record, it is not an auditable preference.
What is the bill actually made of?
The request may incur downstream usage, while the local system pays to retain evidence that explains both the route and the eventual charge. Those costs grow differently. A policy document changes infrequently; attempt records multiply with traffic, retries, and fallbacks. The illustrative calculation above excludes indexing, replication, payloads, and downstream charges. Measure those separately before setting retention windows.
The change that moves the storage term is to retain a compact record per attempt: stable operation and attempt identifiers, workload, policy version, destination, budget reservation identifier, decision outcome, and time. Store detailed request and response content only under a separately justified retention rule. This reduces the size of the repeated record while preserving a link between the decision and the ledger entry used for reconciliation. Do not confuse a short payload-retention window with permission to discard the financial audit trail.
How should provider routing preferences express constraints?
Consider a marketplace payout-reconciliation workload with a daily budget allocation. It may prefer one destination's latency profile, but policy also requires an approved data boundary and enough unreserved budget. Preference orders candidates. Eligibility decides whether any candidate can be used at all.
No candidate, no dispatch.
Treat the cap as admission control, not a prediction based on the invoice. Reserve a conservative upper bound before dispatch, and associate it atomically with a stable idempotency key so concurrent requests cannot each claim the same remaining balance. Settle the reservation against known usage and release any unused portion. Where no defensible upper bound exists, reject the attempt or set an explicit conservative ceiling. This is an architecture rule, not a claim that an external destination provides exactly-once execution. Its limitation is that conservative reservations can deny legitimate work temporarily when usage estimates are wide; loosening the ceiling restores capacity but weakens the guarantee that the cap holds before billing.
A timeout is the awkward case: the remote attempt may have completed even though the caller never saw its result. Reusing the operation key avoids duplicate local reservations, but it does not prove the external operation occurred once. Record transport attempts separately from the logical operation and reconcile ambiguous outcomes before releasing their reserved budget. That distinction matters more than a tidy destination list.
An ordered list such as primary, then secondary is insufficient. Every fallback must meet the same hard constraints, and the record must say why the preceding candidate was unavailable or ineligible. Save the policy version as well; explaining yesterday's decision using today's policy silently rewrites the evidence.
Credentials form another boundary, not a preference. Keep destination credentials out of policy documents and logs, scope access to the workloads allowed to use them, and record credential identifiers rather than secret values. OWASP's Secrets Management Cheat Sheet covers access controls, rotation, auditing, and the secret lifecycle. Routing among destinations does not relax those requirements. Expressing this boundary as a policy condition is valuable only if enforcement occurs before the credential is retrieved: documenting a restriction while letting a downstream adapter pick any credential leaves the audit record describing an intention rather than an enforced decision.
A useful acceptance test sends the same logical operation twice with one idempotency key while the first transport attempt has an unknown outcome. The ledger should show one reservation and distinct transport attempts. A further route should remain unavailable unless the reconciliation rule permits it. Then deploy a new policy version and verify old decisions remain explainable under the original version. Also test denial: the first-ranked destination must never receive a request that fails its access constraint.
What should operators stop keeping?
Stop keeping complete payloads by default merely to explain a route. Retain minimal decision and settlement links for the period required by the organization's actual financial, contractual, and regulatory obligations; there is no universal retention period implied here. This has a cost when an incident occurs: after detailed payloads expire, an investigator may prove which policy admitted an attempt and how the reservation settled, but may not reconstruct every input field or response byte. Compliance and incident-response owners should decide explicitly whether that remaining evidence is sufficient.
Reject before dispatch when eligibility or budget evidence is missing; rank only eligible destinations; preserve the explanation and financial outcome. The cap becomes enforceable before an invoice can reveal an overspend.
Further reading
The source below details the secret-handling boundary relevant to routing credentials.
