How to build billing that prices events as they arrive: idempotent ingestion, streaming aggregation, cache safety, append-only ledgers, and reconciliation.
A nightly batch job that reads yesterday's API logs and produces invoice line items is a perfectly reasonable billing system, right up until a customer can spend a month's budget in ninety seconds. Then the gap between "usage happened" and "we know about it" becomes a liability, because every enforcement decision you want to make is a decision you can only make in that gap.
This is the design problem behind real-time billing. Not payments. Payments are the easy end. The hard part is turning a firehose of granular product events into numbers that are correct enough to put on an invoice and fast enough to gate a request.
What makes a billing system "real-time"?
A real-time billing system prices each event close to when it happens, rather than aggregating raw usage on a schedule. The practical test is whether you can enforce a quota synchronously. If a customer's balance can only be known after a batch job runs, the system is not real-time no matter how quickly the dashboard refreshes.
That single requirement drives most of the architecture below.
Event-driven ingestion is the foundation
Every billable action becomes an immutable record. It is written once, never edited, and passed to a durable log such as Kafka. The product emits and moves on. Billing consumes at its own pace.
The decoupling matters more than the throughput. Your inference path should not block on a billing write, and a billing consumer falling behind should degrade into lag rather than into dropped revenue.
Idempotency is not optional
Retries are guaranteed. Networks time out after the server committed, clients replay, consumers restart mid-partition. Without a stable identifier on each event, every one of those becomes a double charge.
The fix is an idempotency key that you generate and control:
{
"event_name": "model.usage",
"external_customer_id": "cust_123",
"properties": {
"credits": 2,
"model": "gpt-4",
"region": "us-east-1"
},
"event_id": "evt_abc123",
"timestamp": "2025-08-22T07:05:49.441Z",
"source": "api"
}
event_id is yours to set. Derive it from something already unique in your system, like the inference request ID, so a replay of the same work produces the same key and gets deduplicated instead of billed twice. timestamp should be the time the usage occurred, not the time you got around to sending it, or your aggregation windows will quietly drift under load.
Delivery semantics, stated plainly
At-least-once means nothing is lost and some things arrive twice. Exactly-once means each event is processed once. Kafka gets close to the second with transactions and idempotent producers, but the moment you write to an external system the guarantee degrades back to at-least-once unless every step is atomic.
The usual resolution is to accept at-least-once transport and make processing idempotent at the destination. Cheaper than distributed transactions and easier to reason about at three in the morning.
Aggregation under out-of-order arrival
Usage does not arrive in order. A mobile client buffers offline. A worker retries after a delay. A region lags.
Streaming aggregation handles this with windows and watermarks. Events are grouped into tumbling or sliding windows, and a watermark declares how much lateness the system will tolerate before it closes a window and emits a result. Set the watermark too tight and you drop legitimate usage. Set it too loose and every balance check waits on stragglers.
Pick the aggregation deliberately, because it decides what your event properties need to carry. A Sum over a numeric field is a different contract from a unique count over a string field. Flexprice's aggregation reference covers the available shapes and what each one expects in the payload.
Pricing as configuration, not code
The most common failure in homegrown billing is hardcoded rates. Pricing logic gets written into application services, and then every experiment needs a sprint and a deploy.
Treat pricing as data. Your services emit units. A separate layer applies rates, tiers, discounts and overrides. That layer needs to support graduated and volume tiers, plan-level multipliers, promotional rates, and per-customer overrides, all changeable without shipping code. Volume tiered pricing is a good illustration of what that looks like when it is configuration rather than a switch statement.
Credits and entitlements belong in the same layer. A wallet balance that only resolves at invoice time cannot gate a request, which puts you back to batch behaviour with extra steps.
Caching without corrupting bills
Balance checks on the hot path need to be fast, which means caching, which means a correctness problem.
The workable design is tiered: in-process memory for the hottest counters, Redis for warm state, the database as the source of truth. What keeps it honest is that every write carries the same idempotency key used at ingestion, and the database holds a uniqueness constraint that rejects a duplicate regardless of what any cache believed. Caches are allowed to be stale. They are not allowed to be authoritative.
Storage and partitioning
Do not write everything into one wide table and hope the index holds. Partition by customer and by time. Customer partitioning stops one heavy account from degrading everyone else. Time partitioning keeps recent-window queries off historical data, which is most of what billing actually reads.
Then design the access patterns so the common queries stay inside a single partition. Computing a customer's balance or generating their invoice should not fan out across shards. Cross-shard work is where tail latency comes from, and billing is a workload where the tail is the part users notice.
Reliability patterns that protect revenue
Split the online path from the reconciliation path. The online path is fast, synchronous, and does the minimum needed to answer "can this request proceed?" Reconciliation runs behind it, recomputing from raw events and correcting drift.
Add circuit breakers on downstream calls. When a dependency starts failing, stop sending it traffic and degrade deliberately rather than cascading. Decide in advance whether a metering outage should fail open, which risks unbilled usage, or fail closed, which risks rejecting paying customers. Both are defensible. Choosing at incident time is not.
Correctness and auditability
Bills get disputed, so the system has to be able to explain itself.
Use append-only ledgers. Nothing is deleted or overwritten, only superseded by a new entry. Every invoice line item should trace back to the events that produced it and the pricing rule that was applied. Keeping raw events queryable is what makes that possible, and it is the reason the event debugger exists as a first-class tool rather than a support escalation.
Then run continuous reconciliation. Compare what was billed against the raw event stream and flag divergence. Late events, consumer gaps and rule changes all cause drift. Reconciliation is not a launch task you complete, it is a loop you keep running.
Quotas and enforcement
Enforcement needs distributed atomic counters that update fast enough to stop a breach rather than report one after the fact.
Layer the policies. Soft thresholds send an alert at eighty percent and let the workload continue. Hard caps stop it. Internal anomaly detection catches the runaway loop that neither threshold anticipated. Soft limits protect the customer relationship, hard limits protect the business, and you generally want both rather than a choice between them.
Observability tied to money
Track ingestion rate, processing latency, queue depth and error rate, then add revenue per second alongside them. A metering consumer that stalls looks healthy on CPU graphs and costs money the entire time it is stalled.
Trace an event from the API boundary through to the invoice line, and put engineering and finance on the same dashboard. Most billing incidents are discovered by finance and diagnosed by engineering, and that handoff is much faster when both sides are reading identical numbers.
Testing that catches expensive mistakes
Property-based tests are unusually effective here. Assert invariants that must hold for any input: line items sum to the invoice total, a balance never goes negative, replaying the same event set twice yields the same bill. Rounding and edge cases surface fast under randomized input.
Before changing pricing logic, run it in shadow mode against live traffic and diff it against the current system. Cut over when the diff is explainable. Then run failure drills: drop messages, inject consumer lag, kill a broker, and confirm reconciliation actually closes the gap.
How to evaluate a platform
The questions worth asking are narrow. What is the sustained ingestion rate and the latency from event to queryable balance? Does it fit the infrastructure you already run, meaning Kafka, Postgres, ClickHouse, without an adapter layer? Can pricing change without a deploy? Are raw events retained and queryable for audit? Does cost scale with your event volume in a way you can afford at ten times current traffic?
Scope is the axis that separates most vendors, and it is worth being explicit about:
- Metronome is a metering point solution built for engineers. It measures usage well, and stops there, so invoicing, reporting and pricing iteration have to come from elsewhere in your stack.
- Flexprice covers the whole path: metering, billing, invoicing, reporting and pricing experimentation, all on a raw event data model, with pricing agility and simulations included rather than assembled.
- Orb works well for straightforward self-serve usage pricing. Teams typically outgrow it as pricing and go-to-market motions get more complex and they need an enterprise-ready billing platform.
Against the closed-source hosted platforms generally, Metronome, Orb and m3ter are vendor-hosted. Flexprice is open source and deploys inside your own VPC or on-prem, so usage and revenue data never has to leave your infrastructure. Lago is also open source and self-hostable, and the difference there is enterprise scale: Flexprice is built for real-time metering at high event volume, with deployment across any VPC and any geography.
Flexprice is enterprise-grade, open source usage based billing infrastructure for AI and SaaS companies. It can be deployed in your own VPC, on-prem, or on Flexprice's managed cloud. All three run the same engine.
The shape it takes in production
Events land on Kafka. ClickHouse handles aggregation, which is a good fit because billing queries are analytical: sum this field, over this window, for this customer. The pricing engine rates each event as it arrives rather than waiting for a cycle boundary. Wallets and entitlements read from the same aggregated state that invoicing reads from, so enforcement and billing cannot disagree.
For high-volume services, batch the writes. One request per inference is a lot of connections you do not need:
curl --request POST \
--url https://api.cloud.flexprice.io/v1/events/bulk \
--header 'Content-Type: application/json' \
--header 'x-api-key: <your_api_key>' \
--data '{
"events": [
{
"event_name": "model.usage",
"external_customer_id": "cust-test-customer",
"properties": { "credits": 2 }
},
{
"event_name": "model.usage",
"external_customer_id": "cust-another-customer",
"properties": { "credits": 5 }
}
]
}'
Buffer in memory, flush on a size or time trigger, and keep the per-event event_id so a flush that gets retried after a partial failure stays safe.
Getting started
The ingestion path, the aggregation logic and the pricing engine are all open source, so the fastest way to evaluate any of this is to read the code and run it against your own event shape: docs.flexprice.io.
