Metering pipelines for usage billing: idempotency, skew, and Stripe sync

Meta description: Architect a usage-based billing system that survives duplicate events, clock skew, and retroactive corrections — with idempotent aggregation and Stripe sync.

Tags: backend microservices saas architecture api

---

TL;DR

Usage-based billing sounds simple until you hit your first duplicate event, retroactive correction, or clock skew bug at 2 AM. The naive implementation — count events, bill at month end — collapses fast in production. This post walks through the architecture that actually holds: idempotent ingestion, windowed aggregation, reliable Stripe sync, and the edge cases most teams discover only after they’ve already significantly undercharged a customer.

---

Why naive metering breaks in production

In my experience building production billing systems, the failure mode is always the same: engineers treat metering like logging. They fire-and-forget events to a queue, aggregate nightly, and push totals to Stripe. It works in staging. It fails in production.

Usage metering is a financial system, not an analytics pipeline. The correctness bar is completely different.

Failure class	Naive implementation	Production reality
Duplicate events	Ignored	Double-billing customers
Clock skew	Assumed negligible	Events land in wrong billing window
Retroactive corrections	Not modeled	Ledger is permanently wrong

---

The event ingestion layer

Your ingestion endpoint must be idempotent by design, not by convention.

Every usage event needs a deterministic, client-generated idempotency key — not a server-assigned UUID. The client owns the identity of the event.

data class UsageEvent(
    val idempotencyKey: String,  // SHA-256(tenantId + resource + timestamp + nonce)
    val tenantId: String,
    val metric: String,
    val quantity: Long,
    val occurredAt: Instant,     // client-side wall clock
    val receivedAt: Instant      // server-side, set at ingestion boundary
)

On the server, store idempotencyKey with a unique constraint. Duplicate submissions return HTTP 200 with the original result — no error, no retry storm. The client never needs to know.

Use occurredAt for billing window assignment, but cap the acceptable skew. Events arriving more than 24 hours late relative to receivedAt should trigger a manual review flag, not silent acceptance. This threshold isn’t arbitrary: NTP drift on well-configured infrastructure is measured in milliseconds, not hours, and most retry window conventions top out well under 24 hours. An event outside that delta is almost certainly a client bug, a deployment artifact, or a manipulation attempt — not legitimate late delivery.

---

Windowed aggregation without losing your mind

Never aggregate raw events at query time for billing. Pre-aggregate into immutable time-window buckets.

// Aggregation worker — runs on a cron, idempotent
fun aggregateWindow(tenantId: String, windowStart: Instant, windowEnd: Instant) {
    val events = eventStore.query(tenantId, windowStart, windowEnd)
    val totals = events.groupBy { it.metric }.mapValues { (_, evts) -> evts.sumOf { it.quantity } }

    // Upsert — safe to re-run
    aggregateStore.upsert(
        AggregateRecord(tenantId, windowStart, windowEnd, totals, computedAt = Instant.now())
    )
}

The upsert pattern here is load-bearing. Re-running aggregation over a closed window must produce the same result. This property is what makes retroactive corrections possible without corrupting your ledger.

---

Stripe sync: the last mile that bites you

The Stripe billing sync is where most architectures introduce silent inconsistency. If your sync job isn’t idempotent against Stripe’s API, you will double-report usage during a retry. Stripe’s idempotency key deduplication window is 24 hours — meaning retries of the same request within that window are safely deduplicated, but outside it, a retry becomes a net-new write. The real risk is a retry loop within that window writing the same usage record twice using a different or missing key.

The fix is Stripe’s Idempotency-Key header on every UsageRecord write, keyed to your internal aggregate ID:

POST /v1/subscription_items/{si_id}/usage_records
Idempotency-Key: agg_{tenantId}_{windowStart}_{metric}

On retry within the deduplication window, Stripe returns the original response. No duplicate record, no double charge. Outside 24 hours, the same key won’t deduplicate — which is why you should never retry a sync job across day boundaries without checking whether the record already exists.

---

Retroactive corrections: model the correction, not the delete

When you discover a metering bug — and you will — the instinct is to patch historical aggregates. Don’t.

Model corrections as signed adjustment events on your ledger:

data class UsageCorrection(
    val originalAggregateId: String,
    val delta: Long,              // negative to reduce, positive to add
    val reason: String,
    val authorizedBy: String,
    val appliedAt: Instant
)

This gives you a full audit trail, makes corrections reversible, and means your aggregate store is append-only. Immutable history is not a nice-to-have — it is the foundation your finance team will rely on during disputes.

---

Three things I’d tell past me

Make idempotency structural, not cultural. Client-generated idempotency keys with unique constraints in your event store eliminate duplicate billing without requiring discipline from every upstream service. You can’t policy your way to correctness in a distributed system.

Separate occurredAt from receivedAt from day one. Billing window assignment lives on occurredAt. Fraud and skew detection lives on the delta between the two. Conflating them is a schema mistake you can’t easily fix later — you’ll end up with a backfill that touches every historical record.

Treat your aggregate store as an append-only ledger. No in-place updates, no deletes. Corrections are signed adjustment records. This single decision will save you from an audit nightmare and make retroactive corrections a routine operation instead of a crisis.