Idempotent API Design for Mobile Payment Flows: Request Fingerprinting, Server-Side Deduplication Windows, and the Exactly-Once Architecture That Prevents Double Charges on Flaky Networks
The problem: networks lie
In my experience building production payment systems, the most dangerous HTTP response is no response at all. A mobile client sends a charge request, the server processes it, the database commits. Then the TCP connection drops before the 200 reaches the client. The client retries. The user gets charged twice.
This is not an edge case. Mobile networks exhibit timeout rates between 1-5% depending on carrier and region. For a payment system processing thousands of transactions daily, that translates to dozens of potential double charges, each one a support ticket, a chargeback risk, and a reason for users to stop trusting you.
The architecture
| Layer | Responsibility | Implementation |
|---|---|---|
| Client | Generate + attach idempotency key | OkHttp/Ktor interceptor |
| Server gate | Deduplicate requests | PostgreSQL ON CONFLICT upsert |
| Concurrency guard | Serialize simultaneous duplicates | SELECT ... FOR UPDATE row lock |
Layer 1: client-side idempotency keys
The client generates a deterministic key before the first attempt and reuses it across retries. The key is a fingerprint of the request intent, not the request metadata.
// Android - OkHttp Interceptor
class IdempotencyInterceptor : Interceptor {
override fun intercept(chain: Interceptor.Chain): Response {
val request = chain.request()
if (request.method == "POST" && request.url.encodedPath.contains("/payments")) {
val body = request.body?.let { it.toBufferedContent() } ?: return chain.proceed(request)
val key = body.sha256().hex()
val newRequest = request.newBuilder()
.header("Idempotency-Key", key)
.build()
return chain.proceed(newRequest)
}
return chain.proceed(request)
}
}For Ktor HttpClient, the equivalent plugin:
val client = HttpClient(OkHttp) {
install(DefaultRequest) {
// Idempotency key attached at call site
}
}
suspend fun submitPayment(payment: PaymentRequest): PaymentResponse {
val idempotencyKey = payment.hashFingerprint()
return client.post("/api/v1/payments") {
header("Idempotency-Key", idempotencyKey)
setBody(payment)
}.body()
}This is the part most people get backwards: derive the key from business-level fields (user ID, amount, merchant, timestamp bucket), not from a random UUID. A random UUID defeats the entire purpose on retry because each attempt generates a new one. I’ve seen teams spend weeks debugging “phantom duplicates” that traced back to this exact mistake.
Layer 2: server-side deduplication with PostgreSQL
The Ktor backend intercepts the idempotency key and performs an atomic upsert before processing:
// Ktor Backend - Route Handler
post("/api/v1/payments") {
val key = call.request.headers["Idempotency-Key"]
?: return@post call.respond(HttpStatusCode.BadRequest, "Missing Idempotency-Key")
val cached = transaction {
IdempotencyRecord.find { IdempotencyTable.key eq key }.firstOrNull()
}
if (cached != null && cached.status == "completed") {
return@post call.respond(HttpStatusCode.OK, cached.responseBody)
}
// Upsert: claim this key or detect conflict
val claimed = transaction {
exec("""
INSERT INTO idempotency_keys (key, status, created_at)
VALUES (?, 'processing', NOW())
ON CONFLICT (key) DO NOTHING
RETURNING key
""".trimIndent(), listOf(key)) { it.next() }
}
if (claimed == null) {
return@post call.respond(HttpStatusCode.Conflict, "Request already in flight")
}
// Process payment, store result, return response
val result = paymentService.charge(call.receive<PaymentRequest>())
transaction {
exec("UPDATE idempotency_keys SET status='completed', response_body=? WHERE key=?",
listOf(Json.encodeToString(result), key))
}
call.respond(HttpStatusCode.OK, result)
}Layer 3: distributed lock for concurrent duplicates
ON CONFLICT DO NOTHING handles sequential duplicates. But what about two identical requests arriving within milliseconds? SELECT ... FOR UPDATE serializes them:
BEGIN;
SELECT * FROM idempotency_keys WHERE key = $1 FOR UPDATE;
-- Only one transaction proceeds; the other blocks until commit
COMMIT;This row-level lock gives you exactly-once semantics even under concurrent pressure.
TTL-based cleanup
Idempotency records shouldn’t live forever. A scheduled job prunes stale entries:
// Ktor scheduled task
fun Application.configureCleanup() {
launch {
while (isActive) {
delay(1.hours)
transaction {
exec("DELETE FROM idempotency_keys WHERE created_at < NOW() - INTERVAL '24 hours'")
}
}
}
}24 hours balances storage cost against retry windows. Most mobile retries resolve within seconds, but offline-first clients may queue requests for hours.
Common mistakes
| Mistake | Consequence | Fix |
|---|---|---|
| Random UUIDs as idempotency keys | Each retry treated as a new request | Derive key from request content hash |
| No server-side storage | Deduplication only works in-memory, lost on restart | Persist to PostgreSQL |
| Missing concurrency guard | Parallel duplicates both succeed | FOR UPDATE row locks |
| No TTL on idempotency records | Table grows unbounded | Scheduled cleanup with 24h window |
What to do with all this
Fingerprint, don’t randomize. Derive idempotency keys from business fields (user, amount, merchant, time bucket) so retries naturally carry the same key without client-side state management.
Make the database your single source of truth. PostgreSQL ON CONFLICT upserts give you atomic deduplication without external dependencies like Redis. That’s one fewer system to operate and monitor, and in my experience, fewer moving parts in the payment path means fewer 3 AM pages.
Lock the row, not the table. SELECT ... FOR UPDATE serializes only concurrent duplicates for the same key, keeping throughput high for everything else. I’ve seen teams reach for table-level locks or external distributed locks first. Don’t. PostgreSQL row locks are battle-tested and fast enough for the vast majority of payment volumes you’ll actually encounter.