Idempotency Keys: The Simple Pattern That Prevents Duplicate API Requests

If you've ever hit "Submit" on a payment form twice and got charged twice, you've experienced what happens when an API lacks idempotency. It's one of the most underappreciated patterns in API design — and one of the most important for anything involving money, email sends, order creation, or any operation that should happen exactly once.

What Is Idempotency?

An operation is idempotent if performing it multiple times produces the same result as performing it once. GET requests are naturally idempotent — fetching a resource ten times doesn't change anything. But POST requests that create resources are not — posting the same order twice will create two orders.

Idempotency keys solve this. The client generates a unique key per logical operation and sends it with each request. The server uses that key to detect duplicates and return the original result instead of processing the request again.

POST /payments
Idempotency-Key: a7f3d9c2-1b4e-4f6a-8e2d-9c3f1a5b7e8d
Content-Type: application/json

{
  "amount": 9900,
  "currency": "usd",
  "customer_id": "cus_abc123"
}

If the network drops after the server processes this request but before the client receives the response, the client can safely retry with the same Idempotency-Key. The server recognizes the key, finds the stored result, and returns it without charging the customer again.

Implementing Idempotency on the Server

Here's a minimal Node.js/Express implementation using Redis as the idempotency store:

const express = require('express');
const redis = require('redis');
const { v4: uuidv4 } = require('uuid');

const app = express();
const client = redis.createClient();
const TTL_SECONDS = 86400; // 24 hours

app.use(express.json());

async function idempotencyMiddleware(req, res, next) {
  const key = req.headers['idempotency-key'];

  if (!key) return next(); // idempotency is optional for this endpoint

  const cacheKey = `idempotency:${req.path}:${key}`;
  const cached = await client.get(cacheKey);

  if (cached) {
    const { status, body } = JSON.parse(cached);
    return res.status(status).json(body);
  }

  // Intercept the response to cache it
  const originalJson = res.json.bind(res);
  res.json = async (body) => {
    await client.setEx(
      cacheKey,
      TTL_SECONDS,
      JSON.stringify({ status: res.statusCode, body })
    );
    return originalJson(body);
  };

  next();
}

app.post('/payments', idempotencyMiddleware, async (req, res) => {
  // Your actual payment logic here
  const payment = await processPayment(req.body);
  res.status(201).json(payment);
});

A few important details:

• Scope the key to the endpoint — the same key value used on /payments and /orders should be treated as separate keys.
• Set a TTL — 24 hours is a common default; Stripe uses 24 hours, others use up to 48.
• Return the original status code — if the original request returned a 422 validation error, return that same 422 on retries, not a 200.

Generating Keys on the Client

Clients should generate a new UUID per logical request, persisting it locally until they receive a definitive success or failure:

import uuid
import httpx

def create_payment(amount: int, currency: str, customer_id: str) -> dict:
    idempotency_key = str(uuid.uuid4())  # generate once, reuse on retry

    for attempt in range(3):
        try:
            response = httpx.post(
                "https://api.example.com/payments",
                headers={"Idempotency-Key": idempotency_key},
                json={
                    "amount": amount,
                    "currency": currency,
                    "customer_id": customer_id,
                },
                timeout=10.0,
            )

            if response.status_code in (200, 201):
                return response.json()

            if response.status_code in (400, 422):
                raise ValueError(f"Bad request: {response.text}")

            # 5xx or network error — retry with the SAME key
        except httpx.TimeoutException:
            if attempt == 2:
                raise

    raise RuntimeError("Payment failed after 3 attempts")

The key insight: generate the UUID before the first attempt and keep using it on retries. If you generate a new UUID on each retry, you've defeated the entire purpose.

Handling Conflicts

What if the same idempotency key is used with different request bodies? This is almost certainly a client bug. The server should return a 409 Conflict with a clear error message:

{
  "error": "idempotency_conflict",
  "message": "This idempotency key was already used with different request parameters."
}

What About Database Transactions?

If your datastore supports it, you can implement idempotency without a separate cache by using a unique constraint on an idempotency_key column:

CREATE TABLE payments (
  id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  idempotency_key VARCHAR(255) UNIQUE,
  amount      INTEGER NOT NULL,
  customer_id VARCHAR(255) NOT NULL,
  created_at  TIMESTAMPTZ DEFAULT now()
);

On conflict, return the existing row. This is simpler and stays consistent with your primary database.

Closing Thoughts

Idempotency keys are a small implementation effort with a huge reliability payoff. Any API endpoint that creates or mutates a resource — payments, orders, email sends, user signups — should support them. Your users' retry logic (and their patience) will thank you.

When you're exploring and testing APIs that use idempotency keys, APIKumo makes it easy to set and manage custom headers like Idempotency-Key across your requests, so you can verify that your server handles retries correctly without writing throw-away test scripts.