Idempotency keys: exactly-once effects over an at-least-once network
Networks lose responses, not just requests. A client that times out cannot know whether its order was created, so it retries – and without protection the server creates a second order. Idempotency keys let clients retry freely while the server guarantees a single effect.
Ce contenu est actuellement disponible uniquement en anglais.
The problem idempotency solves
RFC 9110 defines a method as idempotent when multiple identical requests have the same intended effect on the server as a single one. Clients, proxies and gateways may therefore retry idempotent requests automatically. POST carries no such guarantee, yet it is the method used for the operations where duplicates hurt most: creating orders, submitting transfers to a downstream system, sending messages, provisioning resources.
The failure is not exotic. A mobile client on a weak connection sends a request, the server commits the change, and the response is lost. From the client’s point of view the request failed. Retrying is the correct reaction for the user, but it is unsafe for the system. Idempotency keys resolve that conflict at the protocol level instead of leaving it to user interface warnings.
How the mechanism works
The client generates a unique key – typically a UUID – for each logical operation and sends it in a header such as Idempotency-Key. The server processes the first request normally and stores, under that key, a fingerprint of the request and the final response. Any later request with the same key is answered from storage without repeating the side effect.
- Scope: keys are unique per client or account, never globally, so one client cannot collide with or probe another’s keys.
- Fingerprint: a hash of the method, path and canonicalised body. A retry with the same key but a different body is a client error and should be rejected, commonly with
422. - Stored response: status code, relevant headers and body, so the replay is indistinguishable from the original.
- Retention: long enough to cover realistic retry windows – often 24 hours – and documented for clients.
Concurrency: the hard part
The naïve implementation checks whether the key exists, processes the request and then stores the result. Two retries arriving at the same moment both pass the check and both execute. The key record must therefore be claimed atomically before any work starts, for example with an insert into a table that has a unique constraint on (client_id, idempotency_key).
The record moves through explicit states: in progress, completed with a stored response, or failed in a way that allows a retry. A second request that finds the key in progress should not wait indefinitely; returning 409 Conflict with a short Retry-After keeps the protocol simple. If the server crashes while a key is in progress, a lease timeout lets a later retry take over – but only when the side effect itself is safe to repeat or can be checked.
Making the side effect itself safe
An idempotency record only protects the boundary of your service. If processing calls a downstream system, that call needs its own protection: pass the same key (or a derived one) downstream, or design the operation so that it can be checked before being repeated. The strongest pattern is to commit the key record, the business change and an outbox event in one local database transaction. Then either everything happened or nothing did, and events are published from the outbox afterwards.
Responses that depend on time – for example a status that changes later – should be replayed as they were originally sent. Clients that want the current state should read the resource, not repeat the write.
Where to implement it
Gateways can enforce the presence and format of keys on routes that require them and reject requests without one. Storing responses at the gateway is possible but couples the gateway to business semantics; most teams keep the authoritative idempotency store inside the service that owns the side effect, close to its database transaction. The gateway’s retry policy can then rely on the key: a request carrying a valid key may be retried, one without it may not.
Document the behaviour precisely in the API description: which routes require keys, the header name, the retention period, the response to a mismatched body and the response to a concurrent duplicate.
Testing and observability
Idempotency bugs hide in timing, so tests must create the timing deliberately. Useful cases include: two identical requests sent concurrently with the same key; a retry after the first request has committed but before its response is stored; a retry with the same key and a modified body; a retry after the retention period; and a crash injected between the business commit and the response. Each case should end with exactly one business effect and a predictable response.
In production, count replays per route and per client. A sudden rise usually points to network problems or a client bug that retries too aggressively, and a client that frequently reuses keys with different bodies is almost certainly generating keys incorrectly – for example per session instead of per operation. Log the key alongside the request ID in traces, so support teams can answer “did my order go through?” without guessing.
Finally, include the key in audit records for sensitive operations. When a dispute arises about whether an instruction was sent once or twice, the stored key, fingerprint and timestamps provide a precise answer.
Exemple: Creating an order safely
A client sends POST /orders with Idempotency-Key: 7c1e…. The service inserts a row (client 42, 7c1e…, fingerprint, in_progress); the unique constraint guarantees that only one request wins. In the same transaction it creates the order and writes an OrderCreated event to the outbox, then marks the key as completed with the 201 response. The response is lost on the network. Ten seconds later the client retries with the same key and body; the service finds the completed record and returns the stored 201 with the same order ID. A buggy client that reuses the key with a different amount receives 422, and no second order exists.
Pièges courants
- Checking for the key and inserting it in two separate steps, which lets concurrent duplicates through.
- Making keys globally unique instead of scoping them per client.
- Ignoring the request body, so a reused key silently returns a response for a different request.
- Retaining keys for less time than clients actually retry.
- Protecting the API boundary but calling downstream systems without their own idempotency.
Questions et réponses
Should clients generate the key, or the server?
The client. Only the client knows that two requests represent the same logical operation; a server-generated key would arrive too late to help with the first lost response.
Can we use the idempotency key as the resource ID?
It is better to keep them separate. The resource ID is a server concept with its own format and lifecycle; the key is a client retry token with a retention period.
What about PUT and DELETE?
They are idempotent by definition if implemented correctly, so keys are usually unnecessary. Make sure the implementation really is: a DELETE that also sends a notification each time is not idempotent in effect.
Liste de contrôle
- List the non-idempotent routes where duplicates cause harm.
- Require an idempotency key on those routes and document the header.
- Scope keys per client and claim them atomically with a unique constraint.
- Store a request fingerprint and reject mismatched reuse.
- Commit key, business change and outbox event in one transaction.
- Define retention and the behaviour for concurrent duplicates.
- Allow gateway retries only for requests that carry a key.
Sources primaires
- RFC 9110 – HTTP Semantics, section 9.2.2 Idempotent Methods
- IETF HTTPAPI working group, “The Idempotency-Key HTTP Header Field”, Internet-Draft (work in progress at the time of writing)
- RFC 6585 – Additional HTTP Status Codes
- Kleppmann, M., Designing Data-Intensive Applications, O’Reilly, 2017 – chapters on transactions and stream processing