Idempotent Receiver
Distributed systems deliver messages more than once.
That sentence surprises people the first time they build a message-driven system. A queue promises durability. A broker promises retries. Everything feels reliable.
And then a message shows up twice.
The duplication is rarely malicious. It is usually the result of a retry. A consumer processes a message successfully, but the acknowledgement fails. The broker assumes the message was not handled and delivers it again.
From the broker’s perspective, this is safety. From the application’s perspective, it can be disaster.
Consider what happens when a message represents a side effect.
A “ChargeCustomer” event is processed twice.
An “OrderShipped” email is sent twice.
A “ProvisionSubscription” command creates two subscriptions.
At-least-once delivery guarantees that a message will be delivered. It does not guarantee that it will be delivered only once.
The Idempotent Receiver pattern exists to make duplicate delivery harmless.
The Nature of At-Least-Once Delivery
Message brokers favour safety over precision. If there is uncertainty about whether a message was processed, they err on the side of redelivery.
This behaviour protects against message loss. It does not protect against duplicate work.
In many systems, that tradeoff is acceptable because consumers are designed to tolerate duplicates.
That tolerance is not accidental. It is deliberate.
Designing for Safe Repetition
An operation is idempotent if applying it multiple times produces the same result as applying it once.
Reading a value is idempotent.
Setting a flag to true is idempotent.
Charging a credit card is not.
The Idempotent Receiver pattern ensures that each message is processed in a way that repeated handling does not create repeated side effects.
The most common technique is simple: record which message IDs have already been processed.
Each message carries a unique identifier. Before performing any side effects, the consumer checks whether that identifier has been seen before. If it has, the message is ignored. If it has not, the consumer proceeds and records the ID as processed.
In practice, this is often enforced with a database constraint.
A table might contain a processed_messages record keyed by message_id. The consumer attempts to insert the ID as part of its transaction. If the insert fails due to a unique constraint violation, the message has already been handled.
The database becomes the arbiter of uniqueness.
The pattern shifts the burden from the broker to the consumer. The broker may deliver twice. The consumer ensures the side effect occurs once.
A Subtle but Crucial Boundary
Idempotency is not only about tracking message IDs. It also influences how side effects are structured.
Instead of writing code that says “create a record,” an idempotent design might say “ensure this record exists.” Instead of “increment balance,” it might say “set balance to this calculated value.”
The goal is to make repetition safe, even in edge cases where state changes and message ordering interact in unexpected ways.
In systems that charge money, send emails, or provision infrastructure, this discipline matters. Duplicate delivery without idempotency becomes visible to users very quickly.
Tradeoffs and Costs
The pattern introduces overhead. Processed message IDs must be stored. Storage grows over time and may require cleanup or partitioning strategies. Each side-effecting consumer must be written carefully.
The complexity does not live in one place. It spreads across every consumer that performs irreversible actions.
Yet the alternative is fragile behaviour that depends on the hope that duplicates never occur.
In distributed systems, hope is not a strategy.
Where It Fits
The Idempotent Receiver pattern is essential in any system that relies on at-least-once delivery semantics and performs side effects that cannot safely repeat.
Payment processing, notification systems, inventory updates, provisioning workflows, and event-driven microservices all benefit from it.
In contrast, purely functional transformations or systems that can tolerate duplicate processing without consequence may not need explicit tracking.
The key question is simple: what happens if this message runs twice?
If the answer is “nothing bad,” you are safe.
If the answer makes you uneasy, idempotency is required.
The Larger Lesson
Message brokers prioritise durability. They guarantee that a message will be delivered.
It is the consumer’s responsibility to guarantee that repeated delivery does not cause harm.
The Idempotent Receiver pattern acknowledges a fundamental truth of distributed systems:
Delivery is not the same as execution.
Safety lives at the edges.