Two users race to claim the username alice under a UNIQUE(username) constraint. Two independent leaders each check their local table, find alice free, and both GRANT — the constraint is silently violated and the conflict only surfaces at merge. Route the same two claims through one totally-ordered log partition and a single consumer grants one, rejects the other — exactly one owner.
Two users race to claim the username alice; a UNIQUE(username) constraint says exactly one may own it. A tempting design: two datacenters, each a leader that enforces the constraint locally, replicating asynchronously. Model it — two independent leaders, each with its own taken set, each check-then-set against its local table — and both requests find alice free and both GRANTED. The constraint is silently violated; the conflict is only discovered at replication merge, by which point two users have been told they own alice. Now route every claim through one totally-ordered log partition (partition by hash of the username), consumed single-threaded: the consumer sees a definite first writer, grants one and REJECTS the rest — exactly one owner. Same two claims; the only difference is whether the contenders share an order.
Designing Data-Intensive Applications, 2nd ed. (Kleppmann & Riccomini, 2025), Chapter 13 — The Future of Data Systems (stream processing / a philosophy of streaming systems), §"Uniqueness constraints require consensus" and §"Uniqueness in log-based messaging":
Enforcing a uniqueness constraint requires consensus [...] The most common way of achieving this consensus is to make a single node the leader, and put it in charge of making all the decisions.
The book rules out asynchronous multi-leader replication for uniqueness because different leaders can concurrently accept conflicting writes, and offers the log-based route: funnel all requests for a key through one partition of a totally-ordered log and let a single consumer decide. Here you watch both.
The surprise is that a local uniqueness check is not a uniqueness constraint. Each line notes where the idea shows up.
UNIQUE(username) must admit exactly one owner across the whole system, not per node. It shows up as the two-owner outcome in Model 1: each leader's local constraint held, yet the global one broke.user-1 via req-1, user-2 via req-2) both claim alice.
A uniqueness constraint requires all contenders for a name to agree who was first. Two ways to try to supply that agreement:
alice free, and both grant. The disagreement surfaces only when the two logs merge — after both users were promised the name. A compare-and-set inside each leader would not help: the leaders don't share the register it would test.alice is appended to the same partition (chosen by hash of the username) and a single consumer reads it in order. The log is the shared order: it has a definite first entry, so the consumer grants that one and rejects every later one. One order, one first writer, one owner.Both models run against the identical two claims. The whole thing is one script, code/uniqueness_needs_total_order.py.
cd study/ddia/ch13/code
python3 uniqueness_needs_total_order.pyDo not read the output yet — make each prediction first.
Predict. Two datacenters, each a leader enforcing UNIQUE(username) on its own local table, replicating asynchronously. Two users concurrently claim alice, one request routed to each leader. How many of the two claims are GRANTED?
Both GRANTED. Each leader's local check passed — from L1's view alice was free, and from L2's view alice was free too. Neither local constraint was violated; the global one was.
Predict. Replication now merges the two leaders' indexes. Does the merge catch the conflict in time to keep the constraint — and how many owners does alice end up with?
Two owners — the constraint is violated. The merge detects the conflict, but detection is too late: both grants already committed and both users were told they own alice. Async replication moves conflict discovery to after the decision, which is exactly when a uniqueness constraint can no longer be enforced.
Predict. Now route both claims through a single log partition (chosen by hash of the username) and consume it single-threaded. How many of the two claims are GRANTED?
Exactly one owner. The log gives the two claims a definite order; the single consumer grants offset 0 and, seeing alice already taken, rejects offset 1. The constraint holds before anyone is told they won — no merge, no conflict, no double grant.
Predict. Summarize: how many owners does each model produce, and what is the one structural difference that decides it?
2 vs 1 — the difference is a shared order. Nothing about the claims changed; what changed is whether the contenders were forced through one order. That order is the consensus the constraint needs.
alice ends with 2 owners — UNIQUE(username) violated, and the conflict only surfaces at merge.alice has 1 owner — UNIQUE(username) held.A uniqueness constraint is a consensus problem wearing a schema keyword. To grant alice to exactly one claimant, the system must linearize the claims: put them in a single order and let the first one win. Everything turns on whether that order exists at decision time.
Asynchronous multi-leader replication has no such order. Each leader is an independent decider with its own copy of the index, and by construction it commits a write before replicating it. So when two claims land at the same instant on different leaders, each check-then-set runs against a table that has not yet heard about the other — both read alice free, both write, both grant. The conflict is real, but it becomes visible only when the two logs merge, and a merge that runs after both commits can at best detect a double grant, never prevent it. (A compare-and-set inside a leader doesn't rescue this: CAS serializes contenders on one register, and the two leaders are testing two different registers. Local atomicity buys nothing when the deciders are independent.) That is precisely why the book rules async multi-leader out for uniqueness.
Routing every claim for a key through one totally-ordered log partition is the consensus that was missing. Appending to a single partition assigns the concurrent claims a definite sequence — offset 0, then offset 1 — and a single-threaded consumer reading that sequence sees an unambiguous first writer. It grants offset 0 and rejects everything after, so at most one claim is ever granted. The log doesn't detect the conflict after the fact; it dissolves it, because there is never a moment when two deciders both believe alice is free. Uniqueness = agreeing on a first writer = a total order over the contenders.
UNIQUE index on one primary does the job. The anomaly appears only when two or more nodes decide the same key without a shared order. And the fix does not require a global bottleneck: partitioning the log by the unique key keeps every key's decisions serial on its own partition while different keys proceed in parallel — you serialize claim('alice') against other alice claims, not against claim('bob'). The constraint scales precisely because uniqueness is per-key, so the order it needs is per-key too.
partition_for to spread keys across partitions and confirm that alice and bob claims land on different partitions and decide in parallel — uniqueness held per key, no global lock.Sources: DDIA 2e, Ch. 13, §"Uniqueness constraints require consensus", §"Uniqueness in log-based messaging" · Ch. 6 §"Handling write conflicts" · Ch. 10 §"Consensus".