Kafka guarantees message ordering within partitions. The catch: your partition key determines which partition each event lands in, and a wrong key silently scrambles the order your consumers depend on.
You ship three events for one order: OrderCreated, OrderPaid, OrderCancelled. The consumer reads them and decides the order's final state. In staging, every order ends up correct. In production, a few orders a day end up Paid when they should be Cancelled. The events are all there. The payloads are right. They just arrived in the wrong order.
The bug isn't in your consumer. It's in the partition key you picked for the producer. Kafka gave you exactly the ordering guarantee it promised. It just wasn't the one you assumed.
The Guarantee Kafka Actually Makes
Kafka orders messages within a single partition. That's the whole promise. Messages in partition 3 come out in the order they went into partition 3. There is no ordering across partitions. None. A topic with 12 partitions is 12 independent ordered logs. Consumers read each partition in order, but two partitions advance independently. If OrderPaid lands in partition 4 and OrderCancelled lands in partition 7, the two consumer threads reading those partitions race. Whichever thread is ahead wins, and "ahead" depends on consumer lag, rebalances, and which broker answered first.
So the question that decides your ordering isn't "does Kafka preserve order." It's "which partition does each event land in." And that is decided entirely by the partition key.
The default partitioner hashes the key and takes it modulo the partition count:
Comments
Please log in or register to join the discussion