Kafka Producer Acks Explained: Replicas, ISR, and Write Guarantees

When building distributed systems with Apache Kafka, one of the most critical decisions you'll make is how to configure producer acknowledgments. This seemingly simple setting has profound implications for your system's reliability, performance, and data integrity. Let's explore the mechanics behind Kafka's acks configuration and how it interacts with replication to provide different levels of durability guarantees.

The Foundation: Replicas and ISR

Before diving into acknowledgments, we need to understand two fundamental Kafka concepts that directly influence reliability: Replicas and ISR (In-Sync Replicas).

When you create a Kafka topic, each partition is replicated across multiple brokers. This replication is controlled by the replication.factor setting. For example, with replication.factor = 3, each partition has:

• 1 Leader replica
• 2 Follower replicas

Together they form the replica set. Replicas are all copies of a partition across brokers and remain fixed when the topic is created.

However, not all replicas are necessarily "in sync." The ISR is a dynamic subset of replicas that are:

• Alive
• Fully caught up with the leader

If a follower falls behind or becomes unavailable, Kafka removes it from the ISR. This dynamic nature is crucial because Kafka uses ISR—not all replicas—when determining if a write is successful.

Understanding Producer Acknowledgments

The acks configuration controls how many brokers must confirm a write before the producer considers it successful. There are three main settings:

• acks=0
• acks=1
• acks=all (or acks=-1)

Each option provides a different balance between throughput, durability, and latency.

acks = 0: Fire and Forget

With this configuration, the producer does not wait for any acknowledgment from the broker. It simply sends the record and moves on.

Characteristics:

• No confirmation from the broker
• Highest throughput
• Minimal network overhead
• Possible data loss

What Happens Internally:

1. Producer sends the message
2. Producer immediately considers the write successful
3. Broker response is not waited for

If the leader broker is unavailable or crashes, the producer will not know. Retries don't help much here because the producer never receives failure feedback.

Typical Use Cases:

• Logs
• Metrics
• Non-critical telemetry data

acks = 1: Leader Acknowledgment

Here the producer waits for acknowledgment from the leader broker only. The leader writes the message to its local log and then sends the acknowledgment back to the producer.

What Happens Internally:

1. Producer sends record to leader
2. Leader writes record to its log
3. Leader sends acknowledgment
4. Replication to followers happens asynchronously

Risk Scenario: If the leader crashes before followers replicate the message, the data may be lost.

Characteristics:

• Good throughput
• Moderate durability
• Lower latency compared to acks=all

Historically, acks=1 was the default configuration in Kafka up to version 2.x.

acks = all: Full Durability

This is the strongest durability guarantee Kafka offers for producers. Here the leader waits for all replicas in the ISR to acknowledge the write.

However, this works together with another configuration: min.insync.replicas.

min.insync.replicas

This defines the minimum number of ISR replicas that must acknowledge a write. The default value is min.insync.replicas = 1.

If the ISR size drops below this value, the broker rejects the write.

Write Flow with acks=all:

1. Leader appends the record to its log
2. Leader replicates the record to all ISR members
3. Leader waits for acknowledgments
4. Write succeeds only if: ISR size >= min.insync.replicas

If this condition fails, the broker returns errors such as:

• NOT_ENOUGH_REPLICAS
• NOT_ENOUGH_REPLICAS_AFTER_APPEND

The producer then receives an exception.

Kafka 3.x Default Behavior

This is a detail that often confuses people. Even in Kafka 3.x:

• acks=1 still appears as the default producer configuration

However, enable.idempotence=true is enabled by default starting from Kafka 3.0. Idempotent producers automatically require acks=all.

So although the configuration may show acks=1, producers effectively behave like acks=all unless idempotence is disabled.

Write Availability vs Durability

Now let's connect these settings to cluster availability. Assume replication.factor = 3.

With acks=0 or acks=1

As long as the leader broker is available, writes can succeed. Even if follower replicas are down, the producer can still write.

With acks=all

Write availability now depends on min.insync.replicas.

Case 1: min.insync.replicas = 1

• Writes succeed as long as the leader is alive
• This means two brokers can fail and writes will still be accepted

Case 2: min.insync.replicas = 2

• Now at least: Leader + One follower must be present in ISR
• This means one broker failure can be tolerated

Case 3: min.insync.replicas = 3

• All replicas must acknowledge the write
• This means no broker failure can be tolerated

While technically possible, this setup is rarely used because Kafka systems are designed to tolerate node failures. However, it may be used in scenarios where:

• Extremely high durability is required
• Write unavailability is preferred over potential data loss

General Rule for Failure Tolerance: If:

• replication.factor = N
• min.insync.replicas = M

Then the system can tolerate: N - M brokers failing while still accepting writes when using acks=all.

The ISR size can be at most equal to the replication factor, but may shrink dynamically if followers fall behind.

Choosing the Right Acknowledgment Strategy

A quick rule of thumb used in many production systems:

Setting	Throughput	Durability	Typical Usage
acks=0	Highest	Lowest	Logs, metrics
acks=1	High	Moderate	General workloads
acks=all	Lower	Highest	Financial or critical data

Most modern Kafka deployments prefer:

• acks=all
• enable.idempotence=true
• min.insync.replicas >= 2

This combination provides strong guarantees without sacrificing too much availability.

Closing Thoughts

Producer acknowledgments directly impact data durability and system availability. Understanding the relationship between:

• Replication factor
• ISR
• min.insync.replicas
• Producer acks

is essential when designing reliable Kafka pipelines.

Misconfiguring these settings can either lead to:

• Data loss
• Unnecessary write failures
• Reduced throughput

A balanced configuration ensures both reliability and performance in production systems.

Summary

Kafka producer acknowledgments determine when a write is considered successful:

• acks=0 prioritizes throughput but risks data loss
• acks=1 waits for leader acknowledgment
• acks=all ensures replication across ISR members

Combined with replication.factor and min.insync.replicas, these settings control the balance between durability and availability.

What's Next?

Now that we understand acknowledgments and replication, the next step is exploring:

• Retries
• Idempotent producers

These features build on top of the acknowledgment mechanism and are critical for building robust event-driven systems.

Originally published on my personal blog: Kafka Producer Acks Explained