Can a Kafka consumer read messages immediately when acks=0, before the leader syncs with followers? - Stack Overflow

admin管理员组

文章数量:1302379

If I set acks=0 in Kafka, my producer will write messages one after another without waiting for any acknowledgment from the leader broker or the replicas/followers. However, my question is whether a consumer can start reading these messages before the leader synchronizes with the followers.

Specifically, if the producer sends messages continuously, can the consumer read them immediately, or does the leader need to synchronize with the followers first before the consumer can access the messages?

Could you please point me to the relevant Kafka documentation that addresses this?

If I set acks=0 in Kafka, my producer will write messages one after another without waiting for any acknowledgment from the leader broker or the replicas/followers. However, my question is whether a consumer can start reading these messages before the leader synchronizes with the followers.

Specifically, if the producer sends messages continuously, can the consumer read them immediately, or does the leader need to synchronize with the followers first before the consumer can access the messages?

Could you please point me to the relevant Kafka documentation that addresses this?

• apache-kafka

Share Improve this question Follow asked Feb 10 at 19:55 Ciprian Ciprian 7311 gold badge11 silver badge66 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

As you said, if ack set to 0, producer will not wait for a response from the broker and that message will be immediately available for consumers to read. In short ack value means "how many brokers should the producer wait to receive a response?". Those brokers are "in sync" brokers for that topic.

Then what is in-sync brokers? In-sync brokers keep data synchronously, unlike replication brokers. A replicated data may be written to other brokers a few minutes, or hours later but that is not the case for in-sync brokers. It is defined by min.insync.replicas parameter. By default it's 1, meaning only 1 broker has the synced message.

It matters for ack's too, since ack value can not be higher than in.sync.replicas. If your ack is 2 and in.sync.replicas 3, producer is going to wait for the broker it directly connected and 1 other sync broker to complete the producing process.

In short;

• When ack is 0, the message will be immediately available to consume, regardless of the number of in.sync.replicas or replication.factor.
• When ack is 1 (by default), producer is going to wait for the acknowledgement from the broker it is connected.
• When ack is more than 1 (let's say 3), in.sync.replicas must be equal or greater than ack value. The producer is going to wait for the acknowledgement from the brokers with a total number given in ack parameter. These brokers are called "in sync" brokers.
• ack ≤ in.sync.replicas ≤ replication.factor

For documentation:

https://docs.confluent.io/kafka/design/replication.html#in-sync-replicas-and-producer-acks https://docs.confluent.io/platform/current/installation/configuration/topic-configs.html#min-insync-replicas

本文标签：