Producer Acknowledgements (acks)

Producers can choose to receive acknowledgement of data writes:

That means to have the confirmation from the kafka broker that the write did successfully happen.

So we have three settings of acks values:

acks=0, producer won't wait for acknowledgement (possible data loss):

Which means that the producer is not even going to wait or ask for an acknowledgement, that means we have a possible data loss because if a broker goes down, we won't know about it.

acks=1, producer will wait for leader acknowlwdgement (limited data loss):

In this case, producer is going to wait for the leader broker to acknowledgement and that means we have limited data loss.

acks=all, leader + replica acknowledgement (no data loss) :

The producer require the leader as well as all replica (in sync replica) to acknowlwdge the write. Which is going to provide you a guarantee of no data loss under certain circumstances.

Kafka Topic Durability

For a topic replication factor of 3, topic data durbility can withstand 2 brokers loss:

If you have a kafka topic with a replication factor of three then the topic can withstand two two broker loss.

In block diagram, we can see that we lost Broker-102 but still the topic data available to us, because it was on other brokers.

As a rule for a replication factor of N, you can permanently lose up to N-1 brokers and still recover your data:

It means, if you chose a replication factor of N, then you can permanently lose up to N-1 brokers and still have a copy of your data somewhere in your cluster, and that is a very important feature of kafka.