educative.io

Kafka keys and adding new partitions

Courses
,
#1

However, adding more partitions to an existing topic doesn’t guarantee that new messages with a given key will land in the same partition as before.

Why is this not guaranteed? If a key maps to a partition before the adding new partition, what is preventing it from mapping to the same partition after the mapping?

Generally, new partitions shouldn’t be added to a topic if partitioning by keys is important.

Why is this not recommended?

#2

Hi @dgdp

Can you specify the lesson with chapter no. in the course which states these lines? I am not able to find it.

#3

Hi Usman,

This is in the Partitions lesson of the Basics chapter:

https://www.educative.io/courses/scalable-data-pipelines-kafka/3j9oDjE93lx#:~:text=However%2C%20adding%20more%20partitions%20to%20an%20existing%20topic%20doesn%E2%80%99t%20guarantee%20that%20new%20messages%20with%20a%20given%20key%20will%20land%20in%20the%20same%20partition%20as%20before.%20Generally%2C%20new%20partitions%20shouldn%E2%80%99t%20be%20added%20to%20a%20topic%20if%20partitioning%20by%20keys%20is%20important.