What is a Kafka topic and how does partitioning work?

Beginner

Answer

A topic is a category or stream name to which producers send messages and from which consumers read messages. Topics are divided into partitions for scalability and parallelism.

Partitioning benefits:

  • Enables parallel processing by multiple consumers
  • Distributes data across multiple brokers
  • Provides ordering guarantees within each partition
  • Allows horizontal scaling

Example: A topic "user-events" with 3 partitions can have messages distributed across partitions 0, 1, and 2 based on a key or round-robin.