What are Kafka Headers and when would you use them?

Beginner

Answer

Kafka Headers are optional metadata key-value pairs attached to each message record.

Use cases:

  • Routing: Route messages based on headers
  • Tracing: Add correlation IDs for distributed tracing
  • Security: Include authentication tokens
  • Content metadata: MIME types, encoding information
  • Source identification: Origin system information
// Producer adding headers
ProducerRecord<String, String> record = new ProducerRecord<>(
    "my-topic", "key", "value");
record.headers().add("correlation-id", "12345".getBytes());
record.headers().add("source-system", "order-service".getBytes());

Benefits: Headers don't affect partitioning and allow metadata without modifying message payload.