Common consumer patterns and use cases
The following pages detail standard patterns for Streams reading messages from Kafka.
These patterns have the following assumptions:
- One consumer operator consumes messages from one single topic with a string message, for example a JSON message
- For a production environment, the consumer starts consuming at the default start position
- Kafka guarantees no ordering of messages accross partitions.
- Consume all partitions
- A single KafkaConsumer operator consumes all messages from a topic regardless of the number of partitions.
- Kafka consumer group
- Multiple KafkaConsumer operators consume from the same topic(s) where the topic partitions are automatically distributed over the consumer operators.
- Assigned partitions
- Multiple KafkaConsumer operators reading from the same topic(s).