Toolkit com.teracloud.streams.kafka 5.0.0
General Information
The Kafka toolkit integrates Streams with Apache Kafka servers allow applications to read messages from Kafka topics and write messages into topics. The toolkit contains Kafka client libraries which can be used on the following server versions:
- 3.7+
The Kafka toolkit contains two operators:
- The KafkaConsumer source operator consumes messages from a Kafka topic and creates tuples which can be processed by downstream operators in the Streams application.
The relation between Kafka messages and tuples is one-to-one
- The KafkaProducer sink operator creates Kafka messages from tuples recevied.
The relation between tuples and Kafka messages can be one-to-one or one-to-many if multiple topics are specified.
Both operators must be configured to communicate with the Kafka brokers.
- Configuring operators to communicate with Kafka
- The Kafka operators must be configured in order to connect to the Kafka servers.
- Common consumer patterns and use cases
- The following pages detail standard patterns for Streams reading messages from Kafka.
- Improving throughput
- This page demonstrates a technique to improve the overall throughput of a Streams application that is consuming messages from Kafka.
- Version
- 5.0.0
- Required Product Version
- 7.2.0.0