kafka-streams-reduce

Kafka Streams Reduce

This module demonstrates the following:

• The use of the Kafka Streams DSL, including reduce(), groupBy(), toStream() and peek().
• Unit testing using Topology Test Driver.

In this module, records of type <String, KafkaUser> are streamed from a topic named USER_TOPIC. The following tasks are performed:

1. Group the stream by nationality using the groupBy() function.
2. Reduce each group by keeping the KafkaUser with the maximum age for each nationality using the reduce() function.
3. Write the resulting KafkaUser objects to a new topic named USER_REDUCE_TOPIC.

Prerequisites

To compile and run this demo, you will need the following:

• Java 21
• Maven
• Docker

Running the Application

To run the application manually:

• Start a Confluent Platform in a Docker environment.
• Produce records of type <String, KafkaUser> to a topic named USER_TOPIC. You can use the producer user to do this.
• Start the Kafka Streams application.

To run the application in Docker, use the following command:

docker-compose up -d

This command will start the following services in Docker:

• 1 Kafka broker (KRaft mode)
• 1 Schema registry
• 1 Control Center
• 1 producer User
• 1 Kafka Streams Reduce