omearacomearac
84

Reactive Kafka Microservice Template

screenshot of Reactive Kafka Microservice Template

An example of a microservice template which uses akka/reactive-kafka to stream JSON messages

Github

Overview

The Reactive Kafka Microservice Template serves as an excellent starting point for developers looking to integrate Apache Kafka with Akka Streams for microservice communication. This template does not include domain-specific business logic, offering a flexible framework that can be easily customized to fit different use cases. Its primary focus lies on demonstrating the essential features of consumer and producer streams using the Reactive-Kafka library, making it a practical tool for those looking to adopt reactive programming paradigms.

The architecture is designed to showcase how Akka actors can effectively handle backpressure in a streaming context, with clearly defined roles for both producing and consuming data. During application startup, the microservice initializes multiple streams and actors that interact with Kafka, allowing for real-time data processing and effective message handling. With an intuitive Akka HTTP API to control stream operations, this template simplifies the management of message flow in a distributed system.

Features

  • Backpressure Management: Utilizes Akka's backpressure mechanisms to handle message flow between producer and consumer streams, ensuring efficient resource utilization.

  • Reactive Streams: Implements Akka Streams for both producing and consuming messages, allowing for a reactive approach to stream processing.

  • Actor Integration: Connects Akka Actors seamlessly with Kafka, leveraging actor-based concurrency for processing messages in real-time.

  • Real-Time Event Sourcing: Allows for local event sourcing through the Akka EventBus, facilitating the publication and consumption of events within the system.

  • Akka HTTP Interface: Provides a simple HTTP API to control the lifecycle of the consumer streams, enabling dynamic starting and stopping of message processing.

  • JSON (Un)Marshalling: Supports the conversion of messages to and from JSON format, making it easy to interact with external systems.

  • Example Topics: Pre-defined Kafka topics (TestDataChannel and AppEventChannel) demonstrate how to manage multiple streams within the Kafka message broker.

  • Testing Framework: Includes setup for unit and integration testing of Akka HTTP, Actors, and Streams, ensuring reliability and helping to catch issues early in the development process.