Getting intel from Samza-based applications (Kafka topics and internal state stores) is not a straightforward task, and it usually requires lot of manual intervention (e.g., manually consuming the Kafka topics with the console consumer utility). This may work initially, but as soon as we get more records in the topics, correlating the information quickly turns into a complex task. In addition, if Avro (or another encoding) is used, then using the "console consumer" tools is not the best option.
SaPiDa is a set of micro-services providing a Dashboard for Samza-based Pipelines, based on Akka HTTP and the Akka Streams library Reactive-Kafka.
| Kafka | Akka version | SaPiDa |
|---|---|---|
| 0.11.x | 2.5.x | release 0.10 |
- SBT 1.x for building.
- Docker Compose for spinning up Kafka, Zookeeper, and a schema registry.
Contributions are very welcome!
If you seen an issue that you would like to have it fixed, the best way to make it true is to help out by submitting a pull request. Are you looking to contribute but don't know where to start? ticket marked as "help wanted" or "good first issue" are a good starting point.
Please refer to CONTRIBUTING.md for more details.
SaPiDa is Open Source and available under the Apache 2 License.