Plugin that reads and reports kafka topic lag
- Topic producer rate - events per min for each topic being produced.
- Topic/group consumer rate - events per min consuming rate for each group/topic tuple.
- Group/topic/partition lag - difference between the offset and the head of the topic for each group/topic/patition tuple.
- Group/topic lag summary - same as above but a summary across all partitions (faster dashboard loading when you have a lot of partitions).
- Offset age - time in milliseconds since the offset has been updated (data used to alert when a consumer has stopped functioning)
This plugin can be loaded into Kairos to monitor Kafka offsets in a variety of ways:
- Load plugin on your existing Kairos nodes.
- Load plugin on a dedicated Kairos node that writes directly to C*.
- Load plugin on a dedicated Kairos node that uses the remote datastore to forward the data to your existing Kairos ingest nodes.
- Create a custom datastore to forward the data to whatever you like.