-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Description
We have only FileEventSink for persistent event storage. However, it is only effective for local debugging of a small set of recorded events.
We need to add a proper DB support to store events so that frontend clients can access past events also.
Candidate databases for now:
- ClickHouse: It is a column store DB with best performance for append only writes and range queries.
- There is chDB which is lightweight version. It can be used for local dev environment and small deployments.
- It has native support of JSON also. This way Two columns can be timestamp and task name which will be used for efficient queries and other attributes can live in the JSON field.
- Does not support direct modification effiicently. But does have batch delete/updates . Can be important for keeping particular time range data.
- TimescaleDB (PostgreSQL-based): Maturity of PostgreSQL, hybrid row and column hypertable.
- Overall looks more mature than ClickHouse, but does not seem as specialised for our use case as ClickHouse.
- MongoDB: Native BSON support, widely used for JSON objects.
- Good community support and integrations.
- MongoDB Atlas gives free hosted servers for small projects.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels