This is the full release of the pseudo ETL system. It comes with two data sources, two data sinks and a simple console user interface.
Implemented features
- Data sinks
- Console: container where consumed messages are printed to STDOUT.
- PostgreSQL: container where consumed messages are saved to a PostgreSQL database table.
- Data sources:
- Simulation: container which generates messages with random data when queried.
- File: container which reads messages on a one by one basis from a JSON file.
- ETL interface: extracts messages from a data source and dumps them in a data sink on a one by one basis. ETL tasks are run until the data source is depleted.
- Console front-end: simple text-based console user interface, used to run ETL tasks.