Data.Rentgen

What is Data.Rentgen?

Data.Rentgen is a DataLineage service compatible with OpenLineage specification.

Note: service is under active development, and is not ready to use.

Collect lineage events produced by OpenLineage clients & integrations (Spark, Airflow, Flink, custom ones).
Store operation-grained events (instead of job grained Marquez), for better detalization.
Provide API for run ↔ dataset lineage, as well as parent run → children run lineage.
Support handling large amounts of lineage events, using Kafka as event buffer and storing data in tables partitioned by event timestamp.

Name		Name	Last commit message	Last commit date
Latest commit History 144 Commits
.github		.github
data_rentgen		data_rentgen
docker		docker
docs		docs
tests		tests
.dockerignore		.dockerignore
.env.docker		.env.docker
.env.local		.env.local
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
.readthedocs.yaml		.readthedocs.yaml
.spdx-license-header.txt		.spdx-license-header.txt
CONTRIBUTING.rst		CONTRIBUTING.rst
LICENSE.txt		LICENSE.txt
Makefile		Makefile
README.rst		README.rst
SECURITY.rst		SECURITY.rst
codecov.yml		codecov.yml
docker-compose.test.yml		docker-compose.test.yml
docker-compose.yml		docker-compose.yml
poetry.lock		poetry.lock
pyproject.toml		pyproject.toml