Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add openlineage adapter #1123

Merged
merged 7 commits into from
Sep 10, 2024
Merged

Add openlineage adapter #1123

merged 7 commits into from
Sep 10, 2024

Commits on Sep 10, 2024

  1. WIP

    @skrawcz @elijahbenizzy
    skrawcz authored and elijahbenizzy committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    bfc6a4b View commit details
    Browse the repository at this point in the history

  2. Adds more facets 

    Seems to work with marquez!
    @skrawcz @elijahbenizzy
    skrawcz authored and elijahbenizzy committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    b9b4966 View commit details
    Browse the repository at this point in the history

  3. Adds open lineage tracker, example, and adds to docs 

    This gets the V1 version going.

Main assumptions:
 - we utilize the utils functions for metadata

This is the basis for someone quickly and easily
implementing OL.

TODO:  version the utils metadata schema
TODO: add some unit tests.
    @skrawcz @elijahbenizzy
    skrawcz authored and elijahbenizzy committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    0b48090 View commit details
    Browse the repository at this point in the history

  4. Adds openlineage to docs requirements and as optional target

    @skrawcz @elijahbenizzy
    skrawcz authored and elijahbenizzy committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    f4cb881 View commit details
    Browse the repository at this point in the history

  5. Fix up rst docs

    @skrawcz @elijahbenizzy
    skrawcz authored and elijahbenizzy committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    15d39d3 View commit details
    Browse the repository at this point in the history

  6. Fixes materialization link

    @skrawcz @elijahbenizzy
    skrawcz authored and elijahbenizzy committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    e34fb27 View commit details
    Browse the repository at this point in the history

  7. Fixes up compatibility in python3.10+ for traceback counting

    @elijahbenizzy
    elijahbenizzy committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    6b056ca View commit details
    Browse the repository at this point in the history