You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
Hi Team
I am working on a use case where i have to create dbt models on redshift.
On top of these models we have to build data lineage graph on marquez.
My dbt model has 2 tables (one staging table and other one fact table)
Staging table-it queries data from aws data catalog like this
`
{{ config(materialized='view',bind=False) }}
with source_data as (
SELECT book,hfm_code,entity
FROM
"awsdatacatalog"."database_name"."table_name" limit 1000
)
select *
from source_data
fact table model looks like this
{{ config(materialized='table') }}
SELECT book,hfm_code
FROM
{{ ref('staging_fa_capital_local_af') }}
where entity = '5001'
`
my expectation is that i should see 3 nodes in the lineage graph
1st node: representing source as glue table
2nd node:representing the staging table
3rd node:representing the fact table
However lineage medata data does not provide any information about source coming from external database like awsdata catalog
Lineage event generated is:
Expected Behavior
my expectation is that i should see 3 nodes in the lineage graph
1st node: representing source as glue table
2nd node:representing the staging table
3rd node:representing the fact table
Steps To Reproduce
my expectation is that i should see 3 nodes in the lineage graph
1st node: representing source as glue table
2nd node:representing the staging table
3rd node:representing the fact table
Relevant log output
No response
Environment
- OS:
- Python:
- dbt-core:
- dbt-redshift:
Additional Context
No response
The text was updated successfully, but these errors were encountered:
Is this a new bug in dbt-redshift?
Current Behavior
Hi Team
I am working on a use case where i have to create dbt models on redshift.
On top of these models we have to build data lineage graph on marquez.
My dbt model has 2 tables (one staging table and other one fact table)
Staging table-it queries data from aws data catalog like this
`
{{ config(materialized='view',bind=False) }}
with source_data as (
)
select *
from source_data
fact table model looks like this
{{ config(materialized='table') }}
SELECT book,hfm_code
FROM
{{ ref('staging_fa_capital_local_af') }}
where entity = '5001'
`
my expectation is that i should see 3 nodes in the lineage graph
1st node: representing source as glue table
2nd node:representing the staging table
3rd node:representing the fact table
However lineage medata data does not provide any information about source coming from external database like awsdata catalog
Lineage event generated is:
Expected Behavior
my expectation is that i should see 3 nodes in the lineage graph
1st node: representing source as glue table
2nd node:representing the staging table
3rd node:representing the fact table
Steps To Reproduce
my expectation is that i should see 3 nodes in the lineage graph
1st node: representing source as glue table
2nd node:representing the staging table
3rd node:representing the fact table
Relevant log output
No response
Environment
Additional Context
No response
The text was updated successfully, but these errors were encountered: