Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NTD: staging and mart tables for 2022 agency data #3605

Merged
merged 3 commits into from
Dec 19, 2024

Conversation

charlie-costanzo
Copy link
Member

Description

Following up on #3604, this PR creates a staging and mart table for the ingest of external_ntd__annual_reporting.2022__annual_database_agency_information

Type of change

  • New feature

How has this been tested?

Test once the external table is created

Post-merge follow-ups

  • No action required

@charlie-costanzo charlie-costanzo added the data-pipeline-ingestion-and-modeling Ingesting, parsing and modeling data. Evan Siroky is product owner. label Dec 18, 2024
@charlie-costanzo charlie-costanzo self-assigned this Dec 18, 2024
Comment on lines +178 to +179
- name: stg_ntd__2023_agency_information
- name: stg_ntd__2023_contractual_relationships
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it is erroring because these two tables as duplicated on this file. Just need to delete them from lines 501, 502

Copy link
Contributor

@erikamov erikamov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Just fixing the failing check and it will be good to go!

@charlie-costanzo charlie-costanzo marked this pull request as ready for review December 18, 2024 22:01
@charlie-costanzo charlie-costanzo force-pushed the ntd-2022-agency-data-staging-mart branch from 744179d to fb6491b Compare December 19, 2024 03:29
Copy link

Warehouse report 📦

Checks/potential follow-ups

Checks indicate the following action items may be necessary.

  • For new models, do they all have a surrogate primary key that is tested to be not-null and unique?

New models 🌱

calitp_warehouse.mart.ntd_annual_reporting.dim_2022_agency_information

calitp_warehouse.staging.ntd_annual_reporting.stg_ntd__2022_agency_information

DAG

Legend (in order of precedence)

Resource type Indicator Resolution
Large table-materialized model Orange Make the model incremental
Large model without partitioning or clustering Orange Add partitioning and/or clustering
View with more than one child Yellow Materialize as a table or incremental
Incremental Light green
Table Green
View White

@charlie-costanzo charlie-costanzo merged commit 36c4beb into main Dec 19, 2024
4 checks passed
@charlie-costanzo charlie-costanzo deleted the ntd-2022-agency-data-staging-mart branch December 19, 2024 03:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data-pipeline-ingestion-and-modeling Ingesting, parsing and modeling data. Evan Siroky is product owner.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants