Skip to content

Commit

Permalink
feat(dim_organizations): Replaces ntd_id_2022 from calculation to usi…
Browse files Browse the repository at this point in the history
…ng airtable (#3367)

Co-authored-by: V <vevetron@gmail.com>
  • Loading branch information
vevetron and V authored Jun 11, 2024
1 parent 5c61b0f commit 329b926
Show file tree
Hide file tree
Showing 6 changed files with 8 additions and 4 deletions.
1 change: 1 addition & 0 deletions airflow/.gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
logs/*
airflow.sh
Original file line number Diff line number Diff line change
Expand Up @@ -254,3 +254,6 @@ schema_fields:
- name: raw_ntd_id
type: STRING
mode: NULLABLE
- name: ntd_id_2022
type: STRING
mode: NULLABLE
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ int_transit_database__organizations_dim AS (
hq_county_geography,
is_public_entity,
raw_ntd_id,
ntd_id_2022,
public_currently_operating,
public_currently_operating_fixed_route,
_is_current,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ models:
renamed to `_deprecated__ntd_agency_to_organizations`).
- name: ntd_id_2022
description: |
Just pulls out the 5 digit component of the raw_ntd_id
Pulled from airtable, the 5 digit component of raw_ntd_id
- name: name
description: Organization name
- name: organization_type
Expand Down
4 changes: 1 addition & 3 deletions warehouse/models/mart/transit_database/dim_organizations.sql
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,7 @@ dim_organizations AS (
WHEN _valid_from >= '2023-05-23' THEN raw_ntd_id
ELSE ntd_to_org.ntd_id
END AS ntd_id,
IF(LENGTH(ntd_id) >= 10,
SUBSTR(ntd_id, -5),
ntd_id) AS ntd_id_2022,
ntd_id_2022,
public_currently_operating,
public_currently_operating_fixed_route,
_is_current,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ stg_transit_database__organizations AS (
hq_county_geography,
is_public_entity = "Yes" AS is_public_entity,
raw_ntd_id,
ntd_id_2022,
public_currently_operating = "Yes" AS public_currently_operating,
public_currently_operating_fixed_route = "Yes" AS public_currently_operating_fixed_route,
FROM once_daily_organizations
Expand Down

0 comments on commit 329b926

Please sign in to comment.