Skip to content

feat(opendevdata): use committed_at for developer validity periods instead of ingestion time #6170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kenyiu wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat(opendevdata): use committed_at for developer validity periods instead of ingestion time #6170

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ce2334a
fix: update timestamp fields from created_at to committed_at in model…
kenyiu Jan 13, 2026
fe61f3f
Merge remote-tracking branch 'refs/remotes/origin/main'
kenyiu Jan 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ...models/intermediate/domain-specific/ddp/commits/int_opendevdata__commits_with_repo_id.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ MODEL (
description 'OpenDevData commits enriched with canonical repo_id',
dialect trino,
kind FULL,
partitioned_by DAY("created_at"),
partitioned_by DAY("committed_at"),
tags (
"opendevdata",
"ddp",
Expand All @@ -18,7 +18,7 @@ SELECT
odc.id,
odc.repo_id AS opendevdata_repo_id,
odc.sha1,
odc.created_at,
odc.created_at AS opendevdata_created_at,
odc.additions,
odc.deletions,
odc.authored_at,
Expand Down
2 changes: 1 addition & 1 deletion ...qlmesh/models/intermediate/domain-specific/ddp/developers/int_opendevdata__developers.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ WITH developer_history AS (
'@',
SPLIT_PART(commits.commit_author_email, '@', 2)
) AS hashed_author_email,
MIN(commits.created_at) AS valid_from
MIN(commits.committed_at) AS valid_from
FROM oso.stg_opendevdata__commits AS commits
INNER JOIN oso.stg_opendevdata__canonical_developers AS devs
ON commits.canonical_developer_id = devs.id
Expand Down
4 changes: 2 additions & 2 deletions warehouse/oso_sqlmesh/tests/test_int_opendevdata__commits_with_repo_id.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ test_int_opendevdata__commits_with_repo_id:
- id: 1
opendevdata_repo_id: 10
sha1: "sha1"
created_at: "2024-01-01 00:00:00"
opendevdata_created_at: "2024-01-01 00:00:00"
additions: 10
deletions: 5
authored_at: "2024-01-01 00:00:00"
Expand All @@ -95,7 +95,7 @@ test_int_opendevdata__commits_with_repo_id:
- id: 2
opendevdata_repo_id: 10
sha1: "sha2"
created_at: "2024-01-01 00:00:00"
opendevdata_created_at: "2024-01-01 00:00:00"
additions: 20
deletions: 10
authored_at: "2024-01-01 00:00:00"
Expand Down
10 changes: 5 additions & 5 deletions warehouse/oso_sqlmesh/tests/test_int_opendevdata__developers.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ test_int_opendevdata__developers_single_entry:
- canonical_developer_id: 101
commit_author_name: "Open Dev"
commit_author_email: "dev@open.com"
created_at: "2024-01-01 10:00:00"
committed_at: "2024-01-01 10:00:00"
oso.stg_opendevdata__canonical_developers:
rows:
- id: 101
Expand All @@ -30,16 +30,16 @@ test_int_opendevdata__developers_scd_logic:
- canonical_developer_id: 202
commit_author_name: "Original Name"
commit_author_email: "original@test.com"
created_at: "2024-01-01 10:00:00"
committed_at: "2024-01-01 10:00:00"
- canonical_developer_id: 202
commit_author_name: "New Name"
commit_author_email: "original@test.com"
created_at: "2024-02-01 12:00:00"
committed_at: "2024-02-01 12:00:00"
# Duplicate record for first state
- canonical_developer_id: 202
commit_author_name: "Original Name"
commit_author_email: "original@test.com"
created_at: "2024-01-15 10:00:00"
committed_at: "2024-01-15 10:00:00"
oso.stg_opendevdata__canonical_developers:
rows:
- id: 202
Expand Down Expand Up @@ -70,7 +70,7 @@ test_int_opendevdata__developers_filter_nulls:
- canonical_developer_id: null
commit_author_name: "Ghost"
commit_author_email: "ghost@test.com"
created_at: "2024-01-01 10:00:00"
committed_at: "2024-01-01 10:00:00"
oso.stg_opendevdata__canonical_developers:
rows: []
outputs:
Expand Down