[DO NOT merge] - Set retention on client_level tables for glean_usage tables #6815

alekhyamoz · 2025-01-15T16:01:36Z

As part of the data retention project, the final set of work would be setting the partitions before 775 days (on client-level tables) expire.
This PR will be merged only after January 28th.

As part of this PR 2 changes will be implemented

Set the partition expiration to 775
Add a label table_type: client_level

Please note that first_seen tables are excluded from setting retention.

Reviewer please valid for the tables you own or use. Also, please don't hesitate to tag the appropriate owners.

scholtzan · 2025-01-15T16:57:07Z

sql_generators/glean_usage/templates/baseline_clients_daily_v1.metadata.yaml

@@ -19,6 +19,7 @@ bigquery:
    type: day
    field: submission_date
    require_partition_filter: true
+    expiration_days: 775


Just out of curiosity, what number should be used for expiration days? I have seen 780 as well:

bigquery-etl/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml

Line 23 in b81a7a0

expiration_days: 780

In the recent data retention sync meeting, we discussed and decided to use 775. The above PR was before a number was finalized. Open to suggestions or discussions if needed.

Should we change the 780 to 775 in a separate PR? To make it consistent?

I have changed 780 to 775 for a couple of tables. This file is deprecated and I will add it to my to-be-deleted list
ql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_v1/metadata.yaml#L23

badboy

r+ for the events_stream tables.

bani

LGTM

scholtzan · 2025-01-31T19:37:47Z

sql/moz-fx-data-shared-prod/telemetry_derived/clients_daily_event_v1/metadata.yaml

 owners:
-  - jklukas@mozilla.com
+  - ascholtz@mozilla.com


should this table have expiration set?

… a separate PR

alekhyamoz · 2025-02-04T20:23:48Z

@kwindau thank you for working on creating a separate PR for baseline tables. The tables will be needed for ToS aggregate work

dataops-ci-bot · 2025-02-04T20:41:55Z

Integration report for "Moving baseline_clients_daily_v1 and baseline_clients_last_seen_v1 to a separate PR"

`sql.diff`

Click to expand!

diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_main_summary.py /tmp/workspace/generated-sql/dags/bqetl_main_summary.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_main_summary.py	2025-02-04 20:25:08.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_main_summary.py	2025-02-04 20:41:05.000000000 +0000
@@ -399,12 +399,8 @@
         destination_table="clients_daily_event_v1",
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
-        owner="jklukas@mozilla.com",
-        email=[
-            "ascholtz@mozilla.com",
-            "jklukas@mozilla.com",
-            "telemetry-alerts@mozilla.com",
-        ],
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
         start_date=datetime.datetime(2021, 1, 19, 0, 0),
         date_partition_parameter="submission_date",
         depends_on_past=False,
@@ -631,12 +627,8 @@
         destination_table="clients_last_seen_event_v1",
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
-        owner="jklukas@mozilla.com",
-        email=[
-            "ascholtz@mozilla.com",
-            "jklukas@mozilla.com",
-            "telemetry-alerts@mozilla.com",
-        ],
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
         start_date=datetime.datetime(2021, 1, 19, 0, 0),
         date_partition_parameter="submission_date",
         depends_on_past=True,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/events_stream_v1/metadata.yaml	2025-02-04 20:21:10.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:10.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -26,7 +27,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/events_stream_v1/metadata.yaml	2025-02-04 20:20:47.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:26.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:20:47.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:26.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/events_stream_v1/metadata.yaml	2025-02-04 20:20:55.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:26.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -26,7 +27,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/ads_backend_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/ads_backend_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/ads_backend_derived/events_stream_v1/metadata.yaml	2025-02-04 20:21:10.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/ads_backend_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:18.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/events_stream_v1/metadata.yaml	2025-02-04 20:20:47.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:26.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:20:55.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:10.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/events_stream_v1/metadata.yaml	2025-02-04 20:20:55.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:10.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:20:55.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:10.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:20:55.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:36:10.000000000 +0000
@@ -11,6 +11,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -21,7 +22,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-02-04 20:18:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-02-04 20:30:10.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-02-04 20:18:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-02-04 20:30:02.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/events_stream_v1/metadata.yaml	2025-02-04 20:20:48.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:26.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-02-04 20:18:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-02-04 20:29:50.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-02-04 20:18:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-02-04 20:29:35.000000000 +0000
@@ -48,6 +48,10 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
+  description: null
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:20:48.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:18.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:20:48.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:18.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:20:48.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:36:18.000000000 +0000
@@ -11,6 +11,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -21,7 +22,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/events_stream_v1/metadata.yaml	2025-02-04 20:20:55.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:27.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:20:55.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:27.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml	2025-02-04 20:18:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml	2025-02-04 20:30:19.000000000 +0000
@@ -1,64 +1,66 @@
 fields:
-- description: Unique ID for the client installation.
-  mode: NULLABLE
-  name: client_id
+- name: client_id
   type: STRING
-- description: Sample ID - A number ranging from 0 - 99 based on client ID; used to pull a small sample of data related to a subset of clients over time
   mode: NULLABLE
-  name: sample_id
-  type: INT64
-- description: Submission Date
+  description: Unique ID for the client installation.
+- name: sample_id
+  type: INTEGER
   mode: NULLABLE
-  name: submission_date
+  description: Sample ID - A number ranging from 0 - 99 based on client ID; used to
+    pull a small sample of data related to a subset of clients over time
+- name: submission_date
   type: DATE
-- description: First Seen Date - The date this client was first seen
   mode: NULLABLE
-  name: first_seen_date
+  description: Submission Date
+- name: first_seen_date
   type: DATE
-- description: Days Since First Seen - The number of days since the client was first seen
   mode: NULLABLE
-  name: days_since_first_seen
-  type: INT64
-- description: Days Since Active
+  description: First Seen Date - The date this client was first seen
+- name: days_since_first_seen
+  type: INTEGER
   mode: NULLABLE
-  name: days_since_active
-  type: INT64
-- description: First Reported Country - The country this client ID was first reported from
+  description: Days Since First Seen - The number of days since the client was first
+    seen
+- name: days_since_active
+  type: INTEGER
   mode: NULLABLE
-  name: first_reported_country
+  description: Days Since Active
+- name: first_reported_country
   type: STRING
-- description: Attribution
   mode: NULLABLE
-  name: attribution
+  description: First Reported Country - The country this client ID was first reported
+    from
+- name: attribution
   type: RECORD
+  mode: NULLABLE
   fields:
-  - mode: NULLABLE
-    name: source
+  - name: source
     type: STRING
+    mode: NULLABLE
     description: Attribution Source
-  - mode: NULLABLE
-    name: medium
+  - name: medium
     type: STRING
+    mode: NULLABLE
     description: Attribution Medium
-  - mode: NULLABLE
-    name: campaign
+  - name: campaign
     type: STRING
+    mode: NULLABLE
     description: Attribution Campaign
-  - mode: NULLABLE
-    name: content
+  - name: content
     type: STRING
+    mode: NULLABLE
     description: Attribution Content
-  - mode: NULLABLE
-    name: experiment
+  - name: experiment
     type: STRING
+    mode: NULLABLE
     description: Attribution Experiment
-  - mode: NULLABLE
-    name: variation
+  - name: variation
     type: STRING
+    mode: NULLABLE
     description: Attribution Variation
-  - mode: NULLABLE
-    name: dltoken
+  - name: dltoken
     type: STRING
+    mode: NULLABLE
     description: Attribution Download Token
   - name: dlsource
     type: STRING
@@ -68,40 +70,43 @@
     type: STRING
     mode: NULLABLE
     description: Attribution UA
-- description: Active
-  mode: NULLABLE
-  name: active
-  type: INT64
-- description: Ad Clicks - The number of ad clicks from this client on the submission date
+  description: Attribution
+- name: active
+  type: INTEGER
   mode: NULLABLE
-  name: ad_clicks
-  type: INT64
-- description: Total Historic Ad Clicks - The number of ad clicks from this client on or before the submission date
+  description: Active
+- name: ad_clicks
+  type: INTEGER
   mode: NULLABLE
-  name: total_historic_ad_clicks
-  type: INT64
-- description: Days Seen Bytes
+  description: Ad Clicks - The number of ad clicks from this client on the submission
+    date
+- name: total_historic_ad_clicks
+  type: INTEGER
   mode: NULLABLE
-  name: days_seen_bytes
+  description: Total Historic Ad Clicks - The number of ad clicks from this client
+    on or before the submission date
+- name: days_seen_bytes
   type: BYTES
-- description: Pattern
   mode: NULLABLE
-  name: pattern
+  description: Days Seen Bytes
+- name: pattern
   type: INTEGER
-- description: Death Time
   mode: NULLABLE
-  name: death_time
+  description: Pattern
+- name: death_time
   type: INTEGER
-- description: Max Days
   mode: NULLABLE
-  name: max_days
+  description: Death Time
+- name: max_days
   type: INTEGER
-- description: Markov States
   mode: NULLABLE
-  name: markov_states
+  description: Max Days
+- name: markov_states
   type: RECORD
-  fields:
-  - description: Desktop States V1
     mode: NULLABLE
-    name: desktop_states_v1
+  fields:
+  - name: desktop_states_v1
     type: STRING
+    mode: NULLABLE
+    description: Desktop States V1
+  description: Markov States
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml	2025-02-04 20:18:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml	2025-02-04 20:30:07.000000000 +0000
@@ -1,48 +1,47 @@
 fields:
-- description: Submission Timestamp
-  mode: NULLABLE
-  name: submission_timestamp
+- name: submission_timestamp
   type: TIMESTAMP
-- description: Normalized Country Code, Examples - US, AR, BR, etc.
   mode: NULLABLE
-  name: normalized_country_code
+  description: Submission Timestamp
+- name: normalized_country_code
   type: STRING
-- description: Normalized Channel, Examples - release, nightly, aurora, esr, beta
   mode: NULLABLE
-  name: normalized_channel
+  description: Normalized Country Code, Examples - US, AR, BR, etc.
+- name: normalized_channel
   type: STRING
-- description: Document ID
   mode: NULLABLE
-  name: document_id
+  description: Normalized Channel, Examples - release, nightly, aurora, esr, beta
+- name: document_id
   type: STRING
-- description: Pocket Enabled
   mode: NULLABLE
-  name: pocket_enabled
+  description: Document ID
+- name: pocket_enabled
   type: BOOLEAN
-- description: Pocket Sponsored Stories Enabled
   mode: NULLABLE
-  name: pocket_sponsored_stories_enabled
+  description: Pocket Enabled
+- name: pocket_sponsored_stories_enabled
   type: BOOLEAN
-- description: Newtab Locale
   mode: NULLABLE
-  name: newtab_locale
+  description: Pocket Sponsored Stories Enabled
+- name: newtab_locale
   type: STRING
-- description: App Build
   mode: NULLABLE
-  name: app_build
+  description: Newtab Locale
+- name: app_build
   type: STRING
-- description: App Display Version
   mode: NULLABLE
-  name: app_display_version
+  description: App Build
+- name: app_display_version
   type: STRING
-- description: Client ID
   mode: NULLABLE
-  name: client_id
+  description: App Display Version
+- name: client_id
   type: STRING
+  mode: NULLABLE
+  description: Client ID
 - name: events
   type: RECORD
   mode: REPEATED
-  description: Events
   fields:
   - name: category
     type: STRING
@@ -51,7 +50,6 @@
   - name: extra
     type: RECORD
     mode: REPEATED
-    description: Extras
     fields:
     - name: key
       type: STRING
@@ -61,6 +59,7 @@
       type: STRING
       mode: NULLABLE
       description: Value
+    description: Extras
   - name: name
     type: STRING
     mode: NULLABLE
@@ -69,3 +68,4 @@
     type: INTEGER
     mode: NULLABLE
     description: Event Timestamp
+  description: Events
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/events_stream_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:11.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:11.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/events_stream_v1/metadata.yaml	2025-02-04 20:20:49.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:27.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:20:49.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:27.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:21:02.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:27.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/events_stream_v1/metadata.yaml	2025-02-04 20:21:02.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:27.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:21:02.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:27.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:21:02.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:36:27.000000000 +0000
@@ -11,6 +11,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -21,7 +22,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:20.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/events_stream_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:20.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:20.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:36:20.000000000 +0000
@@ -11,6 +11,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -21,7 +22,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/pageload_1pct_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/pageload_1pct_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/pageload_1pct_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/pageload_1pct_v1/metadata.yaml	2025-02-04 20:36:20.000000000 +0000
@@ -7,6 +7,7 @@
 - dpalmeiro@mozilla.com
 labels:
   incremental: true
+  table_type: client_level
   dag: bqetl_pageload_v1
   owner1: wichan
   owner2: acreskey
@@ -21,7 +22,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: 780.0
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/pageload_nightly_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/pageload_nightly_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/pageload_nightly_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/pageload_nightly_v1/metadata.yaml	2025-02-04 20:36:20.000000000 +0000
@@ -7,6 +7,7 @@
 - dpalmeiro@mozilla.com
 labels:
   incremental: true
+  table_type: client_level
   dag: bqetl_pageload_v1
   owner1: wichan
   owner2: acreskey
@@ -21,7 +22,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: 780.0
+    expiration_days: 775.0
   range_partitioning: null
   clustering: null
 workgroup_access:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:21:02.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:29.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:21:02.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:29.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:21:02.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:36:29.000000000 +0000
@@ -11,6 +11,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -21,7 +22,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:29.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:29.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:36:29.000000000 +0000
@@ -11,6 +11,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -21,7 +22,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:20:56.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:11.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:20:56.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:11.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:20:56.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:36:11.000000000 +0000
@@ -11,6 +11,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -21,7 +22,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:20:49.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:30.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:20:49.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:30.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:20:49.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:36:30.000000000 +0000
@@ -11,6 +11,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -21,7 +22,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:28.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:28.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:36:28.000000000 +0000
@@ -11,6 +11,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -21,7 +22,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_translations_derived/events_stream_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_translations_derived/events_stream_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_translations_derived/events_stream_v1/metadata.yaml	2025-02-04 20:21:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_translations_derived/events_stream_v1/metadata.yaml	2025-02-04 20:36:27.000000000 +0000
@@ -10,6 +10,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: jrediger
   owner2: wstuckey
@@ -24,7 +25,7 @@
     type: day
     field: submission_timestamp
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:21:03.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:05.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:21:03.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:05.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:21:03.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-02-04 20:36:05.000000000 +0000
@@ -11,6 +11,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -21,7 +22,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:20:56.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2025-02-04 20:36:05.000000000 +0000
@@ -13,6 +13,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -24,7 +25,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:20:56.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/metrics_clients_daily_v1/metadata.yaml	2025-02-04 20:36:05.000000000 +0000
@@ -6,6 +6,7 @@
 labels:
   incremental: true
   schedule: daily
+  table_type: client_level
   dag: bqetl_glean_usage
   owner1: ascholtz
 scheduling:
@@ -16,7 +17,7 @@
     type: day
     field: submission_date
     require_partition_filter: true
-    expiration_days: null
+    expiration_days: 775.0
   range_partitioning: null
   clustering:
     fields:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/metrics_clients_

⚠️ Only part of the diff is displayed.

Link to full diff

Set retention on client_level tables for glean_usage tables

05d9dc6

alekhyamoz requested review from badboy, scholtzan, akkomar, kik-kik, lucia-vargas-a and a team January 15, 2025 16:01