DO-2075 Added fenix and desktop baseline city seen tables #7974

wwyc · 2025-08-19T23:01:37Z

Description

Initialize the *baseline_city_seen tables by deriving each client’s first-seen and last-seen city, subdivision and country fields from the stable tables.
Note: This one-time initialization logic will no longer apply once city/subdivision/country fields are nulled in the stable tables.
Ongoing updates: After initialization, the tables will be updated daily via ETL using live tables (appending new clients and advancing last-seen values).

Related Tickets & Documents

DO-2075

Reviewer, please follow this checklist

sql/moz-fx-data-shared-prod/fenix_derived/clients_city_seen_v1/schema.yaml

sql/moz-fx-data-shared-prod/fenix_derived/clients_city_seen_v1/metadata.yaml

sql/moz-fx-data-shared-prod/fenix_derived/clients_city_seen_v1/schema.yaml

sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_city_seen_v1/query.sql

sql_generators/clients_city_seen_v1/templates/metadata.yaml

sql/moz-fx-data-shared-prod/fenix_derived/clients_city_seen_v1/metadata.yaml

sql_generators/clients_city_seen_v1/templates/metadata.yaml

sql_generators/clients_city_seen_v1/__init__.py

sql_generators/clients_city_seen_v1/templates/query.sql

sql/moz-fx-data-shared-prod/fenix_derived/clients_city_seen_v1/schema.yaml

wwyc · 2025-10-03T21:02:43Z

@BenWu @soGaussian could you do a final review/approval on this PR? I’d like to merge early next week.

@BenWu is there a time of day you recommend for the merge so initialization doesn’t affect artifact deployments? Thanks!

BenWu · 2025-10-03T21:42:53Z

I'll take a closer look on Monday.

Running it towards the end of the day should be fine so it doesn't block any fixes that need to go out. I didn't see your comment on how long it would take but it should be fine to do the initialization. I don't think running in parallel would make it much faster since the query is large enough to use up most of the slots. I just did a test and a 1% query uses up all 2k slots in the backfill reservation for basically the whole duration of the query.

One thing that we should do is make the initialization run in the backfill reservation instead of analysis-and-etl so there isn't slot contention with etl. I can make that change on Monday.

Also Kiran is planning on doing a large backfill of events_first_seen next week so make sure to coordinate with her so the backfills don't end up fighting for slots

wwyc · 2025-10-03T21:48:44Z

I'll take a closer look on Monday.

Running it towards the end of the day should be fine so it doesn't block any fixes that need to go out. I didn't see your comment on how long it would take but it should be fine to do the initialization. I don't think running in parallel would make it much faster since the query is large enough to use up most of the slots. I just did a test and a 1% query uses up all 2k slots in the backfill reservation for basically the whole duration of the query.

One thing that we should do is make the initialization run in the backfill reservation instead of analysis-and-etl so there isn't slot contention with etl. I can make that change on Monday.

Also Kiran is planning on doing a large backfill of events_first_seen next week so make sure to coordinate with her so the backfills don't end up fighting for slots

Thank you! I will check with kiran. The time it takes to run 2% sample: firefox_desktop: ~319s, fenix: ~111s.

sql_generators/baseline_clients_city_seen_v1/templates/metadata.yaml

sql_generators/baseline_clients_city_seen_v1/templates/query.sql

BenWu · 2025-10-06T22:00:08Z

sql_generators/baseline_clients_city_seen_v1/__init__.py

+        if name
+        in ConfigLoader.get(
+            "generate", "baseline_clients_city_seen", "apps", fallback=[]
+        )


Assuming the allowlist is just for this initial phase, when you add the rest of the apps, you'll also need to check if the app has baseline pings since glean.js and server glean doesn't use them. That doesn't need to be done now but might be needed later.

One way to do that would be (I didn't test this):

"glean-core" in requests.get( f"https://probeinfo.telemetry.mozilla.org/glean/{info[0]['v1_name']}/dependencies" ).json()

glean-usage downloads the schemas but that's slower if you only need to check baseline.

bqetl_project.yaml

…a.yaml Co-authored-by: Ben Wu <12437227+BenWu@users.noreply.github.com>

dataops-ci-bot · 2025-10-07T03:36:59Z

Integration report for "Fix dag tag"

`sql.diff`

Click to expand!

Only in /tmp/workspace/generated-sql/dags/: bqetl_baseline_clients_city_seen.py
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_baseline_clients_city_seen.py /tmp/workspace/generated-sql/dags/bqetl_baseline_clients_city_seen.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_baseline_clients_city_seen.py	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_baseline_clients_city_seen.py	2025-10-07 03:35:55.000000000 +0000
@@ -0,0 +1,77 @@
+# Generated via https://github.com/mozilla/bigquery-etl/blob/main/bigquery_etl/query_scheduling/generate_airflow_dags.py
+
+from airflow import DAG
+from airflow.sensors.external_task import ExternalTaskMarker
+from airflow.sensors.external_task import ExternalTaskSensor
+from airflow.utils.task_group import TaskGroup
+import datetime
+from operators.gcp_container_operator import GKEPodOperator
+from utils.constants import ALLOWED_STATES, FAILED_STATES
+from utils.gcp import bigquery_etl_query, bigquery_dq_check, bigquery_bigeye_check
+
+docs = """
+### bqetl_baseline_clients_city_seen
+
+Built from bigquery-etl repo, [`dags/bqetl_baseline_clients_city_seen.py`](https://github.com/mozilla/bigquery-etl/blob/generated-sql/dags/bqetl_baseline_clients_city_seen.py)
+
+#### Description
+
+Scheduled queries for client city seen tables
+#### Owner
+
+wichan@mozilla.com
+
+#### Tags
+
+* impact/tier_2
+* repo/bigquery-etl
+"""
+
+
+default_args = {
+    "owner": "wichan@mozilla.com",
+    "start_date": datetime.datetime(2025, 9, 20, 0, 0),
+    "end_date": None,
+    "email": ["telemetry-alerts@mozilla.com", "wichan@mozilla.com"],
+    "depends_on_past": False,
+    "retry_delay": datetime.timedelta(seconds=1800),
+    "email_on_failure": True,
+    "email_on_retry": True,
+    "retries": 2,
+    "max_active_tis_per_dag": None,
+}
+
+tags = ["impact/tier_2", "repo/bigquery-etl"]
+
+with DAG(
+    "bqetl_baseline_clients_city_seen",
+    default_args=default_args,
+    schedule_interval="0 4 * * *",
+    doc_md=docs,
+    tags=tags,
+    catchup=False,
+) as dag:
+
+    fenix_baseline_clients_city_seen_v1 = bigquery_etl_query(
+        task_id="fenix_baseline_clients_city_seen_v1",
+        destination_table="baseline_clients_city_seen_v1",
+        dataset_id="fenix_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="wichan@mozilla.com",
+        email=["telemetry-alerts@mozilla.com", "wichan@mozilla.com"],
+        date_partition_parameter=None,
+        depends_on_past=True,
+        parameters=["submission_date:DATE:{{ds}}"],
+    )
+
+    firefox_desktop_baseline_clients_city_seen_v1 = bigquery_etl_query(
+        task_id="firefox_desktop_baseline_clients_city_seen_v1",
+        destination_table="baseline_clients_city_seen_v1",
+        dataset_id="firefox_desktop_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="wichan@mozilla.com",
+        email=["telemetry-alerts@mozilla.com", "wichan@mozilla.com"],
+        date_partition_parameter=None,
+        depends_on_past=True,
+        parameters=["submission_date:DATE:{{ds}}"],
+    )
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix: baseline_clients_city_seen
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived: baseline_clients_city_seen_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop: baseline_clients_city_seen
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived: baseline_clients_city_seen_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:18:33.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-06"
+  DATE(submission_timestamp) >= "2025-10-07"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-07 03:28:58.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-07 03:28:58.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-07 03:28:58.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:18:33.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-06"
+  DATE(submission_timestamp) >= "2025-10-07"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-07 03:28:58.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:18:34.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-06"
+  DATE(submission_timestamp) >= "2025-10-07"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:18:34.000000000 +0000
@@ -119,7 +119,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-06"
+  DATE(submission_timestamp) >= "2025-10-07"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-10-07 03:20:01.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-10-07 03:19:55.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:18:34.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-06"
+  DATE(submission_timestamp) >= "2025-10-07"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -51,7 +51,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-10-07 03:18:34.000000000 +0000
@@ -53,7 +53,7 @@
 FROM
   combined
 WHERE
-  DATE(submission_timestamp) >= "2025-10-06"
+  DATE(submission_timestamp) >= "2025-10-07"
 GROUP BY
   submission_date,
   window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -0,0 +1,14 @@
+friendly_name: Baseline Clients City Seen
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1
+require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/schema.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/schema.yaml	2025-10-07 03:24:59.000000000 +0000
@@ -0,0 +1,58 @@
+fields:
+- name: app_id
+  type: STRING
+  mode: NULLABLE
+  description: ID of the browser application.
+- name: client_id
+  type: STRING
+  mode: NULLABLE
+  description: A UUID uniquely identifying the client.
+- name: sample_id
+  type: INTEGER
+  mode: NULLABLE
+  description: A number, 0-99, that samples by client_id and allows filtering data
+    for analysis.
+- name: first_seen_city_date
+  type: DATE
+  mode: NULLABLE
+  description: Date when the first seen city was captured.
+- name: first_seen_city
+  type: STRING
+  mode: NULLABLE
+  description: City captured on first_seen_city_date.
+- name: first_seen_subdivision1
+  type: STRING
+  mode: NULLABLE
+  description: Major country subdivision, typically a state, province, or county captured
+    on first_seen_city_date.
+- name: first_seen_subdivision2
+  type: STRING
+  mode: NULLABLE
+  description: Second major country subdivision; not applicable for most countries
+    captured on first_seen_city_date.
+- name: first_seen_country
+  type: STRING
+  mode: NULLABLE
+  description: An ISO 3166-1 alpha-2 country code captured on first_seen_city_date.
+- name: last_seen_city_date
+  type: DATE
+  mode: NULLABLE
+  description: Date when the last seen city was captured.
+- name: last_seen_city
+  type: STRING
+  mode: NULLABLE
+  description: City captured on last_seen_city_date.
+- name: last_seen_subdivision1
+  type: STRING
+  mode: NULLABLE
+  description: Major country subdivision, typically a state, province, or county captured
+    on last_seen_city_date.
+- name: last_seen_subdivision2
+  type: STRING
+  mode: NULLABLE
+  description: Second major country subdivision; not applicable for most countries
+    captured on last_seen_city_date.
+- name: last_seen_country
+  type: STRING
+  mode: NULLABLE
+  description: An ISO 3166-1 alpha-2 country code captured on last_seen_city_date.
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/view.sql	2025-10-07 03:19:22.000000000 +0000
@@ -0,0 +1,8 @@
+--- User-facing view. Generated via sql_generators.baseline_clients_city_seen.
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.fenix.baseline_clients_city_seen`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml	2025-10-07 03:28:56.000000000 +0000
@@ -1,6 +1,10 @@
-friendly_name: Broken Site Report
+friendly_name: App-specific view for Glean ping "broken-site-report"
 description: |-
-  Please provide a description for the query
+  This a view that UNIONs the stable ping tables
+  across all channels of the Glean application "Firefox for Android"
+  (org_mozilla_firefox.broken_site_report, org_mozilla_firefox_beta.broken_site_report, org_mozilla_fenix.broken_site_report, org_mozilla_fenix_nightly.broken_site_report, org_mozilla_fennec_aurora.broken_site_report).
+
+  It is used by Looker.
 owners: []
 labels: {}
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml	2025-10-07 03:28:57.000000000 +0000
@@ -1,6 +1,10 @@
-friendly_name: Crash
+friendly_name: App-specific view for Glean ping "crash"
 description: |-
-  Please provide a description for the query
+  This a view that UNIONs the stable ping tables
+  across all channels of the Glean application "Firefox for Android"
+  (org_mozilla_firefox.crash, org_mozilla_firefox_beta.crash, org_mozilla_fenix.crash, org_mozilla_fenix_nightly.crash, org_mozilla_fennec_aurora.crash).
+
+  It is used by Looker.
 owners: []
 labels: {}
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-10-07 03:20:09.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-10-07 03:20:17.000000000 +0000
@@ -48,6 +48,9 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attribution_clients_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attribution_clients_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attribution_clients_v1/metadata.yaml	2025-10-07 03:16:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attribution_clients_v1/metadata.yaml	2025-10-07 03:28:56.000000000 +0000
@@ -41,7 +41,7 @@
   enabled: true
   collection: Operational Checks
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/metadata.yaml	2025-10-07 03:28:56.000000000 +0000
@@ -0,0 +1,50 @@
+friendly_name: Baseline Clients City Seen
+description: |-
+  This table stores the first-seen and last-seen geo attributes for each client_id.
+  The table was initialized from stable tables (with finite retention) in Oct. 2025,
+  so the initial dates reflect the earliest/latest observations within that window.
+  After initialization, the city fields in the stable tables are nullified and
+  the one-time initialization code is no longer relevant.  The table is updated daily using live tables,
+  updating last-seen fields for existing clients and adding new clients with first-seen geo attributes
+  that do not yet exist in the table.
+  This table should not be backfilled beyond 30 days which is the maximum retention of live tables.
+
+  JIRA ticket: https://mozilla-hub.atlassian.net/browse/DENG-9468
+owners:
+- wichan@mozilla.com
+labels:
+  incremental: true
+  schedule: daily
+  table_type: client_level
+  dag: bqetl_baseline_clients_city_seen
+  owner1: wichan
+scheduling:
+  dag_name: bqetl_baseline_clients_city_seen
+  task_name: fenix_baseline_clients_city_seen_v1
+  depends_on_past: true
+  date_partition_parameter: null
+  parameters:
+  - submission_date:DATE:{{ds}}
+bigquery:
+  time_partitioning:
+    type: day
+    field: first_seen_city_date
+    require_partition_filter: false
+    expiration_days: null
+  range_partitioning: null
+  clustering:
+    fields:
+    - sample_id
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  query.sql:
+  - moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1
+  - moz-fx-data-shared-prod.org_mozilla_fenix_live.baseline_v1
+  - moz-fx-data-shared-prod.org_mozilla_fenix_nightly_live.baseline_v1
+  - moz-fx-data-shared-prod.org_mozilla_fennec_aurora_live.baseline_v1
+  - moz-fx-data-shared-prod.org_mozilla_firefox_beta_live.baseline_v1
+  - moz-fx-data-shared-prod.org_mozilla_firefox_live.baseline_v1
+require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/query.sql	2025-10-07 03:19:22.000000000 +0000
@@ -0,0 +1,1153 @@
+ -- Query generated via sql_generators.baseline_clients_city_seen.
+ -- this mimics the logic used in baseline_clients_daily_v1.
+WITH base_org_mozilla_firefox AS (
+  -- Live table dedup logic mimics copy_deduplicate
+  SELECT
+    submission_timestamp,
+    DATE(submission_timestamp) AS submission_date,
+    LOWER(client_info.client_id) AS client_id,
+    sample_id,
+    mozfun.glean.parse_datetime(ping_info.end_time) AS parsed_end_time,
+    `moz-fx-data-shared-prod.udf.glean_timespan_seconds`(
+      metrics.timespan.glean_baseline_duration
+    ) AS duration,
+    metadata.geo.city AS city,
+    metadata.geo.subdivision1 AS subdivision1,
+    metadata.geo.subdivision2 AS subdivision2,
+    metadata.geo.country AS country
+  FROM
+    {% if is_init() %}
+      `moz-fx-data-shared-prod.org_mozilla_firefox_stable.baseline_v1`
+    {% else %}
+      `moz-fx-data-shared-prod.org_mozilla_firefox_live.baseline_v1`
+    {% endif %}
+  WHERE
+    client_info.client_id IS NOT NULL
+    {% if is_init() %}
+      AND sample_id = @sample_id
+      AND DATE(submission_timestamp) <= CURRENT_DATE()
+    {% else %}
+      AND DATE(submission_timestamp) = @submission_date
+      AND 'automation' NOT IN (
+        SELECT
+          TRIM(t)
+        FROM
+          UNNEST(SPLIT(metadata.header.x_source_tags, ',')) t
+      )
+      QUALIFY
+        ROW_NUMBER() OVER (PARTITION BY document_id ORDER BY submission_timestamp) = 1
+    {% endif %}
+),
+overactive_org_mozilla_firefox AS (
+  -- Find client_ids with over 150 000 pings in a day,
+  -- which could cause errors in the next step due to aggregation overflows.
+  SELECT
+    submission_date,
+    client_id
+  FROM
+    base_org_mozilla_firefox
+  WHERE
+    {% if is_init() %}
+      submission_date >= '2018-01-01'
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id
+  HAVING
+    COUNT(*) > 150000
+),
+clients_daily_org_mozilla_firefox AS (
+  SELECT
+    "org_mozilla_firefox" AS app_id,
+    submission_date,
+    client_id,
+    sample_id,
+    `moz-fx-data-shared-prod.udf.mode_last`(
+      ARRAY_AGG(STRUCT(city, subdivision1, subdivision2, country) ORDER BY submission_timestamp)
+    ) AS geo
+  FROM
+    base_org_mozilla_firefox
+  LEFT JOIN
+    overactive_org_mozilla_firefox
+    USING (submission_date, client_id)
+  WHERE
+    overactive_org_mozilla_firefox.client_id IS NULL
+    -- `mode_last` can result in struct with all null values if it’s most frequent (or latest among ties).
+    -- This exclude structs with all null values so there will always be one non-NULL field.
+    AND COALESCE(city, subdivision1, subdivision2, country) IS NOT NULL
+    {% if is_init() %}
+      AND submission_date >= '2018-01-01'
+    {% else %}
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id,
+    sample_id
+),
+{% if is_init() %}
+  clients_city_first_seen_org_mozilla_firefox AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date) = 1
+  ),
+  clients_city_last_seen_org_mozilla_firefox AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date DESC) = 1
+  ),
+{% else %}
+  _previous_org_mozilla_firefox AS (
+    SELECT
+      *
+    FROM
+      `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
+    WHERE
+      app_id = "org_mozilla_firefox"
+  ),
+  _current_org_mozilla_firefox AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox
+    WHERE
+      geo.city IS NOT NULL
+  ),
+{% endif %}
+base_org_mozilla_firefox_beta AS (
+  -- Live table dedup logic mimics copy_deduplicate
+  SELECT
+    submission_timestamp,
+    DATE(submission_timestamp) AS submission_date,
+    LOWER(client_info.client_id) AS client_id,
+    sample_id,
+    mozfun.glean.parse_datetime(ping_info.end_time) AS parsed_end_time,
+    `moz-fx-data-shared-prod.udf.glean_timespan_seconds`(
+      metrics.timespan.glean_baseline_duration
+    ) AS duration,
+    metadata.geo.city AS city,
+    metadata.geo.subdivision1 AS subdivision1,
+    metadata.geo.subdivision2 AS subdivision2,
+    metadata.geo.country AS country
+  FROM
+    {% if is_init() %}
+      `moz-fx-data-shared-prod.org_mozilla_firefox_beta_stable.baseline_v1`
+    {% else %}
+      `moz-fx-data-shared-prod.org_mozilla_firefox_beta_live.baseline_v1`
+    {% endif %}
+  WHERE
+    client_info.client_id IS NOT NULL
+    {% if is_init() %}
+      AND sample_id = @sample_id
+      AND DATE(submission_timestamp) <= CURRENT_DATE()
+    {% else %}
+      AND DATE(submission_timestamp) = @submission_date
+      AND 'automation' NOT IN (
+        SELECT
+          TRIM(t)
+        FROM
+          UNNEST(SPLIT(metadata.header.x_source_tags, ',')) t
+      )
+      QUALIFY
+        ROW_NUMBER() OVER (PARTITION BY document_id ORDER BY submission_timestamp) = 1
+    {% endif %}
+),
+overactive_org_mozilla_firefox_beta AS (
+  -- Find client_ids with over 150 000 pings in a day,
+  -- which could cause errors in the next step due to aggregation overflows.
+  SELECT
+    submission_date,
+    client_id
+  FROM
+    base_org_mozilla_firefox_beta
+  WHERE
+    {% if is_init() %}
+      submission_date >= '2018-01-01'
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id
+  HAVING
+    COUNT(*) > 150000
+),
+clients_daily_org_mozilla_firefox_beta AS (
+  SELECT
+    "org_mozilla_firefox_beta" AS app_id,
+    submission_date,
+    client_id,
+    sample_id,
+    `moz-fx-data-shared-prod.udf.mode_last`(
+      ARRAY_AGG(STRUCT(city, subdivision1, subdivision2, country) ORDER BY submission_timestamp)
+    ) AS geo
+  FROM
+    base_org_mozilla_firefox_beta
+  LEFT JOIN
+    overactive_org_mozilla_firefox_beta
+    USING (submission_date, client_id)
+  WHERE
+    overactive_org_mozilla_firefox_beta.client_id IS NULL
+    -- `mode_last` can result in struct with all null values if it’s most frequent (or latest among ties).
+    -- This exclude structs with all null values so there will always be one non-NULL field.
+    AND COALESCE(city, subdivision1, subdivision2, country) IS NOT NULL
+    {% if is_init() %}
+      AND submission_date >= '2018-01-01'
+    {% else %}
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id,
+    sample_id
+),
+{% if is_init() %}
+  clients_city_first_seen_org_mozilla_firefox_beta AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox_beta
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date) = 1
+  ),
+  clients_city_last_seen_org_mozilla_firefox_beta AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox_beta
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date DESC) = 1
+  ),
+{% else %}
+  _previous_org_mozilla_firefox_beta AS (
+    SELECT
+      *
+    FROM
+      `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
+    WHERE
+      app_id = "org_mozilla_firefox_beta"
+  ),
+  _current_org_mozilla_firefox_beta AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox_beta
+    WHERE
+      geo.city IS NOT NULL
+  ),
+{% endif %}
+base_org_mozilla_fenix AS (
+  -- Live table dedup logic mimics copy_deduplicate
+  SELECT
+    submission_timestamp,
+    DATE(submission_timestamp) AS submission_date,
+    LOWER(client_info.client_id) AS client_id,
+    sample_id,
+    mozfun.glean.parse_datetime(ping_info.end_time) AS parsed_end_time,
+    `moz-fx-data-shared-prod.udf.glean_timespan_seconds`(
+      metrics.timespan.glean_baseline_duration
+    ) AS duration,
+    metadata.geo.city AS city,
+    metadata.geo.subdivision1 AS subdivision1,
+    metadata.geo.subdivision2 AS subdivision2,
+    metadata.geo.country AS country
+  FROM
+    {% if is_init() %}
+      `moz-fx-data-shared-prod.org_mozilla_fenix_stable.baseline_v1`
+    {% else %}
+      `moz-fx-data-shared-prod.org_mozilla_fenix_live.baseline_v1`
+    {% endif %}
+  WHERE
+    client_info.client_id IS NOT NULL
+    {% if is_init() %}
+      AND sample_id = @sample_id
+      AND DATE(submission_timestamp) <= CURRENT_DATE()
+    {% else %}
+      AND DATE(submission_timestamp) = @submission_date
+      AND 'automation' NOT IN (
+        SELECT
+          TRIM(t)
+        FROM
+          UNNEST(SPLIT(metadata.header.x_source_tags, ',')) t
+      )
+      QUALIFY
+        ROW_NUMBER() OVER (PARTITION BY document_id ORDER BY submission_timestamp) = 1
+    {% endif %}
+),
+overactive_org_mozilla_fenix AS (
+  -- Find client_ids with over 150 000 pings in a day,
+  -- which could cause errors in the next step due to aggregation overflows.
+  SELECT
+    submission_date,
+    client_id
+  FROM
+    base_org_mozilla_fenix
+  WHERE
+    {% if is_init() %}
+      submission_date >= '2018-01-01'
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id
+  HAVING
+    COUNT(*) > 150000
+),
+clients_daily_org_mozilla_fenix AS (
+  SELECT
+    "org_mozilla_fenix" AS app_id,
+    submission_date,
+    client_id,
+    sample_id,
+    `moz-fx-data-shared-prod.udf.mode_last`(
+      ARRAY_AGG(STRUCT(city, subdivision1, subdivision2, country) ORDER BY submission_timestamp)
+    ) AS geo
+  FROM
+    base_org_mozilla_fenix
+  LEFT JOIN
+    overactive_org_mozilla_fenix
+    USING (submission_date, client_id)
+  WHERE
+    overactive_org_mozilla_fenix.client_id IS NULL
+    -- `mode_last` can result in struct with all null values if it’s most frequent (or latest among ties).
+    -- This exclude structs with all null values so there will always be one non-NULL field.
+    AND COALESCE(city, subdivision1, subdivision2, country) IS NOT NULL
+    {% if is_init() %}
+      AND submission_date >= '2018-01-01'
+    {% else %}
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id,
+    sample_id
+),
+{% if is_init() %}
+  clients_city_first_seen_org_mozilla_fenix AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date) = 1
+  ),
+  clients_city_last_seen_org_mozilla_fenix AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date DESC) = 1
+  ),
+{% else %}
+  _previous_org_mozilla_fenix AS (
+    SELECT
+      *
+    FROM
+      `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
+    WHERE
+      app_id = "org_mozilla_fenix"
+  ),
+  _current_org_mozilla_fenix AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix
+    WHERE
+      geo.city IS NOT NULL
+  ),
+{% endif %}
+base_org_mozilla_fenix_nightly AS (
+  -- Live table dedup logic mimics copy_deduplicate
+  SELECT
+    submission_timestamp,
+    DATE(submission_timestamp) AS submission_date,
+    LOWER(client_info.client_id) AS client_id,
+    sample_id,
+    mozfun.glean.parse_datetime(ping_info.end_time) AS parsed_end_time,
+    `moz-fx-data-shared-prod.udf.glean_timespan_seconds`(
+      metrics.timespan.glean_baseline_duration
+    ) AS duration,
+    metadata.geo.city AS city,
+    metadata.geo.subdivision1 AS subdivision1,
+    metadata.geo.subdivision2 AS subdivision2,
+    metadata.geo.country AS country
+  FROM
+    {% if is_init() %}
+      `moz-fx-data-shared-prod.org_mozilla_fenix_nightly_stable.baseline_v1`
+    {% else %}
+      `moz-fx-data-shared-prod.org_mozilla_fenix_nightly_live.baseline_v1`
+    {% endif %}
+  WHERE
+    client_info.client_id IS NOT NULL
+    {% if is_init() %}
+      AND sample_id = @sample_id
+      AND DATE(submission_timestamp) <= CURRENT_DATE()
+    {% else %}
+      AND DATE(submission_timestamp) = @submission_date
+      AND 'automation' NOT IN (
+        SELECT
+          TRIM(t)
+        FROM
+          UNNEST(SPLIT(metadata.header.x_source_tags, ',')) t
+      )
+      QUALIFY
+        ROW_NUMBER() OVER (PARTITION BY document_id ORDER BY submission_timestamp) = 1
+    {% endif %}
+),
+overactive_org_mozilla_fenix_nightly AS (
+  -- Find client_ids with over 150 000 pings in a day,
+  -- which could cause errors in the next step due to aggregation overflows.
+  SELECT
+    submission_date,
+    client_id
+  FROM
+    base_org_mozilla_fenix_nightly
+  WHERE
+    {% if is_init() %}
+      submission_date >= '2018-01-01'
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id
+  HAVING
+    COUNT(*) > 150000
+),
+clients_daily_org_mozilla_fenix_nightly AS (
+  SELECT
+    "org_mozilla_fenix_nightly" AS app_id,
+    submission_date,
+    client_id,
+    sample_id,
+    `moz-fx-data-shared-prod.udf.mode_last`(
+      ARRAY_AGG(STRUCT(city, subdivision1, subdivision2, country) ORDER BY submission_timestamp)
+    ) AS geo
+  FROM
+    base_org_mozilla_fenix_nightly
+  LEFT JOIN
+    overactive_org_mozilla_fenix_nightly
+    USING (submission_date, client_id)
+  WHERE
+    overactive_org_mozilla_fenix_nightly.client_id IS NULL
+    -- `mode_last` can result in struct with all null values if it’s most frequent (or latest among ties).
+    -- This exclude structs with all null values so there will always be one non-NULL field.
+    AND COALESCE(city, subdivision1, subdivision2, country) IS NOT NULL
+    {% if is_init() %}
+      AND submission_date >= '2018-01-01'
+    {% else %}
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id,
+    sample_id
+),
+{% if is_init() %}
+  clients_city_first_seen_org_mozilla_fenix_nightly AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix_nightly
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date) = 1
+  ),
+  clients_city_last_seen_org_mozilla_fenix_nightly AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix_nightly
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date DESC) = 1
+  ),
+{% else %}
+  _previous_org_mozilla_fenix_nightly AS (
+    SELECT
+      *
+    FROM
+      `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
+    WHERE
+      app_id = "org_mozilla_fenix_nightly"
+  ),
+  _current_org_mozilla_fenix_nightly AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix_nightly
+    WHERE
+      geo.city IS NOT NULL
+  ),
+{% endif %}
+base_org_mozilla_fennec_aurora AS (
+  -- Live table dedup logic mimics copy_deduplicate
+  SELECT
+    submission_timestamp,
+    DATE(submission_timestamp) AS submission_date,
+    LOWER(client_info.client_id) AS client_id,
+    sample_id,
+    mozfun.glean.parse_datetime(ping_info.end_time) AS parsed_end_time,
+    `moz-fx-data-shared-prod.udf.glean_timespan_seconds`(
+      metrics.timespan.glean_baseline_duration
+    ) AS duration,
+    metadata.geo.city AS city,
+    metadata.geo.subdivision1 AS subdivision1,
+    metadata.geo.subdivision2 AS subdivision2,
+    metadata.geo.country AS country
+  FROM
+    {% if is_init() %}
+      `moz-fx-data-shared-prod.org_mozilla_fennec_aurora_stable.baseline_v1`
+    {% else %}
+      `moz-fx-data-shared-prod.org_mozilla_fennec_aurora_live.baseline_v1`
+    {% endif %}
+  WHERE
+    client_info.client_id IS NOT NULL
+    {% if is_init() %}
+      AND sample_id = @sample_id
+      AND DATE(submission_timestamp) <= CURRENT_DATE()
+    {% else %}
+      AND DATE(submission_timestamp) = @submission_date
+      AND 'automation' NOT IN (
+        SELECT
+          TRIM(t)
+        FROM
+          UNNEST(SPLIT(metadata.header.x_source_tags, ',')) t
+      )
+      QUALIFY
+        ROW_NUMBER() OVER (PARTITION BY document_id ORDER BY submission_timestamp) = 1
+    {% endif %}
+),
+overactive_org_mozilla_fennec_aurora AS (
+  -- Find client_ids with over 150 000 pings in a day,
+  -- which could cause errors in the next step due to aggregation overflows.
+  SELECT
+    submission_date,
+    client_id
+  FROM
+    base_org_mozilla_fennec_aurora
+  WHERE
+    {% if is_init() %}
+      submission_date >= '2018-01-01'
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id
+  HAVING
+    COUNT(*) > 150000
+),
+clients_daily_org_mozilla_fennec_aurora AS (
+  SELECT
+    "org_mozilla_fennec_aurora" AS app_id,
+    submission_date,
+    client_id,
+    sample_id,
+    `moz-fx-data-shared-prod.udf.mode_last`(
+      ARRAY_AGG(STRUCT(city, subdivision1, subdivision2, country) ORDER BY submission_timestamp)
+    ) AS geo
+  FROM
+    base_org_mozilla_fennec_aurora
+  LEFT JOIN
+    overactive_org_mozilla_fennec_aurora
+    USING (submission_date, client_id)
+  WHERE
+    overactive_org_mozilla_fennec_aurora.client_id IS NULL
+    -- `mode_last` can result in struct with all null values if it’s most frequent (or latest among ties).
+    -- This exclude structs with all null values so there will always be one non-NULL field.
+    AND COALESCE(city, subdivision1, subdivision2, country) IS NOT NULL
+    {% if is_init() %}
+      AND submission_date >= '2018-01-01'
+    {% else %}
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id,
+    sample_id
+),
+{% if is_init() %}
+  clients_city_first_seen_org_mozilla_fennec_aurora AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country
+    FROM
+      clients_daily_org_mozilla_fennec_aurora
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date) = 1
+  ),
+  clients_city_last_seen_org_mozilla_fennec_aurora AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fennec_aurora
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date DESC) = 1
+  )
+{% else %}
+  _previous_org_mozilla_fennec_aurora AS (
+    SELECT
+      *
+    FROM
+      `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
+    WHERE
+      app_id = "org_mozilla_fennec_aurora"
+  ),
+  _current_org_mozilla_fennec_aurora AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fennec_aurora
+    WHERE
+      geo.city IS NOT NULL
+  )
+{% endif %}
+{% if is_init() %}
+  SELECT
+    app_id,
+    client_id,
+    sample_id,
+    first_seen_city_date,
+    first_seen_city,
+    first_seen_subdivision1,
+    first_seen_subdivision2,
+    first_seen_country,
+    last_seen_city_date,
+    last_seen_city,
+    last_seen_subdivision1,
+    last_seen_subdivision2,
+    last_seen_country
+  FROM
+    clients_city_first_seen_org_mozilla_firefox cfs
+  FULL OUTER JOIN
+    clients_city_last_seen_org_mozilla_firefox cls
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+    app_id,
+    client_id,
+    sample_id,
+    first_seen_city_date,
+    first_seen_city,
+    first_seen_subdivision1,
+    first_seen_subdivision2,
+    first_seen_country,
+    last_seen_city_date,
+    last_seen_city,
+    last_seen_subdivision1,
+    last_seen_subdivision2,
+    last_seen_country
+  FROM
+    clients_city_first_seen_org_mozilla_firefox_beta cfs
+  FULL OUTER JOIN
+    clients_city_last_seen_org_mozilla_firefox_beta cls
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+    app_id,
+    client_id,
+    sample_id,
+    first_seen_city_date,
+    first_seen_city,
+    first_seen_subdivision1,
+    first_seen_subdivision2,
+    first_seen_country,
+    last_seen_city_date,
+    last_seen_city,
+    last_seen_subdivision1,
+    last_seen_subdivision2,
+    last_seen_country
+  FROM
+    clients_city_first_seen_org_mozilla_fenix cfs
+  FULL OUTER JOIN
+    clients_city_last_seen_org_mozilla_fenix cls
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+    app_id,
+    client_id,
+    sample_id,
+    first_seen_city_date,
+    first_seen_city,
+    first_seen_subdivision1,
+    first_seen_subdivision2,
+    first_seen_country,
+    last_seen_city_date,
+    last_seen_city,
+    last_seen_subdivision1,
+    last_seen_subdivision2,
+    last_seen_country
+  FROM
+    clients_city_first_seen_org_mozilla_fenix_nightly cfs
+  FULL OUTER JOIN
+    clients_city_last_seen_org_mozilla_fenix_nightly cls
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+    app_id,
+    client_id,
+    sample_id,
+    first_seen_city_date,
+    first_seen_city,
+    first_seen_subdivision1,
+    first_seen_subdivision2,
+    first_seen_country,
+    last_seen_city_date,
+    last_seen_city,
+    last_seen_subdivision1,
+    last_seen_subdivision2,
+    last_seen_country
+  FROM
+    clients_city_first_seen_org_mozilla_fennec_aurora cfs
+  FULL OUTER JOIN
+    clients_city_last_seen_org_mozilla_fennec_aurora cls
+    USING (client_id, sample_id, app_id)
+{% else %}
+  SELECT
+-- _p.* fields are NULL for clients that are not yet captured in the baseline_city_seen derived table.
+    IF(_p.app_id IS NULL, _c.app_id, _p.app_id) AS app_id,
+    IF(_p.client_id IS NULL, _c.client_id, _p.client_id) AS client_id,
+    IF(_p.sample_id IS NULL, _c.sample_id, _p.sample_id) AS sample_id,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_city_date,
+      _p.first_seen_city_date
+    ) AS first_seen_city_date,
+    IF(_p.client_id IS NULL, _c.first_seen_city, _p.first_seen_city) AS first_seen_city,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_subdivision1,
+      _p.first_seen_subdivision1
+    ) AS first_seen_subdivision1,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_subdivision2,
+      _p.first_seen_subdivision2
+    ) AS first_seen_subdivision2,
+    IF(_p.client_id IS NULL, _c.first_seen_country, _p.first_seen_country) AS first_seen_country,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_city_date,
+      _p.last_seen_city_date
+    ) AS last_seen_city_date,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_city,
+      _p.last_seen_city
+    ) AS last_seen_city,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_subdivision1,
+      _p.last_seen_subdivision1
+    ) AS last_seen_subdivision1,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_subdivision2,
+      _p.last_seen_subdivision2
+    ) AS last_seen_subdivision2,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_country,
+      _p.last_seen_country
+    ) AS last_seen_country
+  FROM
+    _current_org_mozilla_firefox AS _c
+  FULL JOIN
+    _previous_org_mozilla_firefox AS _p
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+-- _p.* fields are NULL for clients that are not yet captured in the baseline_city_seen derived table.
+    IF(_p.app_id IS NULL, _c.app_id, _p.app_id) AS app_id,
+    IF(_p.client_id IS NULL, _c.client_id, _p.client_id) AS client_id,
+    IF(_p.sample_id IS NULL, _c.sample_id, _p.sample_id) AS sample_id,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_city_date,
+      _p.first_seen_city_date
+    ) AS first_seen_city_date,
+    IF(_p.client_id IS NULL, _c.first_seen_city, _p.first_seen_city) AS first_seen_city,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_subdivision1,
+      _p.first_seen_subdivision1
+    ) AS first_seen_subdivision1,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_subdivision2,
+      _p.first_seen_subdivision2
+    ) AS first_seen_subdivision2,
+    IF(_p.client_id IS NULL, _c.first_seen_country, _p.first_seen_country) AS first_seen_country,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_city_date,
+      _p.last_seen_city_date
+    ) AS last_seen_city_date,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_city,
+      _p.last_seen_city
+    ) AS last_seen_city,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_subdivision1,
+      _p.last_seen_subdivision1
+    ) AS last_seen_subdivision1,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_subdivisio

⚠️ Only part of the diff is displayed.

Link to full diff

soGaussian

The query template looks good to me

wwyc · 2025-10-07T19:11:10Z

@BenWu if i can get an approval from your end perhaps I can merge today? Or would you prefer to merge this one? Thanks for the thorough review!

BenWu

You'll need to delete the existing baseline_clients_city_seen_v1 tables to make the initialization run. You can merge this towards the end of the day and make a note in the data platform slack channel that artifact deployment will be blocked for a few hours

wwyc requested a review from a team as a code owner August 19, 2025 23:01

wwyc marked this pull request as draft August 19, 2025 23:02

wwyc commented Aug 19, 2025

View reviewed changes

sql/moz-fx-data-shared-prod/fenix_derived/clients_city_seen_v1/schema.yaml Outdated Show resolved Hide resolved

wwyc requested a review from BenWu August 19, 2025 23:07

This comment has been minimized.

Sign in to view

lucia-vargas-a reviewed Aug 20, 2025

View reviewed changes

sql/moz-fx-data-shared-prod/fenix_derived/clients_city_seen_v1/metadata.yaml Outdated Show resolved Hide resolved

lucia-vargas-a reviewed Aug 20, 2025

View reviewed changes

sql/moz-fx-data-shared-prod/fenix_derived/clients_city_seen_v1/schema.yaml Outdated Show resolved Hide resolved

lucia-vargas-a reviewed Aug 20, 2025

View reviewed changes

sql/moz-fx-data-shared-prod/fenix_derived/clients_city_seen_v1/schema.yaml Outdated Show resolved Hide resolved

lucia-vargas-a reviewed Aug 20, 2025

View reviewed changes

sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_city_seen_v1/query.sql Outdated Show resolved Hide resolved

lucia-vargas-a reviewed Aug 20, 2025

View reviewed changes

sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_city_seen_v1/query.sql Outdated Show resolved Hide resolved

lucia-vargas-a reviewed Aug 20, 2025

View reviewed changes

sql_generators/clients_city_seen_v1/templates/metadata.yaml Outdated Show resolved Hide resolved

lucia-vargas-a reviewed Aug 20, 2025

View reviewed changes

sql/moz-fx-data-shared-prod/fenix_derived/clients_city_seen_v1/metadata.yaml Outdated Show resolved Hide resolved

lucia-vargas-a reviewed Aug 20, 2025

View reviewed changes

sql_generators/clients_city_seen_v1/templates/metadata.yaml Outdated Show resolved Hide resolved

BenWu reviewed Aug 20, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

wwyc changed the title ~~DO-2075 Added fenix and desktop city seen initial table - DO NOT MERGE~~ DO-2075 Added fenix and desktop city seen initial table - DO NOT MERGE (DRAFT) Aug 27, 2025

This comment has been minimized.

Sign in to view

wwyc force-pushed the do-2075-initialize-client-city-seen branch from 64349bd to dc75f81 Compare August 28, 2025 21:29

This comment has been minimized.

Sign in to view

wwyc requested a review from BenWu August 28, 2025 22:31

wwyc force-pushed the do-2075-initialize-client-city-seen branch from ce4fe26 to ff85557 Compare August 29, 2025 19:16

This comment has been minimized.

Sign in to view

wwyc requested a review from soGaussian September 2, 2025 17:48

This comment was marked as resolved.

Sign in to view

BenWu mentioned this pull request Sep 22, 2025

DENG-9727: Added mode last struct retain nulls udf #8130

Closed

This comment has been minimized.

Sign in to view

wwyc and others added 2 commits October 3, 2025 13:04

Added sample_id parameter

ea8f209

Merge branch 'main' into do-2075-initialize-client-city-seen

b2ede49

wwyc changed the title ~~DO-2075 Added fenix and desktop city seen initial table - DO NOT MERGE (DRAFT)~~ DO-2075 Added fenix and desktop baseline city seen initial tables Oct 3, 2025

wwyc marked this pull request as ready for review October 3, 2025 20:06

wwyc changed the title ~~DO-2075 Added fenix and desktop baseline city seen initial tables~~ DO-2075 Added fenix and desktop baseline city seen tables Oct 3, 2025

Merge branch 'main' into do-2075-initialize-client-city-seen

44e2f77

This comment has been minimized.

Sign in to view

BenWu mentioned this pull request Oct 6, 2025

Lower initialization parallelism and run in backfill slots #8209

Merged

BenWu reviewed Oct 6, 2025

View reviewed changes

wwyc and others added 2 commits October 6, 2025 16:13

Update sql_generators/baseline_clients_city_seen_v1/templates/metadat…

273f95b

…a.yaml Co-authored-by: Ben Wu <12437227+BenWu@users.noreply.github.com>

Removed dates cte

e26f0c1

This comment has been minimized.

Sign in to view

wwyc added 2 commits October 6, 2025 17:29

Merge branch 'main' into do-2075-initialize-client-city-seen

c44d376

Removed trailing spaces

838dd20

This comment has been minimized.

Sign in to view

Fix dag tag

4f558e6

soGaussian reviewed Oct 7, 2025

View reviewed changes

BenWu approved these changes Oct 7, 2025

View reviewed changes

wwyc added this pull request to the merge queue Oct 7, 2025

Merged via the queue into main with commit 4f3d19e Oct 7, 2025
22 checks passed

wwyc deleted the do-2075-initialize-client-city-seen branch October 7, 2025 23:09

This was referenced Oct 8, 2025

DO-2075 Removed init in clients city seen #8223

Closed

DENG-9478: Added init sampling batch size #8258

Merged

wwyc mentioned this pull request Oct 15, 2025

DENG-9478: Added baseline city seen tables for remaining browser apps #8269

Merged

DO-2075 Added fenix and desktop baseline city seen tables #7974

DO-2075 Added fenix and desktop baseline city seen tables #7974

Uh oh!

Conversation

wwyc commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Tickets & Documents

Uh oh!

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment was marked as resolved.

Uh oh!

This comment has been minimized.

This comment has been minimized.

wwyc commented Oct 3, 2025

Uh oh!

BenWu commented Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wwyc commented Oct 3, 2025

Uh oh!

Uh oh!

Uh oh!

BenWu Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

dataops-ci-bot commented Oct 7, 2025

Integration report for "Fix dag tag"

sql.diff

Uh oh!

soGaussian left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wwyc commented Oct 7, 2025

Uh oh!

BenWu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

wwyc commented Aug 19, 2025 •

edited

Loading

BenWu commented Oct 3, 2025 •

edited

Loading

`sql.diff`

soGaussian left a comment •

edited

Loading