initial route variance identification work

[csuyat-dot]

initial work related to #924.

Determined that fct_monthly_routes, dim_providers_gtfs_data and helpers.import_scheduled_trips may give us the best chance at observing route variances over time.

Started with joining fct_monthly_routes and dim_providers_gtfs_data. Filtered down to April 2023 to September 2023 for Sacramento Schedule. Observed that 1 gtfs shcedule dataset key was found.

Then looked into helpers.import_shcedule_trips. Also filtered down to the same timeframe and provider. but found 2 gtfs schedule dataset keys and no data was found for June 2023.

Tested joining the fct_month_routes/dim_providers_gfts_data & helpers.import_schedule_trips, but wondering if helpers.import_schedule_trips needs further investigation.

[github-actions]

nbviewer URLs for impacted notebooks:

• bus_service_increase/B2_chart_bus_arrivals_by_tract.ipynb
• conveyal_update/check_download_feeds.ipynb
• conveyal_update/old_PrepGTFSFeeds.ipynb
• finding_transfers/test_process_conveyal.ipynb
• gtfs_quality_check/gtfs_checking.ipynb
• gtfs_schedule/05_transit_density.ipynb
• gtfs_schedule/06_Amanda_2023_route_identification_over_time_approach_2-Copy1.ipynb
• gtfs_schedule/06_Amanda_route_identification.ipynb
• gtfs_schedule/06_Amanda_route_identification_over_time.ipynb
• gtfs_schedule/06_shapes_on_roads.ipynb
• gtfs_schedule/07_route_variance_identification.ipynb
• high_quality_transit_areas/check3_hqta_points.ipynb
• ntd/ntd_id_changes.ipynb
• ntd/ntd_test_failures.ipynb
• open_data/check_exported_data.ipynb
• project_list/add_LP2000.ipynb
• project_list/add_LRTP_congestion.ipynb
• project_list/add_ctips_11_21_2023.ipynb
• project_list/add_lp2000_10_27_2023.ipynb
• project_list/add_sb1.ipynb
• quarterly_performance_objective/current_quarter_report.ipynb
• quarterly_performance_objective/historical_report.ipynb
• quarterly_performance_objective/historical_service_hours_v2.ipynb
• rt_scheduled_v_ran/01_scheduled_stop_times.ipynb
• rt_scheduled_v_ran/02_metrics_muni_only.ipynb
• rt_scheduled_v_ran/03_metrics_all_ops.ipynb

[github-actions]

nbviewer URLs for impacted notebooks:

• bus_service_increase/B2_chart_bus_arrivals_by_tract.ipynb
• conveyal_update/check_download_feeds.ipynb
• conveyal_update/old_PrepGTFSFeeds.ipynb
• finding_transfers/test_process_conveyal.ipynb
• gtfs_quality_check/gtfs_checking.ipynb
• gtfs_schedule/05_transit_density.ipynb
• gtfs_schedule/06_Amanda_2023_route_identification_over_time_approach_2-Copy1.ipynb
• gtfs_schedule/06_Amanda_route_identification.ipynb
• gtfs_schedule/06_Amanda_route_identification_over_time.ipynb
• gtfs_schedule/06_shapes_on_roads.ipynb
• gtfs_schedule/07_route_variance_identification.ipynb
• high_quality_transit_areas/check3_hqta_points.ipynb
• ntd/ntd_id_changes.ipynb
• ntd/ntd_test_failures.ipynb
• open_data/check_exported_data.ipynb
• project_list/add_LP2000.ipynb
• project_list/add_LRTP_congestion.ipynb
• project_list/add_ctips_11_21_2023.ipynb
• project_list/add_lp2000_10_27_2023.ipynb
• project_list/add_sb1.ipynb
• quarterly_performance_objective/current_quarter_report.ipynb
• quarterly_performance_objective/historical_report.ipynb
• quarterly_performance_objective/historical_service_hours_v2.ipynb
• rt_scheduled_v_ran/01_scheduled_stop_times.ipynb
• rt_scheduled_v_ran/02_metrics_muni_only.ipynb
• rt_scheduled_v_ran/03_metrics_all_ops.ipynb

[tiffanychu90]

Notes here for continuing work on this issue:

• don't use fct_monthly_routes for this, compile across multiple dates using helpers.import_scheduled_trips.
• differences we're interested in (if we have 4 dates):
  • route_id: check nunique(gtfs_dataset_key-route_id combination) > 1.
  • if nunique == 4, then route_id is changing every month, if nunique == 2, then route_id changes once in this period
  • check nunique(gtfs_dataset_key-route_long_name-route_short_name) > 1
• pick a couple examples to show where this is true for all operators