Skip to content

Epic - deprecate legacy speedmaps and review associated modules #1308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
2 of 6 tasks
edasmalchi opened this issue Nov 26, 2024 · 4 comments
Open
2 of 6 tasks

Epic - deprecate legacy speedmaps and review associated modules #1308

edasmalchi opened this issue Nov 26, 2024 · 4 comments
Assignees
Labels
epic Representing research requests - large segments of work and their dependencies gtfs-rt Work related to GTFS-Realtime

Comments

@edasmalchi
Copy link
Member

edasmalchi commented Nov 26, 2024

Epic Information - deprecate legacy speedmaps and review associated modules

Summary

  • Now that speedmaps are deployed via the rt_segment_speeds pipeline, it's soon time to finally deprecate the old code. We do need to ensure we can generate the SCCP/LPP Level of Transit Delay metric via the new pipeline first, and let's also use this opportunity to clean up shared_utils.rt_utils.

Reviewers [Stakeholders]

  1. @tiffanychu90
  2. @edasmalchi

Issues/parts

  • create functions to run SCCP/LPP level of transit delay metric using rt_segment_speeds pipeline
  • consolidate time of day related functions (including shared_utils.rt_utils.categorize_time_of_day) and definitions in shared_utils.time_helpers, migrate existing uses
  • consolidate and document functions related to webapp mapping (some in ca_transit_speed_maps/speedmap_utils for now) largely duplicated in Research Task - Mapping app? #1140
  • remove rt_delay/rt_analysis package
  • review shared_utils.rt_utils for deprecated functions, other opportunities to consolidate
  • review docs site, remove rt_analysis module docs

Deliverables

@edasmalchi edasmalchi added epic Representing research requests - large segments of work and their dependencies gtfs-rt Work related to GTFS-Realtime labels Nov 26, 2024
@edasmalchi edasmalchi self-assigned this Nov 26, 2024
@edasmalchi
Copy link
Member Author

Hi @tiffanychu90, @KatrinaMKaiser mentioned that you'd like an update on this one.

I'm not actively working on it now, so broadly speaking it shouldn't block anything. I still think consolidating shared_utils time of day functions could be worthwhile, and I could take that on starting in a few weeks.

I actually don't think I should delete rt_delay/rt_analysis quite yet -- only because it could still be useful for looking at data from the early days of the v2 warehouse/before the rt_segment_speeds pipeline was running. Of course it could be substantially slimmed down since we aren't running it on new data.

@tiffanychu90
Copy link
Member

tiffanychu90 commented Mar 25, 2025

@edasmalchi : Can you remove the portions that need to be removed?

I'm noticing several instances now where the same underlying issue is being referred in different ways and noticed in different areas. I'm going to do a pass through everything, from gtfs_funnel, through segment_speeds/rt_vs_schedule, to gtfs_digest to see where the same thing is getting passed around and address at the most upstream.

  • caltrans district: 1349/1405.
  • checking for published operators for open data...this is done in gtfs_funnel now, but there's a broader need to have it for digest too.
  • with 1379, the reference was that 2025 was a new year, but that's not the only reference, and i want to refactor rt_dates
    • there are 3 categories of dates: v2 dates with all files, v2 dates that are one-off and do not have all files, and v1 dates that do not and should not have all files. the dates that are getting called repeatedly elsewhere are brittle against these 3 varied categories, so i've addressed the one-off dates and removed them already.
    • v1 dates would sit better in rt_delay, and work with a specific subset of tables. if in the future, we add back those dates and run it all through the entirety of the pipeline, those dates would also make their way back to rt_dates overall because all the files would be generated.
  • I will be thinking more broadly how all the functions are being used and see if any can be moved / consolidated, etc, so having the things you're going to remove in rt_utils already removed will help clear up what's not being used.

@edasmalchi
Copy link
Member Author

Sure thing, I'll take a pass at removing/consolidating

@edasmalchi
Copy link
Member Author

@tiffanychu90 just scrubbed a bunch from rt_utils via #1433

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
epic Representing research requests - large segments of work and their dependencies gtfs-rt Work related to GTFS-Realtime
Projects
None yet
Development

No branches or pull requests

2 participants