Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clean up segment speeds methodology and refactor trip speeds and route speeds #946

Merged
merged 16 commits into from
Nov 7, 2023
Merged

Clean up segment speeds methodology and refactor trip speeds and route speeds #946

merged 16 commits into from
Nov 7, 2023

Conversation

tiffanychu90
Copy link
Member

@tiffanychu90 tiffanychu90 commented Nov 7, 2023 •
edited
Loading

segment speeds

  • Cleanup: remove scripts related to sjoin methodology
  • Deprecate no longer used functions in segment_speed_utils
  • Refactor: avg_speeds, trip_speeds and route_speeds. p20, p50, p80 quantiles are too slow to calculate in pandas, switch to numpy
  • Weave trip_speeds and route_speeds into existing avg_speeds and export scripts instead of 2 standalone scripts afterwards
  • Leave the cut_stop_segments alone for now, see if new ideas pop up once we move to road segments
  • New pipeline from nearest_vp to export takes roughly 30 min Research Request - Performant vp -> speeds with interpolation method #939
  • Backfill Mar-Oct speeds parquets in GCS
  • Epic - Daskify RT segment speeds #592

road segments

  • add segment origin and destination to road segments. analogous to stops along shapes are origin/destination/cutpoints along road segments

tiffanychu90 added 16 commits November 1, 2023 21:49
add subseq stop sequence to stop_times_direction
3dd4af4
add subseq stop sequence to stop_times_direction and stops_projected,…
ed1f694 
… run for all months
rerun stop_times_direction and prep_stop_segments for all months
bbc0775
set up scripts to loop through multiple dates and test
3fe8616
mar-aug run for new speeds pipeline
e8227ab
use np percetile for avg speeds, run export for all months
e81b04a
rename scripts
92f61cb
(remove): sjoin methodology scripts
bb87732
(remove): sjoin methodology speed calculation script
3e15b2a
(deprecate): no longer used functions in segment_speed_utils
ae66f40
get C3 trip route speed ready to be refactored, maybe point at differ…
3f9e96c 
…ent source file
(remove): loop_utils used for exploratory
335ad13
rename sjoin_vp_segments to sjoin_shapes_roads
ffa6382
trip speeds and route speeds for all months
83c774d
remove C2, C3 scripts for trip/route speeds, work into avg and export…
60e64e3 
… scripts
filter to weirdest loop/inlining shapes
52e7914
@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 7, 2023

nbviewer URLs for impacted notebooks:

4 similar comments
@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 7, 2023

nbviewer URLs for impacted notebooks:

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 7, 2023

nbviewer URLs for impacted notebooks:

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 7, 2023

nbviewer URLs for impacted notebooks:

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 7, 2023

nbviewer URLs for impacted notebooks:

@tiffanychu90 tiffanychu90 merged commit 9fe222f into main Nov 7, 2023
7 checks passed
@tiffanychu90 tiffanychu90 deleted the speeds-on-roads branch November 7, 2023 23:27
@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 7, 2023

nbviewer URLs for impacted notebooks:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@tiffanychu90