Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VMT -> Ridership model for TTTF, some further work on transfers #1025

Merged
merged 139 commits into from
Feb 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
139 commits
Select commit Hold shift + click to select a range
65fc6a8
add raster nb
edasmalchi Jan 16, 2024
04a10f6
run route typologies for rest of months in 2023
Jan 2, 2024
5fcacf1
compile schedule metrics for all dates in 2023
Jan 2, 2024
60d47aa
add gtfs-segments pkg
Jan 11, 2024
561b5be
put together all the route-direction tables we have
Jan 11, 2024
99b2d48
add single day avg to route-direction script
Jan 11, 2024
e5b8c44
checked data for 1/16 and added new function to return entire complet…
amandaha8 Jan 16, 2024
62acff2
add crosswalk from gtfs_key to org for all analysis dates
Jan 12, 2024
e8b3f31
(helpers): deprecate unused imports for segments, old functions to pa…
Jan 12, 2024
1a09c5d
(helpers): consolidate stop_times import, get_pandas defaults to true
Jan 12, 2024
b2f5319
(helpers): add crosswalk import function
Jan 12, 2024
faffe1b
(segment_speed_utils): refactor, add time_helpers, move sched_rt_util…
Jan 13, 2024
c4a2613
clearer logic for averaging for single and multi
Jan 13, 2024
dcc2aca
(remove): single and multi day avg scripts
Jan 13, 2024
2fd9d49
grab more org columns for crosswalk
Jan 13, 2024
b6f5c38
rename avg speeds script, it now does all aggregations
Jan 13, 2024
0a5c39b
remove unused functions in export
Jan 16, 2024
6a7c074
logs from refactored exports
Jan 16, 2024
d543ee7
(remove): export.py, use average_speeds now
Jan 16, 2024
a1fe2aa
compre pkg with our stop segments
Jan 16, 2024
e4260b1
rerun and select simlar routes to compare
Jan 17, 2024
8a19bdc
update normal stop segments script to use gtfs-segments
Jan 17, 2024
e36fa68
(remove): special stop segments, concat stop segments script
Jan 17, 2024
751eb0b
(remove): prep_stop_segments...need to select the shape used based on…
Jan 17, 2024
3ab9bf4
(segment_speed_utils): remove unused functions for array handling, no…
Jan 17, 2024
1c71686
(gtfs_funnel): remove stop_xnorm, stop_ynorm from all intermed parquets
Jan 17, 2024
76ff649
run multiple days for stop_segments
Jan 17, 2024
15bb175
first draft of compiling list of sb125
amandaha8 Jan 17, 2024
225005a
added lp2000, refined harmonization script
amandaha8 Jan 18, 2024
04df361
double checked monetary vals, filtered out for tircp, filtered out on…
amandaha8 Jan 19, 2024
c2dec00
ctips work for january
amandaha8 Jan 19, 2024
be62c87
new branch, folder and notebook
csuyat-dot Jan 5, 2024
88bad67
cleaning. snake case, thousand comma, testing string split
csuyat-dot Jan 5, 2024
4d12914
cleaning values in bus count col.
csuyat-dot Jan 5, 2024
4d7f28e
cleaning bus desc column
csuyat-dot Jan 6, 2024
85f5ccc
completed cleaning data. saved to gcs
csuyat-dot Jan 6, 2024
3c34b3a
start of data analysis
csuyat-dot Jan 8, 2024
fa04d45
removed thousands seperator comma from df
csuyat-dot Jan 8, 2024
1eb60f7
cleaned up project type column
csuyat-dot Jan 8, 2024
34f64c4
starting to analyze data
csuyat-dot Jan 8, 2024
3713852
analysis contd
csuyat-dot Jan 9, 2024
fd6a3a1
more cleaning
csuyat-dot Jan 9, 2024
b893acf
some analyses, some summary
csuyat-dot Jan 9, 2024
3e05c34
analysis contd
csuyat-dot Jan 9, 2024
c75d57a
more analysis
csuyat-dot Jan 9, 2024
962efbd
start itrcp notebook
csuyat-dot Jan 12, 2024
1a4c0a3
start DGS usage report notebook
csuyat-dot Jan 12, 2024
3617171
attempt to import script
csuyat-dot Jan 13, 2024
1b29c7a
reading on tircp excel sheet
csuyat-dot Jan 16, 2024
5ead90e
cleaning
csuyat-dot Jan 17, 2024
4820d36
got cost per bus, experimenting with plotting std dev
csuyat-dot Jan 17, 2024
bad483a
cleaning
csuyat-dot Jan 17, 2024
43e2935
cleaning. exporting cost per bus
csuyat-dot Jan 17, 2024
7b1995f
allocation data has some $0. started similar work on project sheet
csuyat-dot Jan 17, 2024
d725420
working on tircp project data
csuyat-dot Jan 18, 2024
fbbd0bc
set up to clean agency column
csuyat-dot Jan 18, 2024
4bd0d29
got tircp project sheet cost per bus
csuyat-dot Jan 18, 2024
9cda9b4
start of concat notebook
csuyat-dot Jan 18, 2024
e147d3e
working on distribution chart
csuyat-dot Jan 18, 2024
8c04765
summary variables
csuyat-dot Jan 18, 2024
b7b183a
summary stats
csuyat-dot Jan 18, 2024
d55239e
started to add charts
csuyat-dot Jan 18, 2024
5eda3e0
working on adding bus type and propulsion type
csuyat-dot Jan 18, 2024
41ae0f9
progress
csuyat-dot Jan 19, 2024
ecc059b
started read me
csuyat-dot Jan 19, 2024
ce46c5c
function that extracts propulsion type and bus size type from fta data
csuyat-dot Jan 20, 2024
0db00aa
checked data & modified func for incompleteness
amandaha8 Jan 19, 2024
823734a
rerun signal analysis, add override option, docs + gcs pending
edasmalchi Jan 17, 2024
3f74982
calculate overall score as sum of median subscores
edasmalchi Jan 18, 2024
d304310
fix timedelta bug, pending rerun
edasmalchi Jan 18, 2024
377f0aa
helper function for total timespan
edasmalchi Jan 18, 2024
0d3fc3f
signal variation map, legends, patch centroid
edasmalchi Jan 19, 2024
c3760d0
add methodology docs, move excel sheet to gcs
edasmalchi Jan 22, 2024
d8986ce
format
edasmalchi Jan 22, 2024
535a60e
added .env.sample, changed mtc pull
Jan 22, 2024
a4106f3
work towards matching warehouse and Conveyal feed keys
edasmalchi Jan 24, 2024
500aaf2
add conveyal region awareness, debug
edasmalchi Jan 25, 2024
8332b24
wip
edasmalchi Jan 25, 2024
3eafff5
try using nearest neighbors on small trip
Jan 18, 2024
7d78be3
condense vp into lines
Jan 19, 2024
9b1db36
(segment_speed_utils): add vp_transform and neighbor functions
Jan 19, 2024
24d3e0c
get nearest snap
Jan 20, 2024
a2109bf
np vectorize across row wise nearest neighbor search
Jan 22, 2024
e184817
move working script into old
Jan 22, 2024
7e84dc6
remove test script
Jan 22, 2024
0c70f6c
grab vp_idx and timestamp boundaries
Jan 23, 2024
87af1cb
refactor interpolate script after nearest neighbors
Jan 24, 2024
07e54a8
add selecting stop segments heuristic for 1 set of cutpoints per shape
Jan 24, 2024
c85bafc
(remove): shapely project vp
Jan 24, 2024
9123106
try segmentize on jan2024
Jan 24, 2024
679d630
run through interpolate step
Jan 24, 2024
2fe7f73
remove unused logs
Jan 24, 2024
3ca5601
calculate speed from current to subseq stop
Jan 24, 2024
77d8842
run speeds through averaging
Jan 25, 2024
04e07f4
try condenser for direction differently
Jan 25, 2024
f65ebb1
refactor vp_transform for simpler approach
Jan 25, 2024
3555913
cut segments for mar2023-jan2024, run nearest for sep2023-dec2023
Jan 26, 2024
d6c2a93
update config, now that we parse for parent, stem for sub-folders
Jan 26, 2024
00a8fc5
add segment geometry, move averaging speeds to segment_calcs
Jan 26, 2024
806fefe
jan open data
Jan 18, 2024
ac364c7
use helper functions for imports
Jan 26, 2024
2facfdc
stage datasets for geoportal to consistent url
Jan 26, 2024
d172457
remove uri from traffic_ops, add route_name for speeds_route_dir
Jan 26, 2024
ff73408
speeds has metadata.yml and data_dictionary.yml updates
Jan 26, 2024
27ea5a6
add more columns for renaming dict
Jan 26, 2024
df697f6
update use of gtfs_schedule_wrangling
Jan 26, 2024
cf18069
update to use jan date
Jan 26, 2024
5eff4a1
still working on cleaning df
csuyat-dot Jan 22, 2024
875ddc8
start tircp cleaning
csuyat-dot Jan 22, 2024
25b6319
tircp cleaning
csuyat-dot Jan 23, 2024
a123aff
cleaning and value updating
csuyat-dot Jan 23, 2024
6534695
working on prop_type col
csuyat-dot Jan 23, 2024
320ccf0
compelted prop type and bus size col
csuyat-dot Jan 23, 2024
a65ef96
organizing
csuyat-dot Jan 23, 2024
027c47b
edits to cost_per_bus notebook
csuyat-dot Jan 23, 2024
60d3642
read in tircp and fta data. fta data still need some cleaning in prop…
csuyat-dot Jan 24, 2024
e96b771
adjusted prop type column to consolidate more zero and low emission c…
csuyat-dot Jan 24, 2024
789eac5
cleaned and concat fta and tircp df
csuyat-dot Jan 24, 2024
86c06fb
beginning to analyze concat df
csuyat-dot Jan 24, 2024
a6f03a6
fixed data validation on prop_type
csuyat-dot Jan 24, 2024
fc4bafc
add code review snippets
Jan 25, 2024
dd9aada
add gdrive link
Jan 25, 2024
6240221
consolidating transformations into functions
csuyat-dot Jan 25, 2024
8e50330
continue turning things into functions
csuyat-dot Jan 26, 2024
c20c325
making charts
csuyat-dot Jan 26, 2024
3d1152e
making more variables for summary and chart improvements
csuyat-dot Jan 26, 2024
a64369c
found error in fta prop_type. fixing
csuyat-dot Jan 26, 2024
d4ece65
still working on fta manual check prop_type rows.
csuyat-dot Jan 27, 2024
29b2c11
look at nov data
Jan 30, 2024
eacf1e8
update url within notebook and update file it pulls from
Jan 30, 2024
48748c3
backfill all the dates with new workflow
Jan 30, 2024
d90bfcd
add segment geom for multiday avg
Jan 30, 2024
eb51dfd
remove unused code
Jan 31, 2024
8f5669a
add vmt sketch notebooks, data now in gcs:
edasmalchi Feb 3, 2024
a9e8a53
rename folder
edasmalchi Feb 3, 2024
c3e4efd
uza summaries, etc
edasmalchi Feb 12, 2024
859f865
add stub readme, clear maps for pr
edasmalchi Feb 14, 2024
b4efec5
resolve conflicts, use versions from main
edasmalchi Feb 14, 2024
7a2ed49
checkout rt segments speeds from main
edasmalchi Feb 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion conveyal_update/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
* Set target date in `conveyal_vars.py`. Region boundaries are also set here, but these should remain static unless the decision is made to use entirely different regions in Conveyal. Target date should be a mid-week day.
* `evaluate_feeds.py` includes functions to check to see which feeds have service defined on the target date, and show feeds without any apparent service, including if that service is apparently captured in another feed. This helps check for potential coverage gaps, likely due to GTFS feed expirations and/or the [publishing future service issue](https://github.com/MobilityData/GTFS_Schedule_Best-Practices/issues/48). You may have to shift the target date around to find the best overall coverage, and/or manually edit important but missing feeds to define service if reasonable.
* `match_feeds_regions.py` matches feeds to Conveyal regions, based on if the feed contains _any_ stops within each region.
* `download_data.py` downloads and zips original GTFS feeds, and additionally generates a shell script that can be used to download, crop, and filter OSM data for each region using Osmosis (not currently able to do so via hub, use other platform).
* `download_data.py` downloads and zips original GTFS feeds, and additionally generates a shell script that can be used to download, crop, and filter OSM data for each region using Osmosis (not currently able to do so via hub, use other platform). Downloaded feeds are labelled `{row.gtfs_dataset_name.replace(" ", "_")}_{row.feed_key}_gtfs.zip`

## Workflow

Expand Down
Loading
Loading