-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update comparison tool #2056
base: main
Are you sure you want to change the base?
Update comparison tool #2056
Commits on Aug 10, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 218fa48 - Browse repository at this point
Copy the full SHA 218fa48View commit details -
Imputing income using geographic neighbors (#1559)
Imputes income field with a light refactor. Needs more refactor and more tests (I spotchecked). Next ticket will check and address but a lot of "narwhal" architecture is here.
Configuration menu - View commit details
-
Copy full SHA for f680d86 - Browse repository at this point
Copy the full SHA f680d86View commit details -
Added HOLC indicator (Historic Redlining Score) from NCRC work; included 3.25 cutoff and low income as part of the housing burden category.
Configuration menu - View commit details
-
Copy full SHA for 3a96001 - Browse repository at this point
Copy the full SHA 3a96001View commit details -
Update backend for Puerto Rico (#1686)
* Update PR threshold count to 10 We now show 10 indicators for PR. See the discussion on the github issue for more info: #1621 * Do not use linguistic iso for Puerto Rico Closes 1350. Co-authored-by: Shelby Switzer <shelbyswitzer@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 2e38aaa - Browse repository at this point
Copy the full SHA 2e38aaaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 92d68ba - Browse repository at this point
Copy the full SHA 92d68baView commit details -
Do not drop Guam and USVI from ETL (#1681)
* Remove code that drops Guam and USVI from ETL * Add back code for dropping rows by FIPS code We may want this functionality, so let's keep it and just make the constant currently be an empty array. Co-authored-by: Shelby Switzer <shelbyswitzer@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for f8a6567 - Browse repository at this point
Copy the full SHA f8a6567View commit details -
Emma nechamkin/holc patch (#1742)
Removing HOLC calculation from score narwhal.
Configuration menu - View commit details
-
Copy full SHA for 002cddf - Browse repository at this point
Copy the full SHA 002cddfView commit details -
Configuration menu - View commit details
-
Copy full SHA for e98282d - Browse repository at this point
Copy the full SHA e98282dView commit details -
Rescaling linguistic isolation (#1750)
Rescales linguistic isolation to drop puerto rico
Configuration menu - View commit details
-
Copy full SHA for 29419dd - Browse repository at this point
Copy the full SHA 29419ddView commit details -
adds leaky underground storage tanks
Configuration menu - View commit details
-
Copy full SHA for daf188c - Browse repository at this point
Copy the full SHA daf188cView commit details -
Changing LHE in tiles to a boolean (#1767)
also includes merging / clean up of the release
Configuration menu - View commit details
-
Copy full SHA for bbb5bbc - Browse repository at this point
Copy the full SHA bbb5bbcView commit details -
Configuration menu - View commit details
-
Copy full SHA for cac1e04 - Browse repository at this point
Copy the full SHA cac1e04View commit details -
Configuration menu - View commit details
-
Copy full SHA for 19d3bde - Browse repository at this point
Copy the full SHA 19d3bdeView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3aa03f1 - Browse repository at this point
Copy the full SHA 3aa03f1View commit details -
Configuration menu - View commit details
-
Copy full SHA for ed9b717 - Browse repository at this point
Copy the full SHA ed9b717View commit details -
Refactor DOE Energy Burden and COI to use YAML (#1796)
* added tribalId for Supplemental dataset (#1804) * Setting zoom levels for tribal map (#1810) * NRI dataset and initial score YAML configuration (#1534) * update be staging gha * NRI dataset and initial score YAML configuration * checkpoint * adding data checks for release branch * passing tests * adding INPUT_EXTRACTED_FILE_NAME to base class * lint * columns to keep and tests * update be staging gha * checkpoint * update be staging gha * NRI dataset and initial score YAML configuration * checkpoint * adding data checks for release branch * passing tests * adding INPUT_EXTRACTED_FILE_NAME to base class * lint * columns to keep and tests * checkpoint * PR Review * renoving source url * tests * stop execution of ETL if there's a YAML schema issue * update be staging gha * adding source url as class var again * clean up * force cache bust * gha cache bust * dynamically set score vars from YAML * docsctrings * removing last updated year - optional reverse percentile * passing tests * sort order * column ordening * PR review * class level vars * Updating DatasetsConfig * fix pylint errors * moving metadata hint back to code Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov> * Correct copy typo (#1809) * Add basic test suite for COI (#1518) * Update COI to use new yaml (#1518) * Add tests for DOE energy budren (1518 * Add dataset config for energy budren (1518) * Refactor ETL to use datasets.yml (#1518) * Add fake GEOIDs to COI tests (#1518) * Refactor _setup_etl_instance_and_run_extract to base (#1518) For the three classes we've done so far, a generic _setup_etl_instance_and_run_extract will work fine, for the moment we can reuse the same setup method until we decide future classes need more flexibility --- but they can also always subclass so... * Add output-path tests (#1518) * Update YAML to match constant (#1518) * Don't blindly set float format (#1518) * Add defaults for extract (#1518) * Run YAML load on all subclasses (#1518) * Update description fields (#1518) * Update YAML per final format (#1518) * Update fixture tract IDs (#1518) * Update base class refactor (#1518) Now that NRI is final I needed to make a small number of updates to my refactored code. * Remove old comment (#1518) * Fix type signature and return (#1518) * Update per code review (#1518) Co-authored-by: Jorge Escobar <83969469+esfoobar-usds@users.noreply.github.com> Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov> Co-authored-by: Vim <86254807+vim-usds@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 9635ef5 - Browse repository at this point
Copy the full SHA 9635ef5View commit details
Commits on Aug 11, 2022
-
Configuration menu - View commit details
-
Copy full SHA for d55b7c0 - Browse repository at this point
Copy the full SHA d55b7c0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 485a9a8 - Browse repository at this point
Copy the full SHA 485a9a8View commit details -
Imputing income using geographic neighbors (#1559)
Imputes income field with a light refactor. Needs more refactor and more tests (I spotchecked). Next ticket will check and address but a lot of "narwhal" architecture is here.
Configuration menu - View commit details
-
Copy full SHA for f047ca9 - Browse repository at this point
Copy the full SHA f047ca9View commit details -
Added HOLC indicator (Historic Redlining Score) from NCRC work; included 3.25 cutoff and low income as part of the housing burden category.
Configuration menu - View commit details
-
Copy full SHA for 1782d02 - Browse repository at this point
Copy the full SHA 1782d02View commit details -
Update backend for Puerto Rico (#1686)
* Update PR threshold count to 10 We now show 10 indicators for PR. See the discussion on the github issue for more info: #1621 * Do not use linguistic iso for Puerto Rico Closes 1350. Co-authored-by: Shelby Switzer <shelbyswitzer@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 05748c9 - Browse repository at this point
Copy the full SHA 05748c9View commit details -
Configuration menu - View commit details
-
Copy full SHA for b41a287 - Browse repository at this point
Copy the full SHA b41a287View commit details -
Do not drop Guam and USVI from ETL (#1681)
* Remove code that drops Guam and USVI from ETL * Add back code for dropping rows by FIPS code We may want this functionality, so let's keep it and just make the constant currently be an empty array. Co-authored-by: Shelby Switzer <shelbyswitzer@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 3071815 - Browse repository at this point
Copy the full SHA 3071815View commit details -
Emma nechamkin/holc patch (#1742)
Removing HOLC calculation from score narwhal.
Configuration menu - View commit details
-
Copy full SHA for 7559cf4 - Browse repository at this point
Copy the full SHA 7559cf4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2ab24c6 - Browse repository at this point
Copy the full SHA 2ab24c6View commit details -
Rescaling linguistic isolation (#1750)
Rescales linguistic isolation to drop puerto rico
Configuration menu - View commit details
-
Copy full SHA for f6efdd4 - Browse repository at this point
Copy the full SHA f6efdd4View commit details -
adds leaky underground storage tanks
Configuration menu - View commit details
-
Copy full SHA for b0a7284 - Browse repository at this point
Copy the full SHA b0a7284View commit details -
Changing LHE in tiles to a boolean (#1767)
also includes merging / clean up of the release
Configuration menu - View commit details
-
Copy full SHA for 0d90ae5 - Browse repository at this point
Copy the full SHA 0d90ae5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8c75190 - Browse repository at this point
Copy the full SHA 8c75190View commit details -
Configuration menu - View commit details
-
Copy full SHA for 15450cf - Browse repository at this point
Copy the full SHA 15450cfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4f6a1b5 - Browse repository at this point
Copy the full SHA 4f6a1b5View commit details -
Configuration menu - View commit details
-
Copy full SHA for baa591a - Browse repository at this point
Copy the full SHA baa591aView commit details -
Refactor DOE Energy Burden and COI to use YAML (#1796)
* added tribalId for Supplemental dataset (#1804) * Setting zoom levels for tribal map (#1810) * NRI dataset and initial score YAML configuration (#1534) * update be staging gha * NRI dataset and initial score YAML configuration * checkpoint * adding data checks for release branch * passing tests * adding INPUT_EXTRACTED_FILE_NAME to base class * lint * columns to keep and tests * update be staging gha * checkpoint * update be staging gha * NRI dataset and initial score YAML configuration * checkpoint * adding data checks for release branch * passing tests * adding INPUT_EXTRACTED_FILE_NAME to base class * lint * columns to keep and tests * checkpoint * PR Review * renoving source url * tests * stop execution of ETL if there's a YAML schema issue * update be staging gha * adding source url as class var again * clean up * force cache bust * gha cache bust * dynamically set score vars from YAML * docsctrings * removing last updated year - optional reverse percentile * passing tests * sort order * column ordening * PR review * class level vars * Updating DatasetsConfig * fix pylint errors * moving metadata hint back to code Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov> * Correct copy typo (#1809) * Add basic test suite for COI (#1518) * Update COI to use new yaml (#1518) * Add tests for DOE energy budren (1518 * Add dataset config for energy budren (1518) * Refactor ETL to use datasets.yml (#1518) * Add fake GEOIDs to COI tests (#1518) * Refactor _setup_etl_instance_and_run_extract to base (#1518) For the three classes we've done so far, a generic _setup_etl_instance_and_run_extract will work fine, for the moment we can reuse the same setup method until we decide future classes need more flexibility --- but they can also always subclass so... * Add output-path tests (#1518) * Update YAML to match constant (#1518) * Don't blindly set float format (#1518) * Add defaults for extract (#1518) * Run YAML load on all subclasses (#1518) * Update description fields (#1518) * Update YAML per final format (#1518) * Update fixture tract IDs (#1518) * Update base class refactor (#1518) Now that NRI is final I needed to make a small number of updates to my refactored code. * Remove old comment (#1518) * Fix type signature and return (#1518) * Update per code review (#1518) Co-authored-by: Jorge Escobar <83969469+esfoobar-usds@users.noreply.github.com> Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov> Co-authored-by: Vim <86254807+vim-usds@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 97e1754 - Browse repository at this point
Copy the full SHA 97e1754View commit details -
Configuration menu - View commit details
-
Copy full SHA for 94cdc47 - Browse repository at this point
Copy the full SHA 94cdc47View commit details -
Configuration menu - View commit details
-
Copy full SHA for dcda155 - Browse repository at this point
Copy the full SHA dcda155View commit details -
updated to fix linting errors (#1818)
Cleans and updates base branch
Configuration menu - View commit details
-
Copy full SHA for 481a2a0 - Browse repository at this point
Copy the full SHA 481a2a0View commit details
Commits on Aug 16, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 13e7908 - Browse repository at this point
Copy the full SHA 13e7908View commit details -
* Add spatial join method (#1871) Since we'll need to figure out the tracts for a large number of points in future tickets, add a utility to handle grabbing the tract geometries and adding tract data to a point dataset. * Add FUDS, also jupyter lab (#1871) * Add YAML configs for FUDS (#1871) * Allow input geoid to be optional (#1871) * Add FUDS ETL, tests, test-datae noteobook (#1871) This adds the ETL class for Formerly Used Defense Sites (FUDS). This is different from most other ETLs since these FUDS are not provided by tract, but instead by geographic point, so we need to assign FUDS to tracts and then do calculations from there. * Floats -> Ints, as I intended (#1871) * Floats -> Ints, as I intended (#1871) * Formatting fixes (#1871) * Add test false positive GEOIDs (#1871) * Add gdal binaries (#1871) * Refactor pandas code to be more idiomatic (#1871) Per Emma, the more pandas-y way of doing my counts is using np.where to add the values i need, then groupby and size. It is definitely more compact, and also I think more correct! * Update configs per Emma suggestions (#1871) * Type fixed! (#1871) * Remove spurious import from vscode (#1871) * Snapshot update after changing col name (#1871) * Move up GDAL (#1871) * Adjust geojson strategy (#1871) * Try running census separately first (#1871) * Fix import order (#1871) * Cleanup cache strategy (#1871) * Download census data from S3 instead of re-calculating (#1871) * Clarify pandas code per Emma (#1871)
Configuration menu - View commit details
-
Copy full SHA for d5fbb80 - Browse repository at this point
Copy the full SHA d5fbb80View commit details -
Configuration menu - View commit details
-
Copy full SHA for d6c04b1 - Browse repository at this point
Copy the full SHA d6c04b1View commit details -
Merge branch 'emma-nechamkin/release/score-narwhal' of https://github…
….com/usds/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for 9321798 - Browse repository at this point
Copy the full SHA 9321798View commit details -
Adding DOT composite to travel score (#1820)
This adds the DOT dataset to the ETL and to the score. Note that currently we take a percentile of an average of percentiles.
Configuration menu - View commit details
-
Copy full SHA for ebac552 - Browse repository at this point
Copy the full SHA ebac552View commit details
Commits on Aug 17, 2022
-
Adding first street foundation data (#1823)
Adding FSF flood and wildfire risk datasets to the score.
Configuration menu - View commit details
-
Copy full SHA for 5e378ae - Browse repository at this point
Copy the full SHA 5e378aeView commit details -
Configuration menu - View commit details
-
Copy full SHA for 981a36c - Browse repository at this point
Copy the full SHA 981a36cView commit details -
Add abandoned mine lands data (#1824)
* Add notebook to generate test data (#1780) * Add Abandoned Mine Land data (#1780) Using a similar structure but simpler apporach compared to FUDs, add an indicator for whether a tract has an abandonded mine. * Adding some detail to dataset readmes Just a thought! * Apply feedback from revieiw (#1780) * Fixup bad string that broke test (#1780) * Update a string that I should have renamed (#1780) * Reduce number of threads to reduce memory pressure (#1780) * Try not running geo data (#1780) * Run the high-memory sets separately (#1780) * Actually deduplicate (#1780) * Add flag for memory intensive ETLs (#1780) * Document new flag for datasets (#1780) * Add flag for new datasets fro rebase (#1780) Co-authored-by: Emma Nechamkin <97977170+emma-nechamkin@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for 49623e4 - Browse repository at this point
Copy the full SHA 49623e4View commit details -
Merge branch 'emma-nechamkin/release/score-narwhal' of github.com:usd…
…s/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for 2e05b1d - Browse repository at this point
Copy the full SHA 2e05b1dView commit details -
Adding NLCD's natural space indicator end to end to the score.
Configuration menu - View commit details
-
Copy full SHA for 7d89d41 - Browse repository at this point
Copy the full SHA 7d89d41View commit details -
Configuration menu - View commit details
-
Copy full SHA for 88dc2e5 - Browse repository at this point
Copy the full SHA 88dc2e5View commit details
Commits on Aug 18, 2022
-
Add donut hole calculation to score (#1828)
Adds adjacency index to the pipeline. Requires thorough QA
Configuration menu - View commit details
-
Copy full SHA for 6e41e0d - Browse repository at this point
Copy the full SHA 6e41e0dView commit details -
Adding eamlis and fuds data to legacy pollution in score (#1832)
Update to add EAMLIS and FUDS data to score
Configuration menu - View commit details
-
Copy full SHA for cb4866b - Browse repository at this point
Copy the full SHA cb4866bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3ba1c62 - Browse repository at this point
Copy the full SHA 3ba1c62View commit details -
Quick fix to kitchen or plumbing indicator
Yikes! I think I messed something up and dropped the pctile field suffix from when the KP score gets calculated. Fixing right quick.
Configuration menu - View commit details
-
Copy full SHA for 1ee26bf - Browse repository at this point
Copy the full SHA 1ee26bfView commit details
Commits on Aug 19, 2022
-
Added additional flags for the front end based on our conversation in stand up this morning.
Configuration menu - View commit details
-
Copy full SHA for d892bce - Browse repository at this point
Copy the full SHA d892bceView commit details -
Configuration menu - View commit details
-
Copy full SHA for ad1ce2b - Browse repository at this point
Copy the full SHA ad1ce2bView commit details -
Configuration menu - View commit details
-
Copy full SHA for e6385c1 - Browse repository at this point
Copy the full SHA e6385c1View commit details
Commits on Aug 22, 2022
-
Issue 1827: Add demographics to tiles and download files (#1833)
* Adding demographics for use in sidebar and download files
Configuration menu - View commit details
-
Copy full SHA for 4bf7773 - Browse repository at this point
Copy the full SHA 4bf7773View commit details
Commits on Aug 23, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 6418335 - Browse repository at this point
Copy the full SHA 6418335View commit details
Commits on Aug 25, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 637b8c3 - Browse repository at this point
Copy the full SHA 637b8c3View commit details -
Configuration menu - View commit details
-
Copy full SHA for d3efcbd - Browse repository at this point
Copy the full SHA d3efcbdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4a25a28 - Browse repository at this point
Copy the full SHA 4a25a28View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9a2193d - Browse repository at this point
Copy the full SHA 9a2193dView commit details -
Configuration menu - View commit details
-
Copy full SHA for d16d010 - Browse repository at this point
Copy the full SHA d16d010View commit details -
Configuration menu - View commit details
-
Copy full SHA for b63c465 - Browse repository at this point
Copy the full SHA b63c465View commit details -
Merge branch 'emma-nechamkin/1849-calculation-tests' of github.com:us…
…ds/justice40-tool into emma-nechamkin/1849-calculation-tests
Configuration menu - View commit details
-
Copy full SHA for 41c3766 - Browse repository at this point
Copy the full SHA 41c3766View commit details -
Merge branch 'emma-nechamkin/1849-calculation-tests' of github.com:us…
…ds/justice40-tool into emma-nechamkin/1849-calculation-tests
Configuration menu - View commit details
-
Copy full SHA for cb13388 - Browse repository at this point
Copy the full SHA cb13388View commit details -
Merge branch 'emma-nechamkin/1849-calculation-tests' of github.com:us…
…ds/justice40-tool into emma-nechamkin/1849-calculation-tests
Configuration menu - View commit details
-
Copy full SHA for 16209cc - Browse repository at this point
Copy the full SHA 16209ccView commit details -
Configuration menu - View commit details
-
Copy full SHA for c524447 - Browse repository at this point
Copy the full SHA c524447View commit details
Commits on Aug 26, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 15b4f5b - Browse repository at this point
Copy the full SHA 15b4f5bView commit details -
Configuration menu - View commit details
-
Copy full SHA for e539db8 - Browse repository at this point
Copy the full SHA e539db8View commit details -
* update Python version on README; tuple typing fix * Alaska tribal points fix (#1821) * Bump mistune from 0.8.4 to 2.0.3 in /data/data-pipeline (#1777) Bumps [mistune](https://github.com/lepture/mistune) from 0.8.4 to 2.0.3. - [Release notes](https://github.com/lepture/mistune/releases) - [Changelog](https://github.com/lepture/mistune/blob/master/docs/changes.rst) - [Commits](lepture/mistune@v0.8.4...v2.0.3) --- updated-dependencies: - dependency-name: mistune dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * poetry update * initial pass of score tests * add threshold tests * added ses threshold (not donut, not island) * testing suite -- stopping for the day * added test for lead proxy indicator * Refactor score tests to make them less verbose and more direct (#1865) * Cleanup tests slightly before refactor (#1846) * Refactor score calculations tests * Feedback from review * Refactor output tests like calculatoin tests (#1846) (#1870) * Reorganize files (#1846) * Switch from lru_cache to fixture scorpes (#1846) * Add tests for all factors (#1846) * Mark smoketests and run as part of be deply (#1846) * Update renamed var (#1846) * Switch from named tuple to dataclass (#1846) This is annoying, but pylint in python3.8 was crashing parsing the named tuple. We weren't using any namedtuple-specific features, so I made the type a dataclass just to get pylint to behave. * Add default timout to requests (#1846) * Fix type (#1846) * Fix merge mistake on poetry.lock (#1846) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Jorge Escobar <jorge.e.escobar@omb.eop.gov> Co-authored-by: Jorge Escobar <83969469+esfoobar-usds@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matt Bowen <83967628+mattbowen-usds@users.noreply.github.com> Co-authored-by: matt bowen <matthew.r.bowen@omb.eop.gov>
Configuration menu - View commit details
-
Copy full SHA for 1c4d3e4 - Browse repository at this point
Copy the full SHA 1c4d3e4View commit details
Commits on Aug 30, 2022
-
Merge branch 'emma-nechamkin/release/score-narwhal' of github.com:usd…
…s/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for 6e575c6 - Browse repository at this point
Copy the full SHA 6e575c6View commit details
Commits on Aug 31, 2022
-
Merge branch 'emma-nechamkin/1849-calculation-tests' of github.com:us…
…ds/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for 7c6a907 - Browse repository at this point
Copy the full SHA 7c6a907View commit details -
just testing that the boolean is preserved on gha (#1867)
* updated with hopefully a fix; coercing aml, fuds, hrs to booleans for the raw value to preserve null character.
Configuration menu - View commit details
-
Copy full SHA for b0b7ff0 - Browse repository at this point
Copy the full SHA b0b7ff0View commit details -
Adding tests to ensure proper calculations (#1871)
* just testing that the boolean is preserved on gha * checking drop tracts works * adding a check to the agvalue calculation for nri * updated with error messages
Configuration menu - View commit details
-
Copy full SHA for 5201f9e - Browse repository at this point
Copy the full SHA 5201f9eView commit details -
Merge branch 'emma-nechamkin/release/score-narwhal' of github.com:usd…
…s/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for b7af13b - Browse repository at this point
Copy the full SHA b7af13bView commit details
Commits on Sep 1, 2022
-
* Alaska tribal points fix (#1821) * tribal tiles fix * disabling child opportunity * lint * removing COI * removing commented out code
Configuration menu - View commit details
-
Copy full SHA for ccd72e2 - Browse repository at this point
Copy the full SHA ccd72e2View commit details -
* temp update * updating with fips check * adding check on pfs * updating with pfs test * Update test_tiles_smoketests.py * Fix lint errors (#1848) * Add column names test (#1848) * Mark tests as smoketests (#1848) * Move to other score-related tests (#1848) * Recast Total threshold criteria exceeded to int (#1848) In writing tests to verify the output of the tiles csv matches the final score CSV, I noticed TC/Total threshold criteria exceeded was getting cast from an int64 to a float64 in the process of PostScoreETL. I tracked it down to the line where we merge the score dataframe with constants.DATA_CENSUS_CSV_FILE_PATH --- there where > 100 tracts in the national census CSV that don't exist in the score, so those ended up with a Total threshhold count of np.nan, which is a float, and thereby cast those columns to float. For the moment I just cast it back. * No need for low memeory (#1848) * Add additional tests of tiles.csv (#1848) * Drop pre-2010 rows before computing score (#1848) Note this is probably NOT the optimal place for this change; it might make more sense for each source to filter its own tracts down to the acceptable tract list. However, that would be a pretty invasive change, where this is central and plenty of other things are happening in score transform that could be moved to sources, so for today, here's where the change will live. * Fix typo (#1848) * Switch from filter to inner join (#1848) * Remove no-op lines from tiles (#1848) * Apply feedback from review, linter (#1848) * Check the values oeverything in the frame (#1848) * Refactor checker class (#1848) * Add test for state names (#1848) * cleanup from reviewing my own code (#1848) * Fix lint error (#1858) * Apply Emma's feedback from review (#1848) * Remove refs to national_df (#1848) * Account for new, fake nullable bools in tiles (#1848) To handle a geojson limitation, Emma converted some nullable boolean colunms to float64 in the tiles export with the values {0.0, 1.0, nan}, giving us the same expressiveness. Sadly, this broke my assumption that all columns between the score and tiles csvs would have the same dtypes, so I need to account for these new, fake bools in my test. * Use equals instead of my worse version (#1848) * Missed a spot where we called _create_score_data (#1848) * Update per safety (#1848) Co-authored-by: matt bowen <matthew.r.bowen@omb.eop.gov>
Configuration menu - View commit details
-
Copy full SHA for 9c0e199 - Browse repository at this point
Copy the full SHA 9c0e199View commit details
Commits on Sep 6, 2022
-
Add tests to make sure each source makes it to the score correctly (#…
…1878) * Remove unused persistent poverty from score (#1835) * Test a few datasets for overlap in the final score (#1835) * Add remaining data sources (#1853) * Apply code-review feedback (#1835) * Rearrange a little for readabililty (#1835) * Add tract test (#1835) * Add test for score values (#1835) * Check for unmatched source tracts (#1835) * Cleanup numeric code to plaintext (#1835) * Make import more obvious (#1835)
Configuration menu - View commit details
-
Copy full SHA for d41153d - Browse repository at this point
Copy the full SHA d41153dView commit details
Commits on Sep 7, 2022
-
Merge branch 'emma-nechamkin/release/score-narwhal' of github.com:usd…
…s/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for 31eac41 - Browse repository at this point
Copy the full SHA 31eac41View commit details -
Updating traffic barriers to include low pop threshold (#1889)
Changing the traffic barriers to only be included for places with recorded population
Configuration menu - View commit details
-
Copy full SHA for 426328e - Browse repository at this point
Copy the full SHA 426328eView commit details
Commits on Sep 8, 2022
-
Configuration menu - View commit details
-
Copy full SHA for fb4c484 - Browse repository at this point
Copy the full SHA fb4c484View commit details -
Merge branch 'emma-nechamkin/release/score-narwhal' of github.com:usd…
…s/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for e78c6d0 - Browse repository at this point
Copy the full SHA e78c6d0View commit details
Commits on Sep 10, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 6e9c44e - Browse repository at this point
Copy the full SHA 6e9c44eView commit details
Commits on Sep 12, 2022
-
Merge branch 'emma-nechamkin/release/score-narwhal' of github.com:usd…
…s/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for 7ea2ae8 - Browse repository at this point
Copy the full SHA 7ea2ae8View commit details -
Removing low pop tracts from FEMA population loss (#1898)
dropping 0 population from FEMA
Configuration menu - View commit details
-
Copy full SHA for 60164c8 - Browse repository at this point
Copy the full SHA 60164c8View commit details
Commits on Sep 15, 2022
-
Merge branch 'emma-nechamkin/release/score-narwhal' of github.com:usd…
…s/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for fe4e8e5 - Browse repository at this point
Copy the full SHA fe4e8e5View commit details -
This code causes no functional change to the code. It does two things: 1. Uses difference instead of - to improve code style for working with sets. 2. Removes the line EXPECTED_MISSING_STATES = ["02", "15"], which is now redundant because of the line I added (in a previous pull request) of ALASKA_AND_HAWAII_EXPECTED_IN_DATA = False.
Configuration menu - View commit details
-
Copy full SHA for 4d02525 - Browse repository at this point
Copy the full SHA 4d02525View commit details
Commits on Sep 19, 2022
-
Add tests for all non-census sources (#1899)
* Refactor CDC life-expectancy (1554) * Update to new tract list (#1554) * Adjust for tests (#1848) * Add tests for cdc_places (#1848) * Add EJScreen tests (#1848) * Add tests for HUD housing (#1848) * Add tests for GeoCorr (#1848) * Add persistent poverty tests (#1848) * Update for sources without zips, for new validation (#1848) * Update tests for new multi-CSV but (#1848) Lucas updated the CDC life expectancy data to handle a bug where two states are missing from the US Overall download. Since virtually none of our other ETL classes download multiple CSVs directly like this, it required a pretty invasive new mocking strategy. * Add basic tests for nature deprived (#1848) * Add wildfire tests (#1848) * Add flood risk tests (#1848) * Add DOT travel tests (#1848) * Add historic redlining tests (#1848) * Add tests for ME and WI (#1848) * Update now that validation exists (#1848) * Adjust for validation (#1848) * Add health insurance back to cdc places (#1848) Ooops * Update tests with new field (#1848) * Test for blank tract removal (#1848) * Add tracts for clipping behavior * Test clipping and zfill behavior (#1848) * Fix bad test assumption (#1848) * Simplify class, add test for tract padding (#1848) * Fix percentage inversion, update tests (#1848) Looking through the transformations, I noticed that we were subtracting a percentage that is usually between 0-100 from 1 instead of 100, and so were endind up with some surprising results. Confirmed with lucasmbrown-usds * Add note about first street data (#1848)
Configuration menu - View commit details
-
Copy full SHA for 876655d - Browse repository at this point
Copy the full SHA 876655dView commit details
Commits on Sep 20, 2022
-
Issue 1900: Tribal overlap with Census tracts (#1903)
* working notebook * updating notebook * wip * fixing broken tests * adding tribal overlap files * WIP * WIP * WIP, calculated count and names * working * partial cleanup * partial cleanup * updating field names * fixing bug * removing pyogrio * removing unused imports * updating test fixtures to be more realistic * cleaning up notebook * fixing black * fixing flake8 errors * adding tox instructions * updating etl_score * suppressing warning * Use projected CRSes, ignore geom types (#1900) I looked into this a bit, and in general the geometry type mismatch changes very little about the calculation; we have a mix of multipolygons and polygons. The fastest thing to do is just not keep geom type; I did some runs with it set to both True and False, and they're the same within 9 digits of precision. Logically we just want to overlaps, regardless of how the actual geometries are encoded between the frames, so we can in this case ignore the geom types and feel OKAY. I also moved to projected CRSes, since we are actually trying to do area calculations and so like, we should. Again, the change is small in magnitude but logically more sound. * Readd CDC dataset config (#1900) * adding comments to fips code * delete unnecessary loggers Co-authored-by: matt bowen <matthew.r.bowen@omb.eop.gov>
Configuration menu - View commit details
-
Copy full SHA for aca2261 - Browse repository at this point
Copy the full SHA aca2261View commit details
Commits on Sep 23, 2022
-
Improve score test documentation based on Lucas's feedback (#1835) (#…
Configuration menu - View commit details
-
Copy full SHA for f70f30d - Browse repository at this point
Copy the full SHA f70f30dView commit details -
Configuration menu - View commit details
-
Copy full SHA for d8dd4cf - Browse repository at this point
Copy the full SHA d8dd4cfView commit details -
Add tribal count notebook (#1917) (#1919)
* Add tribal count notebook (#1917) * test without caching * added comment Co-authored-by: lucasmbrown-usds <lucas.m.brown@omb.eop.gov>
Configuration menu - View commit details
-
Copy full SHA for 6e0ef33 - Browse repository at this point
Copy the full SHA 6e0ef33View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9e85375 - Browse repository at this point
Copy the full SHA 9e85375View commit details
Commits on Sep 26, 2022
-
Issue 1910: Do not impute income for 0 population tracts (#1918)
* should be working, has unnecessary loggers * removing loggers and cleaning up * updating ejscreen tests * adding tests and responding to PR feedback * fixing broken smoke test * delete smoketest docs
Configuration menu - View commit details
-
Copy full SHA for 9fb9874 - Browse repository at this point
Copy the full SHA 9fb9874View commit details -
Configuration menu - View commit details
-
Copy full SHA for 15d946c - Browse repository at this point
Copy the full SHA 15d946cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2f61900 - Browse repository at this point
Copy the full SHA 2f61900View commit details
Commits on Sep 27, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 48d961b - Browse repository at this point
Copy the full SHA 48d961bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4da55a9 - Browse repository at this point
Copy the full SHA 4da55a9View commit details
Commits on Sep 28, 2022
-
Update deps safety says are vulnerable (#1937) (#1938)
Co-authored-by: matt bowen <matt@mattbowen.net>
Configuration menu - View commit details
-
Copy full SHA for 0f0d6db - Browse repository at this point
Copy the full SHA 0f0d6dbView commit details
Commits on Sep 29, 2022
-
Add demos for island areas (#1932)
* Backfill population in island areas (#1882) * Update smoketest to account for backfills (#1882) As I wrote in the commend: We backfill island areas with data from the 2010 census, so if THOSE tracts have data beyond the data source, that's to be expected and is fine to pass. If some other state or territory does though, this should fail This ends up being a nice way of documenting that behavior i guess! * Fixup lint issues (#1882) * Add in race demos to 2010 census pull (#1851) * Add backfill data to score (#1851) * Change column name (#1851) * Fill demos after the score (#1851) * Add income back, adjust test (#1882) * Apply code-review feedback (#1851) * Add test for island area backfill (#1851) * Fix bad rename (#1851)
Configuration menu - View commit details
-
Copy full SHA for 8e5ed5b - Browse repository at this point
Copy the full SHA 8e5ed5bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 247db4a - Browse repository at this point
Copy the full SHA 247db4aView commit details -
Merge branch 'emma-nechamkin/release/score-narwhal' of github.com:usd…
…s/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for 7b2a4e5 - Browse repository at this point
Copy the full SHA 7b2a4e5View commit details
Commits on Sep 30, 2022
-
Configuration menu - View commit details
-
Copy full SHA for f4adf17 - Browse repository at this point
Copy the full SHA f4adf17View commit details -
Merge branch 'emma-nechamkin/release/score-narwhal' of github.com:usd…
…s/justice40-tool into emma-nechamkin/release/score-narwhal
Configuration menu - View commit details
-
Copy full SHA for 40efd65 - Browse repository at this point
Copy the full SHA 40efd65View commit details
Commits on Oct 3, 2022
-
Configuration menu - View commit details
-
Copy full SHA for d4ae16b - Browse repository at this point
Copy the full SHA d4ae16bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7c8617d - Browse repository at this point
Copy the full SHA 7c8617dView commit details -
Configuration menu - View commit details
-
Copy full SHA for ecabe79 - Browse repository at this point
Copy the full SHA ecabe79View commit details
Commits on Oct 4, 2022
-
produce a string for the front end to ingest (#1963)
* wip * i believe this works -- let's see the pipeline * updated fixtures
Configuration menu - View commit details
-
Copy full SHA for a438b44 - Browse repository at this point
Copy the full SHA a438b44View commit details -
* updated tile data * ensuring adjli_et in
Configuration menu - View commit details
-
Copy full SHA for 71385a0 - Browse repository at this point
Copy the full SHA 71385a0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1334fcc - Browse repository at this point
Copy the full SHA 1334fccView commit details -
Configuration menu - View commit details
-
Copy full SHA for baa34ec - Browse repository at this point
Copy the full SHA baa34ecView commit details -
Issue 105: Configure and run
black
and other pre-commit hooks (clea……n branch) (#1962) * Configure and run `black` and other pre-commit hooks Co-authored-by: matt bowen <matthew.r.bowen@omb.eop.gov>
Configuration menu - View commit details
-
Copy full SHA for 6e6223c - Browse repository at this point
Copy the full SHA 6e6223cView commit details
Commits on Oct 5, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 6505d49 - Browse repository at this point
Copy the full SHA 6505d49View commit details
Commits on Oct 6, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 72de938 - Browse repository at this point
Copy the full SHA 72de938View commit details
Commits on Oct 7, 2022
-
Configuration menu - View commit details
-
Copy full SHA for e2641fe - Browse repository at this point
Copy the full SHA e2641feView commit details -
Configuration menu - View commit details
-
Copy full SHA for d89c516 - Browse repository at this point
Copy the full SHA d89c516View commit details
Commits on Oct 12, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 8b611ed - Browse repository at this point
Copy the full SHA 8b611edView commit details
Commits on Oct 13, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 743d3ce - Browse repository at this point
Copy the full SHA 743d3ceView commit details -
Configuration menu - View commit details
-
Copy full SHA for 841a26d - Browse repository at this point
Copy the full SHA 841a26dView commit details
Commits on Oct 21, 2022
-
* data source location * toml * cdc_places * cdc_svi_index * url updates * child oppy and dot travel * up to hud_recap * completed ticket * cache bust * hud_recap * us_army_fuds
Configuration menu - View commit details
-
Copy full SHA for d975118 - Browse repository at this point
Copy the full SHA d975118View commit details -
Remove vars the frontend doesn't use (#2020) (#2022)
I did a pretty rough and simple analysis of the variables we put in the tiles and grepped the frontend code to see if (1) they're ever accessed and (2) if they're used, even if they're read once. I removed everything I noticed was not accessed.
Configuration menu - View commit details
-
Copy full SHA for d17f612 - Browse repository at this point
Copy the full SHA d17f612View commit details
Commits on Oct 25, 2022
-
Disable file size limits on tiles (#2031)
* Disable file size limits on tiles * Remove print debugs I know.
Configuration menu - View commit details
-
Copy full SHA for dbea349 - Browse repository at this point
Copy the full SHA dbea349View commit details
Commits on Oct 26, 2022
-
Update file name pattern (#2037) (#2038)
* Update file name pattern (#2037) * Remove ETL from generation (2037) I looked more carefully, and this ETL step isn't used in the score, so there's no need to run it every time. Per previous steps, I removed it from constants so the code is there it won't run by default.
Configuration menu - View commit details
-
Copy full SHA for b5cea27 - Browse repository at this point
Copy the full SHA b5cea27View commit details -
Configuration menu - View commit details
-
Copy full SHA for e51af9d - Browse repository at this point
Copy the full SHA e51af9dView commit details
Commits on Nov 1, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 289024d - Browse repository at this point
Copy the full SHA 289024dView commit details