Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up validation for all indicators #725

Open
7 of 11 tasks
sgsmob opened this issue Jan 25, 2021 · 2 comments · Fixed by #954
Open
7 of 11 tasks

Set up validation for all indicators #725

sgsmob opened this issue Jan 25, 2021 · 2 comments · Fixed by #954

Comments

@sgsmob
Copy link
Contributor

sgsmob commented Jan 25, 2021

Indicators:

Inactive Indicators:

  • cdc_covidnet
  • combo_cases
  • google_health
  • hhs_facilities
  • quidel
  • safegraph

This should basically involve:

  1. Set up bare-bones params.json that specifies:
  • global.data_source (the indicator name)
  • global.span_length = 14
  • dynamic.smoothed_signals
  • dynamic.max_expected_lag
  • dynamic.min_expected_lag
  1. Get the indicator output by running it or downloading some cached files from somewhere
  2. Run the validation from the indicator directory using python -m delphi_validation
  3. Investigate the output and use it to tune the remaining validation parameters
@qx-teo
Copy link
Contributor

qx-teo commented Aug 20, 2021

Validation parameters have been added (and implemented) for all but one indicator ran using automation.

Exceptions include:

  • nchs_mortality: Indicator provides weekly data, which requires modification of the validator (see Adapt Validator for Weekly Data #1117)
  • cron indicators: doctor-visits, fb-survey, hospital-admissions, indicator-combination. Of all these, params.json.template files have been created, but validation isn't currently implemented in the pipeline yet.

@krivard krivard added the Priority-P1 Hope to do; lab will survive without it label Aug 24, 2021
@nmdefries
Copy link
Contributor

Validation is rolled out to all daily signals in dry-run mode. Acquisition blocking is turned off because validation is too sensitive for it to be useful.

@nmdefries nmdefries removed the Priority-P1 Hope to do; lab will survive without it label Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants