Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chng smoothed_adj_outpatient_covid contains spikes/outliers #1915

Open
brookslogan opened this issue Dec 11, 2023 · 4 comments
Open

chng smoothed_adj_outpatient_covid contains spikes/outliers #1915

brookslogan opened this issue Dec 11, 2023 · 4 comments
Labels
data quality Missing data, weird data, broken data
Milestone

Comments

@brookslogan
Copy link

Actual Behavior:

chng smoothed_adj_outpatient_covid signal has several spikes/outliers:

2023-12-11-102746_1912x955_scrot

Zooming in on one of these, it appears to impact only a single day:

2023-12-11-103026_401x827_scrot

Expected behavior

smoothed_adj_outpatient_covid should be smooth.

Context

I was browsing around signals in EpiVis to try to sanity-check COVID-19 hospitalization forecasts.

@brookslogan brookslogan added the data quality Missing data, weird data, broken data label Dec 11, 2023
@melange396
Copy link
Contributor

wild unsubstantiated hunch -- this may be related to:

@dshemetov
Copy link
Contributor

dshemetov commented Jul 10, 2024

It might be due to bad Weekday model fitting, but Clarabel replaced ECOS this May, so I don't think it's responsible for these.

@melange396
Copy link
Contributor

Clarabel is the fix for ECOS instability, and these spikes happened well before May, so it makes sense to me that ECOS is the reason for them. The PR i linked was intended to force doctor_vists off of ECOS in order to prevent similar "spikes" that caused validation failures (dr_visits was still running a pre-1.5 version of cvxpy in late May when #1966 was created). I could be missing something though. ¯\_(ツ)_/¯

@dshemetov
Copy link
Contributor

Ah I misunderstood, thought that issue was about accidentally using Clarabel.

@melange396 melange396 added this to the Data quality milestone Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data quality Missing data, weird data, broken data
Projects
None yet
Development

No branches or pull requests

4 participants