Skip to content

Commit

Permalink
Merge pull request #224 from micokoch/source
Browse files Browse the repository at this point in the history
updating alloscore publication
  • Loading branch information
nickreich authored Dec 20, 2024
2 parents 7667783 + 4b2cae9 commit e5fec86
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 26 deletions.
56 changes: 30 additions & 26 deletions _data/publications.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,33 @@
# 2024
#2025

# New publications

# 2024

- title: 'Evaluating infectious disease forecasts with allocation scoring rules'
slug: alloscore
authors: Gerding A, Reich NG, Rogers B, Ray EL
preprint: https://arxiv.org/abs/2312.16201
pdf: /pdfs/publications/alloscore.pdf #[update if pdf becomes open access]
year: 2024
journal: 'Journal of the Royal Statistical Society Series A: Statistics in Society'
github: aaronger/utility-eval-papers
doi: 10.1093/jrsssa/qnae136
keywords: forecasting, covid-19
abstract: >
Recent years have seen increasing efforts to forecast infectious disease burdens, with a primary
goal being to help public health workers make informed policy decisions. However, there has been
only limited discussion of how predominant forecast evaluation metrics might indicate the success
of policies based in part on those forecasts. We explore one possible tether between forecasts
and policy: the allocation of limited medical resources so as to minimize unmet need. We use
probabilistic forecasts of disease burden in each of several regions to determine optimal resource
allocations, and then we score forecasts according to how much unmet need their associated
allocations would have allowed. We illustrate with forecasts of COVID-19 hospitalizations in the
U.S., and we find that the forecast skill ranking given by this allocation scoring rule can vary
substantially from the ranking given by the weighted interval score. We see this as evidence that
the allocation scoring rule detects forecast value that is missed by traditional accuracy measures
and that the general strategy of designing scoring rules that are directly linked to policy
performance is a promising direction for epidemic forecast evaluation.
- title: 'Beyond forecast leaderboards: Measuring individual model importance based on contribution to ensemble accuracy'
slug: model-importance
Expand Down Expand Up @@ -141,31 +170,6 @@
# 2023

- title: 'Evaluating infectious disease forecasts with allocation scoring rules'
slug: alloscore
authors: Gerding A, Reich NG, Rogers B, Ray EL
preprint: https://arxiv.org/abs/2312.16201
pdf: https://arxiv.org/pdf/2312.16201
year: 2023
journal: arXiv
github: aaronger/utility-eval-papers
#doi: 10.48550/arXiv.2312.16201 [update when published]
keywords: forecasting, covid-19
abstract: >
Recent years have seen increasing efforts to forecast infectious disease burdens, with a primary
goal being to help public health workers make informed policy decisions. However, there has only
been limited discussion of how predominant forecast evaluation metrics might indicate the success
of policies based in part on those forecasts. We explore one possible tether between forecasts
and policy: the allocation of limited medical resources so as to minimize unmet need. We use
probabilistic forecasts of disease burden in each of several regions to determine optimal resource
allocations, and then we score forecasts according to how much unmet need their associated
allocations would have allowed. We illustrate with forecasts of COVID-19 hospitalizations in the
US, and we find that the forecast skill ranking given by this allocation scoring rule can vary
substantially from the ranking given by the weighted interval score. We see this as evidence that the
allocation scoring rule detects forecast value that is missed by traditional accuracy measures and
that the general strategy of designing scoring rules that are directly linked to policy performance
is a promising direction for epidemic forecast evaluation.
- title: 'Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty'
slug: covid_scenario_hub_eval
authors: Howerton E, Contamin L, Mullany LC, Qin M, Reich NG, ... Viboud C, Lessler J
Expand Down
Binary file added pdfs/publications/alloscore.pdf
Binary file not shown.

0 comments on commit e5fec86

Please sign in to comment.