-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add chatzilena et al as a replication example #464
Add chatzilena et al as a replication example #464
Conversation
Remove Statsplots and convert all plots to CairoMakie in Mishra replication
Try this Pull Request!Open Julia and type: import Pkg
Pkg.activate(temp=true)
Pkg.add(url="https://github.com/CDCgov/Rt-without-renewal", rev="446-add-chatzilena-et-al-as-a-replication-example", subdir="EpiAware")
using EpiAware |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #464 +/- ##
=======================================
Coverage 90.02% 90.02%
=======================================
Files 57 57
Lines 742 742
=======================================
Hits 668 668
Misses 74 74 ☔ View full report in Codecov by Sentry. |
Benchmark resultJudge resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
EpiAware/docs/src/showcase/replications/chatzilena-2019/index.jl
Outdated
Show resolved
Hide resolved
EpiAware/docs/src/showcase/replications/chatzilena-2019/index.jl
Outdated
Show resolved
Hide resolved
EpiAware/docs/src/showcase/replications/chatzilena-2019/index.jl
Outdated
Show resolved
Hide resolved
EpiAware/docs/src/showcase/replications/chatzilena-2019/index.jl
Outdated
Show resolved
Hide resolved
EpiAware/docs/src/showcase/replications/chatzilena-2019/index.jl
Outdated
Show resolved
Hide resolved
EpiAware/docs/src/showcase/replications/chatzilena-2019/index.jl
Outdated
Show resolved
Hide resolved
EpiAware/docs/src/showcase/replications/chatzilena-2019/index.jl
Outdated
Show resolved
Hide resolved
This comment was marked as resolved.
This comment was marked as resolved.
Co-authored-by: Sam Abbott <azw1@cdc.gov>
EpiAware/docs/src/showcase/replications/chatzilena-2019/index.jl
Outdated
Show resolved
Hide resolved
EpiAware/docs/src/showcase/replications/chatzilena-2019/index.jl
Outdated
Show resolved
Hide resolved
EpiAware/docs/src/showcase/replications/chatzilena-2019/index.jl
Outdated
Show resolved
Hide resolved
Benchmark resultJudge resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
Hey, @damonbayer . Was just wondering if you have any thoughts on whether this is a good intro to SciML + EpiAware inference? |
Benchmark resultJudge resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
I've nearly addressed the comments from @seabbs . An issue to note, which will be an issue to address, is that in an experiment with using the |
Pertinent to above and @seabbs comment on using After these changes, an experiment to not use initial values to start NUTS still converges onto the correct posterior distributions, so at least in this context this is a big improvement in the stability of the model. I know @damonbayer has wrangled with this in Bayesian inference of ODEs so I wonder what your thoughts on using special functions to stabilise sampling are? I've left the initial condition finding steps in the note because it still improves things a bit, and I think is a good example. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks great now. Love it. Looking forward to v2 once we had added more integration to EpiAware.
Benchmark resultJudge resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTarget
Baseline
Target resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Baseline resultBenchmark Report for /home/runner/work/Rt-without-renewal/Rt-without-renewalJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfo
Runtime information
|
This PR replicates two parts of the demonstration of epidemic inference using stan from Chatzilena et al 2019:
In both cases, I use a combination of SciML and EpiAware functionality which worked well.
Closes #446
Aside comments
The prior predictive checking (not done in Chatzilena et al) reveals very poor prior specification in the sense that I don't think anyone would reasonably expect the prior marginal median for numbers of ill children.
This means that I use a combination of MLE and MAP optimisation to start the NUTS sampling in reasonable parts of parameter space; in practise I would change the prior specifications but I felt this deviated from replication/comparison.
Doing the pre-sampling optimisation means that we are getting to reasonable ess is about 10-60x faster than reported in the paper (acknowledging here that the paper is 4 years old and other confounding factors). However, without this optimization the method is prone to stochastic failures. Therefore, pre-sampling steps are quite important to consider going forwards if we use this kind of approach in applications.