How fast is the importance sampling?

[drphilmarshall]

At the moment we are doing experiments on mock datasets of 200 supernovae. What would it take to repeat these on 100,000 supernovae? Some benchmarking tests could be interesting - and let us know if we need to be thinking about going to NERSC any time soon.

[kbarbary]

Three things along this line:

• We should make it easy to sample individual SNe in parallel (e.g., the trivially parallelizable part of the problem).
• For the global sampling, use the threads argument in emcee. Also, we should investigate other sampling methods or samplers. Maybe @MichelleLochner has some input here...
• It would be nice to have an option to use a gaussian approximation instead of the samples for each SN. I imagine this would be WAY faster.

[kbarbary]

Cc @sofiatti

[drphilmarshall]

Re your third bullet: integrating out the individual supernova parameters is, in general, expensive. It would be interesting to know when (ie at what number of samples per SN) a Gaussian approximation starts to out-perform importance sampling. Also, note that the conditional PDF (for the true parameters of the SNe) will also have to be Gaussian (or a GMM) for the integrals to stay analytic - and even then, you still have to do the required covariance matrix manipulations. So I'm not sure it'll be WAY faster, unless we are trying to use some very large number of samples.

Which brings me to: can we do the benchmark tests as a function of Nsamples as well as Nsne, please? We saw some signs in the MBI project that you could get by with very few samples per object indeed...