Describe the task. It can be a feature, documentation, etc.

Currently we can score ensembles in the evaluation package, and generate line plots of average scores across e.g. forecast steps. It might be a nice feature to plot skill (and spread) spatially, to evaluate whether ensembles are skillful in expected spatial areas, and whether the uncertainty learned by the model is also appropriate. Ideally this could be passed as an option to the eval config (plot_spread_skill or plot_skill maybe : true), and maps would be produced with e.g. ensemble mean RMSE for the globe, averaged across the number of samples specified. Same for spread, sqrt of average ensemble variance per point/grid cell.

It is possible that these maps, where we average across samples (to get e.g. average ensemble mean RMSE spatially) would look quite odd/not be useful for datasets with different points for each sample. We might need to do some coarse regridding and averaging? Perhaps can think about this later.

Some very draughty, out-of-date code is in shmh40/dev/spread-skill-plot.

Hedgedoc URL, if you are keeping notes, plots, logs in hedgedoc.

No response

Area

• datasets, data readers, data preparation and transfer
• model
• science
• infrastructure and engineering
• evaluation, export and visualization
• documentation