You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Co-authored-by: Diego Cammarano <diegocamm@gmail.com>
Co-authored-by: Bettina Gier <gier@uni-bremen.de>
Co-authored-by: Manuel Schlund <32543114+schlunma@users.noreply.github.com>
Co-authored-by: Bouwe Andela <b.andela@esciencecenter.nl>
Co-authored-by: Manuel Schlund <manuel.schlund@dlr.de>
Some of the results of this diagnostics can also be reproduced utilizing
8
+
python diagnostics:
9
+
Portrait plot: :ref:`recipe_portrait`
10
+
Monitoring: :ref:`recipe_monitor`
11
+
6
12
Overview
7
13
--------
8
14
9
-
The goal is to create a standard recipe for the calculation of performance metrics to quantify the ability of the models to reproduce the climatological mean annual cycle for selected "Essential Climate Variables" (ECVs) plus some additional corresponding diagnostics and plots to better understand and interpret the results.
15
+
The goal is to create a standard recipe for the calculation of performance
16
+
metrics to quantify the ability of the models to reproduce the climatological
17
+
mean annual cycle for selected "Essential Climate Variables" (ECVs) plus some
18
+
additional corresponding diagnostics and plots to better understand and
19
+
interpret the results.
20
+
21
+
The recipe can be used to calculate performance metrics at different vertical
22
+
levels (e.g., 5, 30, 200, 850 hPa as in
23
+
`Gleckler et al. (2008) <http://dx.doi.org/10.1029/2007JD008972>`_) and in
24
+
different regions. As an additional reference, we consider
25
+
`Righi et al. (2015) <https://doi.org/10.5194/gmd-8-733-2015>`_.
10
26
11
-
The recipe can be used to calculate performance metrics at different vertical levels (e.g., 5, 30, 200, 850 hPa as in `Gleckler et al. (2008) <http://dx.doi.org/10.1029/2007JD008972>`_ and in different regions. As an additional reference, we consider `Righi et al. (2015) <https://doi.org/10.5194/gmd-8-733-2015>`_.
12
27
13
28
Available recipes and diagnostics
14
29
-----------------------------------
@@ -21,12 +36,19 @@ Recipes are stored in recipes/
21
36
22
37
Diagnostics are stored in diag_scripts/perfmetrics/
23
38
24
-
* main.ncl: calculates and (optionally) plots annual/seasonal cycles, zonal means, lat-lon fields and time-lat-lon fields. The calculated fields can also be plotted as difference w.r.t. a given reference dataset. main.ncl also calculates RMSD, bias and taylor metrics. Input data have to be regridded to a common grid in the preprocessor. Each plot type is created by a separated routine, as detailed below.
39
+
* main.ncl: calculates and (optionally) plots annual/seasonal cycles, zonal
40
+
means, lat-lon fields and time-lat-lon fields. The calculated fields can also
41
+
be plotted as difference w.r.t. a given reference dataset. main.ncl also
42
+
calculates RMSD, bias and taylor metrics. Input data have to be regridded to
43
+
a common grid in the preprocessor. Each plot type is created by a separated
44
+
routine, as detailed below.
25
45
* cycle.ncl: creates an annual/seasonal cycle plot.
26
46
* zonal.ncl: creates a zonal (lat-pressure) plot.
27
47
* latlon.ncl: creates a lat-lon plot.
28
-
* cycle_latlon.ncl: precalculates the metrics for a time-lat-lon field, with different options for normalization.
29
-
* collect.ncl: collects and plots the metrics previously calculated by cycle_latlon.ncl.
48
+
* cycle_latlon.ncl: precalculates the metrics for a time-lat-lon field, with
49
+
different options for normalization.
50
+
* collect.ncl: collects and plots the metrics previously calculated by
* stippling: uses stippling to mark statistically significant differences (default: False = mask out non-significant differences in gray)
55
-
* show_global_avg: diplays the global avaerage of the input field as string at the top-right of lat-lon plots (default: False)
56
-
* annots: choose the annotation style, e.g. ```alias``` which would display the alias of the dataset as title (applies to plot_type zonal and cycle_zonal)
79
+
* stippling: uses stippling to mark statistically significant differences
80
+
(default: False = mask out non-significant differences in gray)
81
+
* show_global_avg: displays the global avaerage of the input field as string
82
+
at the top-right of lat-lon plots (default: False)
83
+
* annots: choose the annotation style, e.g. ```alias``` which would display
84
+
the alias of the dataset as title (applies to plot_type zonal and cycle_zonal)
57
85
* metric: chosen grading metric(s) (if calc_grading is True)
58
86
* normalization: metric normalization (for RMSD and BIAS metrics only)
59
87
* abs_levs: list of contour levels for absolute plot
@@ -114,8 +142,8 @@ User settings in recipe
114
142
115
143
*Optional settings (scripts)*
116
144
117
-
* label_lo: adds lower triange for values outside range
118
-
* label_hi: adds upper triange for values outside range
145
+
* label_lo: adds lower triangle for values outside range
146
+
* label_hi: adds upper triangle for values outside range
119
147
* cm_interval: min and max color of the color table
120
148
* cm_reverse: reverses the color table
121
149
* sort: sorts datasets in alphabetic order (excluding MMM)
@@ -157,14 +185,21 @@ Variables
157
185
Observations and reformat scripts
158
186
---------------------------------
159
187
160
-
The following list shows the currently used observational data sets for this recipe with their variable names and the reference to their respective reformat scripts in parentheses. Please note that obs4MIPs data can be used directly without any reformating. For non-obs4MIPs data use `esmvaltool data info DATASET` or see headers of cmorization scripts (in `/esmvaltool/cmorizers/data/formatters/datasets/
161
-
<https://github.com/ESMValGroup/ESMValTool/blob/main/esmvaltool/cmorizers/data/formatters/datasets/>`_) for downloading and processing instructions.
188
+
The following list shows the currently used observational data sets for this
189
+
recipe with their variable names and the reference to their respective reformat
190
+
scripts in parentheses. Please note that obs4MIPs data can be used directly
191
+
without any reformatitng. For non-obs4MIPs data use `esmvaltool data info DATASET`
192
+
or see headers of cmorization scripts (in `/esmvaltool/cmorizers/data/formatters/datasets/
@@ -190,27 +225,38 @@ The following list shows the currently used observational data sets for this rec
190
225
References
191
226
----------
192
227
193
-
* Gleckler, P. J., K. E. Taylor, and C. Doutriaux, Performance metrics for climate models, J. Geophys. Res., 113, D06104, doi: 10.1029/2007JD008972 (2008).
228
+
* Gleckler, P. J., K. E. Taylor, and C. Doutriaux, Performance metrics for climate models, J.
and Cionni, I.: Quantitative evaluation of ozone and selected climate parameters in a set of EMAC simulations,
233
+
Geosci. Model Dev., 8, 733, doi: 10.5194/gmd-8-733-2015 (2015).
194
234
195
-
* Righi, M., Eyring, V., Klinger, C., Frank, F., Gottschaldt, K.-D., Jöckel, P., and Cionni, I.: Quantitative evaluation of oone and selected climate parameters in a set of EMAC simulations, Geosci. Model Dev., 8, 733, doi: 10.5194/gmd-8-733-2015 (2015).
Annual cycle of globally averaged temperature at 850 hPa (time period 1980-2005) for different CMIP5 models (historical simulation) (thin colored lines) in comparison to ERA-Interim (thick yellow line) and NCEP-NCAR-R1 (thick black dashed line) reanalysis data.
242
+
Annual cycle of globally averaged temperature at 850 hPa (time period 1980-2005)
243
+
for different CMIP5 models (historical simulation) (thin colored lines) in comparison to
244
+
ERA-Interim (thick yellow line) and NCEP-NCAR-R1 (thick black dashed line) reanalysis data.
Taylor diagram of globally averaged temperature at 850 hPa (ta) and longwave cloud radiative effect (lwcre) for different CMIP5 models (historical simulation, 1980-2005). Reference data (REF) are ERA-Interim for temperature (1980-2005) and CERES-EBAF (2001-2012) for longwave cloud radiative effect.
249
+
Taylor diagram of globally averaged temperature at 850 hPa (ta) and longwave cloud
250
+
radiative effect (lwcre) for different CMIP5 models (historical simulation, 1980-2005).
251
+
Reference data (REF) are ERA-Interim for temperature (1980-2005) and CERES-EBAF (2001-2012)
Difference in annual mean of zonally averaged temperature (time period 1980-2005) between the CMIP5 model MPI-ESM-MR (historical simulation) and ERA-Interim. Stippled areas indicdate differences that are statistically significant at a 95% confidence level.
257
+
Difference in annual mean of zonally averaged temperature (time period 1980-2005) between the
258
+
CMIP5 model MPI-ESM-MR (historical simulation) and ERA-Interim. Stippled areas indicdate
259
+
differences that are statistically significant at a 95% confidence level.
Relative space-time root-mean-square deviation (RMSD) calculated from the climatological seasonal cycle of CMIP5 simulations. A relative performance is displayed, with blue shading indicating better and red shading indicating worse performance than the median of all model results. A diagonal split of a grid square shows the relative error with respect to the reference data set (lower right triangle) and the alternative data set (upper left triangle). White boxes are used when data are not available for a given model and variable.
270
+
Relative space-time root-mean-square deviation (RMSD) calculated from the climatological
271
+
seasonal cycle of CMIP5 simulations. A relative performance is displayed, with blue shading
272
+
indicating better and red shading indicating worse performance than the median of all model results.
273
+
A diagonal split of a grid square shows the relative error with respect to the reference data set
274
+
(lower right triangle) and the alternative data set (upper left triangle).
275
+
White boxes are used when data are not available for a given model and variable.
0 commit comments