Skip to content

Commit

Permalink
revised files to be pre-commit tests compliant
Browse files Browse the repository at this point in the history
  • Loading branch information
@lee1043
lee1043 committed Nov 3, 2021
1 parent 788f711 commit f038f99
Show file tree
Hide file tree
Showing 23 changed files with 235 additions and 235 deletions.
2 changes: 1 addition & 1 deletion LICENSE
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
BSD 3-Clause License

Copyright (c) 2021, Lawrence Livermore National Security, LLC
Copyright (c) 2021, Lawrence Livermore National Security, LLC
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
6 changes: 3 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<img src="share/pcmdi/PMPLogo_1359x1146px_300dpi.png" width="15%" height="15%" align="right" />
<h5 align="right"></h5>
<h5 align="right"></h5>

PCMDI metrics package (PMP)
======
Expand All @@ -11,9 +11,9 @@ PCMDI metrics package (PMP)
<!--[![CircleCI](https://circleci.com/gh/PCMDI/pcmdi_metrics.svg?style=svg)](https://circleci.com/gh/PCMDI/pcmdi_metrics) -->
<!--[![Coverage Status](https://coveralls.io/repos/github/PCMDI/pcmdi_metrics/badge.svg)](https://coveralls.io/github/PCMDI/pcmdi_metrics) -->

The PCMDI metrics package is used to provide "quick-look" objective comparisons of Earth System Models (ESMs) with one another and available observations. Results are produced in the context of all model simulations contributed to CMIP6 and earlier CMIP phases. Among other purposes, this enables modeling groups to evaluate changes during the development cycle in the context of the structural error distribution of the multi-model ensemble. Currently, the comparisons emphasize metrics of large- to global-scale annual cycle and both tropcial and extra-tropical modes of variability. Ongoing work in v1.x development branches include established statistics for ENSO, MJO, regional monsoons, and high frequency characteristics of simulated precipitation.
The PCMDI metrics package is used to provide "quick-look" objective comparisons of Earth System Models (ESMs) with one another and available observations. Results are produced in the context of all model simulations contributed to CMIP6 and earlier CMIP phases. Among other purposes, this enables modeling groups to evaluate changes during the development cycle in the context of the structural error distribution of the multi-model ensemble. Currently, the comparisons emphasize metrics of large- to global-scale annual cycle and both tropcial and extra-tropical modes of variability. Ongoing work in v1.x development branches include established statistics for ENSO, MJO, regional monsoons, and high frequency characteristics of simulated precipitation.

**PCMDI uses the PMP to produce [quick-look simulation summaries across generations of CMIP](https://cmec.llnl.gov/results/physical.html)**
**PCMDI uses the PMP to produce [quick-look simulation summaries across generations of CMIP](https://cmec.llnl.gov/results/physical.html)**

The metrics package consists of four parts: 1) Analysis software, 2) an observationally-based database of global (or near global, land or ocean) [time series and climatologies](https://github.com/PCMDI/PCMDIobs-cmor-tables/tree/master/catalogue), 3) a database of performance metrics computed for CMIP models and 4) [package documentation and interactive demos](http://pcmdi.github.io/pcmdi_metrics/).

Expand Down
98 changes: 49 additions & 49 deletions cmec/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,72 +1,72 @@
README
## CMEC driver
The cmec-driver software can be obtained from the [cmec-driver repository](https://github.com/cmecmetrics/cmec-driver). A [wiki](https://github.com/cmecmetrics/cmec-driver/wiki) provides more instructions about the installation and set up. While some instructions are provided below for running cmec-driver, the wiki has a more complete set of [workflow instructions](https://github.com/cmecmetrics/cmec-driver/wiki/PCMDI-Metrics-Package).
The cmec-driver software can be obtained from the [cmec-driver repository](https://github.com/cmecmetrics/cmec-driver). A [wiki](https://github.com/cmecmetrics/cmec-driver/wiki) provides more instructions about the installation and set up. While some instructions are provided below for running cmec-driver, the wiki has a more complete set of [workflow instructions](https://github.com/cmecmetrics/cmec-driver/wiki/PCMDI-Metrics-Package).

## Mean Climate
1. Edit settings in cmec.json.
There are a few parameters you do NOT need to set:
`reference_data_path` is assumed to be $CMEC_OBS_DATA
`test_data_path` is assumed to be $CMEC_MODEL_DATA
`metrics_output_path` is assumed to be $CMEC_WK_DIR
There are a few parameters you do NOT need to set:
`reference_data_path` is assumed to be $CMEC_OBS_DATA
`test_data_path` is assumed to be $CMEC_MODEL_DATA
`metrics_output_path` is assumed to be $CMEC_WK_DIR
Set `compute_climatologies: true` to generate on-the-fly AC files from timeseries.
2. Move or link observational data to your chosen "obs" dir
For example:
`ln -s PCMDIobs2_clims obs`
3. Move or link model data to your chosen "model" directory
For example:
`ln -s model_data_directory model`
4. If the observational data file structure has changed, edit the observational data catalogue. Put the path to the new catalogue in the `custom_observations` parameter in cmec.json.
5. Run cmec driver: `python cmec-driver.py run -obs <obs dir> <model dir> <output dir> PMP/mean_climate`
2. Move or link observational data to your chosen "obs" dir
For example:
`ln -s PCMDIobs2_clims obs`
3. Move or link model data to your chosen "model" directory
For example:
`ln -s model_data_directory model`
4. If the observational data file structure has changed, edit the observational data catalogue. Put the path to the new catalogue in the `custom_observations` parameter in cmec.json.
5. Run cmec driver: `python cmec-driver.py run -obs <obs dir> <model dir> <output dir> PMP/mean_climate`

## Modes of variability
1. Edit settings in cmec.json.
`modpath` and `reference_data_path` are relative to the CMEC $CMEC_MODEL_DATA and $CMEC_OBS_DATA directories, respectively.
The `ObsUnitsAdjust` and `ModUnitsAdjust` tuples should be encased in quotes (e.g. `"ObsUnitsAdjust": "(True, 'divide', 100.0)"`)
2. Move or link observational data to your chosen "obs" dir
3. Move or link model data to your chosen "model" dir
4. Run cmec driver: `python cmec-driver.py run -obs <obs dir> <model dir> <output dir> PMP/variability_modes`
## Modes of variability
1. Edit settings in cmec.json.
`modpath` and `reference_data_path` are relative to the CMEC $CMEC_MODEL_DATA and $CMEC_OBS_DATA directories, respectively.
The `ObsUnitsAdjust` and `ModUnitsAdjust` tuples should be encased in quotes (e.g. `"ObsUnitsAdjust": "(True, 'divide', 100.0)"`)
2. Move or link observational data to your chosen "obs" dir
3. Move or link model data to your chosen "model" dir
4. Run cmec driver: `python cmec-driver.py run -obs <obs dir> <model dir> <output dir> PMP/variability_modes`

## MJO
1. Edit settings in cmec.json
2. Move or link observational data to your chosen "obs" dir
3. Move or link model data to your chosen "model" dir
4. Run cmec driver: `python cmec-driver.py run -obs <obs dir> <model dir> <output dir> PMP/mjo`
1. Edit settings in cmec.json
2. Move or link observational data to your chosen "obs" dir
3. Move or link model data to your chosen "model" dir
4. Run cmec driver: `python cmec-driver.py run -obs <obs dir> <model dir> <output dir> PMP/mjo`

## Monsoon (Wang)
1. Edit settings in cmec.json.
`test_data_path` and `reference_data_path` are relative to the CMEC $CMEC_MODEL_DATA and $CMEC_OBS_DATA directories, respectively.
The `threshold` variable needs to be in the same units as your input data. The default is 2.5 mm / 86400 = 2.894e-05 for model input with units of kg m-2 s-1.
2. Move or link observational data to your chosen "obs" dir
3. Move or link model data to your chosen "model" dir
4. Run cmec driver: `python cmec-driver.py run -obs <obs dir> <model dir> <output dir> PMP/monsoon_wang`
1. Edit settings in cmec.json.
`test_data_path` and `reference_data_path` are relative to the CMEC $CMEC_MODEL_DATA and $CMEC_OBS_DATA directories, respectively.
The `threshold` variable needs to be in the same units as your input data. The default is 2.5 mm / 86400 = 2.894e-05 for model input with units of kg m-2 s-1.
2. Move or link observational data to your chosen "obs" dir
3. Move or link model data to your chosen "model" dir
4. Run cmec driver: `python cmec-driver.py run -obs <obs dir> <model dir> <output dir> PMP/monsoon_wang`

## Monsoon (Sperber)
1. Edit settings in cmec.json
`modpath` and `reference_data_path` are relative to the CMEC $CMEC_MODEL_DATA and $CMEC_OBS_DATA directories, respectively.
`modpath_lf` and `reference_data_lf` path are also relative to the above directories.
The `ObsUnitsAdjust` and `ModUnitsAdjust` tuples should be encased in quotes (e.g. `"ObsUnitsAdjust": "(True, 'divide', 100.0)"`)
3. Move or link observational data to your chosen "obs" dir
4. Move or link model data to your chosen "model" dir
5. Run cmec driver: python `cmec-driver.py run -obs <obs dir> <model dir> <output dir> PMP/monsoon_sperber`
1. Edit settings in cmec.json
`modpath` and `reference_data_path` are relative to the CMEC $CMEC_MODEL_DATA and $CMEC_OBS_DATA directories, respectively.
`modpath_lf` and `reference_data_lf` path are also relative to the above directories.
The `ObsUnitsAdjust` and `ModUnitsAdjust` tuples should be encased in quotes (e.g. `"ObsUnitsAdjust": "(True, 'divide', 100.0)"`)
3. Move or link observational data to your chosen "obs" dir
4. Move or link model data to your chosen "model" dir
5. Run cmec driver: python `cmec-driver.py run -obs <obs dir> <model dir> <output dir> PMP/monsoon_sperber`

## Diurnal Cycle of Precipitation
1. Edit settings in cmec.json
`filename_template` is relative to your $CMEC_MODEL_DATA directory. This should be 3hr precipitation data.
`filename_template` is relative to your $CMEC_MODEL_DATA directory. This should be 3hr precipitation data.
Do not set `modpath` as this is your $CMEC_MODEL_DATA directory.
2. Move or link model data to your chosen "model" dir
3. Run cmec driver: `python cmec-driver.py run <model dir> <output dir> PMP/diurnal_cycle`
2. Move or link model data to your chosen "model" dir
3. Run cmec driver: `python cmec-driver.py run <model dir> <output dir> PMP/diurnal_cycle`

### Running multiple metrics
Follow all but the final step for your chosen metrics. To run all the metrics via cmec-driver, run:
`cmec-driver.py run -obs <obs dir> <model dir> <output dir> <list metrics names>`
Follow all but the final step for your chosen metrics. To run all the metrics via cmec-driver, run:
`cmec-driver.py run -obs <obs dir> <model dir> <output dir> <list metrics names>`
For example:
`python cmec-driver.py run -obs obs model output PMP/mjo PMP/monsoon_wang PMP/diurnal_cycle`
It is not guaranteed that the metrics will run in the same order that they are listed in your run statement.
`python cmec-driver.py run -obs obs model output PMP/mjo PMP/monsoon_wang PMP/diurnal_cycle`
It is not guaranteed that the metrics will run in the same order that they are listed in your run statement.

### Other tips for the configuration file
True and False values in the JSON standard are represented by `true` and `false`. For the PMP metrics, users can also use the strings `"True"` and `"False"`, though this is not recommended.
The PMP parameter files accept python functions and datatypes that are not valid JSON objects. Any parameter values that are function calls, tuples, or other non-JSON types should be encased in quotes.
The "datetime", "glob", and "os" packages are available in the parameter files generated during this workflow. These packages can be used to set parameter values in cmec.json. For example, a user can set `"case_id": "datetime.datetime.now().strftime('v%Y%m%d')"` to have the case_id reflect the date.
True and False values in the JSON standard are represented by `true` and `false`. For the PMP metrics, users can also use the strings `"True"` and `"False"`, though this is not recommended.
The PMP parameter files accept python functions and datatypes that are not valid JSON objects. Any parameter values that are function calls, tuples, or other non-JSON types should be encased in quotes.
The "datetime", "glob", and "os" packages are available in the parameter files generated during this workflow. These packages can be used to set parameter values in cmec.json. For example, a user can set `"case_id": "datetime.datetime.now().strftime('v%Y%m%d')"` to have the case_id reflect the date.

# Information for Developers
This folder contains settings files and scripts as an interface between the PMP and cmec-driver. It is organized so that each metric is a standalone cmec-driver configuration within the PMP module. Each metric has its own folder containing a settings JSON, a driver script, and a metadata script. There is an additional "scripts" folder for shared code. Finally, there is a file called "contents.json" that is located at the top level of the PMP repo.
Expand All @@ -76,7 +76,7 @@ CMEC driver first looks at the contents.json file to find the settings file for

## Steps for adding a new metric
1. Create a folder for the metric under cmec/.
2. Write a driver bash script for the metric. This script should contain the entire workflow for generating the metric, including activating the PMP conda environment, generating a parameter file, running the metric, and generating metadata, as needed (for example, see mean_climate/pmp_mean_climate_driver.sh.
2. Write a driver bash script for the metric. This script should contain the entire workflow for generating the metric, including activating the PMP conda environment, generating a parameter file, running the metric, and generating metadata, as needed (for example, see mean_climate/pmp_mean_climate_driver.sh.
3. Write a settings file (for example, see mean_climate/pmp_mean_climate.json):
- The required keys are "settings", "varlist", and "obslist".
- The required keys under settings are "name", "long_name", and "driver".
Expand All @@ -85,6 +85,6 @@ CMEC driver first looks at the contents.json file to find the settings file for
5. Add the settings file path to pcmdi_metrics/contents.json in the "contents" list.
6. If there is any special processing that needs to happen to the input parameters, add that code to scripts/pmp_param_generator.py
7. If any other scripts are created to help run the metric in cmec-driver, save them under scripts/ or in the metric folder.
8. If the metric does not output its own metadata JSON, write a script to generate one (for example, see mean_climate/mean_climate_output.py). This script should be included in the driver script workflow (for example, see mean_climate/pmp_mean_climate_driver.sh).
8. If the metric does not output its own metadata JSON, write a script to generate one (for example, see mean_climate/mean_climate_output.py). This script should be included in the driver script workflow (for example, see mean_climate/pmp_mean_climate_driver.sh).
9. (Optional) Add an html page that displays the results of your metric.
10. Test by running your new metric in cmec-driver. Check that it produces the output files that you expect and that they are correctly documented in your output metadata file.
2 changes: 1 addition & 1 deletion cmec/mjo/mjo_output.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
"filename": result_file.split(".")[0]+"_cmec.json",
"long_name": "MJO model statistics (CMEC format)",
"description": "MJJASO and NDJFMA season MJO metrics in CMEC formatted JSON"
}
}
}

# Add optional netcdf and plot info
Expand Down
16 changes: 8 additions & 8 deletions doc/jupyter/Demo/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ The jupyter dashboard will be launched in your browser. From there, navigate to
The notebooks listed below contain command line examples that can be copied from the notebook, customized, and run in the terminal. These commands are found in cells with a *%%bash* header.

## What's in the notebooks
[Demo 0](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_0_download_data.ipynb): Set data and output directories. Download sample data and use cdscan to create xml. Generate parameter files.
[Demo 1](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_1_mean_climate.ipynb): Run the Mean Climate driver and explore the many options for customization.
[Demo 1a](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_1a_compute_climatologies.ipynb): Learn how to create annual climatology files for running the mean climate driver with custom data.
[Demo 2a](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_2a_monsoon_wang.ipynb): Run the Monsoon (Wang) driver and its options.
[Demo 2b](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_2b_monsoon_sperber.ipynb): Run the Monsoon (Sperber) driver with options. Maps the monsoon regions and displays output graphic.
[Demo 3](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_3_diurnal_cycle.ipynb): Demonstrates the full workflow for running the Diurnal Cycle metrics.
[Demo 4](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_4_modes_of_variability.ipynb): Install scipy and eofs in the kernel environment, run the modes of variability driver with the NAM, and view an output image.
[Demo 5](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_5_mjo_metrics.ipynb): Run the MJO driver with options and view an output image.
[Demo 0](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_0_download_data.ipynb): Set data and output directories. Download sample data and use cdscan to create xml. Generate parameter files.
[Demo 1](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_1_mean_climate.ipynb): Run the Mean Climate driver and explore the many options for customization.
[Demo 1a](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_1a_compute_climatologies.ipynb): Learn how to create annual climatology files for running the mean climate driver with custom data.
[Demo 2a](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_2a_monsoon_wang.ipynb): Run the Monsoon (Wang) driver and its options.
[Demo 2b](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_2b_monsoon_sperber.ipynb): Run the Monsoon (Sperber) driver with options. Maps the monsoon regions and displays output graphic.
[Demo 3](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_3_diurnal_cycle.ipynb): Demonstrates the full workflow for running the Diurnal Cycle metrics.
[Demo 4](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_4_modes_of_variability.ipynb): Install scipy and eofs in the kernel environment, run the modes of variability driver with the NAM, and view an output image.
[Demo 5](https://github.com/PCMDI/pcmdi_metrics/blob/master/doc/jupyter/Demo/Demo_5_mjo_metrics.ipynb): Run the MJO driver with options and view an output image.
2 changes: 1 addition & 1 deletion doc/jupyter/Demo/basic_annual_cycle_param.py.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import os
vars = ['rlut']

# START AND END DATES FOR CLIMATOLOGY
start = '2003-01'
start = '2003-01'
end = '2018-12'

# INPUT DATASET - CAN BE MODEL OR OBSERVATIONS
Expand Down
2 changes: 1 addition & 1 deletion doc/jupyter/Demo/basic_mov_param.py.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ results_dir = os.path.join(
'$OUTPUT_DIR$',
"%(case_id)")

# OUTPUT OPTIONS
# OUTPUT OPTIONS
nc_out = True # Write output in NetCDF
plot = True # Create map graphics
update_json = True # False
Expand Down
Loading

0 comments on commit f038f99

Please sign in to comment.