Skip to content

Files

Latest commit

 

History

History
62 lines (43 loc) · 5.57 KB

HelpDoc.md

File metadata and controls

62 lines (43 loc) · 5.57 KB

How to run the backend scripts

This page includes additional information on how to run each of the included scripts to update the OOI Data Team Portal database. For more information, please see Readme.md.

Note, these scrips make use of the library files found in datateam directory. Most functions in the library are fairly straightforward, loading data from CSV files from various associated GitHib repos (see the readme), and additional methods for inserting the parsed information into the database.

The scripts below provide high-level methods to batch load all of the data at once.

load-annotations.py

This script retrieves the latest annotations from the OOI Data Portal (aka uFrame) and adds them to the Data Team Portal database. In order for this script to work, you will need to specify your OOINet API username and password in config.cfg, as well as your mysql database configuration.

Options

  • -s --server Specifies which database connection to use from the configuration file. Specify either "development" (the default) or "production".

load-dailystatys.py

This script loads the daily instrument statistics information calculated by the scripts available in the ooi-stats repo.

Options

  • -s --server Specifies which database connection to use from the configuration file. Specify either "development" (the default) or "production".
  • -d --directory The directory where the stats output can be found.

Example

./load-dailystats.py -s production -d /home/knuth/ooi_stats/stats/output/

load-data.py

This script can be used to load and update most of the metadata needed for the OOI Data Team portal. It uses information found in various other GitHub repositories managed by the OOI program.

Options

Example

./load-data.py -s production -o regions

load-status.py

This script will ping the OOI Data Portal (via the API) to request the last 48 hours of data for each instrument. If data is returned, it will update the current_status column in the Data Team Portal.

Note that this script only checks instruments that do not already have a current_status defined in infrastructure/instruments.csv. This allows non-real-time instruments (e.g. lost, suspended, etc.) to be excluded from the status check. It can also be used to exclude non-essential instruments (e.g. engineering ones).

The preferred_stream and preferred_parameter columns in the instruments file are used to specify which stream/parameter should be checked to verify whether an instrument is operation in real-time. The method (telemetered or streamed) is automatically determined by the script.

In order for this script to work, you will need to specify your OOINet API username and password in config.cfg, as well as your mysql database configuration.

This script will also output a dated CSV file containing the status of all instruments checked.

Options

  • -s --server Specifies which database connection to use from the configuration file. Specify either "development" (the default) or "production".