Skip to content

Latest commit

 

History

History
189 lines (132 loc) · 8.06 KB

CHANGELOG.md

File metadata and controls

189 lines (132 loc) · 8.06 KB

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

unreleased

Added

  • Add support for MLFlow logging and metrics tracking. #77 @khintz

  • Add support for multi-node training. #103 @simonkamuk @sadamov

  • Add option to clamp output prediction using limits specified in config file #92 @SimonKamuk

Fixed

  • Only print on rank 0 to avoid duplicates of all print statements. #103 @simonkamuk @sadamov

  • Fix MLFlow exception import introduced in #77. #111 @observingClouds

  • Fix duplicate tensor copy to CPU #106 @observingClouds

  • Fix bug where the inverse_softplus used in clamping caused nans in the gradients #123 @SimonKamuk

  • Add standardization to state diff stats from mdp datastore #122 @SimonKamuk

  • Set ci/cd badges to refer to the new test matrix #130 @SimonKamuk

Maintenance

  • update AWS GPU ci/cd to use ami with larger (200GB) root volume and ensure nvme drive is used for pip venvn #126, @leifdenby

  • update ci/cd testing setup to install torch version compatible with neural-lam dependencies #115, @leifdenby

  • switch to new npyfiles MEPS and mdp DANRA test datasets which are coincident in time and space (on cropped ~100x100 grid-point domain) #110, @leifdenby

  • use dynamic versioning based on git tags and commit hashes #118, @observingClouds

  • add detect_anomaly=True to pl.Trainer in test_training.py #124, @SimonKamuk

v0.3.0

This release introduces Datastores to represent input data from different sources (including zarr and numpy) while keeping graph generation within neural-lam.

Added

  • Introduce Datastores to represent input data from different sources, including zarr and numpy. #66 @leifdenby @sadamov

  • Implement standardization of static features when loaded in ARModel #96 @joeloskarsson

Fixed

  • Fix wandb environment variable disabling wandb during tests. Now correctly uses WANDB_MODE=disabled. #94 @joeloskarsson

  • Fix bugs introduced with datastores functionality relating visualation plots #91 @leifdenby

v0.2.0

Added

  • Added tests for loading dataset, creating graph, and training model based on reduced MEPS dataset stored on AWS S3, along with automatic running of tests on push/PR to GitHub, including push to main branch. Added caching of test data to speed up running tests. #38 #55 @SimonKamuk

  • Replaced constants.py with data_config.yaml for data configuration management #31 @sadamov

  • new metrics (nll and crps_gauss) and metrics submodule, stddiv output option c14b6b4 @joeloskarsson

  • ability to "watch" metrics and log c14b6b4 @joeloskarsson

  • pre-commit setup for linting and formatting #6, #8 @sadamov, @joeloskarsson

  • added github pull-request template to ease contribution and review process #53, @leifdenby

  • ci/cd setup for running both CPU and GPU-based testing both with pdm and pip based installs #37, @khintz, @leifdenby

Changed

  • Clarify routine around requesting reviewer and assignee in PR template #74 @joeloskarsson

  • Argument Parser updated to use action="store_true" instead of 0/1 for boolean arguments. (#72) @ErikLarssonDev

  • Optional multi-core/GPU support for statistics calculation in create_parameter_weights.py #22 @sadamov

  • Robust restoration of optimizer and scheduler using ckpt_path #17 @sadamov

  • Updated scripts and modules to use data_config.yaml instead of constants.py #31 @sadamov

  • Added new flags in train_model.py for configuration previously in constants.py #31 @sadamov

  • moved batch-static features ("water cover") into forcing component return by WeatherDataset #13 @joeloskarsson

  • change validation metric from mae to rmse c14b6b4 @joeloskarsson

  • change RMSE definition to compute sqrt after all averaging #10 @joeloskarsson

Removed

  • WeatherDataset(torch.Dataset) no longer returns "batch-static" component of training item (only prev_state, target_state and forcing), the batch static features are instead included in forcing #13 @joeloskarsson

Maintenance

  • simplify pre-commit setup by 1) reducing linting to only cover static analysis excluding imports from external dependencies (this will be handled in build/test cicd action introduced later), 2) pinning versions of linting tools in pre-commit config (and remove from requirements.txt) and 3) using github action to run pre-commit. #29 @leifdenby

  • change copyright formulation in license to encompass all contributors #47 @joeloskarsson

  • Fix incorrect ordering of x- and y-dimensions in comments describing tensor shapes for MEPS data #52 @joeloskarsson

  • Cap numpy version to < 2.0.0 (this cap was removed in #37, see below) #68 @joeloskarsson

  • Remove numpy < 2.0.0 version cap #37 @leifdenby

  • turn neural-lam into a python package by moving all *.py-files into the neural_lam/ source directory and updating imports accordingly. This means all cli functions are now invoke through the package name, e.g. python -m neural_lam.train_model instead of python train_model.py (and can be done anywhere once the package has been installed). #32, @leifdenby

  • move from requirements.txt to pyproject.toml for defining package dependencies. #37, @leifdenby

  • Add slack and new publication info to readme #78 @joeloskarsson

v0.1.0

First tagged release of neural-lam, matching Oskarsson et al 2023 publication (https://arxiv.org/abs/2309.17370)