changelog.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

Added

Changed

Deprecated

Fixed

0.0.5 - 2024-11-04

This release adds a minor improvement of the structured matrix sub-module.

Added

• .shape property of a StructuredMatrix (PR)

0.0.4 - 2024-07-03

This release adds a new interface function to SINGD's structured matrix sub-module. This is a useful addition for our SIRFShampoo optimizer. The SINGD optimizer is not affected by this feature.

Added

• Method to extract a structured matrix from X @ X.T (PR)

0.0.3 - 2024-06-24

This release requires torch>=2.2.0 and ships internal refactorings to update newly supported operations in half precision.

Added

• Functionality to compute the Frobenius norm of a structured matrix (interface PR, efficient implementation PR)

Changed

• Update .bib entry to ICML 2024 (PR)

Deprecated

• Use torch>=2.2.0 and update its newly added support for matmul, conv1d, and einsum in half precision on CPU (PR)

Fixed

No bug fixes

0.0.2 - 2023-12-11

This release adds support for neural networks with in-place activations and also comes with performance improvements for convolutions, as well as improvements regarding numerical stability in half precision.

Added

New features:

• Support Conv2d layers with dilation != 1 (PR)
• Support neural networks with inplace activation functions (PR)

Performance improvements:

• Speed up input processing for Conv2d with groups != 1 (PR)
• Speed up computation of averaged patches for KFAC-reduce (kfac_approx='reduce') in Conv2d using the tensor network approach of Dangel, 2023 (PR)

Changed

• Move un-scaling of H_C into the update step to improve numerical stability when using half precision + gradient scaling (PR)

Deprecated

No deprecations

Fixed

No bug fixes

0.0.1 - 2023-10-31

Initial release