Skip to content

Latest commit

 

History

History
43 lines (29 loc) · 1.3 KB

README.md

File metadata and controls

43 lines (29 loc) · 1.3 KB
Raw

(Unofficial) Mindspore Optimizer

An implementation of the Mindspore optimizer that functions similarly to PyTorch’s optimizer.

Highlights:

  • support native FP16 / BF16 training, or AMP training.
  • support group learning rate / group weight decay.
  • some optimizers may offer better speed compared with official ones. (e.g., adafactor), Feel free to experiment!

Support Optimizers

Environment

  • python >= 3.9
  • mindspore >= 2.2.14

Test

To test the optimizer, follow these steps using the CIFAR-10 dataset:

  1. Download the CIFAR-10 binary dataset and unzip it with the following command:
wget -c https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/cifar-10-binary.tar.gz -P tests/data/ && tar xzf tests/data/cifar-10-binary.tar.gz -C tests/data/
  1. Install mindcv >= 0.3.0 by
pip install mindcv
  1. Start training by running:
PYTHONPATH=$(pwd):$PYTHONPATH python tests/train.py -n adafactor

During training, the results will be displayed in the terminal, and an additional loss curve plot named loss.jpg will be saved.