Skip to content

Keras AdamW 1.1
Compare
Choose a tag to compare
@OverLordGoldDragon OverLordGoldDragon released this 27 Oct 16:28
· 38 commits to master since this release
v1.1
d9a36ed
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

New features:

  • Full compatibility with TensorFlow 2.0.0 and Keras 2.3.0 (keras + tf.keras)
  • Full compatibility with TensorFlow 1.14.0 and Keras 2.2.5 (keras + tf.keras)^
  • Run-based weight decay normalization scheme, normalizing over arbitrary # of iterations independent of LR scheduler (e.g. over all epochs) (total_iterations_wd)
  • utils now contain all common optimizer ops, e.g. _apply_weight_decays()
  • Reference implementation compatible with tf.python.keras, TF2 + Keras 2.3.0; see docstring
  • Cleaner code

^ - also compatible w/ TensorFlow 1.13.0 & 1.15.0, Keras 2.2.3-2.2.4

BUGFIXES:

  • lr_multipliers were not being applied for AdamW
  • SGDW was missing K.symbolic wrapper
  • and total_iterations != 0 was missing for NadamW and SGDW, yielding NaN optimizer weights for total_iterations = 0
  • TF reset seed method was incompatible w/ TF 2.0.0; added support for both

Misc changes:

  • Updated README.md w/ info on total_iterations_wd
  • Added README.md to keras_adamw on module version selection
  • Added unit tests for respective optimizers
  • Revamped test.sh; each test has a dedicated folder to ease coverage isolation
  • Revamped .travis.yml:
    • Added versioning builds w/ TF_KERAS, TF_EAGER, TF_VERSION, and KERAS_VERSION flags
    • Removed boolean KERAS flag; repo is based on Keras
    • Reorder buggy setuptools & conda update -all commands
  • Moved testing module requirements from requirements.txt to requirements-test.txt
Assets 2
Loading