Alpha v0.2

[LouisCastricato]

Complete revamp of our initial release.

New features:

• Hydra models, 20x faster than vanilla PPO with minimal performance hits at large scales
• Massively revamped API, significantly less boiler plate.
• Save/load callbacks.
• Greatly improved orchestrator.
• Better commented RL code, easier to understand whats going on.
• Cool examples, including architext and simulacra.
• Better extendability, and standardized styling.

Features coming soon:

• Megatron support! we're already working on this.
• More interesting examples that are relevant to production use cases of TRLX.
• Better integration of W&B, including sweeps.
• Evaluation and benchmarking.

:)

Autogenerated release notes below:

What's Changed

• Fix typo by @mrm8488 in Fix typo #2
• Create LICENSE by @LouisCastricato in Create LICENSE #3
• QOL fixes by @LouisCastricato in QOL fixes #5
• stage ilql by @reciprocated in stage ilql #6
• Adds style file and reward function capabilities to ppo orchestrator by @LouisCastricato in Adds style file and reward function capabilities to ppo orchestrator  #8
• Update ppo value head + print logs by @Dahoas in Update ppo value head + print logs #11
• Make ilql respect the config & remove sin by @reciprocated in Make ilql respect the config & remove sin #22
• Docs by @shahbuland in Docs #31
• Implemented hydra heads + adaptive kl by @Dahoas in Implemented hydra heads + adaptive kl #33
• Add pre-commit with black by @cat-state in Add pre-commit with black #36
• [update] Improve package setup by @jon-tow in [update] Improve package setup #42
• Add initial issue templates by @jon-tow in Add initial issue templates #45
• Some readme improvements by @thedch in Some readme improvements #44
• Add initial GitHub workflows by @jon-tow in Add initial GitHub workflows #43
• [docs] Add CONTRIBUTING.md by @jon-tow in [docs] Add CONTRIBUTING.md #52
• Simplify api by @reciprocated in Simplify api #24

New Contributors

• @mrm8488 made their first contribution in Fix typo #2
• @LouisCastricato made their first contribution in Create LICENSE #3
• @reciprocated made their first contribution in stage ilql #6
• @Dahoas made their first contribution in Update ppo value head + print logs #11
• @shahbuland made their first contribution in Docs #31
• @cat-state made their first contribution in Add pre-commit with black #36
• @jon-tow made their first contribution in [update] Improve package setup #42
• @thedch made their first contribution in Some readme improvements #44

Full Changelog: https://github.com/CarperAI/trlx/commits/v0.2

---

This discussion was created from the release Alpha v0.2.