This release follows the advent of Gemma 2 2b, the smallest Gemma 2 model available to date. Its small size is perfect for limited hardware resources or as a speed-up companion to the larger variants through assisted generation!
Highlights:
- 💎 Gemma 2 2b support
- ⚡
speed
preset (usestorch.compile
) - 👪 CLI uses the 2b to speed up the larger models, through assisted generation
Thank you to all contributors since the previous release: @sanchit-gandhi @gante @shapito27 @Vaibhavs10