Pinned Loading

1. EleutherAI/lm-evaluation-harness EleutherAI/lm-evaluation-harness Public

   A framework for few-shot evaluation of language models.

   Python 7.3k 2k

2. transformer_shmap transformer_shmap Public

   Tensor Parallelism with JAX + Shard Map

   Python 11 1

3. ZeRO-transformer ZeRO-transformer Public

   Two implementations of ZeRO-1 optimizer sharding in JAX

   Python 13

4. fast_sequential_scan fast_sequential_scan Public

   A fast sequential scan on GPU

   Cuda

5. tritonformer tritonformer Public

   Differentiable transformer in Triton, matching the performance of PyTorch + cuDNN/cuBLAS

   Python 4

6. hawk-pytorch hawk-pytorch Public

   PyTorch implementation of Hawk from "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models" (https://arxiv.org/abs/2402.19427). Compatible with torch.compile.

   Python