GitHub - MyDarapy/SmolLM-experiments-with-grouped-query-attention: (Unofficial) building Hugging Face SmolLM-blazingly fast and remarkably powerful small language model with PyTorch implementation of grouped query attention (GQA)

MyDarapy / SmolLM-experiments-with-grouped-query-attention Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

(Unofficial) building Hugging Face SmolLM-blazingly fast and remarkably powerful small language model with PyTorch implementation of grouped query attention (GQA)

0 stars 0 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 8 Commits
Embeddings.py		Embeddings.py
SwiGLU FFN.py		SwiGLU FFN.py
grouped query attention.py		grouped query attention.py
transformer.py		transformer.py

About

(Unofficial) building Hugging Face SmolLM-blazingly fast and remarkably powerful small language model with PyTorch implementation of grouped query attention (GQA)

transformer attention smol huggingface ml-efficiency llm grouped-query-attention smol-lm huggingface-smol-lm

Activity

0 stars

2 watching

0 forks

Report repository