Hello, I'm Shivendra. I like to code and make videos. Check out some of my previous work at Vakya
1- SmallLanguageModel [completed, closed]: Making a LLM from scratch all the way from generating raw training data to tokenizing it, creating a model & then training it.
2- Enigma-1.5b [completed, closed]: Transformer model trained on raw DNA data to predict the next letter of the DNA.
3- Axon [completed, updating]: Numpy from scratch in python without any external library. Also with a tiny scalar level autograd axon.micro
4- Axgrad [in progress]: Lightweight Tensor manipulation library like PyTorch written in c/c++, cuda & python.
5- Axon.drop [in progress]: A small & lightweight, experimental Tensor manipulation library wrapped on top of Scalar level autograd, written in c/c++, cuda & python.
6- WebGraze [completed, updating]: A Python-based library for webscraping & generating/downloading data from various sources on the internet for training ml models.
7- Synapse [completed, to be updated]: A free platform for streaming music & audio/podcasts, based on Youtube V3 API.
8- Micrograd.c [completed, no updates]: Micrograd by Karpathy written in C & C++.
9- AVA [in progress, backlogged]: A multimodal ai system inspired by AVA from Ex-Machina, but currently more like OpenAi's 4o, (I started working on it prior to the model launch).
10- Shred [in progress]: BPE tokenizer library using C based code at the backend for fast execution speeds with Python interface for usage.
11- Enigma2 [in progress, backlogged]: Second version of Enigma to predict & classify DNA & proteins more accurately using Transformers.
Instagram | Twitter | LinkedIn | Youtube