📚Modern CUDA Learn Notes with PyTorch: Tensor/CUDA Cores, 📖150+ CUDA Kernels with PyTorch bindings, 📖HGEMM/SGEMM (95%~99% cuBLAS performance), 📖100+ LLM/CUDA Blogs.

Root Mean Square Layer Normalization

Code for the paper "On the Expressivity Role of LayerNorm in Transformers' Attention" (Findings of ACL'2023)

Implement layer normalization GRU in pytorch

Some common CUDA kernel implementations (Not the fastest).

MNIST Digit Prediction using Batch Normalization, Group Normalization, Layer Normalization and L1-L2 Regularizations

WGAN with feedback from discriminator& LayerNorm instead of BatchNorm

Fundamentals of Artificial Intelligence and Deep Learning Frameworks