🎉 Modern CUDA Learn Notes with PyTorch: CUDA Cores, Tensor Cores, fp32/tf32, fp16/bf16, fp8/int8, flash_attn, rope, sgemm, hgemm, sgemv, warp/block reduce, elementwise, softmax, layernorm, rmsnorm.

Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.

An implementation of SGEMV with performance comparable to cuBLAS.

Matilda is a library to repeatedly multiply a constant matrix with a variable vector

Highly optimized DGEMV on CPU with both serial and parallel performance better than MKL and OpenBLAS.