This is the repo for the MixKABRN Neural Network (Mixture of Kolmogorov-Arnold Bit Retentive Networks), and an attempt at first adapting it for training on text, and later adjust it for other modalities.
ai neural-network model architecture moe bitnet mixture-of-experts ai-models llms retnet retentive-network kolmogorov-arnold-networks
-
Updated
May 14, 2024 - Python