This repository hosts the next-generation neural architecture for PNB (Project Neuro-Bit). The goal is to integrate four cutting-edge, non-mainstream neural paradigms to build an Intel XMX-accelerated, Cache-less, Bio-inspired Dynamic Neural Architecture.
This project is built upon four fundamental pillars, designed to overcome the limitations of traditional deep learning (memory bottlenecks, lack of adaptability):
- Concept: Infinite-depth networks using only a single layer's parameters. Solves for the fixed point (Equilibrium) via recursive iteration.
- Advantage: O(1) Memory. Perfect for "Cache-less" operation as it eliminates the need to store intermediate activations for hundreds of layers.
- Concept: Reinterpreting the Transformer Attention mechanism as a massive-capacity "Associative Memory Network".
- Advantage: One-shot Retrieval. Provides the network with powerful memory and pattern completion capabilities.
- Concept: Introduces "Astrocytes" (Star-shaped Glial Cells) as a third component. They do not transmit signals but listen to neuronal activity and dynamically regulate synaptic weights (Plasticity).
- Advantage: Adaptability. Allows the network to self-regulate to the "Edge of Chaos", the state of maximal computational power.
- Concept: A physics-based energy learning rule to replace Backpropagation.
- Advantage: Bio-plausible & Ultra-low Memory. No need to store a massive Computation Graph for differentiation; learning is driven by the energy difference between two states (free phase vs. clamped phase).
- Backbone: DEQ for infinite-depth recursive structure (XMX-accelerated dense compute).
- Memory: Hopfield Layer replacing traditional layers for associative recall.
- Dynamics: Astrocyte mechanism for dynamic weight regulation.
- Learning: EqProp for training, enabling true Cache-less On-chip Learning.
- Status: Pre-Alpha / Architectural Design Phase
- Target Hardware: Intel Arc / Xeon (XMX Acceleration)