Authors: Alessandro Fella, Lucia Depaoli, Lorenzo Mandolito and Simone Mistrali
Project on Hopfield Network Model for Laboratory of Computational Physics at UniPD.
A Hopefiled network is a form of recurrent artificial neural network and a type of spin glass system (Disordered system) based on the Ising model. Hopfield networks serve as content-addressable ("associative") memory systems with binary threshold nodes, or with continuous variables. Hopfield networks also provide a model for understanding human memory.
This spin system has many nontrivial minima, it is a neural
network where patterns are intentionally generated by an external agent, by encoding them in the coupling matrix between neurons, which are biologically realized by axons. In this case each
represents a synaptic efficiency, i.e. the kind of transmission of the axon from
neuron to neuron
. We will map neurons to spins and
to their coupling, thus translating patterns into energetic minima.
Biologically, a neuron is activated when the incoming electrical signal overcomes a threshold. We define neuron states as
and a local field collecting all other impulses as
Given a pattern , with
, this is essentially a spin configuration. We can retrieve the couplings matrix in the following way:
where in the number of patterns. For semplicity we apply the so called Hebb’s rule, so we neglect the auto-interation
.
We use the following update rule (i.e sign rule):
In this project we basically try to figure out if this type of model works fine using different patterns (1-D and 2-D), i.e given a corrupted pattern (i.e. a pattern
with probability q<1 , otherwise
with probability
) it can reconstruct the original pattern
with the sign rule.
The first part shows a simple example using random generated patterns made by and
numbers and random corrupted patterns, defined by a grade of corruption
respect to the original ones. We have see the averaged mean of pattern retrived by varying the number of patterns given different probability of corruption.
The second part is focused on a simple 2-D toy model, we use as a pattern the number of the handwritten MINST database. We try different approach:
- We use the vanilla database with poor results.
- We crop the images and in this case we can see that the model can correct retrieve up to 5 images using
images.
- We crop and vary the number of patterns.
- We crop the images to
pixels and apply a so called nearest neighbor
, we set to 0 the interaction between pixels which are farther away than a given distance.
To properly run this project please clone this directory and from a terminal run this command:
python3 -m pip install -r requirements.txt
Contact Simone Mistrali at simone.mistrali at gmail.com for any questions or comments.
To properly see the formulas please set the light theme on Github settings.