rules There are various different learning rules that can be used to store information in the memory of the Hopfield network. It is desirable for a learning rule to have both of the following two properties: Local: A learning rule is local if each weight is updated using information available to neurons on either side of the connection that is associated with that particular weight. Incremental: New patterns can be learned without using information from the old patterns that have been also used for training. That is, when a new pattern is used for training, the new values for the weights only depend on the old values and on the new pattern. These properties are desirable, since a learning rule satisfying them is more biologically plausible. For example, since the human brain is always learning new concepts, one can reason that human learning is incremental. A learning system that was not incremental would generally be trained only once, with a huge batch of training data. Hebbian learning rule for Hopfield networks The Hebbian Theory was introduced by Donald Hebb in 1949, in order to explain "associative learning", in which simultaneous activation of neuron cells leads to pronounced increases in synaptic strength between those cells. It is often summarized as "Neurons that fire together, wire together. Neurons that fire out of sync, fail to link". The Hebbian rule is both local and incremental. For the Hopfield networks, it is implemented in the following manner, when learning binary patterns: where represents bit i from pattern . If the bits corresponding to neurons i and j are equal in pattern , then the product will be positive. This would, in turn, have a positive effect on the weight and the values of i and j will tend to become equal. The opposite happens if the bits corresponding to neurons i and j are different. Storkey learning rule This rule was introduced by Amos Storkey in 1997 and is both local and incremental. Storkey also showed that a Hopfield network trained using this rule has a greater capacity than a corresponding network trained using the Hebbian rule. The weight matrix of an attractor neural network is said to follow the Storkey learning rule if it obeys: where is a form of local field at neuron i. This learning rule is local, since the synapses take into account only neurons at their sides. The rule