the third terms look superficially different. In equation () it is a Legendre transform of the Lagrangian for the feature neurons, while in () the third term is an integral of the inverse activation function. Nevertheless, these two expressions are in fact equivalent, since the derivatives of a function and its Legendre transform are inverse functions of each other. The easiest way to see that these two terms are equal explicitly is to differentiate each one with respect to . The results of these differentiations for both expressions are equal to . Thus, the two expressions are equal up to an additive constant. This completes the proof that the classical Hopfield Network with continuous states is a special limiting case of the modern Hopfield network () with energy (). General formulation of the modern Hopfield network Biological neural networks have a large degree of heterogeneity in terms of different cell types. This section describes a mathematical model of a fully connected modern Hopfield network assuming the extreme degree of heterogeneity: every single neuron is different. Specifically, an energy function and the corresponding dynamical equations are described assuming that each neuron has its own activation function and kinetic time scale. The network is assumed to be fully connected, so that every neuron is connected to every other neuron using a symmetric matrix of weights , indices and enumerate different neurons in the network, see Fig.3. The easiest way to mathematically formulate this problem is to define the architecture through a Lagrangian function that depends on the activities of all the neurons in the network. The activation function for each neuron is defined as a partial derivative of the Lagrangian with respect to that neuron's activity From the biological perspective one can think about as an axonal output of the neuron . In the simplest case, when the Lagrangian is additive for different neurons, this definition results in the activation that is a non-linear function of that neuron's activity. For non-additive Lagrangians this activation function can depend on the activities of a group of neurons. For instance, it can contain contrastive (softmax) or divisive normalization. The dynamical equations describing temporal evolution of a given neuron are given by This equation belongs to the class of models called firing rate models in neuroscience. Each neuron collects the axonal outputs from all the neurons, weights them with the synaptic coefficients and produces its own