initial conditions. For Hopfield Networks, however, this is not the case - the dynamical trajectories always converge to a fixed point attractor state. This property is achieved because these equations are specifically engineered so that they have an underlying energy function The terms grouped into square brackets represent a Legendre transform of the Lagrangian function with respect to the states of the neurons. If the Hessian matrices of the Lagrangian functions are positive semi-definite, the energy function is guaranteed to decrease on the dynamical trajectory This property makes it possible to prove that the system of dynamical equations describing temporal evolution of neurons' activities will eventually reach a fixed point attractor state. In certain situations one can assume that the dynamics of hidden neurons equilibrates at a much faster time scale compared to the feature neurons, . In this case the steady state solution of the second equation in the system () can be used to express the currents of the hidden units through the outputs of the feature neurons. This makes it possible to reduce the general theory () to an effective theory for feature neurons only. The resulting effective update rules and the energies for various common choices of the Lagrangian functions are shown in Fig.2. In the case of log-sum-exponential Lagrangian function the update rule (if applied once) for the states of the feature neurons is the attention mechanism commonly used in many modern AI systems (see Ref. for the derivation of this result from the continuous time formulation). Relationship to classical Hopfield network with continuous variables Classical formulation of continuous Hopfield Networks can be understood as a special limiting case of the modern Hopfield networks with one hidden layer. Continuous Hopfield Networks for neurons with graded response are typically described by the dynamical equations and the energy function where , and is the inverse of the activation function . This model is a special limit of the class of models that is called models A, with the following choice of the Lagrangian functions that, according to the definition (), leads to the activation functions If we integrate out the hidden neurons the system of equations () reduces to the equations on the feature neurons () with , and the general expression for the energy () reduces to the effective energy While the first two terms in equation () are the same as those in equation (),