Continuous Time Analysis of Momentum Methods

Abstract: Gradient descent-based optimization methods underpin the parameter training of neural networks, and hence comprise a significant component in the impressive test results found in a number of applications. Introducing stochasticity is key to their success in practical problems, and there is some understanding of the role of stochastic gradient descent in this context. Momentum modifications of gradient descent such as Polyak's Heavy Ball method (HB) and Nesterov's method of accelerated gradients (NAG), are also widely adopted. In this work our focus is on understanding the role of momentum in the training of neural networks, concentrating on the common situation in which the momentum contribution is fixed at each step of the algorithm. To expose the ideas simply we work in the deterministic setting.Our approach is to derive continuous time approximations of the discrete algorithms; these continuous time approximations provide insights into the mechanisms at play within the discrete algorithms. We prove three such approximations. Firstly we show that standard implementations of fixed momentum methods approximate a time-rescaled gradient descent flow, asymptotically as the learning rate shrinks to zero; this result does not distinguish momentum methods from pure gradient descent, in the limit of vanishing learning rate. We then proceed to prove two results aimed at understanding the observed practical advantages of fixed momentum methods over gradient descent. We achieve this by proving approximations to continuous time limits in which the small but fixed learning rate appears as a parameter. Furthermore in a third result we show that the momentum methods admit an exponentially attractive invariant manifold on which the dynamics reduces, approximately, to a gradient flow with respect to a modified loss function.

\section{Introduction}
\label{sec:I}

\subsection{Background and Literature Review}
\label{ssec:B}

At the core of many machine learning tasks is solution of the optimization problem
\begin{equation}
\label{eq:min}
\mathop{\mathrm{arg\,min\,\,}}_{u \in \mathbb{R}^d} \Phi(u)
\end{equation}
where \(\Phi: \mathbb{R}^d \to \mathbb{R}\) is an objective (or loss) function that is, in general, non-convex
and differentiable. Finding global minima of such objective functions is an important
and challenging task with a long history, one in which the use of stochasticity
has played a prominent role for many decades, with papers in the early
development of machine learning \cite{geman1987stochastic,styblinski1990experiments},
together with concomitant theoretical analyses
for both discrete \cite{bertsimas1993simulated}
and continuous problems \cite{kushner1987asymptotic,kushner2012stochastic}.
Recent successes in the training of deep neural networks have built on this
older work, leveraging the enormous computer power now available, together with
empirical experience about good design choices for the architecture of the networks;
reviews may be found in \cite{deeplearningbook,deeplearningnature}.
Gradient descent plays a prominent conceptual role in many algorithms, following
from the observation that the equation
\begin{equation}
\label{eq:GD}
\frac{du}{dt}=-\nabla \Phi(u)
\end{equation}
will decrease $\Phi$ along trajectories.
The most widely adopted methods use stochastic gradient decent (SGD), a concept
introduced in \cite{robbinsmonroe}; the basic idea is to use
gradient decent steps based on a noisy approximation to the gradient of \(\Phi\).
Building on deep work in the convex optimization literature, momentum-based
modifications to stochastic gradient decent have also become widely used in
optimization. Most notable
amongst these momentum-based methods are the Heavy Ball Method (HB),
due to \cite{polyakhevyball}, and Nesterov's method of accelerated gradients (NAG) \cite{originalnesterov}.
To the best of our knowledge, the first application of HB to neural network training appears in \cite{firsthb}. More
recent work, such  as \cite{importanceofinitmom}, has even argued for the indispensability of such momentum
based methods for the field of deep learning.

From these two basic variants on gradient decent, there have come a plethora of
adaptive methods, incorporating momentum-like ideas, such
as Adam \cite{adam}, Adagrad \cite{adagrad}, and RMSProp \cite{rmsprop}. There is no
consensus on which method performs best and results vary based on application. The recent work of
\cite{marginaladaptive} argues that the rudimentary, non-adaptive schemes SGD, HB, and NAG result
in solutions with the greatest generalization performance for supervised learning applications with
deep neural network models.

There is a natural physical analogy for momentum methods, namely that they relate to
a damped second order Hamiltonian dynamic with potential $\Phi$:
\begin{equation}
\label{eq:HAM}
m\frac{d^2u}{dt^2}+\gamma(t)\frac{du}{dt}+\nabla \Phi(u)=0.
\end{equation}
This perspective goes back to Polyak's original work \cite{polyakhevyball,polyakoptimization}
and was further expanded on in \cite{wronghb}, although no proof was given.
For NAG, the work of \cite{odenesterov} proves that the method approximates a damped Hamiltonian system
of precisely this form, with a time-dependent damping coefficient.
The analysis in \cite{odenesterov} holds  when
the momentum factor is chosen according to the rule
\begin{equation}
\label{eq:lambda_time}
\lambda = \lambda_n =  \frac{n}{n + 3},
\end{equation}
where $n$ is the iteration count; this choice was proposed
in the original work of \cite{originalnesterov} and results in
a choice of $\lambda$
which is asymptotic to $1$.  In the setting where
\(\Phi\) is \(\mu\)-strongly convex, it is proposed in
\cite{nesterovbook} that the momentum factor
is fixed and chosen close to $1$; specifically it is proposed that
\begin{equation}
\label{eq:lambda_mu}
\lambda = \frac{1 - \sqrt{\mu h}}{1+ \sqrt{\mu h}}
\end{equation}
where \(h > 0\) is the time-step (learning rate).
In \cite{wilsoneq}, a limiting equation for both HB and NAG of the
form
\[\ddot{u} + 2\sqrt{\mu}\dot{u} + \nabla \Phi(u) = 0\]
is derived under the assumption that \(\lambda\) is fixed with respect
to iteration number $n$, and dependent on the time-step $h$ as specified
in \eqref{eq:lambda_mu};
convergence is obtained to order \(\mathcal{O}(h^{1/2})\).
Using insight from this limiting equation it is possible to
choose the optimal value of $\mu$ to maximize the convergence
rate in the neighborhood of a locally strongly convex objective
function.  Further related work is developed in \cite{shi2018understanding}
where separate limiting equations for HB and NAG  are derived both in
the cases of \(\lambda\)
given by \eqref{eq:lambda_time} and \eqref{eq:lambda_mu}, obtaining
convergence to order \(\mathcal{O}(h^{3/2})\). Much work has
also gone into analyzing these methods in the discrete setting, without
appeal to the continuous time limits, see
\cite{hu2017dissipativity,lessard2016analysis}, as well as in the
stochastic setting, establishing
how the effect on the generalization error, for
example,  \cite{gadat2018stochastic,loizou2017linearly,yang2016unified}.
In this paper, however, our focus is on the use of continuous time limits
as a methodology to explain optimization algorithms.

In many machine learning applications, especially for deep learning,
NAG and HB are often used with a constant momentum factor \(\lambda\)
that is chosen independently of the iteration count $n$
(contrary to \eqref{eq:lambda_time}) and independently of the
learning rate $h$ (contrary to \eqref{eq:lambda_mu}).
In fact, popular books on the subject such as \cite{deeplearningbook}
introduce the methods in this way, and popular articles, such as \cite{originalresnet} to name one of many,
simply state the value of the constant momentum factor used in their experiments.
Widely used deep learning libraries such as Tensorflow \cite{tensorflow} and
PyTorch \cite{pytorch} implement the methods with a fixed choice
of momentum factor.
Momentum based methods used in this way,
with fixed momentum, have not been carefully analyzed.
We will undertake
such an analysis, using ideas from numerical analysis, and in particular
the concept of {\em modified equations} \cite{griffiths1986scope,chartier2007numerical}
and from the theory of {\em attractive invariant
manifolds} \cite{hirsch2006invariant,wiggins2013normally};
both ideas are explained in the text \cite{stuart1998dynamical}.
It is noteworthy that the {\em  high resolution ODE approximation}
described in \cite{shi2018understanding} may be viewed as
a rediscovery of the method of modified equations.
We emphasize the fact that our work is not at odds
with any previous analyses of these methods, rather, we consider a
setting which is widely adopted in deep learning applications
and has not been subjected to continuous time analysis to date.

\begin{remark}
Since publication of this article in \cite{kovachki2021continuous},
we became aware of related, and earlier, work by \cite{farazmand2018multiscale}.
Farazmand starts from the Bregman Lagrangian introduced in
\cite{wibisono2016variational} and uses ideas from geometric singular
perturbation theory to derive an invariant manifold. The work leads
to a more general description of the invariant manifold than the one
given by our equation \eqref{eq:approxg2}. Farazmand's work was published
in \cite{farazmand2020multiscale}.
\end{remark}

\subsection{Our Contribution}

We study momentum-based optimization algorithms for the minimization task
\eqref{eq:min}, with learning rate independent momentum, fixed at every
iteration step, focusing on deterministic methods for clarity of exposition.
Our approach is to derive continuous time approximations of the discrete
algorithms; these continuous time approximations provide insights into the
mechanisms at play within the discrete algorithms.
We prove three such approximations. The first shows that the asymptotic
limit of the momentum methods, as learning rate approaches zero,
is simply a rescaled gradient flow \eqref{eq:GD}. The second two
approximations include small perturbations to the rescaled gradient flow,
on the order of the learning rate, and give insight into the behavior
of momentum methods when implemented with momentum and fixed learning
rate. Through these approximation theorems, and accompanying
numerical experiments, we make the following contributions to the
understanding of momentum methods as often implemented within
machine learning:

\begin{itemize}

\item We show that momentum-based methods
with a fixed momentum factor, satisfy, in the continuous-time limit obtained
by sending the learning rate to zero,
a rescaled version of the gradient flow equation \eqref{eq:GD}.

\item We show that such methods also approximate a damped
Hamiltonian system of the form \eqref{eq:HAM}, with small mass $m$ (on
the order of the learning rate) and constant damping $\gamma(t)=\gamma$;
this approximation has the same order of accuracy as the approximation
of the rescaled equation \eqref{eq:GD} but provides a better
qualitative understanding of the fixed learning rate momentum
algorithm in its transient phase.

\item We also show that, for the approximate Hamiltonian system, the dynamics
admit an exponentially attractive invariant manifold,
locally representable as a graph mapping co-ordinates to their velocities.
The map generating this graph describes
a gradient flow in a potential which is a small (on the order of the learning rate)
perturbation of $\Phi$ -- see \eqref{eq:mp}; the correction to the potential is convexifying,
does not change the global minimum, and provides insight into the
fixed learning rate momentum algorithm beyond its initial transient
phase.

\item We provide numerical experiments which illustrate the foregoing considerations, for simple linear test problems,
and for the MNIST digit classification problem; in the latter case we
consider SGD and thereby demonstrate that the conclusions of our theory
have relevance for understanding the stochastic setting as well.

\end{itemize}

Taken together our results are interesting because they demonstrate that
the popular belief that (fixed) momentum methods resemble the dynamics induced
by \eqref{eq:HAM} is misleading. Whilst it is true, the mass in the approximating
equation is small and
as a consequence understanding the dynamics as gradient flows \eqref{eq:GD},
with modified potential, is more instructive.
In fact, in the first application of HB to neural networks
described in \cite{firsthb}, the authors state that
\say{[their] experience has been that [one] get[s] the same solutions by setting [the momentum factor to zero]
and reducing the size of [the learning rate].} However our theorems should
not be understood to imply that there is no practical difference between momentum
methods (with fixed learning rate) and SGD. There is indeed a practical difference
as has been demonstrated in numerous papers throughout the machine learning
literature, and our experiments in Section \ref{ssec:DL} further confirm this.
We show that while these methods have the same transient dynamics, they are approximated differently. Our results demonstrate that, although momentum methods
behave like a gradient descent algorithm, asymptotically, this algorithm
has a modified potential. Furthermore, although this modified potential  \eqref{eq:approxg2}
is on the order of the learning rate, the fact that the learning rate is often
chosen as large as possible, constrained by numerical stability, means that
the correction to the potential may be significant. Our results may be
interpreted as indicating that the practical success of momentum methods stems
from the fact that they provide a more stable
discretization to \eqref{eq:GD} than the forward Euler method employed in SGD. The damped Hamiltonian
dynamic \eqref{eq:visco}, as well the modified potential, give insight into
how this manifests. Our work gives further theoretical justification for the exploration of
the use of different numerical integrators for the purposes of optimization such as those performed
in \cite{scieur2017integration,betancourt2018symplectic,runge-kuttadisc}.

While our analysis is confined to the non-stochastic case to simplify the exposition,
the results will, with some care, extend to the stochastic setting using ideas from
averaging and homogenization \cite{PavS} as well as continuum analyses of SGD as in \cite{weinansgd,sgdsemigroup};
indeed, in the stochastic setting, sharp uniform in
time error estimates are to be expected for empirical
averages \cite{mat10,bach17}.
To demonstrate that our analysis is
indeed relevant in the stochastic setting, we train a deep autoencoder with mini-batching (stochastic)
and verify that our convergence results still hold. The details of this experiment
are given in section \ref{ssec:DL}.
Furthermore we also confine our analysis to fixed learning rate, and impose global
bounds on the relevant derivatives of $\Phi$; this further simplifies the exposition of
the key ideas, but is not essential to them; with considerably more analysis the ideas
exposed in this paper will transfer to adaptive time-stepping methods and much less
restrictive classes of \(\Phi\).

The paper is organized as follows. Section \ref{sec:M}
introduces the optimization procedures and states the
convergence result to a rescaled gradient flow. In section \ref{sec:ME} we
derive the modified, second-order equation and
state convergence of the schemes to this equation.
Section \ref{sec:IM}  asserts the existence of an attractive invariant manifold,
demonstrating that it results in a gradient flow with respect to a small
perturbation of $\Phi$. In section \ref{ssec:DL}, we train a deep autoencoder,
showing that our results hold in a stochastic setting with Assumption \ref{assump:one} violated.
We conclude in section \ref{sec:C}.
All proofs of theorems are given in the appendices so that the ideas of the
theorems can be presented clearly within the main body of the text.

\subsection{Notation}
\label{ssec:N}

We use $|\cdot|$ to denote the Euclidean norm on $\mathbb{R}^d.$
We define \(f : \mathbb{R}^d \to \mathbb{R}^d\) by \(f(u) \coloneqq - \nabla \Phi(u)\) for any \(u \in \mathbb{R}^d\).
Given parameter $\lambda \in [0,1)$ we define
 \(\bar{\lambda} \coloneqq (1-\lambda)^{-1}\).

For two Banach spaces \(A,B\), and $A_0$ a subset in $A$, we denote by \(C^k(A_0;B)\) the set of \(k\)-times continuously differentiable functions
with domain \(A_0\) and range \(B\). For a function \(u \in C^k(A_0;B)\), we let
\(D^ju\) denote its \(j\)-th (total) Fr{\'e}chet derivative for \(j=1,\dots,k\).
For a function \({u \in C^k([0,\infty), \mathbb{R}^d)}\), we denote its
derivatives by \(\frac{du}{dt}, \frac{d^2 u}{dt^2},\) etc. or equivalently by \(\dot{u}, \ddot{u},\) etc.

To simplify our proofs, we make the following assumption
about the objective function.

\begin{assumption}
\label{assump:one}
Suppose \(\Phi \in C^3(\mathbb{R}^d;\mathbb{R})\) with uniformly bounded derivatives. Namely,
there exist constants \(B_0,B_1,B_2 > 0\) such that
\[\|D^{j-1}f\| = \|D^j \Phi \| \leq B_{j-1}\]
for \(j=1,2,3\) where \(\|\cdot\|\) denotes any appropriate operator norm.
\end{assumption}

We again stress that this assumption is not key to developing the ideas in this work, but is
rather a simplification used to make our results global. Without Assumption \ref{assump:one},
and no further assumption on \(\Phi\) such as convexity, one could only hope to give local results
i.e. in the neighborhood of a critical point of \(\Phi\). Such analysis could indeed
be carried out (see for example \cite{carr2012applications}), but we choose not to do so here for
the sake of clarity of exposition. In section \ref{ssec:DL}, we give a practical example
where this assumption is violated and yet the behavior is as predicted
by our theory.

Finally we observe that the nomenclature ``learning rate'' is now prevalent
in machine learning, and so we use it in this paper; it refers to the
object commonly referred to as ``time-step'' in the field of numerical analysis.

\section{Momentum Methods and Convergence to Gradient Flow}
\label{sec:M}

In subsection \ref{ssec:MR1} we state Theorem \ref{thm:conv_to_gf}
concerning the convergence of a class of momentum methods to a rescaled gradient
flow. Subsection \ref{ssec:Link} demonstrates that the HB and NAG methods
are special cases of our general class of momentum methods, and gives
intuition for proof of Theorem \ref{thm:conv_to_gf}; the proof itself
is given in Appendix A.  Subsection \ref{ssec:N1} contains a
numerical illustration of  Theorem \ref{thm:conv_to_gf}.

\subsection{Main Result}
\label{ssec:MR1}

The standard Euler discretization of \eqref{eq:GD} gives the discrete time optimization
scheme
\begin{equation}
\label{eq:gd}
\mathsf{u}_{n+1} = \mathsf{u}_n + h f(\mathsf{u}_n), \quad n=0,1,2,\dots\,.
\end{equation}
Implementation of this scheme requires an initial guess \(\mathsf{u}_0 \in \mathbb{R}^d\).
For simplicity we consider a fixed learning rate \(h > 0\).
Equation \eqref{eq:GD} has a unique solution
\(u \in C^3([0,\infty);\mathbb{R}^d)\) under Assumption \ref{assump:one} and
for $u_n=u(nh)$
$$\sup_{0 \le nh \le T} |\mathsf{u}_n-u_n| \le C(T)h;$$
see \cite{stuart1998dynamical}, for example.

In this section we consider a general class of momentum methods for the minimization
task \eqref{eq:min} which can be written in the form, for some \(a \geq 0\) and
$\lambda \in (0,1)$,
\begin{align}
\label{eq:general_discrete}
\begin{split}
\mathsf{u}_{n+1} &= \mathsf{u}_n + \lambda(\mathsf{u}_n - \mathsf{u}_{n-1}) + hf(\mathsf{u}_n + a(\mathsf{u}_n - \mathsf{u}_{n-1})), \quad n=0,1,2,\dots\,, \\
\mathsf{u}_1 &= \mathsf{u}_0 + hf(\mathsf{u}_0)\,.
\end{split}
\end{align}
Again, implementation of this scheme requires an an initial guess
\(\mathsf{u}_0 \in \mathbb{R}^d\).
The parameter choice \(a=0\) gives HB and \(a = \lambda\) gives NAG.
In Appendix A we prove the following:

\begin{theorem}
\label{thm:conv_to_gf}
Suppose Assumption \ref{assump:one} holds and let \(u \in C^3([0,\infty);\mathbb{R}^d)\)
be the solution to
\begin{align}
\label{eq:rgf}
\begin{split}
&\dtfrac{u} = -(1-\lambda)^{-1}\nabla \Phi(u) \\
&u(0) = \mathsf{u}_0
\end{split}
\end{align}
with \(\lambda \in (0,1)\). For \(n=0,1,2,\dots\)
let \(\mathsf{u}_n\) be the sequence given by \eqref{eq:general_discrete}
and define \(u_n \coloneqq u(nh)\). Then for
any \(T \geq 0\), there is a constant \(C = C(T) > 0\) such that
\[\sup_{0 \leq nh \leq T} |u_n - \mathsf{u}_n| \leq Ch.\]
\end{theorem}

Note that \eqref{eq:rgf} is simply a
sped-up version of \eqref{eq:GD}: if \(v\) solves \eqref{eq:GD} and \(w\) solves
\eqref{eq:rgf} then \(v(t) = w((1-\lambda)t)\) for any \(t \in [0,\infty)\).
This demonstrates that introduction of momentum in the form used within
both HB and NAG results in numerical methods that do not differ substantially
from gradient descent.

\subsection{Link to HB and NAG}
\label{ssec:Link}

The HB method is usually written as a
two-step scheme taking the form (\cite{importanceofinitmom})
\begin{align*}
\mathsf{v}_{n+1} &= \lambda \mathsf{v}_n + h f(\mathsf{u}_n) \\
\mathsf{u}_{n+1} &= \mathsf{u}_n + \mathsf{v}_{n+1}
\end{align*}
with \(\mathsf{v}_0 = 0\), \(\lambda \in (0,1)\) the momentum factor, and \(h > 0\) the learning rate. We can re-write this update
as
\begin{align*}
\mathsf{u}_{n+1} &= \mathsf{u}_n + \lambda \mathsf{v}_n + hf(\mathsf{u}_n) \\
&= \mathsf{u}_n + \lambda (\mathsf{u}_n - \mathsf{u}_{n-1}) + h f(\mathsf{u}_n)
\end{align*}
hence the method reads
\begin{align}
\label{eq:hb_discrete}
\begin{split}
\mathsf{u}_{n+1} &= \mathsf{u}_n + \lambda (\mathsf{u}_n - \mathsf{u}_{n-1}) + h f(\mathsf{u}_n) \\
\mathsf{u}_1 &= \mathsf{u}_0 + hf(\mathsf{u}_0).
\end{split}
\end{align}

Similarly NAG is usually written as (\cite{importanceofinitmom})
\begin{align*}
\mathsf{v}_{n+1} &= \lambda \mathsf{v}_n + h f(\mathsf{u}_n + \lambda \mathsf{v}_n) \\
\mathsf{u}_{n+1} &= \mathsf{u}_n + \mathsf{v}_{n+1}
\end{align*}
with \(\mathsf{v}_0 = 0\). Define \(\mathsf{w}_n \coloneqq \mathsf{u}_n + \lambda \mathsf{v}_n\) then
\begin{align*}
\mathsf{w}_{n+1} &= \mathsf{u}_{n+1} + \lambda \mathsf{v}_{n+1} \\
&= \mathsf{u}_{n+1} + \lambda (\mathsf{u}_{n+1} - \mathsf{u}_n)
\end{align*}
and
\begin{align*}
\mathsf{u}_{n+1} &= \mathsf{u}_n + \lambda \mathsf{v}_n + hf(\mathsf{u}_n + \lambda \mathsf{v}_n) \\
&= \mathsf{u}_n + (\mathsf{w}_n - \mathsf{u}_n) + h f(\mathsf{w}_n) \\
&= \mathsf{w}_n + h f(\mathsf{w}_n).
\end{align*}
Hence the method may be written as
\begin{align}
\label{eq:nag_discrete}
\begin{split}
\mathsf{u}_{n+1} &= \mathsf{u}_n + \lambda(\mathsf{u}_n - \mathsf{u}_{n-1}) + hf(\mathsf{u}_n + \lambda(\mathsf{u}_n - \mathsf{u}_{n-1})) \\
\mathsf{u}_1 &= \mathsf{u}_0 + hf(\mathsf{u}_0).
\end{split}
\end{align}

It is clear that \eqref{eq:hb_discrete} and \eqref{eq:nag_discrete} are special cases
of \eqref{eq:general_discrete}
with \(a=0\) giving HB and \(a = \lambda\) giving NAG.
To intuitively understand Theorem \ref{thm:conv_to_gf}, re-write \eqref{eq:rgf} as
\[\frac{du}{dt} - \lambda \frac{du}{dt} = f(u).\]
If we discretize the \(du/dt\) term using forward differences and the \(-\lambda du/dt\) term using backward differences,
we obtain
\[\frac{u(t+h) - u(t)}{h} - \lambda \frac{u(t) - u(t-h)}{h} \approx f(u(t)) \approx f \left( u(t) + ha \frac{u(t) - u(t-h)}{h} \right )\]
with the second approximate equality coming from the Taylor expansion of \(f\). This can be rearranged as
\[u(t+h) \approx u(t) + \lambda (u(t) - u(t-h)) + h f(u(t) + a(u(t)-u(t-h)))\]
which has the form of \eqref{eq:general_discrete} with the identification \(\mathsf{u}_n \approx u(nh)\).

\begin{figure}[t]
    \centering
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hbgf_traj_k5}
        \caption{HB: \(\kappa = 5\)}
    \end{subfigure}
    ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hbgf_traj_k10}
        \caption{HB: \(\kappa = 10\)}
    \end{subfigure}
     ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hbgf_traj_k20}
        \caption{HB: \(\kappa = 20\)}
    \end{subfigure}

    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{naggf_traj_k5}
        \caption{NAG: \(\kappa = 5\)}
    \end{subfigure}
    ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{naggf_traj_k10}
        \caption{NAG: \(\kappa = 10\)}
    \end{subfigure}
     ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{naggf_traj_k20}
        \caption{NAG: \(\kappa = 20\)}
    \end{subfigure}

    \caption{Comparison of trajectories for HB and NAG with the gradient flow \eqref{eq:rgf} on the two-dimensional problem
    \(\Phi(u) = \frac{1}{2} \langle u, Q u \rangle\) with \(\lambda=0.9\) fixed. We vary the condition
    number of \(Q\) as well as the learning rate \(h\).}
    \label{fig:gf_traj}
\end{figure}

\begin{figure}[t]
    \centering
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hbgf_conv}
        \caption{HB}
    \end{subfigure}
    ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{naggf_conv}
        \caption{NAG}
    \end{subfigure}

    \caption{The numerical rate of convergence, as a function of the learning rate \(h\), of HB and NAG to the gradient flow \eqref{eq:rgf}
    for the problem described in Figure \ref{fig:gf_traj}.}
    \label{fig:gf_conv}
\end{figure}

\subsection{Numerical Illustration}
\label{ssec:N1}

Figure \ref{fig:gf_traj} compares trajectories of the momentum numerical method
\eqref{eq:general_discrete} with the rescaled gradient flow \eqref{eq:rgf}, for
the two-dimensional problem \(\Phi(u) = \frac{1}{2} \langle u, Q u \rangle\).
We pick \(Q\) to be positive-definite so that the minimum is achieved at the point \((0,0)^T\)
and make it diagonal so that we can easily control its condition number.
In particular, the condition number of \(Q\) is given as
\[\kappa = \frac{\max \{Q_{11}, Q_{22}\} }{\min \{Q_{11}, Q_{22}\} }.\]
We see that, as the condition number is increased, both HB and NAG exhibit
more pronounced transient oscillations and are thus further away from
the trajectory of \eqref{eq:rgf}, however, as the learning rate \(h\) is
decreased, the oscillations dampen and the trajectories match more and more
closely. This observation from Figure \ref{fig:gf_traj} is quantified in
Figure \ref{fig:gf_conv} where we estimate the rate of convergence, as a function
of \(h\), which is defined as
\[\Delta = \log_2 \frac{\|\mathsf{u}^{(h)} - u\|_\infty}{\|\mathsf{u}^{(h/2)} - u\|_\infty}\]
where \(\mathsf{u}^{(\alpha)}\) is the numerical solution using time-step \(\alpha\). The figure shows
that the rate of convergence is indeed close to $1$, as predicted by our theory.
In summary the behavior of the momentum methods is precisely that of a rescaled
gradient flow, but with initial transient oscillations which capture momentum
effects, but disappear as the learning rate is decreased.
We model these oscillations in the next section
via use of a modified equation.

\section{Modified Equations}
\label{sec:ME}

The previous section demonstrates how the momentum methods approximate a
time rescaled version of the gradient flow \eqref{eq:GD}. In this section
we show how the same methods may also be viewed as approximations of
the damped Hamiltonian system \eqref{eq:HAM}, with mass $m$ on the
order of the learning rate, using the method of modified equations.
In subsection \ref{ssec:MR2} we state and discuss the main result of the section,
Theorem \ref{thm:conv_to_visco}.
Subsection \ref{ssec:Link2} gives
intuition for proof of Theorem \ref{thm:conv_to_visco}; the proof itself
is given in Appendix B. And the section also contains
comments on generalizing the idea of modified equations.
In subsection \ref{ssec:N2} we describe a numerical
illustration of Theorem \ref{thm:conv_to_visco}.

\subsection{Main Result}
\label{ssec:MR2}

The main result of this section quantifies the sense in which momentum methods
do, in fact, approximate a damped Hamiltonian system; it is proved in Appendix B.

\begin{theorem}
\label{thm:conv_to_visco}
Fix $\lambda \in (0,1)$ and assume that $a \ge 0$ is chosen so that
$\alpha \coloneqq \frac12(1 + \lambda - 2a(1-\lambda))$ is strictly positive.
Suppose Assumption \ref{assump:one} holds and let \(u \in C^4([0,\infty);\mathbb{R}^d)\) be the solution to
\begin{align}
\label{eq:visco}
\begin{split}
& h \alpha \frac{d^2u}{dt^2} + (1-\lambda)\frac{du}{dt} = -\nabla \Phi(u) \\
&u(0) = \mathsf{u}_0, \quad
\frac{du}{dt}(0) = \mathsf{u}_0'.
\end{split}
\end{align}
Suppose further that \(h \leq (1-\lambda)^2/2 \alpha B_1\).
For \(n=0,1,2,\dots\) let \(\mathsf{u}_n\) be the sequence given by \eqref{eq:general_discrete} and define \(u_n \coloneqq u(nh)\). Then for
any \(T \geq 0\), there is a constant \(C = C(T) > 0\) such that
\[\sup_{0 \leq nh \leq T} |u_n - \mathsf{u}_n| \leq Ch.\]
\end{theorem}

Theorem \ref{thm:conv_to_gf} demonstrates the same order of convergence, namely
${\mathcal O}(h)$, to the rescaled gradient flow equation \eqref{eq:rgf},
obtained from \eqref{eq:visco} simply by setting $h=0.$ In the standard method
of modified equations the limit system (here \eqref{eq:rgf})
is perturbed by small terms (in terms of the assumed small learning rate)
and an increased  rate of convergence is obtained to the modified equation
(here  \eqref{eq:visco}). In our setting however, because the small modification is to
a higher derivative (here second) than appears in the limit equation (here first
order), an increased
rate of convergence is not obtained.  This is due to the nature of the modified
equation, whose solution has derivatives that are inversely proportional to
powers of \(h\); this fact is quantified in Lemma \ref{lemma:visco_bound}
from Appendix B. It is precisely because the modified equation does not lead to
a higher rate of convergence that the initial parameter $\mathsf{u}_0'$ is arbitrary; the same
rate of convergence is obtained no matter what value it takes.

It is natural to ask, therefore, what
is learned from the convergence result in Theorem \ref{thm:conv_to_visco}. The
answer is that, although the modified equation \eqref{eq:visco} is approximated
at the same order as the limit equation \eqref{eq:rgf}, it actually contains
considerably more qualitative information about the dynamics of the system,
particularly in the early transient phase of the algorithm; this will be
illustrated in subsection \ref{ssec:N2}.
Indeed we will make a specific choice of $\mathsf{u}_0'$ in our numerical experiments,
namely
\begin{equation}
\label{eq:beta}
\frac{du}{dt}(0) = \frac{1 - 2\alpha}{2\alpha - \lambda + 1} f(\mathsf{u}_0),
\end{equation}
to better match the transient dynamics.

\subsection{Intuition and Wider Context}
\label{ssec:Link2}

\subsubsection{Idea Behind The Modified Equations}

In this subsection, we show that the scheme \eqref{eq:general_discrete} exhibits momentum, in the sense
of approximating a momentum equation, but the size of the momentum term is on the order of the step size \(h\).
To see this intuitively, we add and subtract \(\mathsf{u}_n - \mathsf{u}_{n-1}\) to the right hand size of \eqref{eq:general_discrete} then
we can rearrange it to obtain
\[h \frac{\mathsf{u}_{n+1} - 2\mathsf{u}_n + \mathsf{u}_{n-1}}{h^2} + (1-\lambda) \frac{\mathsf{u}_n - \mathsf{u}_{n-1}}{h} = f(\mathsf{u}_n + a(\mathsf{u}_n - \mathsf{u}_{n-1})).\]
This can be seen as a second order central difference and first order backward difference discretization of the
momentum equation
\[h \frac{d^2u}{dt^2} + (1-\lambda)\frac{du}{dt} = f(u)\]
noting that the second derivative term has size of order \(h\).

\subsubsection{Higher Order Modified Equations For HB}

We will now show that, for HB, we may derive higher order modified equations that are
consistent with \eqref{eq:hb_discrete}. Taking the limit of these equations yields an
operator that agrees with with our intuition for discretizing \eqref{eq:rgf}. To this end,
suppose \(\Phi \in C^\infty_b (\mathbb{R}^d, \mathbb{R})\) and consider the ODE(s),
\begin{equation}
\label{eq:hb_highorder}
\sum_{k=1}^p \frac{h^{k-1}(1 + (-1)^k \lambda)}{k!} \frac{d^k u}{dt^k} = f(u)
\end{equation}
noting that \(p = 1\) gives \eqref{eq:rgf} and \(p=2\) gives \eqref{eq:visco}. Let \(u \in C^\infty ([0,\infty),\mathbb{R}^d)\)
be the solution to \eqref{eq:hb_highorder} and define \(u_n \coloneqq u(nh)\), \(u_n^{(k)} \coloneqq \frac{d^k u}{dt^k} (nh)\)
for \(n=0,1,2,\dots\) and \(k=1,2,\dots,p\). Taylor expanding yields
\[u_{n \pm 1} = u_n + \sum_{k=1}^p \frac{(\pm 1)^k h^k}{k!} u^{(k)}_n + h^{p+1} I^{\pm}_n\]
where
\[I^{\pm}_n = \frac{(\pm 1)^{p+1}}{p!} \int_0^1 (1-s)^p \frac{d^{p+1}u}{dt^{p+1}} ((n \pm s)h) ds.\]
Then
\begin{align*}
u_{n+1} - u_n - \lambda(u_n - u_{n-1}) &= \sum_{k=1}^p \frac{h^k}{k!} u^{(k)}_n + \lambda \sum_{k=1}^p \frac{ (-1)^k h^k}{k!} u^{(k)}_n + h^{p+1}(I^+_n - \lambda I^-_n) \\
&= h \sum_{k=1}^p \frac{h^{k-1}(1 + (-1)^k \lambda)}{k!} u^{(k)}_n + h^{p+1}(I^+_n - \lambda I^-_n) \\
&= h f(u_n) + h^{p+1}(I^+_n - \lambda I^-_n)
\end{align*}
showing consistency to order \(p+1\). As is the case with \eqref{eq:visco} however, the \(I^\pm_n\) terms will
be inversely proportional to powers of \(h\) hence global accuracy will not improve.

We now study the differential operator on the l.h.s. of \eqref{eq:hb_highorder} as \(p \rightarrow \infty\).
Define the sequence of differential operators \(T_p : C^\infty([0,\infty),\mathbb{R}^d) \to C^\infty([0,\infty),\mathbb{R}^d)\) by
\[T_p u = \sum_{k=1}^p \frac{h^{k-1}(1 + (-1)^k \lambda)}{k!} \frac{d^k u}{dt^k}, \quad \forall u \in C^\infty([0,\infty),\mathbb{R}^d).\]
Taking the Fourier transform yields
\[\mathcal{F}(T_p u)(\omega) = \sum_{k=1}^p \frac{h^{k-1}(1 + (-1)^k \lambda)(i \omega)^k}{k!} \mathcal{F}(u)(\omega)\]
where \(i = \sqrt{-1}\) denotes the imaginary unit. Suppose there is a limiting operator \(T_p \to T\) as \(p \to \infty\)
then taking the limit yields
\[\mathcal{F}(Tu)(\omega) = \frac{1}{h} (e^{ih\omega} + \lambda e^{-ih\omega} - \lambda -1) \mathcal{F}(u)(\omega).\]
Taking the inverse transform and using the convolution theorem, we obtain
\begin{align*}
(Tu)(t) &= \frac{1}{h} \mathcal{F}^{-1} ( e^{ i h \omega} + \lambda e^{- i h \omega} - \lambda - 1  )(t) * u(t) \\
&= \frac{1}{h} \left ( -(1+\lambda) \delta(t) + \lambda \delta(t + h) + \delta(t - h) \right) * u(t) \\
&= \frac{1}{h} \int_{-\infty}^\infty \left ( -(1+\lambda) \delta(t - \tau) + \lambda \delta(t - \tau + h) + \delta(t - \tau - h) \right)u(\tau) \: d\tau \\
&= \frac{1}{h} \left ( -(1+\lambda)u(t) + \lambda u(t -  h) + u(t +  h) \right ) \\
&= \frac{u(t+h) - u(t)}{h} - \lambda \left ( \frac{u(t) - u(t-h)}{h} \right )
\end{align*}
where \(\delta(\cdot)\) denotes the Dirac-delta distribution and we abuse notation by writing its action
as an integral. The above calculation does not prove convergence of \(T_p\) to \(T\), but simply confirms
our intuition that \eqref{eq:hb_discrete} is a forward and backward discretization of \eqref{eq:rgf}.

\begin{figure}[t]
    \centering
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hbham_traj_k5}
        \caption{HB: \(\kappa = 5\)}
    \end{subfigure}
    ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hbham_traj_k10}
        \caption{HB: \(\kappa = 10\)}
    \end{subfigure}
     ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hbham_traj_k20}
        \caption{HB: \(\kappa = 20\)}
    \end{subfigure}

    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{nagham_traj_k5}
        \caption{NAG: \(\kappa = 5\)}
    \end{subfigure}
    ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{nagham_traj_k10}
        \caption{NAG: \(\kappa = 10\)}
    \end{subfigure}
     ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{nagham_traj_k20}
        \caption{NAG: \(\kappa = 20\)}
    \end{subfigure}

    \caption{Comparison of trajectories for HB and NAG with the Hamiltonian dynamic \eqref{eq:visco} on the two-dimensional problem
    \(\Phi(u) = \frac{1}{2} \langle u, Q u \rangle\) with \(\lambda=0.9\) fixed. We vary the condition
    number of \(Q\) as well as the learning rate \(h\).}
    \label{fig:visco_traj}
\end{figure}

\begin{figure}[t]
    \centering
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hbham_conv}
        \caption{HB}
    \end{subfigure}
    ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{nagham_conv}
        \caption{NAG}
    \end{subfigure}

    \caption{The numerical rate of convergence, as a function of the learning rate \(h\), of HB and NAG to the momentum equation \eqref{eq:visco}
    for the problem described in Figure \ref{fig:visco_traj}.}
    \label{fig:visco_conv}
\end{figure}

\subsection{Numerical Illustration}
\label{ssec:N2}

Figure \ref{fig:visco_traj} shows trajectories of \eqref{eq:general_discrete}
and \eqref{eq:visco} for different values of \(a\) and
\(h\) on the two-dimensional problem \(\Phi(u) = \frac{1}{2} \langle u , Q u \rangle\),
varying the condition number of \(Q\).
We make the specific choice of $\mathsf{u}_0'$ implied by the initial
condition \eqref{eq:beta}. Figure \ref{fig:visco_conv} shows
the numerical order of convergence as a function of \(h\), as defined in Section \ref{ssec:N1}, which is near 1, matching our theory. We note that the
oscillations in HB are captured well by \eqref{eq:visco}, except for a slight shift when \(h\) and \(\kappa\) are large. This is due to
our choice of initial condition which cancels the maximum number of terms in the Taylor expansion initially, but the overall rate of convergence remains
\(\mathcal{O}(h)\) due to Lemma \ref{lemma:visco_bound}. Other choices of $\mathsf{u}_0'$
also result in \(\mathcal{O}(h)\) convergence and can be picked on a case-by-case basis
to obtain consistency with different qualitative phenomena of interest in the dynamics.
Note also that \(\alpha|_{a = \lambda} < \alpha|_{a = 0}\). As a result the transient
oscillations in \eqref{eq:visco} are more quickly damped in the NAG case than in
the HB case; this is consistent with the numerical results.
However panels (d)-(f) in Figure \ref{fig:gf_traj} show that \eqref{eq:visco} is not able to
adequately capture the oscillations of NAG when \(h\) is relatively large. We
leave for future work, the task of finding equations that are able to appropriately capture the
oscillations of NAG in the large \(h\) regime.

\section{Invariant Manifold}
\label{sec:IM}

The key lessons of the previous two sections are that the momentum methods
approximate a rescaled gradient flow of the form \eqref{eq:GD} and a damped
Hamiltonian system of the form \eqref{eq:HAM}, with small mass $m$ which
scales with the learning rate, and constant damping $\gamma.$
Both approximations hold with the same
order of accuracy, in terms of the learning rate, and numerics demonstrate that the Hamiltonian system is
particularly useful in providing intuition for the transient regime of the
algorithm. In this section we link the two theorems from the two preceding
sections by showing that the Hamiltonian dynamics with small mass from section
\ref{sec:ME} has
an exponentially attractive invariant manifold on which the dynamics
is, to leading order, a gradient flow.
That gradient flow is a small, in terms of the learning rate,
perturbation of the time-rescaled gradient flow
from section \ref{sec:M}.

\subsection{Main Result}
\label{ssec:MR3}

Define
\begin{equation}
\label{eq:define}
\mathsf{v}_n \coloneqq (\mathsf{u}_n - \mathsf{u}_{n-1})/h
\end{equation}
noting that then \eqref{eq:general_discrete} becomes
\[\mathsf{u}_{n+1} = \mathsf{u}_n + h \lambda \mathsf{v}_n + hf(\mathsf{u}_n + ha\mathsf{v}_n)\]
and
\[\mathsf{v}_{n+1} = \frac{\mathsf{u}_{n+1} - \mathsf{u}_n}{h} = \lambda \mathsf{v}_n + f(\mathsf{u}_n + ha\mathsf{v}_n).\]
Hence we can re-write \eqref{eq:general_discrete} as
\begin{align}
\label{eq:two_step_method}
\begin{split}
\mathsf{u}_{n+1} &= \mathsf{u}_n + h \lambda \mathsf{v}_n + hf(\mathsf{u}_n + ha\mathsf{v}_n) \\
\mathsf{v}_{n+1} &= \lambda \mathsf{v}_n + f(\mathsf{u}_n + ha \mathsf{v}_n).
\end{split}
\end{align}

Note that if $h=0$ then \eqref{eq:two_step_method} shows that
$\mathsf{u}_n=\mathsf{u}_0$ is constant in $n$, and that
$\mathsf{v}_n$ converges to $(1-\lambda)^{-1}f(\mathsf{u}_0).$ This suggests that,
for $h$ small, there is an invariant manifold which is a small perturbation of
the relation $\mathsf{v}_n=\bar{\lambda}f(\mathsf{u}_n)$
and is representable as a graph. Motivated by this, we
look for a function \(g: \mathbb{R}^d \to \mathbb{R}^d\) such that
the manifold
\begin{equation}
\label{eq:thisisit}
\mathsf{v}=\bar{\lambda} f(\mathsf{u}) + hg(\mathsf{u})
\end{equation}
is invariant for the dynamics of the numerical method:
\begin{equation}
\label{eq:highlight}
\mathsf{v}_n = \bar{\lambda} f(\mathsf{u}_n) + hg(\mathsf{u}_n) \Longleftrightarrow \mathsf{v}_{n+1} = \bar{\lambda} f(\mathsf{u}_{n+1}) + hg(\mathsf{u}_{n+1}).
\end{equation}

We will prove the existence of such a function $g$ by use of the contraction
mapping theorem to find fixed point of mapping $T$ defined in subsection \ref{ssec:I3}
below. We seek this fixed point in set $\Gamma$ which we now define:

\begin{definition}
\label{def:Gamma}
Let \(\gamma, \delta > 0\) be as in Lemmas \ref{lemma:gamma}, \ref{lemma:delta}. Define \(\Gamma \coloneqq \Gamma(\gamma,\delta)\) to be the closed
subset of \(C(\mathbb{R}^d;\mathbb{R}^d)\) consisting of \(\gamma\)-bounded functions:
\[\|g\|_\Gamma \coloneqq \sup_{\xi \in \mathbb{R}^d} |g(\xi)| \leq \gamma, \quad \forall g \in \Gamma\]
that are \(\delta\)-Lipshitz:
\[|g(\xi) - g(\eta)| \leq \delta |\xi - \eta|, \quad \forall g \in \Gamma, \xi, \eta \in \mathbb{R}^d.\]
\end{definition}

\begin{theorem}
\label{thm:invman_existence}
Fix $\lambda \in (0,1).$ Suppose that $h$ is chosen small enough so that Assumption \ref{assump:h_small} holds. For \(n=0,1,2,\dots\), let \(\mathsf{u}_n\), \(\mathsf{v}_n\) be the sequences given by \eqref{eq:two_step_method}.
Then there is a \(\tau >0\) such that, for all \(h \in (0,\tau)\),  there is a
unique \(g \in \Gamma\) such that \eqref{eq:highlight} holds. Furthermore,
\[|\mathsf{v}_n - \bar{\lambda}f(\mathsf{u}_n) - hg(\mathsf{u}_n)| \leq (\lambda + h^2 \lambda \delta)^n |\mathsf{v}_0 - \bar{\lambda}f(\mathsf{u}_0) - hg(\mathsf{u}_0)|\]
where \(\lambda + h^2 \lambda \delta < 1\).
\end{theorem}

The statement of Assumption \ref{assump:h_small}, and the proof of the preceding
theorem, are given in Appendix C. The assumption appears somewhat involved at first
glance but inspection reveals that it simply places an upper bound on the
learning rate $h,$ as detailed in Lemmas \ref{lemma:gamma}, \ref{lemma:delta}.
The proof of the theorem rests on the Lemmas \ref{lemma:well_defined},
\ref{lemma:gamma_to_gamma} and \ref{lemma:T_contraction} which establish
that the operator \(T\) is well-defined, maps \(\Gamma\) to \(\Gamma\), and
is a contraction on \(\Gamma\). The operator $T$ is defined, and expressed
in a helpful form for the purposes of analysis, in the next subsection.

In the next subsection
we obtain the leading order approximation for $g$, given in equation \eqref{eq:getg}.
Theorem \ref{thm:invman_existence} implies that the large-time dynamics
are governed by the dynamics on the invariant manifold. Substituting the
leading order approximation for $g$ into the invariant manifold \eqref{eq:thisisit}
and using this expression in the definition \eqref{eq:define}
shows that

\begin{subequations}
\label{eq:another}
\begin{align}
\label{eq:another_a}
\mathsf{v}_n&=-(1-{\lambda})^{-1} \nabla \left ( \Phi(\mathsf{u}_n) + \frac{1}{2} h \bar{\lambda}(\bar{\lambda} - a) |\nabla \Phi(\mathsf{u}_n)|^2 \right ),\\
\label{eq:another_b}
\mathsf{u}_{n}&=\mathsf{u}_{n-1}
 - h(1-{\lambda})^{-1} \nabla \left ( \Phi(\mathsf{u}_n) + \frac{1}{2} h \bar{\lambda}(\bar{\lambda} - a) |\nabla \Phi(\mathsf{u}_n)|^2 \right ).
\end{align}
\end{subequations}
Setting
\begin{equation}
\label{eq:mp_c}
c=\bar{\lambda} \left( \bar{\lambda}-a+\frac{1}{2} \right)
\end{equation}
we see that for large time the dynamics of momentum methods, including
HB and NAG, are approximately those of the modified gradient flow
\begin{equation}
\label{eq:approxg2}
\dtfrac{u}
= - (1-{\lambda})^{-1} \nabla \Phi_h(u)
\end{equation}
with
\begin{equation}
\label{eq:mp}
\Phi_h(u)=\Phi(u)+\frac12 hc|\nabla \Phi(u)|^2.
\end{equation}
To see this we proceed as follows. Note that from \eqref{eq:approxg2}
\[\frac{d^2u}{dt^2} = - \frac{1}{2} (1-\lambda)^{-2} \nabla |\nabla \Phi(u)|^2 + \mathcal{O}(h)\]
then Taylor expansion shows that, for $u_n=u(nh)$,
\begin{align*}
u_n &= u_{n-1} + h \dot{u}_n - \frac{h^2}{2} \ddot{u}_n + \mathcal{O}(h^3) \\
&=u_{n-1} - h \bar{\lambda} \left( \nabla \Phi(u_n) + \frac{1}{2}hc\nabla |\nabla \Phi(u_n)|^2 \right) + \frac{1}{4}h^2 \bar{\lambda}^2 \nabla |\nabla \Phi(u_n)|^2 + \mathcal{O}(h^3)
\end{align*}
where we have used that
$$Df(u)f(u)=\frac12 \nabla \left(|\nabla \Phi(u)|^2\right).$$
Choosing $c=\bar{\lambda}(\bar{\lambda}-a+1/2)$ we see that
\begin{equation}
\label{eq:approxflow_taylor}
u_n=u_{n-1}
 - h(1-{\lambda})^{-1} \nabla \left ( \Phi(u_n) + \frac{1}{2} h \bar{\lambda}(\bar{\lambda} - a) |\nabla \Phi(u_n)|^2 \right ) + \mathcal{O}(h^3).
\end{equation}
Notice that comparison
of \eqref{eq:another_b} and \eqref{eq:approxflow_taylor} shows that, on the invariant manifold, the dynamics
are to \(\mathcal{O}(h^2)\) the same as the equation \eqref{eq:approxg2}; this is because the
truncation error between \eqref{eq:another_b} and \eqref{eq:approxflow_taylor} is \(\mathcal{O}(h^3)\).

Thus we have proved:

\begin{theorem}
\label{thm:conv_to_modifiedgf}
Suppose that the conditions of Theorem \ref{thm:invman_existence} hold.
Then for initial data started on the invariant manifold and
any \(T \geq 0\), there is a constant \(C = C(T) > 0\) such that
\[\sup_{0 \leq nh \leq T} |u_n - \mathsf{u}_n| \leq Ch^2,\]
where $u_n=u(nh)$ solves the modified equation \eqref{eq:approxg2}
with $c=\bar{\lambda}(\bar{\lambda}-a+1/2)$.
\end{theorem}

\subsection{Intuition}
\label{ssec:I3}

We will define mapping \(T: C(\mathbb{R}^d;\mathbb{R}^d) \to C(\mathbb{R}^d;\mathbb{R}^d)\) via the equations
\begin{equation}
\label{eq:two_step}
\begin{split}
p = \xi + h \lambda\bigl(\bar{\lambda} f(\xi) + hg(\xi)\bigr)  +
hf\Bigl(\xi + ha\bigl(\bar{\lambda} f(\xi) + hg(\xi)\bigr)\Bigr) \\
\bar{\lambda} f(p)+h(Tg)(p) = \lambda \bigl(\bar{\lambda} f(\xi) + hg(\xi)\bigr) +
f\Bigl(\xi + ha\bigl(\bar{\lambda} f(\xi) + hg(\xi)\bigr)\Bigr).
\end{split}
\end{equation}
A fixed point of the mapping $g \mapsto Tg$ will give function $g$ so that, under
\eqref{eq:two_step}, identity \eqref{eq:highlight} holds.
Later we will show that, for $g$ in $\Gamma$ and all $h$ sufficiently small,
 $\xi$ can be found from (\ref{eq:two_step}a)
for every $p$, and that thus (\ref{eq:two_step}b) defines a mapping from $g \in \Gamma$
into $Tg \in C(\mathbb{R}^d;\mathbb{R}^d).$ We will then show that, for $h$ sufficiently small,
$T: \Gamma \mapsto \Gamma$ is a contraction.

For any \(g \in C(\mathbb{R}^d;\mathbb{R}^d)\) and \(\xi \in \mathbb{R}^d\) define
\begin{align}
\label{eq:wg}
w_g(\xi) &\coloneqq \bar{\lambda} f(\xi) + h g(\xi) \\
\label{eq:zg}
z_g(\xi) &\coloneqq \lambda w_g(\xi) + f\bigl(\xi + haw_g(\xi)\bigr).
\end{align}
With this notation the fixed point mapping \eqref{eq:two_step} for $g$ may be written
\begin{equation}
\label{eq:two_step2}
\begin{split}
p = \xi + hz_g(\xi),\\
\bar{\lambda} f(p)+h(Tg)(p) = z_g(\xi).
\end{split}
\end{equation}

Then, by Taylor expansion,
\begin{align}
\label{eq:I1}
\begin{split}
f\Bigl(\xi + ha\bigl(\bar{\lambda} f(\xi) + hg(\xi)\bigr)\Bigr) &= f\bigl(\xi + haw_g(\xi)\bigr) \\
&= f(\xi) + ha \int_0^1  Df\bigl(\xi + shaw_g(\xi)\bigr)w_g(\xi) ds \\
&= f(\xi) + ha I^{(1)}_g (\xi)
\end{split}
\end{align}
where the last line defines \(I^{(1)}_g\). Similarly
\begin{align}
\label{eq:I2}
\begin{split}
f(p) &= f(\xi + hz_g(\xi)) \\
&= f(\xi) + h \int_0^1 Df\bigl(\xi + shz_g(\xi)\bigr)z_g(\xi)ds \\
&= f(\xi) + h I^{(2)}_g(\xi),
\end{split}
\end{align}
where the last line now defines \(I^{(2)}_g\).
Then (\ref{eq:two_step}b) becomes
\[\bar{\lambda}\bigl(f(\xi) + h I^{(2)}_g(\xi)\bigr) + h (Tg)(p) = \lambda \bar{\lambda} f(\xi) + h \lambda g(\xi) + f(\xi) + haI^{(1)}_g(\xi)\]
and we see that
\[(Tg)(p) = \lambda g(\xi) + a I^{(1)}_g(\xi) - \bar{\lambda}I^{(2)}_g(\xi).\]
In this light, we can rewrite the defining equations \eqref{eq:two_step} for \(T\) as
\begin{align}
\label{eq:p}
p &= \xi + hz_g(\xi), \\
\label{eq:Tg}
(Tg)(p) &= \lambda g(\xi) + a I^{(1)}_g(\xi) - \bar{\lambda}I^{(2)}_g(\xi).
\end{align}
for any \(\xi \in \mathbb{R}^d\).

Perusal of the above definitions reveals that, to leading order in $h$,
$$w_g(\xi)=z_g(\xi)=\bar{\lambda}f(\xi), I^{(1)}_g(\xi)=I^{(2)}_g(\xi)=\bar{\lambda}
Df(\xi)f(\xi).$$
Thus setting $h=0$ in \eqref{eq:p}, \eqref{eq:Tg} shows that, to leading order
in $h$,
\begin{equation}
\label{eq:getg}
g(p) = \bar{\lambda}^2(a-\bar{\lambda})Df(p)f(p).
\end{equation}
Note that since \(f(p) = - \nabla \Phi(p)\), \(Df\) is the negative Hessian of \(\Phi\) and is thus symmetric.
Hence we can write $g$ in gradient form, leading to
\begin{equation}
\label{eq:getg2}
g(p) = \frac12 \bar{\lambda}^2(a-\bar{\lambda})\nabla\bigl(|\nabla \Phi(p)|^2\bigr).
\end{equation}

\begin{remark}
This modified potential \eqref{eq:mp} also arises in the construction of Lyapunov functions
for the one-stage theta method  -- see Corollary 5.6.2 in \cite{stuart1998dynamical}.
\end{remark}

\begin{figure}[t]
    \centering
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hb_invman_u}
        \caption{HB: \(\mathsf{u}_n\) given by \eqref{eq:two_step_method}}
    \end{subfigure}
    ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hb_invman_v}
        \caption{HB: \(\mathsf{v}_n\) given by \eqref{eq:two_step_method}}
    \end{subfigure}
     ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{hb_invman_e}
        \caption{HB: \(\mathsf{e}_n\) given by \eqref{eq:ima2}}
    \end{subfigure}

     \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{nag_invman_u}
        \caption{NAG: \(\mathsf{u}_n\) given by \eqref{eq:two_step_method}}
    \end{subfigure}
    ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{nag_invman_v}
        \caption{NAG: \(\mathsf{v}_n\) given by \eqref{eq:two_step_method}}
    \end{subfigure}
     ~
    \begin{subfigure}[b]{0.31\textwidth}
        \includegraphics[width=\textwidth]{nag_invman_e}
        \caption{NAG: \(\mathsf{e}_n\) given by \eqref{eq:ima2}}
    \end{subfigure}
    \caption{Invariant manifold for HB and NAG with \(h=2^{-6}\) and \(\lambda = 0.9\) on the two-dimensional problem \(\Phi(u) = \frac{1}{2} \langle u, Q u \rangle\), varying the condition number of \(Q\). Panels (c), (f) show the distance from the invariant manifold for the largest condition number \(\kappa = 20\).}
    \label{fig:invman}
\end{figure}

\subsection{Numerical Illustration}
\label{ssec:NI}

In Figure \ref{fig:invman} panels (a),(b),(d),(e), we plot the components $\mathsf{u}_n$ and $\mathsf{v}_n$
found by solving \eqref{eq:two_step_method} with initial conditions $\mathsf{u}_0=(1,1)^T$ and
$\mathsf{v}_n=(0,0)^T$ in the case where  \(\Phi(u) = \frac{1}{2} \langle u, Q u \rangle\).
These initial conditions correspond to initializing the map off the invariant manifold.
To leading order in $h$ the invariant manifold is given by
(see equation \eqref{eq:another})
\begin{equation}
\label{eq:ima1}
v= - (1-{\lambda})^{-1} \nabla \left ( \Phi(u) + \frac{1}{2} h \bar{\lambda}(\bar{\lambda} - a) |\nabla \Phi(u)|^2 \right ).
\end{equation}
To measure the distance of the trajectory shown in panels (a),(b),(d),(e) from the
invariant manifold we define
\begin{equation}
\label{eq:ima2}
\mathsf{e}_n= \left | \mathsf{v}_n+ (1-{\lambda})^{-1} \nabla \left ( \Phi(\mathsf{u}_n) + \frac{1}{2} h \bar{\lambda}(\bar{\lambda} - a) |\nabla \Phi(\mathsf{u}_n)|^2 \right ) \right |.
\end{equation}
Panels (c),(f) show the evolution of $\mathsf{e}_n$ as well as the (approximate) bound on it found
from substituting the leading order approximation of $g$ into the following
upper bound from Theorem \ref{thm:invman_existence}:
$$(\lambda + h^2 \lambda \delta)^n |\mathsf{v}_0 - \bar{\lambda}f(\mathsf{u}_0) - hg(\mathsf{u}_0)|.$$

\section{Deep Learning Example}
\label{ssec:DL}

Our theory is developed under quite restrictive assumptions, in order
to keep the proofs relatively simple and to allow a clearer conceptual
development. The purpose of the numerical experiments in this section
is twofold: firstly to demonstrate that our theory sheds light on a stochastic
version of gradient descent applied, furthermore,
to a setting in which the objective
function does not satisfy the global assumptions which
facilitate our analysis; and second to show that methods implemented
as we use them here (with learning-rate independent momentum, fixed
at every step of the iteration) can out-perform other choices on
specific problems.

Our numerical experiments in this section are
undertaken with in the context of the example
given in \cite{importanceofinitmom}. We train a deep autoencoder, using
the architecture of \cite{pretraining} on the MNIST dataset \cite{mnist}. Since our work is
concerned only with optimization and not generalization, we present our results only on the
training set of 60,000 images and ignore the testing set. We fix an initialization of the
autoencoder following \cite{xavier} and use it to test every optimization method.
Furthermore, we fix a batch size of 200 and train for 500 epochs, not shuffling the data set during training so that
each method sees the same realization of the noise. We use the mean-squared error as
our loss function.

\begin{figure}[t]
\begin{center}
\begin{tabular}{ |l|c|c|c|c|c|c|c| }
\hline
  & \(h=2^0\) & \(h=2^{-1}\) & \(h=2^{-2}\) & \(h=2^{-3}\) & \(h=2^{-4}\) & \(h=2^{-5}\) & \(h=2^{-6}\) \\
\specialrule{.1em}{.05em}{.05em}
GF & n/a & 4.3948 & 4.5954 & 5.6769 & \bf{7.0049} & \bf{8.6468} & \bf{10.6548} \\
\hline
HB & 3.6775 & 4.0157 & 4.5429 & 5.6447 & 7.0720 & 8.7070 & 10.6848 \\
\hline
NAG & \bf{3.2808} &   \bf{3.7166} &  \bf{4.4579} &   \bf{5.6087} &  7.0557 & 8.6987 &  10.6814 \\
\specialrule{.1em}{.05em}{.05em}
Wilson & 6.7395 &   7.5177 &    8.3491 &   9.2543 &   10.2761 &   11.3776 &   12.4123 \\
\hline
HB-\(\mu\) & 5.7099 &  6.6146 &  7.6202 &  8.6629 &   9.7838 &  11.0039 &  12.1743 \\
\hline
NAG-\(\mu\) & \bf{5.6867} & \bf{6.6033} &  \bf{7.6131} & \bf{8.6556} &  \bf{9.7783} &  \bf{11.0015} &  \bf{12.1738} \\
\hline
\end{tabular}
\end{center}
\caption{Final training errors for the autoencoder on  MNIST for six
training methods over different learning rates. GF refers to equation \eqref{eq:rgf_disc} while
HB and NAG to \eqref{eq:general_discrete} all with fixed \(\lambda = 0.9\).}
\label{fig:final_training}
\end{figure}

\begin{figure}[t]
    \centering
    \begin{subfigure}[b]{0.45\textwidth}
        \includegraphics[width=\textwidth]{gf_conv}
        \caption{HB, NAG to \eqref{eq:rgf_disc}}
    \end{subfigure}
    ~
    \begin{subfigure}[b]{0.45\textwidth}
        \includegraphics[width=\textwidth]{wilson_conv}
        \caption{HB-\(\mu\), NAG-\(\mu\) to \eqref{eq:wilson_disc}}
    \end{subfigure}

    \caption{The numerical rate of convergence for the parameters of
    the autoencoder, as a function of the learning rate h, of HB
    and NAG to \eqref{eq:rgf_disc} (a), as well as of HB-\(\mu\) and NAG-\(\mu\) to \eqref{eq:wilson_disc}
    (b).}
    \label{fig:autoencoder_conv}
\end{figure}

We compare HB and NAG given by \eqref{eq:general_discrete} to the re-scaled gradient
flow \eqref{eq:rgf} which we discretize in the standard way to yield the numerical
method
\begin{equation}
\label{eq:rgf_disc}
\mathsf{u}_{n+1} = \mathsf{u}_n - \frac{h}{(1-\lambda)} \nabla \Phi(\mathsf{u}_n),
\end{equation}
hence the momentum term \(\lambda\) only acts to re-scale the learning rate.
We do not test against equation \eqref{eq:visco} because, to discretize it
faithfully, we would need to use a time-step much lower than \(h\) (because
\eqref{eq:visco} contains a term of order \(h\)), but doing so would mean that
we need to train for many more epochs compared to HB and NAG so that the same
final time is reached. This, in turn, implies that the methods would see
different realization of the noise. Thus, to compare them well, we would need
to perform a Monte Carlo simulation, however, since we do not state any
of our results in a stochastic setting, we leave this for future work.

We also compare our results to those of \cite{wilsoneq} which analyze HB and
NAG in the setting where \(\Phi\) is \(\mu\)-strongly convex and \(\lambda\) is
given by \eqref{eq:lambda_mu} that is
\[
\lambda  = \frac{1 - \sqrt{\mu h}}{1 + \sqrt{\mu h}}.
\]
They obtain the limiting equation
\[\ddot{u} + 2\sqrt{\mu} \dot{u} + \nabla \Phi(u) = 0\]
which we discretize via a split-step method to yield
\begin{align}
\label{eq:wilson_disc}
\begin{split}
\mathsf{u}_{n+1} &= \mathsf{u}_n + \frac{1}{2\sqrt{\mu}} \left ( 1- e^{-2\sqrt{ \mu h}} \right ) \mathsf{v}_n \\
\mathsf{v}_{n+1} &= e^{-2\sqrt{ \mu h}} \mathsf{v}_n - \sqrt{h} \nabla \Phi(\mathsf{u}_{n+1})
\end{split}
\end{align}
where we have mapped the the time-step \(h\) in HB and
NAG to \(\sqrt{h}\) as in done in \cite{wilsoneq}. We choose this discretization
because it allows us to directly solve for the linear parts of the ODE (in
the enlarged state-space), yielding a more accurate approximation than the
forward-Euler method used to obtain \eqref{eq:rgf_disc}. A detailed derivation is given in Appendix D.
We will refer to the method in equation \eqref{eq:wilson_disc} as Wilson.
Further we refer to equation \eqref{eq:general_discrete}
with \(\lambda\) given by \eqref{eq:lambda_mu} and \(a=0\) as HB-\(\mu\) and
equation \eqref{eq:general_discrete} with \(\lambda\) given by \eqref{eq:lambda_mu} and \(a=\lambda\) as NAG-\(\mu\).
Since deep neural networks are not strongly convex, there is no single
optimal choice of $\mu$; we simply set \(\mu=1\) in our experiments.

Figure \ref{fig:final_training} gives the final training errors for each
method for several learning rates. We were unable to train the autoencoder
using \eqref{eq:rgf_disc} with \(h=1\) since \(\lambda=0.9\) implies
an effective learning rate of \(10\) for which the system blows up.
In general,
NAG is the best performing method for relatively
large \(h\) which is an observation that is consistently made in the
deep learning literature. Further, we note that as the learning rate
decreases, the final errors become closer indicating convergence to the
appropriate limiting equations. Figure \ref{fig:final_training} showcases the practical effectiveness of momentum methods as they
provide a way of discretizing the gradient flow \eqref{eq:GD} with a large effective learning
rate that forward Euler cannot accommodate. From this perspective, we can view momentum
methods as providing a more stable discretization to gradient flows in a
manner illustrated by \eqref{eq:approxg2}.
Such a viewpoint informs the works \cite{scieur2017integration,betancourt2018symplectic,runge-kuttadisc}.

To further illustrate the point of convergence to the limiting equation, we
compute the numerical rate of convergence, defined in Section \ref{ssec:N1},
as a function of \(h\)
for the neural network parameters between \eqref{eq:rgf_disc} and HB and NAG
as well as between \eqref{eq:wilson_disc} and HB-\(\mu\) and NAG-\(\mu\). Figure
\ref{fig:autoencoder_conv} gives the results. We note that this rate is around
1 as predicted by our theory while the rate for \eqref{eq:wilson_disc} is around
0.5 which is also consistent with the theory in \cite{wilsoneq}.

\section{Conclusion}
\label{sec:C}

Together, equations \eqref{eq:rgf}, \eqref{eq:visco} and \eqref{eq:approxg2}
describe the dynamical systems which are approximated by momentum methods, when
implemented with fixed momentum,
in a manner made precise by the four theorems in this paper. The insight obtained
from these theorems sheds light on how momentum methods perform optimization tasks.

\acks{Both authors are supported, in part, by the US National Science Foundation (NSF)
grant DMS 1818977, the US Office of Naval Research (ONR) grant N00014-17-1-2079, and
the US Army Research Office (ARO) grant W911NF-12-2-0022. Both authors are also
grateful to the anonymous reviewers for their invaluable suggestions which have
helped to significantly strengthen this work.}

\newpage

Machine Learning (cs.LG); Numerical Analysis (math.NA); Machine Learning (stat.ML)


On a Vizing-type integer domination conjecture

Abstract: Given a simple graph $G$, a dominating set in $G$ is a set of vertices $S$ such that every vertex not in $S$ has a neighbor in $S$. Denote the domination number, which is the size of any minimum dominating set of $G$, by $\gamma(G)$. For any integer $k\ge 1$, a function $f : V (G) \rightarrow \{0, 1, . . ., k\}$ is called a \emph{$\{k\}$-dominating function} if the sum of its function values over any closed neighborhood is at least $k$. The weight of a $\{k\}$-dominating function is the sum of its values over all the vertices. The $\{k\}$-domination number of $G$, $\gamma_{\{k\}}(G)$, is defined to be the minimum weight taken over all $\{k\}$-domination functions. Brešar, Henning, and Klavžar (On integer domination in graphs and Vizing-like problems. \emph{Taiwanese J. Math.} {10(5)} (2006) pp. 1317--1328) asked whether there exists an integer $k\ge 2$ so that $\gamma_{\{k\}}(G\square H)\ge \gamma(G)\gamma(H)$. In this note we use the Roman $\{2\}$-domination number, $\gamma_{R2}$ of Chellali, Haynes, Hedetniemi, and McRae, (Roman $\{2\}$-domination. \emph{Discrete Applied Mathematics} {204} (2016) pp. 22-28.) to prove that if $G$ is a claw-free graph and $H$ is an arbitrary graph, then $\gamma_{\{2\}}(G\square H)\ge \gamma_{R2}(G\square H)\ge \gamma(G)\gamma(H)$, which also implies the conjecture for all $k\ge 2$.

\section{Introduction}
Given a graph $G$, a \emph{dominating set} is a set of vertices $S$ in $G$ with the property that every vertex not in $S$ has a neighbor in $S$. The \emph{domination number} of $G$, written $\gamma(G)$, is the cardinality of a minimum dominating set in $G$. One of the most influential and widely studied conjectures of domination in graphs is \emph{Vizing's conjecture}, originally posed by V.G. Vizing in 1963 \cite{Vizing}. This conjecture states that the domination number of the Cartesian product of graphs $G$ and $H$ is bounded below by the product of the domination numbers of $G$ and $H$.
\begin{conj}{\rm (Vizing's conjecture \cite{Vizing})}\label{Vizing}
For every pair of finite graphs $G$ and $H$,
\begin{equation*}
\gamma(G\, \Box \, H) \geq \gamma(G)\gamma(H).
\end{equation*}
\end{conj}

Attemps to prove Vizing's conjecture are numerous. Indeed, there is a myriad of partial results and weaker formulations. For more on the history of Vizing's conjecture,  we refer the reader to the excellent survey \cite{VizingSurvey}.

In this paper we focus on two domiantion functions which produce values at least as large as the classical domination function: The $2$-domination function and the integer domination function, first defined in \cite{DHLF}, and applied to Cartesian products in \cite{BHK}. In that paper, the authors asked two related Vizing-type questions:

\begin{quest}{\rm (Bre\v{s}ar, Henning, Klav\v{z}ar \cite{BHK})}\label{q1}
For any graphs $G$ and $H$, is it true that
\[\gamma_{\{2\}}(G\, \Box \, H) \geq \gamma(G)\gamma(H)?\]
\end{quest}

and the weaker question

\begin{quest}{\rm (Bre\v{s}ar, Henning, Klav\v{z}ar \cite{BHK})}\label{q2}
Is there a natural number k such that for any pair of graphs $G, H$,
\[\gamma_{\{k\}}(G\, \Box \, H) \geq \gamma(G)\gamma(H)?\]
\end{quest}

These questions also appear in \cite{VizingSurvey}.

In this note we answer these questions in the affirmative for claw-free $G$ and arbitrary $H$ by a vertex labeling method first introduced in~\cite{Krop}. We also show that for claw-free $G$ and any $H$, $\gamma_2(G\, \Box \, H)\ge \gamma(G)\gamma(H)$, where $\gamma_2(G)$ is the $2$-domination number of $G$. In doing so, we provide more evidence for the validity of Vizing's conjecture.

\medskip
\noindent\textbf{Definitions and Notation.}
In this note, all graphs will be considered finite  and simple. Specifically, let $G$ be a graph with vertex set $V(G)$ and edge set $E(G)$. The order and size of $G$ will be denoted by $n(G) = |V(G)|$ and $m(G) = |E(G)|$, respectively. Two vertices $v$ and $w$ in $G$ are adjacent, or neighbors, if $vw\in E(G)$. A set of pairwise non-adjacent vertices in $G$ is an \emph{independent set}, or \emph{stable set}. The open neighborhood of a vertex $v\in V(G)$, written $N_G(v)$, is the set of all neighbors of $v$, whereas the closed neighborhood of $v$ is $N_G[v] = N_G(v)\cup \{v\}$. Let $S\subseteq V(G)$ and $v\in S$. The \emph{open} $S$-\emph{private neighborhood} of $v$ is defined as ${\rm pn}(v,S) = \{w\in V(G): N_G(w)\cap S = \{v\}\}$. A graph $G$ is called \emph{claw-free} if $G$ contains no $K_{1,3}$ as an induced subgraph.

As mentioned previously, a set of vertices $S\subseteq V(G)$ is dominating if every vertex not in $S$ has a neighbor in $S$. If $S$ is a dominating set with the additional property that $S$ is also an independent set, then $S$ is a \emph{independent dominating set}. The cardinality of a minimum independent dominating set in $G$ is the \emph{independent domination number} of $G$, denoted $i(G)$.

A set of vertices $S$ in a graph $G$ is \emph{2-dominating} if every vertex not in $S$ has at least 2 neighbors in $S$. The \emph{2-domination number} of $G$, written $\gamma_2(G)$, is the cardinality of a minimum 2-dominating set in $G$. With this definition, it is clear that every 2-dominating set is also a dominating set, and so, $\gamma_2(G) \ge \gamma(G)$.

For any integer $k\ge 1$, a function $f : V (G) \rightarrow \{0, 1, . . ., k\}$ is called a \emph{$\{k\}$-dominating function} if the sum of its function values over any closed neighborhood is at least $k$. The weight of a $\{k\}$-dominating function is the sum of its values over all the vertices. The $\{k\}$-domination number of $G$, $\gamma_{\{k\}}(G)$, is defined to be the minimum weight taken over all $\{k\}$-domination functions.

For other graph theoretic terminology and definitions, we will typically follow~\cite{MHAYbookTD}.

For a given positive integer $k$, we will also make use of the standard notation $[k] = \{1,\dots, k\}$.

\section{Main Result}
In this section we will prove our main result, but before doing so we will need a useful theorem of Allan and Laskar~\cite{Allan} that states equivalence between the domination number and independent domination number of claw-free graphs. We state this theorem formally as follows.
\begin{thm}{\rm (Allan and Laskar \cite{Allan})}\label{claw-free thm}
If $G$ is a claw-free graph, then $\gamma(G) = i(G)$.
\end{thm}

We will also make use of the following observation.
\begin{ob}\label{claw-free ob}
If $G$ is a claw-free graph and $S$ is a minimum independent dominating  set in $G$, then any vertex not in $S$ is adjacent to either one or two vertices in $S$.
\end{ob}

\begin{defn}
For any integer $k\ge 1$, a function $f : V (G) \rightarrow \{0, 1, . . ., k\}$ is called a \emph{weak $\{k\}$-dominating function} if for $v\in V(G)$ such that $f(v)= 0$, the sum of the function values over the closed neighborhood of $v$ is at least $k$. The weight of a weak $\{k\}$-dominating function is the sum of its values over all the vertices. The weak $\{k\}$-domination number of $G$, $\gamma_{w\{k\}}(G)$, is defined to be the minimum weight taken over all weak $\{k\}$-dominating functions.
\end{defn}

We note that we can make an equivalent formulation of a weak $\{k\}$-dominating function on $G$ by giving multiplicity $\ell$ to a dominating vertex $v$ when $f(v)=\ell$, for $1\le \ell\le k$. We can think of this as having $\ell$ copies of $v$. Call such a distribution of dominating vertices for a weak $\{k\}$-dominating function, a \emph{weak \{k\}-dominating set} of $G$. If $f$ is a weak $\{k\}$-domination function of minimum weight, call the distribution of dominating vertices a \emph{minimum weak $\{k\}$-dominating set}.

We are now ready to present our main result.
\begin{thm}
If $G$ is a claw-free graph and $H$ is a graph, then
\begin{equation*}
\gamma_{w\{2\}}(G\, \Box \, H)\ge \gamma(G)\gamma(H).
\end{equation*}
\end{thm}

\noindent\textbf{Proof. } Let $G$ be a claw-free graph and $H$ be a graph. By Theorem~\ref{claw-free thm}, we may choose an independent dominating set $S\subseteq V(G)$ with $|S| = \gamma(G)$. For notational simplicity, suppose $|S| = k$, and label the vertices of $S$ by $v_1,\dots v_k$. Let $D\subseteq V(G\, \Box \, H)$ be a minimum weak $\{2\}$-dominating set of $G\, \Box \, H$, and so, $|D| = \gamma_{w\{2\}}(G\, \Box \, H)$.

We next devise a labeling scheme for the vertices in $D$ which is split into two separate parts; an initial labeling and a finishing labeling. The following labeling is the initial labeling, and we note that by Observation~\ref{claw-free ob} the following labeling will assign at most 2 entries to each label.
\begin{enumerate}
\item[(1)] For $i\in [k]$, if $(v,h)\in D$ with $v\in \{v_i\}\cup {\rm pn}(v_i,S)$, then label $(v,h)$ by $\{i\}$.

\item[(2)] For distinct $i,j\in [k]$, if $(v,h)\in D$ with $v$ adjacent to both $v_i$ and $v_j$ in $G$, where $(v_i, h') \cap D = \emptyset$ with $h'\in N_H[h]$, and $(v_j, h'')\cap D = \emptyset$ with $h''\in N_H[h]$, then label $(v,h)$ by $\{i,j\}$. If $(v,h)$ contains a vertex of $D$ with multiplicity $2$, label one copy by $\{i\}$ and the other by $\{j\}$.

\item[(3)] For distinct $i,j\in [k]$, if $(v,h)\in D$ with $v$ adjacent to both $v_i$ and $v_j$ in $G$, where $(v_i, h') \cap D = \emptyset$ with $h'\in N_H[h]$, and $(v_j, h'')\cap D \neq \emptyset$ with $h''\in N_H[h]$, then label $(v,h)$ by $\{i\}$.

\item[(4)] For distinct $i,j\in [k]$, if $(v,h)\in D$ with $v$ adjacent to both $v_i$ and $v_j$ in $G$, where $(v_i, h') \cap D \neq \emptyset$ with $h'\in N_H[h]$, and $(v_j, h'')\cap D \neq \emptyset$ with $h''\in N_H[h]$, then label $(v,h)$ by $\{i\}$ or $\{j\}$ arbitrarily.
\end{enumerate}

If $S$ is a perfect independent dominating set of $G$, then all vertices in $G\, \Box \, H$ are assigned labels according to (1) in the above process. In this case the final labeling scheme is not necessary and we may bypass it. The following is the finishing labeling.
 \begin{enumerate}
 \item[(5)] If $(u,h)$ and $(v,h')$ are vertices in $D$ that are both labeled $\{i,j\}$ for distinct $i,j\in [k]$, and $hh'\in E(H)$, then relabel $(u,h)$ by $\{i\}$, and $(v,h')$ by $\{j\}$.

\item[(6)] If $(u,h)$ and $(v,h')$ are vertices in $D$ that are labeled $\{i\}$ and $\{i,j\}$, respectively, for distinct $i,j\in [k]$, and $hh'\in E(H)$, then relabel $(v,h')$ by $\{j\}$.

\item[(7)] If $(u,h)$ and $(v,h)$ are vertices in $D$ that are both labeled $\{i,j\}$ for distinct $i,j\in [k]$, then relabel $(u,h)$ by $\{i\}$, and $(v,h)$ by $\{j\}$.

\item[(8)] If $(u,h)$ and $(v,h)$ are vertices in $D$ with labels $\{i\}$ and $\{i,j\}$, respectively, then relabel $(v,h)$ by $\{j\}$.

\item[(9)] If $(u,h)$ and $(v,h)$ are vertices in $D$ with labels $\{i,j\}$ and $\{j,\ell\}$, respectively, then relabel $(v,h)$ by $\{\ell\}$.

\item[(10)] If $(u,h)$ and $(v,h)$ are vertices of $D$ both labeled $\{i\}$, then we may relabel one of $(u,h)$ or $(v,h)$ by any other label.
\end{enumerate}

\begin{claim}\label{claim}
We may apply labelings (1) - (10) to $D$ and produce a labeling such that each vertex has a label with exactly one entry.
\end{claim}
\noindent\textbf{Proof. } Suppose there exists a vertex of $D$, say $(x_{i_1},h)$, which has been assigned the labeling $\{i_1,i_2\}$. Then, according to labeling (2), $(v_{i_2},h)\cap D = \emptyset$ and $(v_{i_1},h')\cap D = \emptyset$ for all vertices $h$ and $h'$ for which $hh'\in E(H)$. Since $D$ is a 2-dominating set of $G\, \Box \, H$, and so, all vertices not in $D$ have at least two neighbors in $D$, it follows that both $(v_{i_2},h)$ and $(v_{i_1},h')$ have at least one other neighbor distinct from $(x_{i_1},h)$ in $D$. Without loss of generality, we consider the case when the neighbor of $(v_{i_2},h)$ in $D$ which is different than $(x_{i_1},h)$ is the vertex $(v_{i_2},h')$. Since $(x_{i_1},h)$ is adjacent to $(v_{i_2},h')$ in $G\, \Box \, H$, it is clear that $hh'\in E(H)$, and so, by labeling (1), $(v_{i_2},h')$ will have the label $\{i_2\}$. This yields a contradiction since according to labels (3) and (4), $(x_{i_1},h)$ could not have been assigned the labeling $\{i_1,i_2\}$.

We next consider when a vertex $(x_{i_2},h)\in D$ for some $x_{i_2}\in V(G)$, different from $(x_{i_1},h)$, is the neighbor of $(v_{i_2},h)$. First suppose that $(x_{i_2},h)$ is assigned a label that contains $\{i_2\}$ (or $\{i_1\}$). By labeling $(7)$ and $(8)$, two vertices such as $(x_{i_2},h)$ and $(x_{i_1},h)$ would receive labels with one entry, which contradicts the possibility of the label $\{i_1,i_2\}$ on $(x_{i_1},h)$.

Finally, we suppose that $(x_{i_2},h)$ has been assigned the label $i_3$, which is distinct from either $i_1$ or $i_2$, for some $i_3\in[k]$. Let $n$ be the minimal index so that for $2\leq \ell\leq n$, $(x_{i_{\ell-1}},h)$ and $(x_{i_{\ell}},h)$ are adjacent to $(v_{i_{\ell-1}},h)$ and $(x_{i_n},h)$ is adjacent to some vertex $(x_{i_m},h)$ for some $m\in [n-2]$, where $(x_{i_\ell},h)$ is labeled $i_\{\ell+1\}$ for $2\leq \ell \leq n-1$.

We consider the cycle $(v_{i_m},h), (x_{i_m},h), (v_{i_{m+1}},h),\dots, (x_{i_n},h)$. Vertex $(x_{i_n},h)$ may be labeled by $\{i_{n-1}\}$, $\{i_m\}$, or $\{i_m,i_{n-1}\}$. If the label on $(x_{i_n},h)$ contains $\{i_{n-1}\}$, then by labeling (10), we may relabel $(x_{i_{n-1}},h)$ by $\{i_{n-1}\}$, and continue relabeling vertex $(x_{i_\ell},h)$ by $\{i_{\ell}\}$ for $2\leq \ell \leq n-1$. However, by labeling (8), this means $(x_{i_1},h)$ could be labeled by $\{i_1\}$. If the label on $(x_{i_n},h)$ contains $\{i_{m}\}$, then by labeling (10), we may relabel $(x_{i_\ell},h)$ by $\{i_{\ell}\}$ for $2\leq \ell \leq m$. Again, by labeling (8), this means $(x_{i_1},h)$ could be labeled by $\{i_1\}$.

Thus, $(x_{i_1},h)$ could be relabeled by a label with one entry.
{\tiny ($\Box$)}

According to Claim~\ref{claim}, each vertex of $D$ has been assigned a label with a single entry. Choose $i\in[k]$, project all vertices of $D$ labeled $i$ onto $H$, and call the projected vertices $U=\{u_1, \dots, u_{\ell}\}$.

\begin{claim}
The set $U$ is a dominating set of $H$.
\end{claim}
\noindent\textbf{Proof. } By way of contradiction, suppose $U$ is not a dominating set of $H$; that is, there exists $h\in V(H)$ such that $h\notin N[U]$. This means that in $G\, \Box \, H$, $(v_i,h)$ is dominated by some vertex $(v,h)$ of $D$ labeled by $\{j\}$ for some $j\in [k]$ with $j\neq i$. Labelings (3) and (4) could not have been applied in this case, since $h\notin N[U]$. If labeling (2) had been applied to $(v,h)$, and then any or none of the labelings (5), (6), (7), (8), (9), or (10) had been applied, then $(v,h)$ would have been adjacent to some vertex $(u,h)$ labeled $\{i\}$. Since $(v,h)$ is adjacent to $(v_i,h)$ and $(v_j,h)$, labeling (1) does not apply. This produces a contradiction since no labeling could have been applied to $(v,h)$ but all vertices of $D$ are labeled by Claim 1. Thus, $U$ is a dominating set of $H$, and the proof of the claim is finished. {\tiny ($\Box$)}

By Claim 2, for each $i\in[k]$, we may project $D$ onto $H$ and obtain a dominating set of $H$. Recalling $\gamma_{w\{2\}}(G\, \Box \, H) = |D|$, $\gamma(G) = k$, and $|U| \ge \gamma(H)$, we observe the following,
\[
\begin{array}{lcl}
\gamma_{w\{2\}}(G\, \Box \, H) & = & \displaystyle{|D|} \vspace{0.1cm} \\
& \ge & \displaystyle{ \sum_{i=1}^{k}\gamma(H)} \vspace{0.1cm} \\
& = & \displaystyle{\gamma(G)\gamma(H)}.
\end{array}
\]
Thus, the proof of the theorem is complete. $\Box$

Since a $2$-dominating set produces a weak $\{2\}$-dominating function, for any graph $G$, $\gamma_{w\{2\}}(G)\le \gamma_2(G)$. Furthermore, $\gamma_{w\{2\}}(G)\le \gamma_{\{2\}}(G)$ since the $\{2\}$-dominating function is a minimum over a set with more restrictions than the weak $\{2\}$-dominating function. These two observations lead to the following.

\begin{cor}
For any claw-free graph $G$ and any graph $H$,
\begin{align*}
\gamma_{\{2\}}(G\, \Box \, H)\ge \gamma(G)\gamma(H)\\
\gamma_2(G\, \Box \, H)\ge \gamma(G)\gamma(H).
\end{align*}
\end{cor}