Asymptotic enumeration of magic series

Abstract: A magic series is a set of natural numbers that, by virtue of its size, sum, and maximum value, could fill a row of a normal magic square. In this paper, we derive an exact two-dimensional integral representation for the number of magic series of order N. By applying the stationary phase approximation, we develop an expansion in powers of 1/N for the number of magic series and calculate the first few terms. We find excellent agreement between our approximation and the known exact values. Related results are presented for magic cube and hypercube series, bimagic series, and trimagic series.

\section{Introduction}

A (normal) \emph{magic square} of order $N$ is an arrangement of the integers from $1$ to
$N^2$ in an $N\times N$ grid such that each row, column, and diagonal has the
same sum.  The common sum is given by
\begin{equation}
S_1 (N)=\frac{1}{N}\sum_{i=1}^{N^2}i=\frac{1}{2}N (N^2+1).
\end{equation}
A subset of $[N^2]=\{1,2,\ldots,N^2\}$ that could populate a row of a normal magic square
(i.e., an $N$-element subset of $[N^2]$ whose sum is $S_1$) is known as a \emph{magic
square series}.  Analogously, a (normal)
\emph{magic cube} of order $N$ is an arrangement of the integers from $1$ to $N^3$ into a cube
such that each row, column, pillar, and space diagonal has the same sum;
legal subsets for the rows of a normal magic cube are \emph{magic cube series};
and so on for hypercubes of any order.

The number of magic square series of order $N$ has been computed exactly for $N$ as
large as $1000$~\cite{trump-enum}, and is empirically well-described by H. Bottomley's formula:
\begin{equation}
\label{eq:bottomley}
{\cal N}_{\rm{2}} (N) \approx \left(\frac{1}{\pi} \sqrt{\frac{3}{e}} \right)\cdot \frac{\left(Ne\right)^{N}}{N^3 - \frac{3}{5}N^2 +
\frac{2}{7}N}.
\end{equation}
While the leading-order behavior can be given a statistical justification, the
coefficients of the correction terms are purely empirical~\cite{bottomley-eq}.  The goal of the
present paper is to derive the leading-order behavior and the first two
correction terms analytically, and to present a perturbative framework within which
arbitrarily high-order correction terms can be calculated.

The structure of the paper is as follows.  In Section 2, we describe a
probabilistic problem that is equivalent, for various parameter values,
to the enumeration of magic series for squares, cubes, and hypercubes.
By applying the central limit theorem to this probabilistic problem, we find
approximate (Gaussian) formulae for the number of magic series of each type.
In Section 3, we derive an exact two-dimensional integral representation of the desired result,
and show how corrections to the Gaussian formulae can be expressed
as perturbation series in $1/N$.  We compute the first- and second-order
correction terms in these series.  In Section 4, we collect our results
for magic series and compare them to the exact counts that are known.
Finally, in Section 5, we discuss extensions to bimagic and trimagic series,
compute approximations to the number of bimagic square series,
bimagic cube series, and trimagic square series,
and compare these results to known exact counts.

\section{Probabilistic approach}
\subsection{Formulation}
For each $i \ge 1$, let $(X_i, Y_i)$ be a random vector that is equal to $(1, i)$ with probability
$\beta$ and equal to $(0, 0)$ otherwise.
Then the partial sums $(A_m, B_m) = \sum_{i=1}^{m} (X_i, Y_i)$ have the following
interpretation: the probability that $(A_m, B_m)$ is equal to $(A, B)$ is the probability of
selecting $A$ numbers and arriving at a total of $B$, selecting each value in $\{1,2,\ldots,m\}$
for inclusion with probability $\beta$.
When we convert this probability to a \emph{count}, the relevance to enumerating magic
series is clear.  Specifically,
\begin{eqnarray}
\label{statform}
p (A, B) &\equiv& P[ (A_m, B_m) = (A,B) ] \nonumber \\
&=& \beta^{A} (1-\beta)^{m-A} {\cal N} (m, A, B),
\end{eqnarray}
where ${\cal N}(m, A, B)$ is the number of subsets of $\{1, 2, \ldots, m\}$ that have
$A$ elements and sum to $B$.  For appropriately chosen arguments, the
value of $\cal{N}$ is equal to the number of magic series for squares, cubes, or
hypercubes.

For magic series, we will be interested in cases where $A_m$ and $B_m$
take on their mean values, which are
\begin{eqnarray}
\mu_x &=& E[A_m] = \beta \sum_{i=1}^{m} 1 = \beta m \\
\mu_y &=& E[B_m] = \beta \sum_{i=1}^{m} i = \frac{1}{2} \beta m (m + 1).
\end{eqnarray}
Define the ``central probability" to be
\begin{equation}
P^{*}(m, \beta) = p(\mu_x, \mu_y) = p\left(\beta m, \frac{1}{2}\beta
m(m+1)\right).
\end{equation}
The corresponding count is given by
\begin{eqnarray}
\label{prefactor}
{\cal N}^{*}(m, \beta) &=& {\cal N} \left(m, \beta m, \frac{1}{2}\beta m(m+1)\right)
\nonumber
\\ &=& \beta^{-\beta m}(1-\beta)^{-(1-\beta)m} P^{*}(m, \beta).
\end{eqnarray}
The prefactor in Eq.~(\ref{prefactor})
is exact.  We will now develop an approximation scheme for
the central probability $P^{*}(m, \beta)$, beginning with the Gaussian approximation.

\subsection{Gaussian approximation}

As $m$ becomes large, the central limit theorem (the Lyapunov central limit
theorem, specifically, since the summed variables are independent but not identically
distributed) implies that the distribution of $(A_m, B_m)$ approaches a two-dimensional Gaussian.
The center of the Gaussian is $(\mu_x, \mu_y)$.
The components of its covariance matrix are
\begin{eqnarray}
\Sigma_{xx} &=& E[A_m^2] - E[A_m]^2 \nonumber
\\ &=& \beta (1-\beta) m
\\
\Sigma_{xy} &=& E[A_m B_m] - E[A_m] E[B_m]  \nonumber
\\ &=& \frac{1}{2} \beta (1-\beta) m (m + 1)
\\
\Sigma_{yy} &=& E[B_m^2] - E[B_m]^2  \nonumber
\\ &=& \frac{1}{6} \beta (1-\beta) m (m + 1) (2m + 1).
\end{eqnarray}
The limiting distribution of $(A_m, B_m)$ is therefore
\begin{equation}
p_{g}(A,B) = \frac{1}{2\pi\sqrt{\det\Sigma}}\exp(\cdots) = \frac{1}{\pi \beta(1-\beta) m} \sqrt{\frac{3}{m^2-1}}
\exp(\cdots),
\end{equation}
where the exponent is quadratic in the distance from $(A, B)$ to the center of the
distribution, and we have used the following for the square root of the determinant
of the covariance matrix:
\begin{eqnarray}
\sqrt{\det \Sigma} &=& \beta (1-\beta) \sqrt{ \frac{1}{6} m^2 (m+1) (2m+1) - \frac{1}{4} m^2 (m+1)^2 } \nonumber
\\ &=& \frac{1}{2} \beta (1-\beta) m \sqrt{\frac{m^2 - 1}{3}}.
\end{eqnarray}
In the Gaussian approximation, then, the central probability is
\begin{equation}
P^{*}_{g}(m,\beta) = \frac{1}{\pi \beta(1-\beta) m} \sqrt{\frac{3}{m^2-1}}.
\end{equation}
We can obtain results for various interesting cases by
specifying $m$ and $\beta$.

\subsection{Magic square series}

The number of magic square series of order $N$ is given by ${\cal N}_2(N) = {\cal N}(N^2, N,
S_1)$, where $S_1=\frac{1}{2}N(N^2+1)$.
We see that if we choose $m=N^2$ and $\beta = 1/N$, then the center of the
Gaussian is located at $(\mu_x, \mu_y) = (N, S_1)$.  In this case, the prefactor in Eq.~({\ref{prefactor}}) is
\begin{eqnarray}
\label{prefactor-squares}
\beta^{-\beta m}(1-\beta)^{-(1-\beta)m} &=& N^{N}\left(1-\frac{1}{N}\right)^{-(N^2-N)}  \nonumber
\\ &=& N^N \exp\left[-(N^2-N)\ln\left(1-\frac{1}{N}\right)\right] \nonumber
\\ &=& N^N \exp\left[N - \frac{1}{2} - \frac{1}{6N} - \frac{1}{12N^2} + \cdots\right]
\nonumber
\\ &=& \frac{\left(Ne\right)^{N}}{\sqrt{e}} \exp\left[-\frac{1}{6N}-\frac{1}{12N^2}+\cdots\right]
\nonumber
\\ &=& \frac{\left(Ne\right)^{N}}{\sqrt{e}} \left(1 - \frac{1}{6N} - \frac{5}{72N^2} +
\cdots\right),
\end{eqnarray}
where the series is readily extended to any order.  The Gaussian approximation for the number of
magic square series of order $N$ is therefore
\begin{eqnarray}
{\cal N}_{2,g}(N) &=&
\frac{\left(Ne\right)^{N}}{\sqrt{e}} \left(1 - \frac{1}{6N} - \frac{5}{72N^2} +
\cdots\right)\times P_{g}^{*}(N^2, 1/N) \nonumber \\
&=&
\frac{\left(Ne\right)^{N}}{\sqrt{e}} \left(1 - \frac{1}{6N} - \frac{5}{72N^2} +
\cdots\right)\times \frac{1}{\pi (N-1)} \sqrt{\frac{3}{N^4-1}} \nonumber
\\ &=& \left(\frac{1}{\pi} \sqrt{\frac{3}{e}} \right)\cdot \frac{\left(Ne\right)^{N}}{N^3} \left(1 + \frac{5}{6N} + \frac{55}{72N^2} + \cdots\right).
\end{eqnarray}

When we compare this estimate to the actual counts for large $N$,
we find that it is
correct to leading order, but the higher-order correction coefficients are wrong.  (For instance, the coefficient of $1/N$
appears to be $3/5$, not $5/6$.)  This should not be terribly surprising.  First, although the CLT
guarantees a Gaussian distribution in the limit $m\rightarrow\infty$, there are
finite-$m$ corrections that affect the higher-order terms.  Second, as the careful reader will have noted,
the CLT no longer applies when $\beta$ is a function of $m$.  We can only guarantee convergence
to a normal distribution when $\beta$ is fixed.  To address both these problems,
we will need to systematically include non-Gaussian corrections, as we will do in Section 3.

\subsection{Magic cube series}

The number of magic cube series of order $N$ is given by ${\cal N}_3(N) = {\cal N}(N^3, N,
T_1)$, where
\begin{equation}
T_1(N) = \frac{1}{N^2}\sum_{i=1}^{N^3}i = \frac{1}{2}N(N^3+1).
\end{equation}
If we choose $m=N^3$ and $\beta=1/N^2$, then the center of the Gaussian is
located at $(\mu_x, \mu_y) = (N, T_1)$.  The prefactor in this case is
\begin{eqnarray}
\label{prefactor-cubes}
\beta^{-\beta m}(1-\beta)^{-(1-\beta)m} &=& N^{2N}\left(1-\frac{1}{N^2}\right)^{-(N^3-N)}
\nonumber
\\ &=& N^{2N} \exp\left[-(N^3-N)\ln\left(1-\frac{1}{N^2}\right)\right] \nonumber
\\ &=& N^{2N} \exp\left[N - \frac{1}{2N} - \frac{1}{6N^3} + \cdots\right]
\nonumber
\\ &=& \left(N^2 e\right)^{N} \exp\left[-\frac{1}{2N}-\frac{1}{6N^3}+\cdots\right]
\nonumber
\\ &=& \left(N^2 e\right)^{N} \left(1 - \frac{1}{2N} + \frac{1}{8N^2} +
\cdots\right),
\end{eqnarray}
and the Gaussian approximation for the number of magic cube series
of order $N$ is
\begin{eqnarray}
{\cal N}_{3,g}(N) &=& \left(N^2 e\right)^{N}\left(1 - \frac{1}{2N} + \frac{1}{8N^2} +
\cdots\right) \times P_{g}^{*}(N^3, 1/N^2) \nonumber
\\ &=&
\left(N^2 e\right)^{N}\left(1 - \frac{1}{2N} + \frac{1}{8N^2} +
\cdots\right) \times \frac{1}{\pi \left(N-\frac{1}{N}\right)} \sqrt{\frac{3}{N^6-1}}
\nonumber
\\ &=& \left(\frac{\sqrt{3}}{\pi} \right) \cdot \frac{\left(N^2 e\right)^{N}}{N^4}
\left(1 - \frac{1}{2N} + \frac{9}{8N^2} + \cdots\right).
\end{eqnarray}
As with the magic square series, a comparison to the exact counts shows that the leading order here is
correct, while the higher-order terms are wrong.

\subsection{Magic series for $\alpha$-hypercubes}

Generalizing the above by taking $m=N^{\alpha}$ and $\beta=1/N^{\alpha-1}$, we obtain an estimate for
the number of magic series for $\alpha$-hypercubes.  Assuming $\alpha > 3$, the prefactor becomes
\begin{eqnarray}
\beta^{-\beta m}(1-\beta)^{-(1-\beta)m} &=& N^{(\alpha-1) N}\left(1-\frac{1}{N^{\alpha-1}}\right)^{-(N^\alpha-N)}
\nonumber
\\ &=& N^{(\alpha-1)N} \exp\left[-(N^\alpha-N)\ln\left(1-\frac{1}{N^{\alpha-1}}\right)\right]
\nonumber
\\ &=& N^{(\alpha-1)N} \exp\left[N - \frac{1}{2N^{\alpha-2}} + \cdots\right]
\nonumber
\\ &=& \left(N^{\alpha-1} e\right)^{N} \exp\left[-\frac{1}{2N^{\alpha-2}} +\cdots\right]
\nonumber
\\ &=& \left(N^{\alpha-1} e\right)^{N} \left(1 - \frac{1}{2N^{\alpha-2}} + \cdots\right).
\end{eqnarray}
The Gaussian approximation for the number of magic series for $\alpha$-hypercubes of order $N$
($\alpha > 3$) is
\begin{eqnarray}
{\cal N}_{\alpha>3,g}(N) &=& \left(N^{\alpha-1} e\right)^{N}\left(1 - \frac{1}{2N^{\alpha-2}} +
\cdots\right) \times P_{g}^{*}(N^\alpha, 1/N^{\alpha-1}) \nonumber
\\ &=&
\left(N^{\alpha-1} e\right)^{N}\left(1 - \frac{1}{2N^{\alpha-2}} +
\cdots\right) \times \frac{1}{\pi \left(N-\frac{1}{N^{\alpha-1}}\right)} \sqrt{\frac{3}{N^{2\alpha}-1}}
\nonumber
\\ &=& \left(\frac{\sqrt{3}}{\pi} \right) \cdot \frac{\left(N^{\alpha-1} e\right)^{N}}{N^{\alpha + 1}}
\left(1 - \frac{1}{2N^{\alpha-2}} + \cdots\right).
\end{eqnarray}
Again, only the leading-order behavior is correct.

\section{Integral representation}

\subsection{Formulation}
Now consider evaluating the probability distribution $p(x, y)$ without
appealing to the central limit theorem.  The probability distribution is a
convolution of $m$ simple distributions; in Fourier space, this convolution
becomes a product.  Defining $\tilde{p}(k_x, k_y) = \sum_{x,y}\exp(i{\bf k}\cdot{\bf x})
p(x, y)$, we have
\begin{equation}
\tilde{p}(k_x, k_y) = \prod_{j=1}^{m} \tilde{p_j}(k_x, k_y),
\end{equation}
where
\begin{eqnarray}
\tilde{p_j}(k_x, k_y) &=& (1-\beta) + \beta e^{i z_j} \nonumber
\\ &=& \exp\left[\ln \left(1 - \beta(1-e^{iz_j})\right) \right],
\end{eqnarray}
and $z_j=(1, j)\cdot {\bf k}= k_x + jk_y$.  Taking the product over $j$
gives
\begin{equation}
\tilde{p}(k_x, k_y) = \exp\left[\sum_{j=1}^{m}\ln \left(1 -
\beta(1-e^{iz_j})\right)\right].
\end{equation}
The central probability is then given by the inverse Fourier
transform:
\begin{eqnarray}
\label{exact-integral-rep}
P^{*}(m, \beta) &=& p(\mu_x, \mu_y) \nonumber \\
&=& \int_{-\pi}^{\pi}\int_{-\pi}^{\pi}\frac{dk_x dk_y}{(2\pi)^2}\exp\left(-i\mu_x k_x-i\mu_y k_y\right)
\tilde{p}(k_x, k_y) \nonumber \\
&=&
\int\frac{d^2 k}{(2\pi)^2}\exp\left\{\sum_{j=1}^{m}\left[-i\beta z_j + \ln\left(1-\beta(1-e^{iz_j})\right)\right]\right\}.
\end{eqnarray}
This expression, while exact, is not of very much use as it stands, and
will need to be approximated.

\subsection{Stationary phase approximation}

First, expanding in $\beta$, we rewrite the exponent as
\begin{equation}
\beta\sum_{j=1}^{m}(e^{iz_j}-1-iz_j) -
\sum_{k=2}^{\infty}\frac{\beta^k}{k}\sum_{j=1}^{m}(1-e^{iz_j})^k.
\end{equation}
This expansion is convergent for small $\beta$.  We now propose that the primary contribution to
the integral in Eq.~(\ref{exact-integral-rep}) comes from the region around the origin, where the
real part of the exponent is zero and its imaginary part is stationary with
respect to $k$.
This leads us to employ the \emph{stationary phase approximation}, in which the integral of a rapidly oscillating function is approximated
by the local integral(s) at the point(s) of stationary phase.  As the oscillations in
phase become increasingly rapid (which we expect to occur here as $m\rightarrow \infty$),
contributions from all other points are exponentially suppressed.  (In this case, the amplitude of
the integrand is also exponentially suppressed away from the origin.)

Collecting like powers of $z$, we have
\begin{equation}
-\frac{1}{2}\beta(1-\beta)\sum_{j=1}^{m}z_j^2 + \beta\sum_{k=3}^{\infty} \frac{f_k(\beta) \sum_{j=1}^{m}\left(iz_j\right)^k}{k!},
\end{equation}
in the exponent, where each ``vertex correction" function $f_k(\beta)$ is equal to $1$ plus
corrections of order $\beta$.  To be more precise, the first few vertex corrections are
\begin{eqnarray}
f_3(\beta) &=& 1 - 3\beta + 2\beta^2 \\
f_4(\beta) &=& 1 - 7\beta + 12\beta^2 - 6\beta^3.
\end{eqnarray}
Summing over $j$ then gives the exponent as
\begin{eqnarray}
- \frac{\Sigma_{ab}}{2}k_{a}k_{b} - \frac{i \beta f_3(\beta) T_{abc}}{6} k_{a} k_{b} k_{c}
+ \frac{\beta f_4(\beta) S_{abcd}}{24} k_{a}k_{b}k_{c}k_{d} + \ldots,
\end{eqnarray}
where repeated indices are summed over, $\Sigma$ is exactly the same as before, and
\begin{eqnarray}
T_{xxx} &=& \sum_{j=1}^{m} 1 = m  \\
T_{xxy} &=& \sum_{j=1}^{m} j =\frac{1}{2} m(m+1) \\
T_{xyy} &=& \sum_{j=1}^{m} j^2 =\frac{1}{3} m(m+1)\left(m+\frac{1}{2}\right) \\
T_{yyy} &=& \sum_{j=1}^{m} j^3 =\frac{1}{4} m^2(m+1)^2.
\end{eqnarray}
All higher-order vertices follow the same pattern: the value of $V_{abc\ldots}$,
where $V$ is any vertex and $abc\ldots$ are its indices,
is equal to $\sum_{j=1}^{m} j^{n_y(abc\ldots)}$, where $n_y$ counts the number
of $y$'s among the indices.  To leading order in $m$,
\begin{equation}
  V_{abc\ldots} = \frac{m^{1 + n_ y(abc\ldots)}}{1 + n_y(abc\ldots)}.
\end{equation}
The central probability is now given by
\begin{eqnarray}
P^{*}(m, \beta) &=&
\int\frac{d^2 k}{(2\pi)^2}\exp\Big[- \frac{\Sigma_{ab}}{2}k_{a}k_{b} - \frac{i\beta f_3(\beta) V_{abc}}{6} k_{a} k_{b}
k_{c} \nonumber
\\ &&\qquad\qquad+ \frac{\beta f_4(\beta) V_{abcd}}{24} k_{a}k_{b}k_{c}k_{d} + \ldots\Big].
\end{eqnarray}
Using rescaled variables $\tilde{k}_x=\sqrt{m\beta(1-\beta)} k_x$
and $\tilde{k}_y=m\sqrt{m\beta(1-\beta)} k_y$, we remove all $\beta$-dependence
and the leading-order dependence on $m$ from the vertex terms, giving
\begin{eqnarray}
&& P^{*}(m, \beta) = \frac{1}{\beta(1-\beta)m^2} \int\frac{d^2 \tilde{k}}{(2\pi)^2}\exp\Big[- \frac{\tilde{\Sigma}_{ab}}{2}\tilde{k}_{a}\tilde{k}_{b}
\nonumber \\ && \qquad\qquad\qquad - \frac{i \tilde{f}_3(\beta) \tilde{V}_{abc}}{6 \sqrt{\beta m}}  \tilde{k}_{a} \tilde{k}_{b}
\tilde{k}_{c} +
 \frac{\tilde{f}_4(\beta)\tilde{V}_{abcd}}{24 \beta m} \tilde{k}_{a}\tilde{k}_{b}\tilde{k}_{c}\tilde{k}_{d}
+ \ldots\Big].
\end{eqnarray}
Here
\begin{equation}
\tilde{\Sigma} = \left(
\begin{array}{cc}
  1 & \frac{1}{2}(1+\frac{1}{m}) \\
  \frac{1}{2}(1+\frac{1}{m}) & \frac{1}{3} (1+\frac{1}{m})(1+\frac{1}{2m})
\end{array}
\right)
\end{equation}
exactly, and
\begin{equation}
\tilde{V}_{abc\ldots} = \frac{1}{1 + n_y(abc\ldots)}
\end{equation}
up to corrections of order $1/m$, and $\tilde{f}_k(\beta) =
(1-\beta)^{-k/2}f_k(\beta)$.  The first two rescaled vertex corrections are
\begin{eqnarray}
\tilde{f}_3(\beta) &=& 1 - \frac{3}{2}\beta + O(\beta^2) \\
\tilde{f}_4(\beta) &=& 1 - 5\beta + O(\beta^2).
\end{eqnarray}
For simplicity we will drop the tildes on $k$ and work with the rescaled variables from this point
forward.  At this point we have arrived at our main theoretical result:
\begin{eqnarray}
\label{perturb-series}
&&P^{*}(m, \beta) = P_{g}^{*}(m, \beta) \times \nonumber \\ &&\qquad\left\langle \exp\left(-\frac{i\tilde{f}_3(\beta)\tilde{V}_{abc}}{6\sqrt{\beta m}}k_a k_b k_c + \frac{\tilde{f}_4(\beta)\tilde{V}_{abcd}}{24\beta m}k_a k_b k_c k_d + \ldots\right) \right\rangle
\end{eqnarray}
where the expectation value is evaluated with respect to the normalized Gaussian
distribution with covariance matrix $\tilde{\Sigma}^{-1}$.  Because each additional power of $k$ (that is,
each new vertex) comes with a multiplier of $(\beta m)^{-1/2}$, expanding the exponential gives
us an asymptotic series, in powers of $1/(\beta m)$, for the multiplicative correction
to $P_{g}^{*}(m, \beta)$.  Odd powers of $k$ evaluate to zero by symmetry, so
only integer powers of $1/(\beta m)$ appear.

\subsection{First-order correction}

To first order in $1/(\beta m)$, we have
\begin{equation}
  P^{*}(m, \beta) \approx P_g^{*}(m, \beta)\left(1 + \frac{K_1}{\beta m}\right),
\end{equation}
where
\begin{equation}
K_1 = -\frac{\tilde{V}_{abc} \tilde{V}_{def}}{72}\langle k_a k_b k_c k_d k_e k_f \rangle + \frac{\tilde{V}_{abcd}}{24}\langle k_a k_b k_c k_d
\rangle.
\end{equation}
Note that the cubic and quartic (in $k$) corrections both contribute to this order.
The usual diagrammatic approach applies: the ``propagator" is
\begin{equation}
  \langle k_a k_b \rangle = \Pi_{ab},
\end{equation}
where
\begin{equation}
  \Pi \equiv {\tilde{\Sigma}}^{-1} = \left(
  \begin{array}{cc}
    4 & -6 \\ -6 & 12
  \end{array}\right)
\end{equation}
to leading order
in $1/m$, and expectation values of higher powers of $k$ can be evaluated by combining pairs
of $k$'s in every possible way, e.g.,
\begin{equation}
\langle k_a k_b k_c k_d \rangle = \Pi_{ab} \Pi_{cd} + \Pi_{ac} \Pi_{bd} + \Pi_{ad}
\Pi_{bc}.
\end{equation}
Because $\tilde{V}$ is symmetric on its indices, this gives
\begin{equation}
\label{eq:k1}
K_1 = -\frac{\tilde{V}_{abc} \tilde{V}_{def}}{8} \Pi_{ab}\Pi_{cd}\Pi_{ef} -\frac{\tilde{V}_{abc} \tilde{V}_{def}}{12}\Pi_{ad}\Pi_{be}\Pi_{cf} + \frac{\tilde{V}_{abcd}}{8} \Pi_{ab} \Pi_{cd} .
\end{equation}
In terms of diagrams, we are summing over all distinct diagrams
for which the sum of the vertex degrees is equal to twice the number
of vertices plus two.  There are three
such diagrams, corresponding to the three terms in Eq.~(\ref{eq:k1}).
A straightforward summation over the indices yields the correction
coefficient:
\begin{equation}
K_1 = -\frac{1}{8}\cdot 4 -\frac{1}{12}\cdot 4 + \frac{1}{8}\cdot\frac{24}{5} = -\frac{7}{30}.
\end{equation}

\subsection{Higher-order corrections}

The coefficient $K_1$ derived in the previous section is
\emph{universal}, in that it describes the first non-Gaussian correction to $P^{*}$
whenever $\beta\rightarrow 0$ and $(\beta m) \rightarrow \infty$.  In
particular, the same value of $K_1$ applies in counting magic
series for hypercubes of any order.  In each case, the Gaussian approximation
derived in Section 2 must be multiplied by $1 + K_1/(\beta m) = 1 - 7/(30N)$ to
yield the correct expression to order $1/N$.

Higher-order corrections are not universal in the same way, because there are
correction terms in powers of $1/m$ (from approximations made to the propagator and
the vertex), powers of $1/(\beta m)$ (from higher-order diagrams), and
powers of $\beta$ (from the vertex corrections).  Depending on the
dimensionality of the problem, these correction terms scale differently.  For magic
square series, $\beta = 1/(\beta m)=1/N$, and so we need both the next-order terms in $\beta$
and the next round of diagrams to get the coefficient of $1/N^2$ in the large-$N$ expansion.
For magic cube and hypercube series, on the other hand, $\beta \ll 1/N$, and so
only the next round of diagrams are required for order $1/N^2$.  (The approximations made to the
propagator and the vertex produce still finer corrections: they enter the calculation at order $1/N^3$
for magic series for squares, and at even higher orders for cubes and hypercubes.)

So, to correctly determine the next order in $1/N$ we need at most two types of additional correction.  Omitting the
details, we find
\begin{equation}
P^{*}(m, \beta) \approx P^{*}_{g}(m, \beta) \left(1 + \frac{K_1}{\beta m} + \frac{K_2}{m} + \frac{K_3}{(\beta m)^2}\right),
\end{equation}
where
\begin{eqnarray}
K_2 &=& \frac{3\tilde{V}_{abc} \tilde{V}_{def}}{8} \Pi_{ab}\Pi_{cd}\Pi_{ef} +\frac{\tilde{V}_{abc} \tilde{V}_{def}}{4}\Pi_{ad}\Pi_{be}\Pi_{cf} - \frac{5\tilde{V}_{abcd}}{8} \Pi_{ab} \Pi_{cd}
\nonumber \\
&=& \frac{3}{8}\cdot 4 + \frac{1}{4}\cdot 4 - \frac{5}{8}\cdot\frac{24}{5}
\nonumber \\
&=& -\frac{1}{2}
\end{eqnarray}
arises from $\tilde{f}_k(\beta)$-corrections, and includes only the three diagrams
already considered; and
\begin{eqnarray}
K_3 &=&
\frac{\tilde{V}_{abc}\tilde{V}_{def}\tilde{V}_{ghi}\tilde{V}_{jkl}}{31104}\langle k_{abcdefghijkl} \rangle
- \frac{\tilde{V}_{abc}\tilde{V}_{def}\tilde{V}_{ghij}}{1728}\langle k_{abcdefghij}\rangle
\nonumber \\ &&
+ \frac{\tilde{V}_{abc} \tilde{V}_{defgh}}{720} \langle k_{abcdefgh}\rangle
+ \frac{\tilde{V}_{abcd}\tilde{V}_{efgh}}{1152}\langle k_{abcdefgh}\rangle
- \frac{\tilde{V}_{abcdef}}{720}\langle k_{abcdef}\rangle
\nonumber \\  &=& \frac{205}{72} - \frac{31}{6} + \frac{22}{15} + \frac{17}{15} - \frac{29}{105}
\nonumber \\  &=& \frac{11}{2520}
\end{eqnarray}
includes a large number of higher-order diagrams.  The summation was performed with the help
of a computer.

\section{Magic series results}

After multiplying the Gaussian results from Section 2 by the correction series
derived in the previous section, the second-order results for the various
types of magic series are as follows:
\begin{eqnarray}
{\cal N}_{2}(N) &\approx& \left(\frac{1}{\pi} \sqrt{\frac{3}{e}} \right)\cdot \frac{\left(Ne\right)^{N}}{N^3} \left(1 + \frac{3}{5N} +
\frac{31}{420N^2}\right).
\\
{\cal N}_{3}(N) &\approx& \left(\frac{\sqrt{3}}{\pi} \right) \cdot \frac{\left(N^2 e\right)^{N}}{N^4}
\left(1 - \frac{11}{15N} + \frac{157}{126N^2}\right).
\\
{\cal N}_{4}(N) &\approx& \left(\frac{\sqrt{3}}{\pi} \right) \cdot \frac{\left(N^{3} e\right)^{N}}{N^{5}}
\left(1 - \frac{7}{30N} - \frac{1249}{2520N^2}\right).
\\
{\cal N}_{\alpha > 4}(N) &\approx& \left(\frac{\sqrt{3}}{\pi} \right) \cdot \frac{\left(N^{\alpha-1} e\right)^{N}}{N^{\alpha + 1}}
\left(1 - \frac{7}{30N} + \frac{11}{2520N^2}\right).
\end{eqnarray}
It is interesting to note that if the expansion of ${\cal N}_2(N)$ in decreasing powers of $N$
is moved to the denominator, one obtains a very slight correction to Bottomley's
formula, Eq.~(\ref{eq:bottomley}), for the number of magic square series :
\begin{equation}
{\cal N}_{\rm{2}}(N) \approx \left(\frac{1}{\pi} \sqrt{\frac{3}{e}} \right)\cdot \frac{\left(Ne\right)^{N}}{N^3 - \frac{3}{5}N^2 +
\left(\frac{2}{7} + \frac{1}{2100}\right)N}.
\end{equation}
The coefficient $(2/7)$ in Bottomley's empirical formula is replaced by $(2/7) + (1/2100)$.
All coefficients in the above formulae are exact;
all besides the $(3/5)$ in ${\cal N}_2(N)$ are novel, to the author's knowledge.

Table~\ref{table:magic-series} compares these second-order estimates with the exact counts, which are
known at least up to $N=1000$ for squares, $N=200$ for cubes, and $N=20$ for $4$-dimensional
hypercubes~\cite{trump-enum}.  The agreement is excellent: at these largest known values,
our second-order approximations are
correct to (respectively) $11$, $7$, and $4$ decimal places.  Moreover, the
exact values allow us approximate the unknown third-order terms fairly well.
Based on the contents of Table~\ref{table:magic-series}, the third-order terms are roughly
$0.032/N^3$, $-1.44/N^3$, and $1.12/N^3$ for squares, cubes, and $4$-dimensional
hypercubes respectively.

\begin{table}
  \begin{center}
\begin{tabular}{|c|c|c|c|c|}
  \hline
   ${\alpha}$ & N & Est. vs. exact magic series & Rel. residual & Residual $\times N^3$ \\
  \hline
  2 & 500 & $1.148464537\underbar{053} \times 10^{1558}$ & $2.46\times 10^{-10}$ & 0.031 \\
     &        & $1.14846453733617811101 \times 10^{1558}$ & & \\
  \hline
  2 & 700 & $3.66527778\underbar{173} \times 10^{2286}$ & $9.00 \times 10^{-11}$ & 0.031 \\
     &  & $3.66527778205981116969 \times 10^{2286}$ & &\\
\hline
2 & 1000 & $6.591829225\underbar{191} \times 10^{3424}$ & $3.18 \times 10^{-11}$ & 0.032 \\
 &  & $6.59182922540146398832 \times 10^{3424}$ & & \\
  \hline
  \hline
  3 & 100 & $1.4713530\underbar{522}\times 10^{435}$ & $-1.43 \times 10^{-6}$ & -1.43 \\
    &  & $1.47135094282799112413\times 10^{435}$ & & \\
\hline
  3 & 150 & $1.01505\underbar{942}\times 10^{709}$ & $-4.25 \times 10^{-7}$ & -1.44 \\
    &  & $1.01505898472535873940 \times 10^{709}$ & & \\
\hline
  3 & 200 & $6.40624\underbar{667} \times 10^{997}$ & $-1.80 \times 10^{-7}$ & -1.44 \\
    &  & $6.40624551588569444014\times 10^{997}$ & & \\
  \hline
  \hline
  4 & 10 & $1.18\underbar{003}\times 10^{29}$ & $1.10\times 10^{-3}$ & 1.10\\
   &  & $1.18132052487666384802\times 10^{29}$ & & \\
 \hline
  4 & 15 & $1.95\underbar{748}\times 10^{53}$ & $3.28\times 10^{-3}$ & 1.11\\
    &  & $1.95811840766424991031\times 10^{53}$ & & \\
 \hline
   4 & 20 & $9.51\underbar{281}\times 10^{79}$ & $1.39\times 10^{-4}$ & 1.11 \\
   &  & $9.51413048876962267360\times 10^{79}$ & & \\
  \hline
\end{tabular}
\caption{{\bf Comparison of second-order magic series estimates to selected exact values.}  Results are
shown for the number of magic square series ($\alpha=2$), magic cube series ($\alpha=3$), and magic
series for $4$-dimensional hypercubes.  The underlined
numbers are the first incorrect digits in each estimate.  The final column approximates the
coefficient of the $N^{-3}$ term for each $\alpha$; note that these approximations appear
to have converged.}
\label{table:magic-series}
\end{center}
\end{table}

\section{Extension to multimagic series}

\subsection{Bimagic series}

A \emph{bimagic square} is a magic square with the additional property that
the squares of the entries in each row, column, and diagonal have the same sum;
one then defines \emph{bimagic square series} as before.  Moreover, one can define
\emph{bimagic cube series}, etc., as well as trimagic and higher multimagic
squares, cubes, etc., and their associated series.
The method described in this paper can be adapted to enumerate multimagic series
as well.  We limit ourselves here to briefly describing its application
to bimagic and trimagic series.

For bimagic series, we will want start by considering random vectors $(X_i, Y_i, Z_i)$
that are equal to $(1,i,\frac{1}{2}i(i-1))$ with probability $\beta$ and equal
to $(0,0,0)$ otherwise.  (If we were to choose $Z_i=i^2$ instead, $Y_i$ and $Z_i$
would be equal modulo $2$; in the $k$-space analysis, this would lead to multiple points of stationary
phase, complicating the problem.)  The probability distribution of the sum of the first $m$
such vectors is related to the number of bimagic series just as in
Eq.~(\ref{statform}).  This probability distribution approaches a
three-dimensional Gaussian centered at
\begin{eqnarray}
\mu_x &=& \beta\sum_{i=1}^{m}1 = \beta m \\
\mu_y &=& \beta\sum_{i=1}^{m}i = \frac{1}{2}\beta m(m+1) \\
\mu_z &=& \beta\sum_{i=1}^{m}\frac{1}{2}i(i-1) = \frac{1}{6}\beta m(m^2-1).
\end{eqnarray}
Its covariance matrix has components
\begin{eqnarray}
\Sigma_{xx} &=& \beta(1-\beta) m \\
\Sigma_{xy} &=& \frac{1}{2}\beta(1-\beta)m(m+1) \\
\Sigma_{xz} &=& \frac{1}{6}\beta(1-\beta)m(m^2-1) \\
\Sigma_{yy} &=& \frac{1}{3}\beta(1-\beta)m(m+1)\left(m+\frac{1}{2}\right) \\
\Sigma_{yz} &=& \frac{1}{8}\beta(1-\beta)m(m^2-1)\left(m+\frac{2}{3}\right) \\
\Sigma_{zz} &=& \frac{1}{20}\beta(1-\beta)m(m^2-1)\left(m^2 - \frac{2}{3}\right)
\end{eqnarray}
To leading order in $1/m$, this matrix has determinant $\beta^{3} (1-\beta)^{3} m^9/8640$;
the Gaussian approximation for the peak probability is then
\begin{equation}
P^{*}_{g}(m,\beta) = \frac{1}{(2\pi)^{3/2}\sqrt{\det\Sigma}} \approx \frac{6\sqrt{30}}{\pi^{3/2}(1-\beta)^{3/2}\beta^{3/2}m^{9/2}}.
\end{equation}
For bimagic square series, the prefactor is the same as in
Eq.~(\ref{prefactor-squares}), $\beta=1/N$, and $m=N^2$, so the
Gaussian approximation for the number of bimagic square series is
\begin{eqnarray}
{\cal N}_{2,g}^{(2)}(N) &=& \frac{(Ne)^N}{\sqrt{e}}\left(1 - \frac{1}{6N} + \cdots\right)\times \frac{6\sqrt{30}\left(1-\frac{1}{N}\right)^{-3/2}}{\pi^{3/2} N^{15/2}}
\nonumber
\\
&=& \left(\frac{6}{\pi^{3/2}}\sqrt{\frac{30}{e}} \right)\cdot \frac{(Ne)^{N}}{N^{15/2}}\left(1 + \frac{4}{3N} +
\cdots\right).
\end{eqnarray}
For bimagic cube series, the prefactor is the same as in
Eq.~(\ref{prefactor-cubes}), $\beta=1/N^2$, and $m=N^3$, so the Gaussian
approximation here is
\begin{eqnarray}
{\cal N}_{3,g}^{(2)}(N) &=& \left(N^2 e\right)^{N} \left(1 - \frac{1}{2N} +
\cdots\right) \times \frac{6\sqrt{30}\left(1-\frac{1}{N^2}\right)^{-3/2}}{\pi^{3/2}N^{21/2}}
\nonumber \\
&=& \left(\frac{6\sqrt{30}}{\pi^{3/2}} \right)\cdot \frac{\left(N^2 e\right)^{N}}{N^{21/2}}\left(1 - \frac{1}{2N} +
\cdots\right).
\end{eqnarray}

Perturbative corrections to the Gaussian results may be calculated exactly as before.  In
fact, Eq.~(\ref{perturb-series}) is still correct, as is the expression
for the first correction coefficient $K_1$.  The only difference is that the
propagator and vertex have changed.  The propagator is now
\begin{equation}
  \Pi^{(2)} = \left(
\begin{array}{ccc}
  9 & -36 & 60 \\
  -36 & 192 & -360 \\
  60 & -360 & 720 \\
\end{array}\right)
\end{equation}
up to corrections of order $1/m$.  The vertex is given by
\begin{eqnarray}
\tilde{V}^{(2)}_{abc\ldots} &=& \frac{1}{m^{1+n_y(abc\ldots) + 2n_z(abc\ldots)}}\sum_{j=1}^{m}
j^{n_y(abc\ldots)}\left(\frac{1}{2}j(j-1)\right)^{n_z(abc\ldots)} \nonumber \\
&\approx& \left(\frac{1}{2}\right)^{n_z(abc\ldots)}\frac{1}{1 + n_y(abc\ldots) +
2n_z(abc\ldots)},
\end{eqnarray}
also to leading order in $1/m$. We find that
\begin{eqnarray}
K_1^{(2)} &=& -\frac{\tilde{V}_{abc}^{(2)} \tilde{V}_{def}^{(2)}}{8} \Pi_{ab}^{(2)}\Pi_{cd}^{(2)}\Pi_{ef}^{(2)} -\frac{\tilde{V}_{abc}^{(2)} \tilde{V}_{def}^{(2)}}{12}\Pi_{ad}^{(2)}\Pi_{be}^{(2)}\Pi_{cf}^{(2)}
+ \frac{\tilde{V}_{abcd}^{(2)}}{8} \Pi_{ab}^{(2)} \Pi_{cd}^{(2)} \nonumber
\\
&=& -\frac{1}{8}\cdot\frac{2781}{245} - \frac{1}{12}\cdot\frac{2403}{245} + \frac{1}{8}\cdot\frac{423}{35}
\nonumber
\\ &=& -\frac{711}{980}.
\end{eqnarray}
Multiplying $1 + K_1^{(2)}/N$ by the Gaussian approximations gives
\begin{equation}
{\cal N}^{(2)}_{2}(N) \approx  \left(\frac{6}{\pi^{3/2}}\sqrt{\frac{30}{e}} \right)\cdot \frac{(Ne)^{N}}{N^{15/2}}\left(1 +
\frac{1787}{2940N}\right)
\end{equation}
and
\begin{equation}
{\cal N}^{(2)}_{3}(N) \approx  \left(\frac{6\sqrt{30}}{\pi^{3/2}} \right)\cdot \frac{\left(N^2 e\right)^{N}}{N^{21/2}}\left(1
- \frac{1201}{980N}\right)
\end{equation}
for the number of bimagic square series and bimagic cube series, respectively.

Table~\ref{table:bimagic-series} compares our approximations
to some known results (from \cite{boyer-enum}).
The number of bimagic square series is known to $N=28$, at which point our approximation
is correct to $4$ significant digits.  The number of bimagic cube series is only known to
$N=11$; our first-order approximation is no better than the Gaussian approximation at that
value, but should improve thereafter.   We predict the number of bimagic square
series of order $29$ to be $3.9714\times 10^{44}$, and the number of bimagic
cube series of order $12$ to be $3.1966 \times 10^{20}$.

\begin{table}
  \begin{center}
\begin{tabular}{|c|c|c|c|c|}
  \hline
   $\alpha$ & N & Est. vs. exact bimagic series & Rel. Residual & Residual $\times N^2$ \\
\hline
   2 & 25 &  $7.68\underbar{395}\times 10^{35}$ & $9.45\times 10^{-5}$ & 0.059 \\
   &  & $7.68467875608077797721\times 10^{35}$ & & \\
  \hline
   2 & 26 &  $1.077\underbar{803}\times 10^{38}$ & $-6.50\times 10^{-4}$ & -0.44 \\
   &  & $1.07710220763567919575\times 10^{38}$ & & \\
  \hline
   2 & 27 &  $1.588\underbar{739}\times 10^{40}$ & $-4.35\times 10^{-4}$ & -0.32 \\
   &  & $1.58804753475269623163\times 10^{40}$ & & \\
  \hline
   2 & 28 &  $2.455\underbar{567}\times 10^{42}$ & $-4.49\times 10^{-5}$ & -0.035 \\
   &  & $2.45545658112397677916\times 10^{42}$ & & \\
  \hline
  \hline
  3 & 9 & $\underbar{5.93} \times 10^{11}$ & 0.098 & 7.98 \\
    &  & $6.51151145259 \times 10^{11}$ & & \\
\hline
  3 & 10 & $3.\underbar{607} \times 10^{14}$ & -0.038 & -3.75 \\
    &  & $3.47171191981324 \times 10^{14}$ & & \\
\hline
  3 & 11 & $\underbar{2.97}\times 10^{17}$ & 0.060 & 7.29 \\
    &  & $3.15035719463520007\times 10^{17}$ & & \\
  \hline
 \end{tabular}
\caption{{\bf Comparison of first-order bimagic series estimates to selected exact values.}  Results are
shown for the number of bimagic square series ($\alpha=2$) and bimagic cube series ($\alpha=3$).  The underlined
numbers are the first incorrect digits in each estimate.  The final column approximates the
coefficient of the $N^{-2}$ term for each $\alpha$.}
\label{table:bimagic-series}
\end{center}
\end{table}

\subsection{Trimagic series}

A \emph{trimagic square} is a bimagic square with the additional property that
the cubes of the entries in each row, column, and diagonal have the same sum.
To enumerate trimagic series, we will consider random vectors $(X_i, Y_i, Z_i, W_i)$
that are equal to $(1,i,\frac{1}{2}i(i-1), \frac{1}{6}i(i-1)(i-2))$ with probability $\beta$ and equal
to $(0,0,0,0)$ otherwise.  All aspects of the preceding calculation go through
as before, so we will omit the details.  One important difference is that the
expected value of $\sum_{j=1}^{N^2}W_j$ is not an integer when
$N=4k+2$, and so there are no trimagic series for $N$ of this form.  For all
other values of $N$, we find
\begin{equation}
{\cal N}_2^{(3)}(N) \approx \left(\frac{720}{\pi^2}\sqrt{\frac{105}{e}}\right)\cdot\frac{(Ne)^N}{N^{14}}
\end{equation}
to leading order.

The exact values (from \cite{boyer-enum}) are compared to the leading-order approximation in
Table~\ref{table:trimagic-series}.  The agreement is fairly poor at the
accessible values of $N$, and because the errors oscillate in magnitude, the $1/N$
correction cannot improve matters much.  There are clearly non-perturbative
effects that need to be better understood, and these effects are evidently more important
for higher multimagic series.  Indeed, similar (albeit smaller)
oscillatory effects are already apparent in the bimagic series approximation.

\begin{table}
     \begin{center}
\begin{tabular}{|c|c|c|c|}
  \hline
   $\alpha$ & N & Est. vs. exact trimagic series & Exact/estimate \\
  \hline
  2 & 11 &  $2.04\times 10^{4}$  & $1.53$ \\
     &        & $31187$ & \\
  \hline
  2 & 12 &   $5.12\times 10^{5}$     & $4.35$ \\
     &        & $2226896$ & \\
  \hline
   2 & 13 &  $1.54 \times 10^{7}$ & $1.12$ \\
     &  &  $17265701$ & \\
\hline
   2 & 15 &  $2.22\times 10^{10}$ & $3.12$ \\
   &  & $69303997733$ & \\
  \hline
 \end{tabular}
\caption{{\bf Comparison of leading-order trimagic series estimates to selected exact values.}  Results are
shown for the number of trimagic square series ($\alpha=2$) only.}
\label{table:trimagic-series}
 \end{center}
\end{table}

Combinatorics (math.CO); Probability (math.PR)

Asymptotic enumeration of magic series


Dynamical systems gradient method for solving ill-conditioned linear algebraic systems

Abstract: A version of the Dynamical Systems Method (DSM) for solving ill-conditioned linear algebraic systems is studied in this paper. An {\it a priori} and {\it a posteriori} stopping rules are justified. An algorithm for computing the solution using a spectral decomposition of the left-hand side matrix is proposed. Numerical results show that when a spectral decompositon of the left-hand side matrix is available or not computationally expensive to obtain the new method can be considered as an alternative to the Variational Regularization.

\section{Introduction}

The Dynamical Systems Method (DSM) was systematically introduced and
investigated in \cite{R499} as a general method for solving operator
equations, linear and nonlinear, especially ill-posed operator equations.
In several recent publications various versions of the DSM, proposed in
\cite{R499}, were shown to be as efficient and economical as variational
regularization methods. This was demonstrated, for example, for the
problems of solving ill-conditioned linear algebraic systems \cite{R526},
 and stable numerical differentiation of noisy data
\cite{R415}, \cite{R441}, \cite{R530}.

The aim of this paper is to formulate a version of the DSM gradient method
for solving ill-posed linear equations and to demonstrate numerical
efficiency of this method. There
is a large literature on
iterative regularization methods. These methods can be derived from
a suitable version of the DSM by a discretization (see \cite{R499}). In
the
Gauss-Newton-type version of the DSM one has to invert some linear
operator, which is an expensive procedure. The same is true for
regularized Newton-type versions of the DSM and of their iterative
counterparts. In contrast, the DSM gradient method we study in this
paper {\it does not}
require inversion of operators.

We want to solve equation \begin{equation} \label{eqi1} Au=f,
\end{equation} where A is a linear bounded operator in a  Hilbert
space $H$. We assume that \eqref{eqi1} has a solution, possibly nonunique,
and denote by $y$ the unique minimal-norm solution to \eqref{eqi1},
$y\perp \mathcal{N}:=\mathcal{N}(A):= \{u: Au=0\}$, $Ay=f$. We assume that
the range of $A$, $R(A)$, is not closed, so problem \eqref{eqi1} is
ill-posed. Let $f_\delta$, $\|f-f_\delta\|\leq\delta$, be the noisy data.
We want to construct a stable approximation of $y$, given $\{\delta,
f_\delta, A\}$. There are many methods for doing this, see, e.g.,
\cite{I}, \cite{L}, \cite{M}, \cite{R499}, \cite{VV}, to mention a few
books, where variational regularization, quasisolutions, quasiinversion,
iterative regularization, and the DSM are studied.

The DSM version we study in this paper consists of solving the Cauchy problem
\begin{equation}
\label{eqi2}
\dot{u}(t)=-A^*(Au(t)-f),\quad u(0)=u_0,
\quad u_0\perp N,\quad \dot{u}:=\frac{du}{dt},
\end{equation}
where $A^*$ is the adjoint to operator $A$, and proving the
existence of the limit
$\lim_{t\to\infty}u(t)=u(\infty)$, and the relation $u(\infty)=y$, i.e.,
\begin{equation}
\label{eqi3}
\lim_{t\to\infty}\|u(t)-y\|=0.
\end{equation}

If the noisy data $f_\delta$ are given, then we solve the problem
\begin{equation}
\label{eqi4}
\dot{u}_\delta(t)=-A^*(Au_\delta(t)-f_\delta),\quad u_\delta(0)=u_0,
\end{equation}
and prove that, for a suitable stopping time
$t_\delta$, and $u_\delta :=u_\delta(t_\delta)$,
one has
\begin{equation}
\label{eqi5}
\lim_{\delta\to 0}\|u_\delta-y\|=0.
\end{equation}

In Section 2 these results are formulated precisely and
recipes for choosing $t_\delta$ are proposed.

The novel results in this paper include the
proof of the discrepancy principle (Theorem 3), an efficient method
for computing $u_\delta(t_\delta)$ (Section \ref{sec comput}),
and an a priori stopping rule (Theorem 2).

Our presentation is essentially self-contained.

\section{Results}

Suppose $ A: H\to H$ is a linear bounded operator in a Hilbert space $H$.
Assume that equation
\begin{equation}
\label{eq1}
Au=f
\end{equation}
has a solution not necessarily unique. Denote by $y$
the unique minimal-norm solution i.e., $y\perp \mathcal{N}:=\mathcal{N}(A)$.
Consider the following Dynamical Systems Method (DSM)
\begin{equation}
\label{eq2}
\begin{split}
\dot{u}&=-A^*(Au-f),\\
u(0)&=u_0,
\end{split}
\end{equation}
where $u_0\perp \mathcal{N}$ is arbitrary. Denote $T:=A^*A,\, Q:= AA^*$.
The unique solution to \eqref{eq2} is
$$
u(t)=e^{-tT}u_0 + e^{-tT}\int_0^t e^{s T} ds A^*f.
$$
Let us show that any ill-posed linear equation \eqref{eq1} with exact data can be solved by the DSM.

\subsection{Exact data}

\begin{thm}
\label{thm1}
Suppose $u_0\perp \mathcal{N}$. Then problem \eqref{eq2} has a unique solution defined on
$[0,\infty)$, and $u(\infty)=y$, where $u(\infty)=\lim_{t\to\infty} u(t)$.
\end{thm}

\begin{proof}
Denote $w:=u(t)-y,\, w_0=w(0)$. Note that $w_0\perp \mathcal{N}$. One has
\begin{equation}
\label{eq3}
\dot{w}=-Tw,\quad T=A^*A.
\end{equation}
The unique solution to \eqref{eq3} is
$w=e^{-tT}w_0$. Thus,
$$
\|w\|^2=\int_0^{\|T\|} e^{-2t\lambda}d\langle E_\lambda w_0,w_0\rangle.
$$
where $\langle u,v\rangle$ is the inner product in $H$, and $E_\lambda$ is the resolution of the
identity of the selfadjoint operator $T$. Thus,
$$
\|w(\infty)\|^2 =\lim_{t\to \infty}\int_0^{\|T\|} e^{-2t\lambda}d\langle E_\lambda w_0,w_0\rangle=\|P_\mathcal{N}w_0\|^2=0,
$$
where $P_\mathcal{N}=E_0-E_{-0}$ is the orthogonal projector onto $\mathcal{N}$.
Theorem \ref{thm1} is proved.
\end{proof}

\subsection{Noisy data $f_\delta$}
\label{Sec2.2}

Let us solve stably equation \eqref{eq1} assuming that $f$ is not known,
but $f_\delta$, the noisy data, are known,
where $\|f_\delta-f\|\le \delta$. Consider the following
DSM
$$
\dot{u}_\delta = - A^*(Au_\delta -f_\delta),\quad u_\delta(0)=u_0.
$$
Denote
$$
w_\delta:=u_\delta-y,\quad T:=A^*A,\quad w_\delta(0)=w_0:=u_0-y\in
\mathcal{N}^\perp.
$$
Let us prove the following result:

\begin{thm}
\label{thm2}
If $\lim_{\delta\to0}t_\delta = \infty,\,\lim_{\delta\to 0}t_\delta \delta=0$, and
$w_0\perp \mathcal{N}$,
then
$$
\lim_{\delta\to 0}\|w_\delta(t_\delta)\|=0.
$$
\end{thm}

\begin{proof}
One has
\begin{equation}
\label{eq4}
\dot{w}_\delta= -Tw_\delta + \eta_\delta,\quad \, \eta_\delta
=A^*(f_\delta-f),\quad \|\eta_\delta\|\le\|A\|\delta.
\end{equation}
The unique solution of equation \eqref{eq4} is
$$
w_\delta(t)=e^{-tT}w_\delta(0)+\int_0^te^{-(t-s)T}\eta_\delta ds.
$$
Let us show that
$\lim_{t\to\infty} \|w_\delta(t)\|=0$.
One has
\begin{equation}
\label{extra1}
\lim_{t\to\infty} \|w_\delta(t)\| \le \lim_{t\to\infty}\|e^{-tT}w_\delta(0)\|
+\lim_{t\to\infty}\bigg{\|}\int_0^te^{-(t-s)T}\eta_\delta ds\bigg{\|}.
\end{equation}
One uses the spectral theorem and gets:
\begin{equation}
\label{eq5}
\begin{split}
\int_0^te^{-(t-s)T}ds\eta_\delta&=\int_0^t\int_0^{\|T\|} dE_\lambda \eta_\delta e^{-(t-s)\lambda} ds\\
&=\int_0^{\|T\|} e^{-t\lambda}\frac{e^{t\lambda}-1}{\lambda}dE_\lambda\eta_\delta
=\int_0^{\|T\|}\frac{1-e^{-t\lambda}}{\lambda}dE_\lambda\eta_\delta.
\end{split}
\end{equation}
Note that
\begin{equation}
\label{eq6}
0\le\frac{1-e^{-t\lambda}}{\lambda}\le t,\quad \forall \lambda>0, t\ge 0,
\end{equation}
since $1-x\le e^{-x}$ for $x\ge 0$.
From \eqref{eq5} and \eqref{eq6}, one obtains
\begin{equation}
\label{extra2}
\begin{split}
\bigg{\|}\int_0^te^{-(t-s)T}ds\eta_\delta\bigg{\|}^2
&=\int_0^{\|T\|}\big{|}\frac{1-e^{-t\lambda}}{\lambda}\big{|}^2d\langle E_\lambda\eta_\delta,\eta_\delta\rangle\\
&\le t^2\int^{\|T\|} d\langle E_\lambda\eta_\delta,\eta_\delta\rangle\\
&=t^2\|\eta_\delta\|^2.
\end{split}
\end{equation}
Since $\|\eta_\delta\|\le \|A\|\delta$, from \eqref{extra1} and \eqref{extra2}, one gets
$$
\lim_{\delta\to0} \|w_\delta(t_\delta)\| \le \lim_{\delta\to 0}\bigg{(}
\| e^{-t_\delta T}w_\delta(0)\|+t_{\delta}\delta\|A\|\bigg{)}=0.
$$
Here we have used the relation:
$$
\lim_{\delta\to 0}\| e^{-t_\delta T}w_\delta(0)\|=\|P_\mathcal{N}w_0\|=0,
$$
and the last equality holds because $w_0\in \mathcal{N}^\perp$.
Theorem \ref{thm2} is proved.
\end{proof}

From Theorem \ref{thm2}, it follows that the relation $t_\delta=\frac{C}{\delta^\gamma}$,
$\gamma=\text{const},\, \gamma\in(0,1)$ and $C>0$ is a constant,
can be used as an \textit{a priori} stopping rule, i.e., for such $t_\delta$ one has
\begin{equation}\
\label{eq7}
\lim_{\delta\to0}\|u_\delta(t_\delta)-y\|=0.
\end{equation}

\subsection{Discrepancy principle}

Let us consider equation \eqref{eq1} with noisy data $f_\delta$, and a DSM of the form
\begin{equation}
\label{eq8}
\dot{u}_\delta = - A^*A u_\delta + A^*f_\delta,\quad u_\delta(0)=u_0.
\end{equation}
for solving this equation.
Equation \eqref{eq8} has been used in Section~\ref{Sec2.2}.
Recall that $y$ denotes the minimal-norm solution of equation \eqref{eq1}.

\begin{thm}
\label{thm3}
Assume that $\|Au_0-f_\delta\|> C\delta$. The
solution $t_\delta$ to the equation
\begin{equation}
\label{eq9}
h(t):=\|Au_\delta(t)- f_\delta\|=C\delta,\quad C=\text{const},\, C\in (1,2),
\end{equation}
does exist, is unique, and
\begin{equation}
\label{eq10}
\lim_{\delta\to 0} \|u_\delta(t_\delta)-y\|=0.
\end{equation}
\end{thm}

\begin{proof}
Denote
$$
v_\delta(t):=Au_\delta(t)- f_\delta,\quad T:=A^*A,\quad Q=AA^*,\quad
w(t):=u(t)-y,\quad w_0:=u_0-y.
$$
One has
\begin{equation}
\label{eq11}
\begin{split}
\frac{d}{dt}\|v_\delta(t)\|^2
&= 2\mathop{\rm Re} \langle A\dot{u}_\delta(t),Au_\delta(t)-f_\delta \rangle\\
&= 2 \mathop{\rm Re} \langle A[-A^*(Au_\delta(t) - f_\delta)],Au_\delta(t)-f_\delta
\rangle\\
&=-2\|A^*v_\delta(t)\|^2\le 0.
\end{split}
\end{equation}
Thus, $\|v_\delta(t)\|$ is a nonincreasing function.
Let us prove that equation \eqref{eq9} has a solution for $C\in (1,2)$.
Recall the known commutation formulas:
$$
e^{-sT}A^*=A^*e^{-sQ},\, Ae^{-sT}=e^{-tQ}A.
$$
Using these formulas and the representation
$$
u_\delta(t)=e^{-tT}u_0+\int_0^te^{-(t-s)T}A^*f_\delta ds,
$$
one gets:
\begin{align*}
v_\delta(t)
&=Au_\delta(t)-f_\delta\\
&=Ae^{-tT}u_0+A\int_0^te^{-(t-s)T}A^*f_\delta ds -f_\delta \\
&=e^{-t Q}Au_0+e^{-t Q}\int_0^{t} e^{sQ}dsQf_\delta-f_\delta \\
&=e^{-tQ}A(u_0-y)+e^{-tQ}f+e^{-tQ}(e^{tQ}-I)f_\delta - f_\delta\\
&=e^{-t Q}Aw_0 -e^{-t Q}f_\delta +e^{-t Q}f
\end{align*}
Note that
$$
\lim_{t\to\infty}e^{-t Q}Aw_0=\lim_{t\to\infty}Ae^{-t T}w_0 = AP_\mathcal{N}w_0=0.
$$
Here the continuity of $A$, and the following
relation
$$
\lim_{t\to\infty}e^{-tT}w_0=\lim_{t\to\infty}\int_0^{\|T\|}e^{-st}dE_sw_0=(E_0-E_{-0})w_0=P_\mathcal{N}w_0,
$$
were used.
Therefore,
\begin{equation}
\label{eq12}
\lim_{t\to\infty}\|v_\delta(t)\|=\lim_{t\to\infty}\|e^{-t Q}(f-f_\delta)\|\le
\|f-f_\delta\|\le\delta,
\end{equation}
because $\|e^{-tQ}\|\le 1$. The function $h(t)$ is continuous on $[0,\infty)$,
$h(0)=\|Au_0-f_\delta\|>C\delta$, $h(\infty)\le \delta$.
Thus, equation \eqref{eq9} must have a solution $t_\delta$.

Let us prove the uniqueness of $t_\delta$. Without loss of generality we can assume that there exists
$t_1>t_\delta$ such that $\|Au_\delta(t_1)- f_\delta\|=C\delta$. Since $\|v_\delta(t)\|$ is
nonincreasing and $\|v_\delta(t_\delta)\|=\|v_\delta(t_1)\|$, one has
$$
\|v_\delta(t)\|=\|v_\delta(t_\delta)\|,\quad \forall t\in [t_\delta, t_1].
$$
Thus,
\begin{equation}
\label{eq13}
\frac{d}{dt}\|v_\delta(t)\|^2=0,\quad \forall t\in (t_\delta, t_1).
\end{equation}
Using \eqref{eq11} and \eqref{eq13} one obtains
$$
A^*v_\delta(t)=A^*(Au_\delta(t)-f_\delta) = 0,\quad \forall t\in [t_\delta,t_1].
$$
This and \eqref{eq8} impl
\begin{equation}
\label{eq14}
\dot{u_\delta}(t)=0,\quad \forall t\in(t_\delta,t_1).
\end{equation}
One has
\begin{equation}
\label{eq15}
\begin{split}
\dot{u_\delta}(t)&=-Tu_\delta(t)+A^*f_\delta \\
&=-T\bigg{(}e^{-tT}u_0 +\int_0^te^{-(t-s)T}A^*f_\delta ds\bigg{)} + A^*f_\delta\\
&=-Te^{-tT}u_0 - (I-e^{-tT})A^*f_\delta+A^*f_\delta\\
&=-e^{-tT}(Tu_0 - A^*f_\delta).
\end{split}
\end{equation}
From \eqref{eq15} and \eqref{eq14}, one gets $Tu_0 - A^*f=e^{tT}e^{-tT}(Tu_0 - A^*f)=0$.
Note that the operator $e^{tT}$ is an isomorphism for any fixed $t$ since $T$ is selfadjoint and bounded.
Since $Tu_0 - A^*f=0$, by \eqref{eq15} one has $\dot{u_\delta}(t)=0$,\, $u_\delta(t)=u_\delta(0)$,\, $\forall t\ge0$.
Consequently,
$$
C\delta <\|Au_\delta(0)-f_\delta\|=\|Au_\delta(t_\delta)-f_\delta\|
=C\delta.
$$
This is a contradiction which proves
the uniqueness of $t_\delta$.

Let us prove \eqref{eq10}. First, we have the following estimate:
\begin{equation}
\label{eq16}
\begin{split}
\|Au(t_\delta)-f\|&\le \|Au(t_\delta)-Au_\delta(t_\delta)\|+\|Au_\delta(t_\delta)-f_\delta\|+\|f_\delta -f\|\\
&\le \bigg{\|}e^{-t_\delta Q}\int_0^{t_\delta}e^{sQ}Qds \bigg{\|} \|f_\delta-f\|+C\delta+\delta.
\end{split}
\end{equation}
Let us use the inequality:
$$
\big{\|}e^{-t_\delta Q}\int_0^{t_\delta}e^{sQ}Qds \big{\|}=\|I-e^{-t_\delta Q}\|\le 2,
$$
and conclude from \eqref{eq16}, that
\begin{equation}
\label{eq17}
\lim_{\delta\to0}\|Au(t_\delta)-f\|=0.
\end{equation}
Secondly,
we claim that
$$
\lim_{\delta\to0}t_\delta=\infty.
$$
Assume the contrary. Then there exist $t_0>0$ and a sequence
$(t_{\delta_n})_{n=1}^\infty$,
$t_{\delta_n}<t_0$, such that
\begin{equation}
\label{eq18}
\lim_{n\to\infty}\|Au(t_{\delta_n})-f\|=0.
\end{equation}
Analogously to \eqref{eq11}, one proves that
$$
\frac{d\|v\|^2}{dt}\le 0,
$$
where $v(t):=Au(t)-f$.
Thus, $\|v(t)\|$ is nonincreasing.
This and \eqref{eq18} imply the relation $\|v(t_0)\|=\|Au(t_0)-f\|=0$.
Thus,
$$
0=v(t_0)=e^{-t_0Q}A(u_0-y).
$$
This implies $A(u_0-y)=e^{t_0Q}e^{-t_0Q}A(u_0-y)=0$, so $u_0-y\in \mathcal{N}$.
Since $u_0-y\in \mathcal{N}^\perp$, it follows that
 $u_0=y$. This is a contradiction because
$$
C\delta\le\|Au_0-f_\delta\|=\|f-f_\delta\|\le\delta, \quad 1<C<2.
$$
Thus, $\lim_{\delta\to0}t_\delta=\infty$.

Let us continue the proof of \eqref{eq10}. Let
$w_\delta(t):=u_\delta(t)-y$.
We claim that $\|w_\delta(t)\|$ is nonincreasing on $[0,t_\delta]$. One has
\begin{align*}
\frac{d}{dt}\|w_\delta(t)\|^2&=2\mathop{\rm Re} \langle
\dot{u_\delta}(t),u_\delta(t)-y\rangle\\
&= 2 \mathop{\rm Re} \langle -A^*(Au_\delta(t)-f_\delta),u_\delta(t)-y\rangle\\
&= -2 \mathop{\rm Re} \langle Au_\delta(t)-f_\delta,Au_\delta(t)-f_\delta
+f_\delta-Ay\rangle\\
&\le -2 \|Au_\delta(t)-f_\delta\|\bigg{(}\|Au_\delta(t)-f_\delta\| - \|f_\delta -f\|\bigg{)}\\
&\le 0.
\end{align*}
Here we have used the inequalities:
$$
\|Au_\delta(t)-f_\delta\|\ge C\delta >\|f_\delta -Ay\|=\delta,\quad
\forall t\in [0,t_\delta].
$$

Let $\epsilon >0$ be arbitrary small. Since $\lim_{t\to\infty} u(t)=y$, there exists $t_0>0$,
independent of $\delta$, such that
\begin{equation}
\label{eq19}
\| u(t_0)-y\|\le \frac{\epsilon}{2}.
\end{equation}
Since $\lim_{\delta\to0} t_\delta =\infty$, there exists $\delta_0$ such that
$t_\delta>t_0,\, \forall \delta\in (0,\delta_0)$.
Since $\|w_\delta(t)\|$ is nonincreasing
on $[0,t_\delta]$ one has
\begin{equation}
\label{eqdel2}
\|w_\delta(t_\delta)\|\le \|w_\delta(t_0)\|\le \|u_\delta(t_0)-u(t_0)\|+ \|u(t_0)-y\|,\quad
\forall \delta\in(0,\delta_0).
\end{equation}
Note that
\begin{equation}
\label{eq21}
\begin{split}
\|u_\delta(t_0)-u(t_0)\|&=\|e^{-t_0T}\int_0^{t_0}e^{sT}ds A^*(f_\delta-f)\|
\le \|e^{-t_0T}\int_0^{t_0}e^{sT}dsA^*\|\delta.
\end{split}
\end{equation}
Since $e^{-t_0T}\int_0^{t_0}e^{sT}ds A^*$ is a bounded operator for any fixed $t_0$,
one concludes from \eqref{eq21} that $\lim_{\delta\to0}\|u_\delta(t_0)-u(t_0)\|=0$.
Hence, there exists $\delta_1\in(0,\delta_0)$ such that
\begin{equation}
\label{eqdel3}
\|u_\delta(t_0)-u(t_0)\|\le \frac{\epsilon}{2},\quad \forall \delta\in(0,\delta_1).
\end{equation}
From \eqref{eq19}--\eqref{eqdel3}, one obtains
\begin{equation*}
\|u_\delta(t_\delta)-y\|=\|w_\delta(t_\delta)\|\le \frac{\epsilon}{2}+ \frac{\epsilon}{2}=\epsilon,\quad
\forall \delta\in(0,\delta_1).
\end{equation*}
This means that $\lim_{\delta\to0}u_\delta(t_\delta)=y$. Theorem~\ref{thm3} is proved.
\end{proof}

\section{Computing $u_\delta(t_\delta)$}
\label{sec comput}

\subsection{Systems with known spectral decomposition}

One way to solve the Cauchy problem \eqref{eq8} is
to use explicit Euler or Runge-Kutta methods
with a constant or adaptive stepsize $h$.
However, stepsize $h$ for solving \eqref{eq8} by explicit numerical methods
is often smaller than 1 and the stopping time $t_\delta=nh$ may be large.
Therefore, the computation
time, characterized by the number of iterations $n$, for this approach may be large.
This fact is also reported in
\cite{R526}, where one of the most efficient numerical methods for solving ordinary differential equations (ODEs),
the DOPRI45 (see \cite{Hairer}), is used for solving a Cauchy problem in a DSM.
Indeed, the use of explicit Euler method leads to a Landweber iteration
which is known for
slow convergence.
Thus, it may be computationally expensive to compute $u_\delta(t_\delta)$ by
numerical methods for ODEs.

However, when $A$ in \eqref{eq8} is a matrix
and a decomposition $A=USV^*$, where $U$ and $V$ are unitary matrices
and $S$ is a diagonal matrix, is known, it is
possible to compute $u_\delta(t_\delta)$ at a speed comparable to
other methods such as the variational regularization (VR) as it will be
shown below.

We have
\begin{equation}
\label{eqforu}
u_\delta(t)=e^{-tT}u_0 + e^{-tT}\int_0^t e^{s T} ds A^*f_\delta,
\quad T:=A^*A.
\end{equation}
Suppose that a decomposition
\begin{equation}
\label{specdecom}
A=USV^*,
\end{equation}
where $U$ and $V$ are unitary matrices
and $S$ is a diagonal matrix is known.
These matrices possibly contain complex entries. Thus, $T=A^*A=V\bar{S}SV^*$ and $e^T=e^{V\bar{S}SV^*}$.
Using the formula $e^{V\bar{S}SV^*}=V e^{\bar{S}S}V^*$, which
is valid if $V$ is unitary and $\bar{S}S$ is diagonal,
equation~\eqref{eqforu} can be rewritten as
\begin{equation}
u_\delta(t)=Ve^{-t\bar{S}S}V^*u_0 + V\int_0^t e^{(s -t)\bar{S}S} ds \bar{S} U^*f_\delta.
\end{equation}
Here, the overbar stands for complex conjugation.
Choose $u_0=0$. Then
\begin{equation}
\label{forforu}
u_\delta(t)=V\int_0^t e^{(s -t)\bar{S}S} ds \bar{S} h_\delta,\quad h_\delta:=U^*f_\delta.
\end{equation}

Let us assume that
\begin{equation}
\label{eq32pre}
A^*f_\delta \not=0.
\end{equation}
This is a natural assumption. Indeed, if $A^*f_\delta=0$, then by
the definition of
$h_\delta$ in \eqref{forforu}, relation $V^*V=I$, and equation \eqref{specdecom}, one gets
\begin{equation}
\label{giaithich}
\bar{S}h_\delta = \bar{S}U^*f_\delta = V^*V\bar{S}U^*f_\delta =
V^*A^*f_\delta=0.
\end{equation}
Equations \eqref{giaithich} and  \eqref{forforu} imply
$u_\delta(t)\equiv 0$.

The stopping time $t_\delta$ we choose by the following discrepancy
principle:
$$
\|Au_{\delta}(t_\delta)-f_\delta\|=
\bigg{\|}\int_0^{t_\delta} e^{(s -t_\delta)\bar{S}S} ds \bar{S}S h_\delta-h_\delta\bigg{\|}
=\|e^{-t_\delta \bar{S}S}h_\delta\|=C\delta.
$$
where $1<C<2$.

Let us find $t_\delta$ from the equation
\begin{equation}
\label{eq32old}
\phi(t):=\psi(t)-C\delta =0,\qquad \psi(t):=\|e^{-t\bar{S}S}h_\delta\|.
\end{equation}
The existence and uniqueness of the solution $t_\delta$ to equation \eqref{eq32old} follow
from Theorem~\ref{thm3}.

We claim that {\it equation \eqref{eq32old} can be solved by using Newton's iteration
\eqref{eqnewton} for any initial value $t_0$ such that $\phi(t_0)>0.$
}

Let us prove this claim. It is sufficient to prove that
$\phi(t)$ is a monotone strictly convex function. This is proved below.

Without loss of generality, we can assume that $h_\delta$ (see \eqref{eq32old}) is a vector
with real components.
The proof remained essentially the same for $h_\delta$ with complex
components.

First, we claim that
\begin{equation}
\label{vocolen}
\sqrt{\bar{S}S}h_\delta\not=0,\qquad \text{and} \qquad
 \|\sqrt{\bar{S}S}e^{-t \bar{S}S}h_\delta\|\not=0,
\end{equation}
so $\psi(t)>0$.

Indeed, since $e^{-t \bar{S}S}$ is an isomorphism and $e^{-t \bar{S}S}$ commutes
with $\sqrt{\bar{S}S}$ one concludes that $\|\sqrt{\bar{S}S}e^{-t \bar{S}S}h_\delta\|=0$
iff $\sqrt{\bar{S}S}h_\delta=0$. If $\sqrt{\bar{S}S}h_\delta=0$ then $\bar{S}h_\delta=0$, and then,
by equation ~\eqref{giaithich}, $A^*f_\delta=\bar{S}h_\delta=0$.
This contradicts to the assumption \eqref{eq32pre}.

Let us now prove that $\phi$ monotonically decays and is strictly convex.
Then our claim will be proved.

One has
$$
\frac{d}{d t}\langle e^{-t \bar{S}S}h_\delta, e^{-t \bar{S}S}h_\delta\rangle
=-2\langle e^{-t\bar{S}S}h_\delta, \bar{S}S e^{-t \bar{S}S}h_\delta\rangle.
$$
Thus,
\begin{equation}
\label{eq32x}
\dot{\psi}(t)=\frac{d}{dt}\|e^{-t\bar{S}S}h_\delta\|
=\frac{\frac{d}{dt}\| e^{-t\bar{S}S}h_\delta \|^2}{2\| e^{-t\bar{S}S}h_\delta \|}
=-\frac{\langle e^{-t \bar{S}S}h_\delta, \bar{S}S e^{-t \bar{S}S}h_\delta\rangle}
{\|e^{-t \bar{S}S}h_\delta\|}.
\end{equation}
Equation \eqref{eq32x}, relation ~\eqref{vocolen}, and the fact that $\langle e^{-t \bar{S}S}h_\delta, \bar{S}S e^{-t \bar{S}S}h_\delta\rangle = \|\sqrt{\bar{S}S}e^{-t \bar{S}S}h_\delta\|^2$ imply
\begin{equation}
\label{eq36xc}
\dot{\psi}(t) < 0.
\end{equation}
From equation \eqref{eq32x} and the definition of $\psi$ in \eqref{eq32old}, one gets
\begin{equation}
\label{eq34x}
\psi(t)\dot{\psi}(t) = -\langle e^{-t \bar{S}S}h_\delta, \bar{S}S e^{-t \bar{S}S}h_\delta\rangle
\end{equation}
Differentiating equation \eqref{eq34x} with respect to $t$, one obtains
\begin{align*}
\psi(t)\ddot{\psi}(t) + \dot{\psi}^2(t) &= \langle \bar{S}Se^{-t \bar{S}S}h_\delta, \bar{S}S e^{-t \bar{S}S}h_\delta\rangle +
\langle e^{-t \bar{S}S}h_\delta, \bar{S}S\bar{S}S e^{-t \bar{S}S}h_\delta\rangle\\
&= 2 \|\bar{S}Se^{-t \bar{S}S}h_\delta\|^2.
\end{align*}
This equation and equation \eqref{eq32x} imply
\begin{equation}
\label{eq38xc}
\psi(t)\ddot{\psi}(t) = 2 \|\bar{S}Se^{-t \bar{S}S}h_\delta\|^2 -
\frac{\langle e^{-t \bar{S}S}h_\delta, \bar{S}S e^{-t \bar{S}S}h_\delta\rangle^2}
{\|e^{-t \bar{S}S}h_\delta\|^2} \ge \|\bar{S}Se^{-t \bar{S}S}h_\delta\|^2 > 0.
\end{equation}
Here the inequality: $\langle e^{-t \bar{S}S}h_\delta,
\bar{S}S e^{-t \bar{S}S}h_\delta\rangle\le
\|e^{-t \bar{S}S}h_\delta\|\|\bar{S}S e^{-t \bar{S}S}h_\delta\|$ was used.
Since $\psi>0$, inequality \eqref{eq38xc} implies
\begin{equation}
\label{eq39xc}
\ddot{\psi}(t) > 0.
\end{equation}
It follows from inequalities \eqref{eq36xc} and \eqref{eq39xc} that $\phi(t)$
is a strictly convex and decreasing function on $(0,\infty)$.
Therefore, $t_\delta$ can be found by Newton's iterations:
\begin{equation}
\label{eqnewton}
\begin{split}
t_{n+1}&= t_n -\frac{\phi(t_n)}{\dot{\phi}(t_n)}\\
&=t_n +
\frac{\| e^{-t_n
\bar{S}S}h_\delta\| - C\delta}{\langle \bar{S}Se^{-t_n
\bar{S}S}h_\delta,e^{-t_n \bar{S}S}h_\delta\rangle}
\| e^{-t_n\bar{S}S}h_\delta\|,\quad n=0,1,...,
\end{split}
\end{equation}
for any initial guess $t_0$ of $t_\delta$ such that $\phi(t_0)>0$.
Once $t_\delta$ is found, the solution $u_\delta(t_\delta)$ is computed by \eqref{forforu}.

\begin{rem}{\rm
In the decomposition $A = VSU^*$ we do not assume that $U,V$ and $S$ are matrices
with real entries. The singular value decomposition (SVD) is a particular
case of this decomposition.

It is computationally expensive to get the SVD of a matrix in general.
However, there are many problems in which the decomposition
\eqref{specdecom} can be computed
fast using the fast Fourier transform (FFT). Examples include image restoration
problems with circulant block matrices (see \cite{N})
and deconvolution problems.
(see Section~\ref{image sec}).
}
\end{rem}

\subsection{On the choice of $t_0$}

Let us discuss a strategy for choosing the initial value $t_0$ in Newton's iterations
for finding $t_\delta$.
We choose $t_0$ satisfying condition:
\begin{equation}
\label{eqt0}
0<\phi(t_0)=\|e^{-t_0\bar{S}S}h_\delta\|-\delta \le\delta
\end{equation}
by the following strategy
\begin{enumerate}
\item{Choose $t_0:=10\frac{\|h_\delta\|}{\delta}$ as an initial guess for $t_0$.}
\item{Compute $\phi(t_0)$. If $t_0$ satisfying \eqref{eqt0} we are done. Otherwise, we go to step 3.}

\item{If $\phi(t_0)< 0$ and the inequality $\phi(t_0)> \delta$ has not occurred in iteration,
we replace $t_0$ by $\frac{t_0}{10}$  and go back to step 2.
If $\phi(t_0)< 0$ and the inequality $\phi(t_0)> \delta$ has occurred in iteration,
we replace $t_0$ by $\frac{t_0}{3}$ and go back to step 2.
If $\phi(t_0)> \delta$, we go to step 4.}

\item{
If $\phi(t_0)> \delta$ and the inequality $\phi(t_0)< 0$ has not occured in iterations,
 we replace $t_0$ by $3t_0$ and go back to step 2.
If the inequality $\phi(t_0)< 0$ has occured in some iteration before, we stop the iteration and use $t_0$
as an initial guess in
Newton's iterations for finding $t_\delta$.}
\end{enumerate}

\section{Numerical experiments}

In this section results of some numerical experiments with ill-conditioned linear algebraic systems
are reported.
In all the experiments, by DSMG we denote the version of the DSM described in this paper,
by VR we denote the Variational Regularization,
implemented using the discrepancy principle, and by DSM-\cite{R526} we denote
the method developed in \cite{R526}.

\subsection{A linear algebraic system for the computation of second derivatives}

Let us do some numerical experiments with linear algebraic systems arising in a
numerical experiment of computing the second derivative of a noisy function.

The problem is reduced to an integral equation of the first kind.
A linear algebraic system is obtained by a discretization of the integral equation whose kernel $K$ is Green's function
$$
K(s,t)=
\left\{
\begin{matrix}
s(t-1),\quad \text{if}\quad s<t\\
t(s-1),\quad \text{if}\quad s\geq t
\end{matrix}
\right. .
$$
Here $s,t\in[0,1]$.
Using $A_N$ from \cite{R526}, we do some numerical experiments for solving $u_N$ from the linear algebraic system
$A_N u_N=b_{N,\delta}$. In the experiments
the exact right-hand side is computed by the formula $b_N=A_Nu_N$ when $u_N$ is given.
In this test, $u_N$ is computed by
$$
u_N:=\big{(}u(t_{N,1}),u(t_{N,2}),....,u(t_{N,N})\big{)}^T,\qquad t_{N,i}:=\frac{i}{N},\quad i=1,...,N,
$$
where $u(t)$ is a given function.
We use $N=10,20,...,100$ and $b_{N,\delta} = b_N + e_N$,
where $e_N$ is a random vector whose coordinates are independent, normally distributed, with mean 0 and  variance 1,
and scaled so that $\|e_N\|=\delta_{rel}\|b_N\|$.
This linear algebraic system is mildly ill-posed: the condition
number of $A_{100}$ is $1.2158\times10^{4}$.

In Figure~\ref{fig1}, the difference between the exaction and
solution obtained by the DSMG, VR and DSM-\cite{R526} are plotted.
In these experiments, we used $N=100$ and $u(t)=\sin(\pi t)$ with $\delta_{rel}=0.05$
and $\delta_{rel}=0.01$.
Figure~\ref{fig1} shows that the results obtained by
the VR and the DSM-\cite{R526} are very close to each other.
The results obtained by the DSMG are much better than those by
the DSM-\cite{R526} and by the VR.

\begin{figure}[!h!t!b!]
\centerline{
\begin{tabular}{c}
\mbox{\includegraphics[scale=0.92]{540fig0.eps}}
\end{tabular}
}
\caption{\it Plots of differences between the exact solution and solutions obtained
by the DSMG, VR and DSM-\cite{R526}.
}
\label{fig1}
\end{figure}

Table~\ref{table2} presents numerical results when $N$ varies from $10$ to $100$,
$u(t)=\sin(2\pi t)$, and $t\in[0,1]$.
In this experiment the DSMG yields more accurate solutions than the
DSM-\cite{R526} and the VR. The DSMG in this experiment takes more iterations than the
DSM-\cite{R526} and the VR to get a solution.
\begin{table}[h]
\caption{Numerical results for computing second derivatives with $\delta_{rel}=0.01$.}
\label{table2}
\centering
\small
\begin{tabular}{@{  }c@{\hspace{2mm}}|c@{\hspace{2mm}}
@{\hspace{2mm}}c@{\hspace{2mm}}|c@{\hspace{2mm}}c@{\hspace{2mm}}
|c@{\hspace{2mm}}c@{\hspace{2mm}}|l@{}}
\hline
&
\multicolumn{2}{c|}{DSM}
&\multicolumn{2}{c|}{DSM-\cite{R526}}&\multicolumn{2}{c|}{VR}\\
$N$&
n$_{\text{iter}}$&$\frac{\|u_\delta-y\|_{2}}{\|y\|_2}$&
n$_{\text{linsol}}$&$\frac{\|u_\delta-y\|_{2}}{\|y\|_2}$&
n$_{\text{linsol}}$&$\frac{\|u_\delta-y\|_{2}}{\|y\|_2}$\\
\hline
20    &9    &0.0973    &3    &0.1130    &6    &0.1079 \\
30    &5    &0.0831    &4    &0.1316    &6    &0.1160 \\
40    &7    &0.0488    &4    &0.1150    &6    &0.1045 \\
50    &9    &0.0614    &4    &0.1415    &6    &0.1063 \\
60    &6    &0.0419    &4    &0.0919    &6    &0.0817 \\
70    &9    &0.0513    &4    &0.0961    &6    &0.0842 \\
80    &6    &0.0418    &4    &0.1225    &6    &0.0981 \\
90    &7    &0.0287    &4    &0.0919    &7    &0.0840 \\
100   &7    &0.0248    &5    &0.0778    &7    &0.0553 \\
\hline
\end{tabular}
\end{table}

In this experiment the DSMG is implemented using the SVD of $A$ obtained by the function {\it svd} in Matlab.
As already mentioned, the SVD is a special case of the spectral decomposition
\eqref{specdecom}.
It is expensive to compute the SVD, in general. However, there are practically important problems where the spectral decomposition
\eqref{specdecom} can be computed fast (see Section~\ref{image sec} below).
These problems consist of deconvolution problems using the Fast Fourier Transform (FFTs).

The conclusion from this experiment is: the DSMG may yield results with much better accuracy than the VR and DSM-\cite{R526}.
Numerical experiments for various $u(t)$ show that the DSMG competes favorably with the VR and the DSM-\cite{R526}.

\subsection{An application to image restoration}
\label{image sec}

The image degradation process can be modeled by the following equation:
\begin{equation}
\label{eq35}
g_\delta = g+w,\quad g = h\ast f,\quad \|w\|\le \delta,
\end{equation}
where $h$ represents a convolution function that models the blurring that many imaging systems introduce.
For example, camera defocus, motion blur, imperfections of the lenses, all
these phenomenon can be modeled by choosing a suitable $h$.
The functions $g_\delta$, $f$, and $w$ are the observed image, the original signal, and the noise, respectively.
The noise $w$ can be due to the electronics used (thermal and shot noise), the recording medium (film grain),
or the imaging process (photon noise).

In practice $g,h$ and $f$ in equation \eqref{eq35} are often given as functions
 of a discrete argument and equation
\eqref{eq35} can be written in this case as
\begin{equation}
g_{\delta,i} = g_i+w_i = \sum_{j=-\infty}^\infty f_j h_{i-j}+w_i,\quad i\in \mathbb{Z}.
\end{equation}
Note that one (or both) signals $f_j$ and $h_j$ have compact support (finite length).
Suppose that signal $f$ is periodic with period $N$, i.e., $f_{i+N}=f_i$, and $h_j=0$
 for $j< 0$ and $j\ge N$.
Assume that $f$ is represented by a sequence $f_0,...,f_{N-1}$
and $h$ is represented by $h_0,...,h_{N-1}$. Then the convolution $h\ast f$ is periodic signal $g$ with period $N$, and the elements of $g$ are defined as
\begin{equation}
\label{eq36}
g_i = \sum_{j=0}^{N-1} h_j f_{(i-j)\, mod\, N},\quad i=0,1,...,N-1.
\end{equation}
Here $(i-j)\, mod\, N$ is $i-j$ modulo $N$.
The discrete Fourier transform (DFT) of $g$ is defined as the sequence
$$
\hat{g}_k:=\sum_{j=0}^{N-1}g_je^{-i2\pi jk/N},\qquad k=0,1,...,N-1.
$$
Denote $\hat{g}=(\hat{g}_0,....,\hat{g}_{N-1})^T$. Then equation \eqref{eq36} implies
\begin{equation}
\label{eq37}
\hat{g} = \hat{f}\hat{h},\qquad \hat{f}\hat{g}:=(\hat{f}_0\hat{h}_0,\hat{f}_1\hat{h}_1,...,\hat{f}_{N-1}\hat{h}_{N-1})^T.
\end{equation}
Let $\mathop{\rm diag}(a)$ denote a diagonal matrix whose diagonal is $(a_0,...,a_{N-1})$ and other entries are zeros. Then equation \eqref{eq37} can be rewritten as
\begin{equation}
\label{eq38}
\hat{g} = A\hat{f},\qquad A:=\mathop{\rm diag}(\hat{h}).
\end{equation}
Since $A$ is of the form \eqref{specdecom} with $U=V=I$
and $S=\mathop{\rm diag}(\hat{h})$,
 one can use the DSMG method to solve equation \eqref{eq38} stably for $\hat{f}$.

The image restoration test problem we use is taken from \cite{N}.
This test problem was developed at the US Air
Force Phillips Laboratory, Lasers and Imaging Directorate, Kirtland Air Force
Base, New Mexico.
The original and blurred
images have $256\times 256$ pixels, and are shown in Figure~\ref{figimage1}.
These data has been widely used in the literature for testing image restoration
algorithms.

\begin{figure}[!h!t!b!]
\centerline{
\begin{tabular}{c}
\mbox{\includegraphics[scale=1]{540fig1.eps}}
\end{tabular}
}
\caption{\it Original and Blurred-noisy images.
}
\label{figimage1}
\end{figure}

Figure~\ref{figimage2} plots the regularized images by the VR and the DSMG when $\delta_{rel}=0.01$. Again, with an input value for $\delta_{rel}$, the observed blurred-noisy
images is computed by
$$
g_\delta = g + \delta_{rel}\frac{\|g\|}{\|err\|}err,
$$
where $err$ is a vector with random entries normally distributed with mean 0 and variance 1.
In this experiment, it took 5 and 8 iterations for the DSMG and the VR, respectively, to yield numerical results.
From Figure~\ref{figimage2} one concludes that the DSMG is comparable to the VR in terms of
accuracy. The time of computation in this experiment is about the same for the VR and DSMG.

\begin{figure}[!h!t!b!]
\centerline{
\begin{tabular}{c}
\mbox{\includegraphics[scale=1]{540fig2.eps}}
\end{tabular}
}
\caption{\it Regularized images when noise level is 1\%.
}
\label{figimage2}
\end{figure}

Figure~\ref{figimage3} plots the regularized images by the VR and the DSMG when $\delta_{rel}=0.05$.
It took 4 and 7 iterations for the DSMG and the VR, respectively, to yield numerical results.
Figure~\ref{figimage3} shows that the images obtained by the DSMG and the VR are
about the same.

\begin{figure}[!h!t!b!]
\centerline{
\begin{tabular}{c}
\mbox{\includegraphics[scale=1]{540fig3.eps}}
\end{tabular}
}
\caption{\it Regularized images when noise level is 5\%.
}
\label{figimage3}
\end{figure}

The conclusions from this experiment are: the DSMG yields results with
the same accuracy as the VR, and requires less iterations than the VR.
The restored images by
the DSM-\cite{R526} are about the same as those by the VR.

\begin{rem}{\rm
Equation \eqref{eq35} can be reduced to equation \eqref{eq37} whenever one of
the two functions $f$ and $h$ has compact support and the other is periodic.
}
\end{rem}

\section{Concluding remarks}

A version of the Dynamical Systems Method for solving ill-conditioned linear algebraic systems is studied in this paper.
An {\it a priori} and {\it a posteriori} stopping rules are formulated and justified.
An algorithm for computing the solution
in the case when a spectral decomposition of the matrix $A$ is available is presented.
Numerical results show that the DSMG, i.e., the DSM version developed in this paper, yields results comparable to those obtained by the
VR and the DSM-\cite{R526} developed in \cite{R526}, and the DSMG method may yield much more accurate results than the VR method.
It is demonstrated in \cite{N} that
the rate of convergence of the Landweber method can be increased by
using preconditioning techniques.
The rate of convergence of the DSM version, presented in this paper,
might be improved by a similar technique.
The advantage of our method over the steepest descent in \cite{N} is the following:
{\it the stopping time $t_\delta$ can be found
from a discrepancy principle by Newton's iterations for a wide range of  initial guess
$t_0$;
when $t_\delta$ is found one can compute the solution without any iterations.}
Also, our method requires less iterations than the steepest descent in \cite{N}, which is an accelerated version of the Landweber method.