\section{Introduction}
Flat-histogram Monte Carlo simulation algorithms calculate the thermodynamic
properties of various systems over a range of temperatures.  The first histogram
method used a single canonical Monte Carlo simulation to predict properties for
nearby temperatures~\cite{ferrenberg1988new}. While the method effectively
samples a narrow energy range, it proves computationally inefficient at sampling large energy ranges.
Multicanonical methods, introduced by Berg and Neuhaus, enabled flat-histogram
sampling which improved the exploration of configurational space and allowed the
simulation to overcome free-energy barriers~\cite{berg1991multicanonical, berg1992multicanonical}.
These works led to increase in the development of a variety of ``flat'' (or ``broad'') histogram
methods~\cite{penna1996broad, penna1998broad, swendsen1999transition,
wang2001determining, wang2001efficient} which could explore a wider range
of energies.  In addition to obtaining thermodynamic information for the entire
energy range for a single simulation, these approaches cannot be easily trapped
in a local energy minimum like a canonical simulation.

Wang and Landau introduced one of the most widely used flat-histogram
Monte Carlo algorithms that determined the density of states (DOS) for
a statistical system~\cite{wang2001determining,wang2001efficient}. For all of
its power, the method unfortunately requires \emph{a priori.} knowledge of several
user-defined parameters. Thus, for any given system under study, the user needs
to determine the ideal parameters in order to apply the method. The Wang-Landau
algorithm is also known to violate detailed balance (although only for brief
time intervals)~\cite{yan2003fast, shell2002generalization}. With the violation
of detailed balance, convergence of the algorithm is not guaranteed.

Because of the uncertainty of convergence for WL, many studies have been undertaken
to understand how the modification (or update) factor $\gamma$ impacts the
convergence~\cite{zhou2005understanding,lee2006convergence,
belardinelli2007wang}. Belardinelli and Pereyra showed that an update factor that decreases faster than $1/t$ leads to nonconvergence~\cite{belardinelli2007fast, belardinelli2007wang, belardinelli2008analysis, zhou2008optimal}, where $t$ corresponds to the number of moves. Schneider \emph{et al.} outline minor refinements algorithm including scaling the update factor with the number of energy bins~\cite{schneider2017convergence}.
These studies led to the
formation of the $1/t$-WL algorithm and have also led some researchers to follow
WL with a ``production run'' with fixed weights, in order to preserve ergodicity and
detailed balance~\cite{jayasri2005wang, mukhopadhyay2008monte}.
%
To our knowledge, this work is the first test of the convergence
properties of WL followed by a production run with comparison to other
methods.

Liang independently considered whether WL could be treated as a special case of
stochastic approximation whose convergence could be mathematically
proven~\cite{liang2006theory, liang2007stochastic}. In 2007, Liang \emph{et
al.}~\cite{liang2007stochastic} argued that WL can be considered a form of
stochastic approximation Monte Carlo (SAMC). Unlike WL, SAMC can guarantee
convergence (if certain conditions are met). Despite the added benefit of
guaranteed convergence, the method still has a system specific user-defined
variable. Such variables often create difficulty when applying Monte Carlo
methods across arbitrary systems.

% optimized ensemble paragraph
Another challenge that flat-histogram methods face is that the convergence rate is
impacted by energy barriers and bottleneck which can make traversing the phase
space difficult. Nadler \emph{et al.} and Trebst \emph{et al.} systematically
examined optimized ensembles to address performance issues flat-histogram methods
face when confronted with hidden energy barriers~\cite{nadler2007generalized,
nadler2007dynamics, trebst2004optimizing}.
The approach involves using an ensemble that does \emph{not} result in a
flat histogram, but instead optimizes the rate of diffusion between low
energy and high energy states.
The goal of these methods is to make the statistical errors uniform.
%
Optimized ensemble methods typically begin by using a flat-histogram method
to get a first approximation for the weights, which means that any of the
methods
tested in this work could be used as a starting point for an optimized
ensemble simulation.

% replica exchange paragraph

An approach to parallelizing flat-histogram Monte Carlo methods is
the replica-exchange approach, which was pioneered by parallel tempering
algorithms~\cite{geyer1995annealing, hukushima1996exchange,
hansmann1997parallel}.  Vogel \emph{et al.} adapted this approach to develop
the Replica Exchange Wang-Landau (REWL)~\cite{vogel2013rewl} approach.  This
approach for parallelization is sufficiently simple and general that it could
equally be applied to any of the methods explored in this paper.

Kim \emph{et al.} introduced Statistical Temperature Monte Carlo (STMC) and the
related Statistical Temperature Molecular Dynamics (STMD), an adaption of the
WL method that approximates the entropy as a piecewise linear function, which
improves convergence for systems with a continuously varying
energy~\cite{kim2006statistical, kim2007statistical}. STMC applied to WL
requires a temperature range be specified rather than an energy range. Kim
\emph{et al.} extended this work as Replica Exchange Statistical Temperature
Monte Carlo (RESTMC), which uses replica exchange of multiple overlapping STMC
simulations to improve convergence~\cite{kim2009replica}. Recently, Junghans
\emph{et al.} demonstrated a close connection between metadynamics, which was
introduced by Laio and Parinello~\cite{laio2002escaping}, and WL-based Monte
Carlo methods, with STMD forging the connection~\cite{junghans2014molecular}.

The SAD (stochastic approximation with a
dynamic $\gamma$) method as outlined by Pommerenck
et.al~\cite{pommerenck2020stochastic} is a special version of the SAMC algorithm
that dynamically chooses the modification factor rather than relying on
un-physical user-defined parameters. SAD shares the same convergence properties
with SAMC while replacing un-physical user-defined parameters with the
algorithms dynamic choice.

In this work, we compare the convergence properties of five flat-histogram
methods. We detail how each method is implemented and apply the family of
weight-based flat-histogram Monte Carlo methods (pure WL, WL followed by a
``production run'', $1/t$-WL, SAMC, and SAD) to the 2D Ising model.

\section{Ising Model}
The 2D Ising spin-lattice system is widely used as a testbed when
benchmarking or comparing Monte Carlo
methods~\cite{ferdinand1969bounded, wang1999transition, trebst2004optimizing, barash2019estimating}. The 2nd order
phase transition behavior and the ability to directly calculate the
exact solution for finite lattices~\cite{beale1996exact, haggkvist2004computation} make the
system sufficiently interesting for such theoretical comparisons. It is
also important to note that direct comparison of the other methods can
be made with WL as its original implementation was done on this
system~\cite{wang2001determining,wang2001efficient}.
We test the convergence of several flat-histogram methods
on the periodic 2D square lattice ferromagnetic Ising model with identical
nearest neighbor interactions~\cite{landau2004new} ($J_{ij} = J$).
\begin{align}
\mathcal{H} = -J \sum_{\braket{i,j}} \sigma_i \sigma_j - h \sum_i s_i
\end{align}
The $N\times N$ spin system can take on values of $\sigma_i = \pm 1$
for up or down spins respectively. In the absence of a magnetic field ($h =
0$), We can write the Hamiltonian as follows~\cite{onsager1944crystal,
kaufman1949crystal}:
\begin{align}
\mathcal{H} = - J\sum_{\braket{i,j}} \sigma_i \sigma_j
\end{align}
where the sum is over nearest neighbor spin sites. Beale showed that for finite
lattices the Density of States could directly be calculated from the partition
function~\cite{beale1996exact}
\begin{align}
Z = \sum_E g(E) e^{-{\beta E}}
\end{align}
% where $g(E)$ is the multiplicity of the system which is proportional to
% $D(E)$. We can estimate the average deviation from the exact solution
% using the relation~\cite{schneider2017convergence, shakirov2018convergence,
% barash2017control}:
% \begin{align}
% \braket{\epsilon(t)}_E = \frac{1}{N_E - 1} \sum_E \left( \frac{S(E,t) - S_{\text{Beale}(E)}}{S_{\text{Beale}(E)}} \right)
% \end{align}
where $g(E)$ is the multiplicity of the system which is proportional to
$D(E)$. We can compute the maximum deviation in the canonical specific heat capacity $c_V$ from the exact solution~\cite{schneider2017convergence, shakirov2018convergence,
barash2017control,barash2017gpu}:
\begin{align}
\epsilon \equiv \max_T \left|\left(c_V(T)_{\text{MC}} - c_V(T)_{\text{Beale}}\right)\right|
\end{align}
Computing the specific heat capacity $c_V$ presents a difficult challenge for
any Monte Carlo method due to fluctuations in the derivative of the
internal energy around the phase transition. The critical temperature $T_c = 2.
269T$ or $\beta= 0.441J$ comes directly from the Kramers-Wannier
duality~\cite{bhattacharjee1995fifty} and marks the transition from a
disordered to ordered magnetic state. Methods that accurately compute $c_V$
also by extension accurately compute the internal energy.

\section{Flat-histogram methods}\label{sec:histogram}
Flat-histogram methods compute the density of states $D(E)$ over a broad range
of energies by simulating each energy with equivalent accuracy. Flat-histogram
Monte Carlo methods propose randomly chosen ``moves'' which change the state of
the system and must satisfy detailed balance.  Each algorithm differs in how it
determines the probability of accepting a move and in what additional statistics
must be collected in order decide on that probability.

We describe several closely related flat-histogram methods which each rely on a
weight function $w(E)$ to determine $D(E)$.  For these algorithms, the
probability of accepting a move is given by
\begin{equation}
	\mathcal{P}(E_\text{old} \rightarrow E_\text{new})
	= \min\left[1,\frac{w(E_\text{old})}{w(E_\text{new})}\right]
\end{equation}
which biases the simulation in favor of energies with low weights. The result of
weights $w(E)$ that are proportional to $D(E)$ is an entirely flat-histogram. We
can relate the entropy to the weights in the microcanonical ensemble, since the
entropy is defined as $S(E) \equiv k_B\ln(D(E)) \approx \ln w(E)$.

Flat-histogram methods employ a random walk in energy space to estimate $D(E)$.  Each method operates by continuously updating the weights at each
step of the simulation
\begin{equation}
	\ln{w_{t+1}(E)}=\ln{w_{t}(E)}
	+\gamma_t
\end{equation}
where $t$ is number of the current move, $\gamma(t)$ is a move-dependent update
factor, and $E$ is the current energy.  This update causes the random walk to
avoid frequent sampling of the same energies, leading to a rapid exploration
of energy space. Flat-histogram methods differ primarily in how they schedule
the decrease of $\gamma_t$.  Figure~\ref{fig:N32-gamma} shows several
flat-histogram methods each decreasing $\gamma_t$ as a function of moves.
Methods that decrease $\gamma_t$ too rapidly can fail to converge while methods
that decrease too slowly can take infinitely long to converge to the correct
$D(E)$.

\begin{figure}
\includegraphics[width=\columnwidth]{gamma-n32.pdf}
  \caption{
  The update factor $\gamma_t$ versus the iteration number for the $N=32 \times 32$
  system.}
  \label{fig:N32-gamma}
\end{figure}

The Wang-Landau algorithm~\cite{wang2001efficient,wang2001determining,
landau2014guide} explores energy space by setting
$\gamma_{t=0}^{\text{WL}}=1$, and then decreases $\gamma^{\text{WL}}$ in prescribed stages. An energy range of
interest must be specified~\cite{wang2001efficient, schulz2003avoiding,yan2003fast}, which often requires multiple simulations if unknown.
%
The number (``counts'') of moves ending at each energy are stored in a
histogram.  For a sufficiently flat energy histogram (typically user-specified to be 0.8), $\gamma^{\text{WL}}$ is decreased by a specified factor of $\frac12$ and the histogram is reset to zero. The entire process is repeated until $\gamma^{\text{WL}}$ reaches a desired cutoff.

The $1/t$-WL algorithm ensures convergence by preventing the $\gamma_t$ factor
from dropping below $N_S/t$~\cite{belardinelli2008analysis,
schneider2017convergence}. The method follows the standard WL algorithm with two
modifications.  Firstly, when each energy state has been visited once, the histogram is considered flat and $\gamma_t$ is
decreased by a factor of two. Secondly, when
$\gamma^{\text{WL}} < N_S/t$ at time $t_0$, the update factor becomes
$\gamma_t = N_S/t$ for the remainder of the simulation:
\begin{align}
  \gamma_t^{1/t\text{-WL}} = \begin{cases}
     \gamma^{\text{WL}}_t & \gamma^{\text{WL}}_t > \frac{N_S}{t} \\
     \frac{N_S}{t} & t \ge t_0
 \end{cases}
\end{align}
where $t$ is the number of moves, $\gamma^{\text{WL}}_t$ is the Wang-Landau update factor
at move $t$, and $N_S$ is the number of energy bins.

The WL method can be terminated after $\gamma$ reaches a specified
minimum $\gamma^{\min}$ followed by a \emph{production run} in which the
weights are held fixed~\cite{gross2018massively}. WL is used to
generate the weights resulting in a
flat-histogram~\cite{janke2017generalized}.  The entropy is then
computed (up to a constant) by adding the logarithm of the production
histogram to the logarithm of the weights.  The production run thus
satisfies detailed balance, and will ideally be ergodic; however, the
convergence of the simulation is still impacted by the choice of the
minimum $\gamma^{\min}$.

Another weight-based flat-histogram method is
the stochastic approximation Monte Carlo (SAMC) algorithm. SAMC has a simple
schedule by which the update factor $\gamma^{\text{SA}}_t$ is continuously
decreased~\cite{liang2007stochastic, werlich2015stochastic,
schneider2017convergence}.  The update factor is defined in the
original implementation~\cite{liang2007stochastic} in terms of an
arbitrary tunable parameter $t_0$,
\begin{align}
\gamma_{t}^{\text{SA}} =\frac{t_0}{\max(t_0,t)}
\end{align}
where as above $t$ is the number of moves that have been attempted.

The implementation of SAMC is extremely simplistic.
In addition, Liang has proven that the weights converge to the true
density of states~\cite{liang2006theory, liang2007stochastic,
liang2009improving} provided the update factor satisfies
\begin{align}
\sum_{t=1}^\infty \gamma_{t} = \infty \quad\textrm{and}\quad
\sum_{t=1}^\infty \gamma_{t}^\zeta < \infty
\end{align}
where $\zeta > 1$.  Unlike WL methods, the energy range need not be
known \emph{a priori.} and the convergence time depends only on the choice of
parameter $t_0$.
Unfortunately, $t_0$ can be difficult to chose in advance
for arbitrary systems.
Liang \emph{et al.} give a rule of thumb in
which $t_0$ is chosen in the range from $2N_S$ to $100N_S$ where $N_S$
is the number of energy bins~\cite{liang2007stochastic}.  Schneider
\emph{et al.} found and we confirm that for the Ising model this heuristic is helpful for small spin systems, but that larger systems require an even higher
$t_0$ value~\cite{schneider2017convergence}.

Pommerenck \emph{et al.} propose a refinement~\cite{pommerenck2020stochastic} to SAMC
where the update factor is determined dynamically rather than by the user.
Stochastic approximation with a dynamic $\gamma$ (SAD) requires the user to
provide a minimum temperature of interest $T_{\min}$. This is analogous to WL
requiring \emph{a priori.} an energy range of interest; however, this is almost always easier to
identify and is more physical than the SAMC parameter $t_0$. The
update factor is given by:
\begin{align}
  \gamma_{t}^{\text{SAD}} =
     \frac{
       \frac{E_{H}-E_{L}}{T_{\text{min}}} + \frac{t}{t_L}
     }{
       \frac{E_{H}-E_{L}}{T_{\text{min}}} + \frac{t}{N_S}\frac{t}{t_L}
     }
\end{align}
where $E_H$ and $E_L$ are the current estimates for the highest and
lowest interesting energies and $t_L$ is the last time at which an
energy in the range of interest is encountered.
%
SAD only explores the energy range of interest as specified by the minimum
temperature of interest $T_{\min} < T < \infty$. During the simulation the two
energies $E_H$ and $E_L$, are refined such that the range of energies are
conservatively estimated. The weights are calculated for each energy region according to the original prescription.
\begin{enumerate}
\item {$E < E_L$:} $w(E>E_H) = w(E_H)$
\item {$E_L < E < E_H$:} moves are handled the same as other weight-based
methods that are mentioned
\item {$E > E_H$:} $w(E<E_L) = w(E_L)e^{-\frac{E_L-E}{T_{\min}}}$
\end{enumerate}
Each time the simulation changes the value of $E_H$ or $E_L$, the weights
within the new portion of the interesting energy range are updated.

\section{Results}
We test the algorithms on two different system sizes of the 2D Ising model.
The first is a smaller simulation with a lattice size of $N = 32 \times 32$ and
the second has a lattice size of $N = 128 \times 128$. The SAD method explores
the energy space of each system using a minimum reduced temperature of $T_{\text{min}} = 1$. All simulations calculate the minimum important energy $E_{\min}$
and maximum entropy energy $E_{\max}$ (with the exception of the WL methods
where both of these parameters are needed \emph{a priori.}).

% \begin{figure}
% \includegraphics[width=\columnwidth]{N32-entropy-error-default.pdf}
%   \caption{
%   The average entropy error for each MC method for $N=32 \times 32$,
%                $\delta_0 = 0.05\sigma$, and $T_{\min} = 1$
%                as a function of number of iterations run.  The error is
%                averaged over 8 independent simulations, and the best
%                and worst simulations for each method are shown as a
%                semi-transparent shaded area.}\label{fig:n32}
% \end{figure}

\begin{figure}
  \includegraphics[width=\columnwidth]{N32-Cv.pdf}
  \caption{
    The specific heat capacity versus the reciprocal temperature $\beta$ for the $N=32 \times 32$ system for each histogram method at $10^{9}$ moves.}
  \label{fig:N32-cv}
\end{figure}

\subsection{The 32 $\times$ 32 Ising model}
Figure~\ref{fig:N32-gamma} shows the update factor $\gamma_t$ for each
of the flat-histogram methods. All of the update factors initially
start at $\gamma_{t=0} = 1$. SAD dynamically updates $\gamma_t$
throughout the simulation. After about $10^{10}$ moves,
$\gamma^{\text{SAD}}_t$ proceeds as $1/t$. We show
$\gamma^{\text{WL}}_t$ for WL both with and without a production
run. The WL production run begins after $\gamma^{\text{WL}}_t$ has reached
$10^{-4}$. The update factor for $1/t$-WL decreases similarly to WL
before finding all the energy states and switching to $1/t$. All of
the SAMC update factors equal 1 until the number of moves is equal to
$t_0$ at which point they decrease as $1/t$.

All of the methods, except for SAMC, use the same single random number seed and
give a reasonable approximation for the heat capacity peak resulting from the
phase transition after only 10$^9$ moves. Figure~\ref{fig:N32-cv} shows the
specific heat capacity vs. the reciprocal temperature $\beta$ at $10^{9}$ moves.
A temperature range of $1.5k_B$ to $5k_B$ is chosen to highlight
the phase transition at the critical temperature $T_c$.

\begin{figure}
  \includegraphics[width=\columnwidth]{N32-Cv-error-default.pdf}
  \caption{The maximum error in the specific heat capacity for each method for the $N=32 \times 32$ and $T_{\min} = 1$ as a function of number of iterations run.  The maximum error is averaged over 8 independent simulations, and the best and worst simulations for each method are shown as a semi-transparent shaded area.}
  \label{fig:N32-cv-error}
\end{figure}

Figure~\ref{fig:N32-cv-error} shows the maximum error in the heat capacity as a
function of time for this system. The solid/dashed lines represent the average
of the maximum value of the error in the specific heat capacity $c_V$ averaged
over eight simulations using different random number seeds. The range of maximum
errors for each simulation is shown as a shaded region. By
the time $10^8$ moves have been made all but the WL simulation have begun to
converge as $1/\sqrt{t}$. We then see the WL error saturate around $10^9$ moves.

\begin{figure}
\includegraphics[width=\columnwidth]{gamma-n128.pdf}
  \caption{
  The update factor $\gamma_t$ versus the iteration number for the $N=128 \times 128$
  system.}
  \label{fig:N128-gamma}
\end{figure}

\begin{figure}
  \includegraphics[width=\columnwidth]{N128-Cv-error-default.pdf}
  \caption{The maximum error in the specific heat capacity for each method for the $N=128 \times 128$ and $T_{\min} = 1$ as a function of number of iterations run.  The maximum error is averaged over 8 independent simulations, and the best and worst simulations for each method are shown as a semi-transparent shaded area.}
  \label{fig:N128-cv-error}
\end{figure}

% \begin{figure}
% \includegraphics[width=\columnwidth]{N128-entropy-error-default.pdf}
%   \caption{
%   The average entropy error for each MC method for $N=128 \times 128$,
%                $\delta_0 = 0.05\sigma$, and $T_{\min} = 1$
%                as a function of number of iterations run.  The error is
%                averaged over 8 independent simulations, and the best
%                and worst simulations for each method are shown as a
%                semi-transparent shaded ar.}\label{fig:n128}
% \end{figure}

\subsection{128 $\times$ 128 Ising system}
For the larger 2D Ising system, the update factors initially
start at the same $\gamma_{t=0} = 1$; however, all of the methods take longer
to proceed as $1/t$ (with SAD taking $10^{13}$ moves). Fig.~\ref{fig:N128-gamma} shows the update factor $\gamma_t$ for each of the
flat-histogram methods.
We implement two WL simulations followed by a production run each beginning
after $\gamma^{\text{WL}}_t$ has reached $10^{-4}$ and $10^{-6}$ respectively.
Fig.~\ref{fig:N128-cv-error} shows the maximum error in the heat capacity as a
function of time for this system. The solid/dashed lines represent the maximum
value of the error in the specific heat capacity $c_V$ averaged
over eight simulations using different random number seeds. The range of maximum
errors for each simulation is shown as a shaded region. By
the time $10^{13}$ moves have been made all but the WL simulation have begun to
converge as $1/\sqrt{t}$. We then see the WL error saturate around $10^{12}$
moves.  The WL simulation followed by a production run when
$\gamma^{\text{WL}}_t$ reached $10^{-4}$ sometimes dramatically failed to
converge, although five of the eight random seeds converged very
nicely.  This highlights a risk taken when setting $\gamma^{\min}$.
If the weights are insufficiently converged, the production run will
fail to explore all energies, in this case, three of the simulations became ``stuck'' at low energies. With the smaller value of
$\gamma^{\min}=10^{-6}$, the method consistently and efficiently converged.

Figure~\ref{fig:N128-cv-error} shows that the SAD algorithm converges
significantly more slowly than the converging WL methods.
The SAD algorithm on average takes around
$10^9$ moves to identify as important all 8192
negative energy states, which is around twice as long as
the number of moves that
the WL methods require in order to explore all the states in this
energy range.  The main difference between the convergence of these
methods is that the WL-based methods decrease $\gamma_t$ far more rapidly,
which leads to more rapid convergence.  The Wang-Landau approaches can
get away with this because the range of energies of interest is given
as an input rather than an output, allowing a more aggressive schedule
of reduction of $\gamma_t$.  This aggressive behavior is precisely what
requires that Wang-Landau methods be followed by some correction stage
(either $1/t$ or a production run) in order to correct residual errors.

\section{Conclusion}
We find that SAD, $1/t$-WL, and WL followed by a production run (with an
adequately small $\gamma^{\min}$) demonstrate excellent and robust
convergence. They all converge more rapidly than SAMC, and unlike pure
WL do not suffer from error saturation. We find that for larger Ising
systems SAD reduces the update factor more slowly (and conservatively)
than $1/t$-WL and WL followed by a production run. This means that SAD
will take proportionately more moves to converge to the same value as
$1/t$-WL as system size is increased.  While the WL methods are
are given the energy range
\emph{a priori}, rather than a temperature range of interest such as
SAD requires, we find that the SAD histogram counts for energies outside the range of interest are negligible. For the general case in
which the energy range of the system is not known and where a range of
desired temperatures is known, the SAD method is
considerably more convenient, and quite possibly more efficient than a
process involving multiple simulations to determine an energy range
of interest. This work also dramatically demonstrates that WL followed
by a production run performs extremely well and is preferable to pure
WL for ensuring both ergodicity and detailed balance. While this has long
been thought to be the case and that a multi-canonical run where WL is used to
determine the parameters is the only way WL should be used, this research
represents the first detailed comparison among all of these
flat-histogram methods.

\section{Acknowledgments}
We wish to thank Johannes Zierenberg for helpful discussions regarding WL followed by a production run and for insights into properties of a production run at different stages of convergence.


\documentclass[]{article}
\usepackage{caption,subcaption,graphicx,float,url,amsmath,amssymb,tocloft,wasysym,amsthm,thmtools,textcomp,listings,amsfonts,cancel}
\usepackage[hidelinks]{hyperref}
\usepackage[toc,acronym,nonumberlist]{glossaries}
\usepackage[]{algorithm2e}
\setacronymstyle{long-short}
\usepackage{glossaries-extra}
\graphicspath{{figs/}}
\setlength{\cftsubsecindent}{0em}
\setlength{\cftsecnumwidth}{3em}
\setlength{\cftsubsecnumwidth}{3em}
\newcommand\numberthis{\addtocounter{equation}{1}\tag{\theequation}}
\newtheorem{thm}{Theorem}
\newtheorem{cor}[thm]{Corollary}
\setcounter{tocdepth}{1}

%opening
\title{Computation in Complex Systems\\
	Week 3\\
	P or NP? That is the Question.
	}

\begin{document}

\maketitle

\section{Circuits \& Formulae}

Table \ref{table:truth} is the truth table for the two formulae. I have used the symbol $\top$ to show that a formula evaluates to true, $\bot$ for false. The column headed ''AND'' denotes the truth value of $y_1 = x_1 \land x_2$, and that headed ''3-SAT'' denotes the truth value of $ (x_1\lor \bar{y_1}) \land (x_1\lor \bar{y_2}) \land (\bar{x_1} \lor \bar{x_2} \lor y_1)$. The demonstrandum asserts that these two columns are equal. Since the 3-clause is a conjunction, one 'F' in any of the 3 preceding columns is enough to falsify the clause.

\begin{table}[H]
	\caption{Truth Table--Circuits \& Formulae, showing that the columns headed "AND" and "3-clause" are identical. Symbols $\top$ and $\bot$ are used to denote True and False, respectively.}\label{table:truth}
	\begin{center}
		\begin{tabular}{|c|c|c||c|c|c|c||c|}\hline
			$x_1$&$x_2$&$y_1$&AND&$x_1 \lor \bar{y_1}$&$x_2 \lor \bar{y_1}$&$\bar{x_1}\lor\bar{x_2}\lor y_1$&3-clause\\ \hline
			F&F&F&$\top$&T&T&T&$\top$\\ \hline
			F&F&T&$\bot$&F&F&T&$\bot$\\ \hline
			F&T&F&$\top$&T&T&T&$\top$\\ \hline
			F&T&T&$\bot$&F&&&$\bot$\\ \hline
			T&F&F&$\top$&T&T&T&$\top$\\ \hline
			T&F&T&$\bot$&T&F&&$\bot$\\ \hline
			T&T&F&$\bot$&T&T&F&$\bot$\\ \hline
			T&T&T&$\top$&T&T&T&$\top$\\ \hline
		\end{tabular}
	\end{center}
\end{table}

\begin{enumerate}
	\item Given than the three clauses are linked by logical AND statements, all three clauses must be true, or else the clause is false.

	\item If $x_1$ is false, $y_1$ must be false.

	\item If $x_2$ is false, $y_1$ must be false.

	\item If $x_1$ is true and $x_2$ is true, $y_1$ must be true.
\end{enumerate}

\section{Travelling Salesperson}

My reading of this question is that I am supplied with a witness--a path that is alleged to solve the problem.
I shall use this criterion: \begin{quotation}
	A yes/no question is in NP if, whenever the answer is "yes", there is an easily checked proof or "witness" to that fact\cite[Lecture 3.5]{sfi2020computation}.
\end{quotation}

\begin{enumerate}
	\item Is $D<10,000$ (miles)?
	\begin{enumerate}
		\item Suppose someone presents we with a collection of edges,
			$W$, say, which she asserts is a witness.
		\item I can easily check that it includes all nodes and returns to the origin, so it is a path.
		\item I can easily verify that the length of $W$, $L(W)$, say, is less than 10,000. Now there are two possibilities:
		\begin{enumerate}
			\item W is the shortest path, so  $L(D)=L(W)<10000$, or
			\item W is not the shortest path, so $\exists D \mid L(D)<L(W)<10000$.
		\end{enumerate}
		Either way $L(D)<10000$.
		\item This problem is therefore in NP.
	\end{enumerate}

	\item Is D more than 8,000 miles?
		\begin{enumerate}
		\item The previous argument does not work in this case. If someone produces a path that is more 8,000 miles, I have no way of easily knowing whether or not there the minimum path is or less than this one. I can test all possible paths to see whether or not they are shorter than 8,000, and I can save a bit of time by rejecting a path as soon as I find a partial length over 8,000: this may make the process a little shorter, but it remains exponential.
		\item IMHO this decision problem is not NP.
	\end{enumerate}

	\item Is D exactly 9,219 miles?
		\begin{enumerate}
		\item This is not NP.
		\item The argument is similar to the "more than 8,000" argument. Although I might get lucky and discredit the witness if I stumble across a path that is shorter than 9,219, this doesn't make the task of verifying a correct solution any easier.
	\end{enumerate}
\end{enumerate}

\section{Complexity Hierarchy}

From \cite{sfi2020glossary}, a cellular automaton is: \begin{quotation}
	A mathematical or computational system in which simple elements (''cell'') are arrayed in a regular lattice. At a given time step, each cell is in some discrete state, and at each time step, each cell updates its state using a function of its current state and the states of its neighboring cells. To define a particular cellular automaton, one must specify the dimensionality of the lattice, the neighborhood of a cell, the set of possible states, and the state update function used by each cell.
\end{quotation}
I am going to assume that I know the rule each cell uses to update its state, that there a $m$ cells, and the state of each cell is binary--0 or 1. If this is not true, if there the number of states is more than 2 but does not exceed $2^k$ for some $k$, I can replace the cellular automaton with another machine with $k\cdot m$ binary cells.

\begin{thm}
	The problem " What will the state be at $t_{n+x}$?" is in P.
\end{thm}

\begin{proof}
	I need to apply the rule $n$ times to $m$ cells, i.e. $m \cdot n$ times.
	This is linear in $n$ and $m$, i.e. polynomial, so the problem is in P.
\end{proof}

\begin{thm}
	The problem "Does s have a predecessor?" is in NP.
\end{thm}

\begin{proof}
	I can test for the existence of a predecessor by applying the rule to $2^m$ possible states and checking the result, to see whether it matches $s$, st which point I can stop and answer \emph{yes}. This process can require time $\propto 2^m$, so it is exponential.

	OTOH if someone offers me a state which they claim to be a witness that there is a predecessor,  I can test it in time $\propto m$, so the problem is in NP.

\end{proof}

\begin{thm}
	The problem "On a lattice of size $n$, is $s$ on a periodic orbit $stuvw...s...$?" is in PSPACE\footnote{I am discounting the possibility that s \emph{leads in to} a periodic orbit, $stuvwx...u...$, which would require each successor to be tested against $s,t,u,v,w,x...$}.
\end{thm}

\begin{proof}
	\begin{enumerate}
		\item On a lattice of size $n$ there are $2^n$ possible states. Therefore if I am presented with $s$ it could  $2^n$ steps. So we are dealing with exponential time.
		\item Even if someone presents we with a path that is allegedly a witness, it still could potentially have $2^n$ steps, so I cannot test in polynomial time. So we can't say that the problem is in $NP$.
		\item Let us assume that I iterate through all states, starting with $n*'0'$ (i.e. all states zero) and ending with $n*'1'$, working from right to left: I iterate until I return to the initial state $s$, or I reach $n*'1'$. Then I need $n$ memory locations to keep track of where I am in the iteration, so the memory requirement is polynomial.
	\end{enumerate}
\end{proof}

\begin{thm}\label{thm:undecideable}
	The problem ''On a lattice of infinite size, will $s$ ever die out?'' is undecidable.
\end{thm}

\begin{proof}
	All we need to do is show that this is undecidable for a particular finite automaton and a particular $s$. We will chose Rule 110--Table \ref{table:rule110}-- since we know it is universal--\cite{cook2004universality}. It is also clear that "dying out" actually makes sense, as Table \ref{table:rule110} shows that $0\rightarrow 0$: we can interpret "dies out" to mean "all zeroes".
	\begin{table}[H]
		\begin{center}
			\caption{Rule 110\cite{wiki:rule:110}}\label{table:rule110}
			\begin{tabular}{|c|c|c|c|c|c|c|c|}\hline
				111 &110&101&100&011&010&001&000\\ \hline
				0 &	1& 	1& 	0&	1 &	1 &	1& 	0 \\ \hline
			\end{tabular}
		\end{center}
	\end{table}

	I assert that Theorem \ref{thm:undecideable} is equivalent to the halting problem. Since Rule 110 is universal, we can encode the halting problem for any Turing machine by a state, and ask whether it will ever die out. The assumption is that "dying out" is somehow equivalent to "halting", which appears to be to be equivalent to Cook's observation: \cite[page 38]{cook2004universality}
	\begin{quotation}
		All of the above machines had a way of halting, whereas the Turing machines presented here do not, since they emulate Rule 110, which, being a 	cellular automaton, does not have an explicit mode of halting. However, the machines presented here are certainly capable of “effective computation” in the sense of Minsky, and so questions about their behavior, 	such as “Will this sequence of symbols ever appear on the tape?”, are undecidable.
	\end{quotation}

	(I am aware that there is still some hand-waving here, but I don't believe I can do better without thoroughly digesting \cite{cook2004universality}.)
\end{proof}

% bibliography go here

\bibliographystyle{unsrt}
\raggedright
\addcontentsline{toc}{section}{Bibliography}
\bibliography{computations}

\end{document}