%!TEX root = ../../dissertation.tex

\section{Introduction}

There is a growing interest among economists in global nonlinear solution
methods. Value function discretization is the most well-known example of a
global nonlinear solution method, but there are a variety of other iterative
and Euler-equation based methods that differ in the way they approximate,
interpolate, integrate and construct nonlinear solutions. These methods solve
economic models on grids of points that cover a relevant area of the state
space, and they generally produce more accurate solutions than perturbation
methods which typically build approximations around a single point.\footnote{
The accuracy of local perturbation solutions can decrease rapidly away from
steady state even in relatively smooth models; see \cite{KKK2011}. In models
with stronger nonlinearities, such as new Keynesian models, approximation
errors can reach hundreds of percent under empirically relevant
parameterizations; see \cite{JMM2017}. In models of labor search, a global
solution is important for accurate capturing even first moments of equilibrium
dynamics; see \cite{PZ2017}} Also, global methods are useful for solving some
models in which perturbation methods are either not applicable or their
applications are limited.\footnote{ For example, perturbation methods are not
well suitable for analyzing sovereign default problems (e.g.,
\cite{Arellano2008}) and portfolio choice problems (e.g., \cite{HK2013}). There
are perturbation-based methods that can deal with occasionally binding
constraints but they are limited to the first-order approximation. For example,
see \cite{LS2011} and \cite{GI2015}.} Finally, the recent developments of numerical
techniques for dealing with high-dimensional data has greatly extended the
class of models that can be studied with global solution methods. In
particular, it is now possible to construct global nonlinear solutions to
economic models with dozens of state variables that are intractable with
conventional global solution methods (such as value-function discretization)
due to the curse of dimensionality.\footnote{ See \cite{MM2014} for a survey of
numerical techniques that are tractable in problems with high dimensionality
and see \cite{LMM2017} for an example of constructing a global nonlinear
solution to a large-scale projection model of the Bank of Canada.}

However, global nonlinear solution methods are more difficult to automate than
perturbation methods, and their implementation requires more substantial
programming from researchers. In particular, there is still no consensus in the
profession on what software to use for implementing global solution methods,
unlike for perturbation methods where a common choice is the Dynare or IRIS platforms.
Because of this, an important aspect of the implementation of a global solution method is the
choice of programming language to use. Traditionally, Matlab is used in
economics, although some researchers have also used Fortan and C. Recently,
Python and Julia have begun to see a more widespread use in the economics
literature.\footnote{ For example, Python and Julia are used in the
\textit{Quantitative economics} lecture site of Tom Sargent and John
Stachurski. Python is used to produce the \textit{Heterogeneous Agents
Resources and toolKit} (HARK) by the team led by Christopher Carroll.}

In this paper, we pursue two goals: First, we provide a comparison between
solution methods implemented in Matlab, Python and Julia in the context of two
popular applications: a neoclassical growth model and a new Keynesian model.
Our goal is to help economic researchers choose the programming language that
is best suited to their own situation, and, if needed, help them transition
from one programming language to another. The readers can see and compare how
the same algorithm is implemented in different languages and, as a result, will
understand some of the unique aspects of each language and choose the one which
best fits their needs.\footnote{ Additionally, we provide carefully elaborated
notebooks with a description of our code and algorithms on the
\href{http://notes.quantecon.org}{QuantEcon Notebook site}\dots} The
implementation, structure and number of lines of code are similar across all
three languages.

Second, we provide a carefully documented collection of routines for Matlab,
Python and Julia that can be used for constructing global nonlinear solutions
with a special emphasis on modern techniques for problems with high
dimensionality. In particular, our code includes routines for constructing
random and quasi-random grids, low-cost monomial integration methods,
approximating functions using complete polynomials, as well as routines for
computing the numerical accuracy of solutions. Much of this code is generic in
a way which allows it to be easily portable to other applications including
problems with kinks and occasionally binding constraints. Our examples are
solved using a variety of solution techniques including: conventional policy
function iteration, conventional value function iteration, an Euler equation method,
the endogenous grid method of \cite{Carroll2006}, and several variants of the
envelope condition method of \cite{MM2015}.

\cite{AF2014} is closely related to our work. In their paper, the authors
compare 18 different programming languages by implementing the same algorithm --
value-function discretization -- to solve the stochastic neoclassical growth
model and by reporting the CPU time needed to solve the model. They find that
the choice of a programming language plays an important role in computation
time: the fastest language in their study (C++) solves the model over 450 times
faster than the slowest language (R). The present work extends \cite{AF2014} by
analyzing a variety of algorithms and also by solving a larger (in terms of the
number of state variables) model. Similar to \cite{AF2014}, we find that the
choice of a programming language may have important effects in terms of speed,
however, we do not observe such huge differences across the three languages
considered. In addition to speed comparisons, we also discuss the key features
of each language that might be useful for economists.

We build on the existing Matlab codes accompanying \cite{AMMT2016} and
\cite{MM2015}. We improve on these codes, and provide new routines that have
not been analyzed previously in the literature. First, for the neoclassical
growth model, we implement seven different methods in all three programming languages, whereas \cite{AMMT2016} is limited to Matlab implementation of the envelope condition method.
Furthermore, for the new Keynesian model, we modify the method proposed in \cite{MM2015} to operate on random and quasi-random
grids covering a fixed hypercube, instead of operating on the high probability area of the state space. The Matlab code developed in the present paper achieves an almost a 60-time speed up over the original code in \cite{MM2015},
while still producing highly accurate solutions. Our code is sufficiently fast
to be used for the estimation of a moderately-large new Keynesian model with 8
state variables. It takes us just a few seconds to construct the solution, including the model with active zero lower bound on the nominal interest rate.

Our overall experience with each language was comparable, though it is useful
to recognize that each language has it specific strengths. Both Julia and
Python are open-source which facilitates transparency and reproducibility. For
reasons to be discussed later, Julia outperforms both Matlab and Python in
algorithms that require numerical solvers or optimizers. Python benefits from
an active community and strong package ecosystem which makes finding the right
tools easy. Finally, Matlab benefits from extensive documentation, technical
support, and various built in toolboxes. Ultimately, the best choice of
programming language depends on various features including: model size,
solution complexity, ease of writing, and co-author preferences. Regardless of
the programming language chosen, our experience shows that transitioning
between each of the three languages we consider should not require a
substantial learning effort.

The rest of the paper is organized as follows. In Section 2, we describe the
three languages, Matlab, Python and Julia. In Section 3, we outline seven
algorithms we implement to solve a variant of the standard neoclassical
stochastic growth model. In Section 4, we present an algorithm for solving a
medium-scale new Keynesian model with a zero lower bound on nominal interest
rates and describe the results of our analysis. Finally, in Section 5 we
conclude.


\documentclass[runningheads,orivec,oribibl,svgnames,table]{llncs}

\usepackage{lncspaper}

\spnewtheorem{assumption}{Assumption}{\bfseries}{\itshape}
\def\algorithmautorefname{Algorithm}
\def\assumptionautorefname{Assumption}

% A draft watermark is nice while the paper is in preparation
% but it interferes with syncTeX
\usepackage{draftwatermark}
\SetWatermarkScale{8}
\SetWatermarkLightness{.97}

% For the Cahier du GERAD.
%\renewcommand{\year}{2020}     % If you don't want the current year.
\newcommand{\cahiernumber}{00}  % Insert your Cahier du GERAD number.

% For debugging.
%\usepackage{showframe}

% For final version.
%\usepackage{butterma}
%\idline{J.~Doe and E.~Muster (Eds.): Perfect Publishing, LNCS 9999}
%\setcounter{page}{101}

% if you have landscape tables
%\usepackage[figuresright]{rotating}

\title{%
  Your Title Should Be Short and to the Point
}
\titlerunning{%
  Short and to the Point  % will appear in header
}
\author{
  Dominique Orban\inst{1,2}%
  \thanks{Research partially supported by NSERC Discovery Grant 299010-04.}
}
\institute{
  Department of Mathematics and Industrial Engineering,
  \'Ecole Polytechnique,
  Montr\'eal, QC, Canada.
  \and
  GERAD, Montr\'eal, QC, Canada.
  \mailto{dominique.orban@gerad.ca}
}
\authorrunning{Orban}

% Meta-information for the PDF file generated.
\pdfinfo{/Author (Dominique Orban)
         /Title (Insert Title Here)
         /Keywords (keyword1, keyword2, keyword3)}

% For final version, set paper format.
%\pdfpagesattr{/CropBox [92 112 523 778]} % LNCS page: 152x235 mm

\begin{document}

\linenumbers

\pagestyle{myheadings}

\maketitle
\thispagestyle{mytitlepage}   % Cahier du GERAD.
%\thispagestyle{electronic}   % Published version.

\begin{abstract}
  The abstract is reserved to an expert audience.
  It should clearly state what types of problems you consider, what your method is, and why it is novel.
  The abstract should also summarize your main theoretical results, say a few words about implementation, and sum up your findings from numerical experiments.
  The abstract should be written last.
  \smarttodo{write abstract}
\end{abstract}

\keywords{Keyword1, keyword2, keyword3}

% Pour le cahier du GERAD.
%\begin{resume}
%
%\end{resume}
%\textbf{Mots cl\'es :}

%========================================================================

\section{Introduction}
\label{sec:introduction}

We consider the problem
\begin{equation}
  \label{eq:unc}
  \minimize{x \in \R^n} \ f(x),
\end{equation}
where \(f : \R^n \to \R\) is continuously differentiable.

\subsection*{Related Research}

Cite the most relevant references, summarize what their approach and contributions are, and indicate how your method differs.

\subsection*{Notation}

We use Householder notation throughout: capital Latin letters such as \(A\), \(B\), and \(H\), represent matrices, lowercase Latin letters such as \(s\), \(x\), and \(y\) represent vectors in \(\R^n\), and lowercase greek letter such as \(\alpha\), \(\beta\) and \(\gamma\) represent scalars.

%========================================================================

%% this structure is only an example

\section{Background and Assumptions}
\label{sec:background}

\begin{assumption}
  \label{ass:f-C1}
  The function \(f\) is continuously differentiable over \(\R^n\).
\end{assumption}

%========================================================================

\section{Complete Algorithm}
\label{sec:algorithm}

We summarize the complete process as \autoref{alg:name-of-your-algorithm}.

\begin{algorithm}[t]
  \caption{Give a short name or description of the algorithm}
  \label{alg:name-of-your-algorithm}
  \begin{algorithmic}[1]
    \Require \(x_0 \in \R^n\)

    \State choose \(\epsilon > 0\), set \(k = 0\)
      \Comment{change as necessary}

    \For{$k = 1, 2, \ldots$}

      \State

      \State

    \EndFor
  \end{algorithmic}
\end{algorithm}

%========================================================================

\section{Convergence Analysis}
\label{sec:convergence}

\begin{lemma}
  Lemmas are used for preliminary results.
\end{lemma}

\begin{proposition}
  A proposition is an important result established by someone else.
\end{proposition}

\begin{theorem}
  Theorems are used to emphasize important results that you established, and distinguish them from the results in Propositions.

  Let \autoref{ass:f-C1} be satisfied.
  Then \(\dots\)
\end{theorem}

\begin{proof}
  Here is the proof.
  \qed
\end{proof}

%========================================================================

\section{Implementation and Numerical Results}
\label{sec:implementation}

\begin{itemize}
  \item describe implementation;
  \item brief explanation of the packages used;
  \item values of algorithmic parameters, choice of accuracy tolerances, stopping conditions, etc.;
  \item description and size of test problems;
  \item etc.
\end{itemize}

%========================================================================

\section{Discussion}
\label{sec:discussion}

A discussion is better than conclusions.
Contrast your finding with those from the literature and justify the statements you made in the abstract and introduction.
Finish with a brief statement of future work.

%========================================================================

\section{Notes}
\label{sec:notes}

You can keep notes in this section and remove the section altogether when you are ready to submit.

Using Natbib with author-year style, you can work citations into your sentences.
For example, \cite{wright-orban-2002} study the existence of the central path under the Mangasarian-Fromovitz constraint qualification.

%========================================================================

%% The Appendices part is started with the command \appendix;
%% appendix sections are then done as normal sections
%% \appendix

%% \section{}
%% \label{}

%% References
%% We use Natbib, which is accepted by SIAM and Springer journals
%% see the cheat sheet: http://merkel.texture.rocks/Latex/natbib.php

% \cite is the same as \citet
% \citet{jon90}	                  -->    	Jones et al. (1990)
% \citet[chap. 2]{jon90}	        -->    	Jones et al. (1990, chap. 2)
% \citep{jon90}	                  -->    	(Jones et al., 1990)
% \citep[chap. 2]{jon90}	        -->    	(Jones et al., 1990, chap. 2)
% \citep[see][]{jon90}	          -->    	(see Jones et al., 1990)
% \citep[see][chap. 2]{jon90}	    -->    	(see Jones et al., 1990, chap. 2)
% \citet*{jon90}	                -->    	Jones, Baker, and Williams (1990)
% \citep*{jon90}	                -->    	(Jones, Baker, and Williams, 1990)

%% Following citation commands can be used in the body text:
%% Usage of \cite is as follows:
%%   \cite{key}         ==>>  [#]
%%   \cite[chap. 2]{key} ==>> [#, chap. 2]
%%

%% References with BibTeX database:
% - each reference should have a DOI
% - use the strings provided for the journal name

\bibliographystyle{abbrvnat}
\bibliography{abbrv,\jobname}

\newpage

\hypertarget{contents}{}  % so clicking on [toc] in the header leads here
\tableofcontents
\listoftodos

\end{document}


%!TEX root = ../thesis.tex
%*******************************************************************************
%****************************** Second Chapter *********************************
%*******************************************************************************

\chapter{SAT Solvers} \label{chap:solvers}

\ifpdf
    \graphicspath{{Chapter2/Figs/Raster/}{Chapter2/Figs/PDF/}{Chapter2/Figs/}}
\else
    \graphicspath{{Chapter2/Figs/Vector/}{Chapter2/Figs/}}
\fi

\section{Backtracking Based Algorithms}
Currently there are predominantly two classes of SAT solvers: those based on
backtracking search and those based on stochastic local search \cite{sat2018descriptions}. This chapter
will go over the two different classes, giving the algorithms that they are
based on followed by an overview of the details put into implementations of
a solver.

\subsection[DPLL]{Davis–Putnam–Logemann–Loveland} \label{sec:dpll}
One of the first SAT solving algorithms came from a desire to show the validity of
statements made in first order predicate logic. Here, we shall only concern ourselves
with the portion that is applicable to SAT formulas. Before introducing
Davis Putnam Logemann Loveland (DPLL),\nomenclature[z-DPLL]{DPLL}{Davis Putnam Logemann Loveland}
we shall consider its predecessor,
the Davis-Putnam procedure (DP)\cite{davis1960computing}.
\nomenclature[z-DP]{DP}{Davis Putnam procedure}
\nomenclature[x-equisat]{F $\approx$ G}{Boolean formulas F and G are equisatisfiable}
This procedure uses a resolution based approach and is based on 3 rules:

\begin{enumerate}
    \item (One Literal Clause Rule) If a formula has a clause containing a single unassigned literal,
    eliminate all clauses that contain this literal, and remove any occurrence of the negated
    literal from the remaining clauses.
    This is also referred to as unit propagation or boolean constraint propagation.
    A clause with only one unassigned literal, is sometimes referred to as a unit
    clause.
    \begin{equation*}
        (a) \land (a \lor b \lor c) \land (\neg a \lor \neg b \lor c) \approx (\neg b \lor c)
    \end{equation*}
    \item (Pure Literal Rule) If a variable in a formula occurs either only
    as a literal in its positive form or only as a literal in its negative form.
    Then eliminate all clauses containing this literal.
    \begin{equation*}
        (a \lor b) \land (a \lor \neg b) \land (b \lor c) \approx (b \lor c)
    \end{equation*}
    \item (Resolution Rule) If there exist two clauses which share a variable where in one
    clause the literal is negated and in the other it is not. The two clauses can be replaced
    by a single clause containing the union of their literals excluding the shared variable.
    \begin{equation*}
        (a \lor x)\land(b \lor \neg x) \approx (a \lor b)
    \end{equation*}
\end{enumerate}

The simplified formulas that result from the application of these rules
are not necessarily equivalent to the original formulas, but they are
equisatisfiable. Therefore if a formula is simplified to the point it contains
no clauses then it is trivially satisfiable and the original formula is also
satisfiable. Observe that it is always possible to reduce a satisfiable formula
to the empty formula, since if the resolution rule cannot be applied then
the pure literal rule can remove all clauses.

To see this procedure in action consider checking whether the following
formula in CNF is satisfiable.

\begin{equation} \label{eq:DP_example}
    (a \lor b) \land (\neg a \lor b) \land (a \lor \neg b)
\end{equation}

As we see in Equation \ref{eq:DP_example}, we can apply the resolution rule
to the first two clauses. This yield the following simplification.

\begin{equation}
    (b) \land (a \lor \neg b)
\end{equation}

From here we see that we can apply the one literal clause rule to eliminate
the first clause. Note that the last clause will be simplified to just $(a)$.
Hence we can apply the one literal clause rule again to get the empty formula.
At this point the procedure finishes. Since we obtained the empty formula
we know that the original formula was satisfiable.

If a formula is unsatisfiable then the input formula will eventually
simplify to a formula containing the empty clause, which cannot be satisfied,
at this point the procedure would halt \cite{davis1960computing}.

Consider Equation \ref{eq:DP_example}, but with the added clause $(\neg a \lor \neg b)$.
This modified formula is now unsatisfiable and if we apply the same rules to
this new formula we will, instead of ending up with $(a)$, we will have
$(a) \land (\neg a)$. At this point, applying the pure literal rule to either
$a$ or $\neg a$ will leave one clause empty.

\subsubsection{Moving to DPLL}
An issue was that the
resolution rule could cause the size of clauses to increase rapidly. This
was an issue that prevented the procedure to implemented due to memory constraints.
The solution was to replace the resolution rule by the splitting rule which
can be stated as the following. Let $x$ be some variable in a formula, the formula
is inconsistent if and only if the formula is inconsistent when $x = \text{true}$ and
when $x = \text{false}$ \cite{davis1962machine}. This is theoretically equivalent
to the resolution rule, however this rule allowed for better memory usage in
practice.

The substitution of the resolution rule for the splitting rule forms the basis
for DPLL with most improvements focusing on finding efficient data
structures for implementing DPLL and on developing good heuristics for which
variable to branch on.

An important observation is that the splitting rule defines a tree with
the original formula as the root and the two children of the root are
the two formulas obtained from the splitting rule. Hence a formula is
satisfiable if and only if one of the leaves in the tree is satisfiable.
DPLL moves through the tree in a pre-order like fashion, this is called
chronological backtracking \cite{biere2009conflict}. However, note that
the order in which the tree is traversed and how the algorithm backtracks
does not affect the correctness of the algorithm as long as we can guarantee that every node in the tree will be visited eventually.
A run of DPLL on Equation \ref{eq:DP_example} can be seen in Figure \ref{fig:run_dpll}.

\begin{figure}
    \centering
    \begin{tikzpicture}
        \node[draw] (A) at (0,0)     {$(a \lor b) \land (\neg a \lor b) \land (a \lor \neg b)$};
        \node[draw] (B) at (-2,-2)   {$(b) \land (\neg b)$};
        \node[draw] (C) at (2,-2)    {$(b)$};
        \node[draw] (D) at (-2,-4) {CONFLICT};
        \node[draw, align=center] (E) at (2,-4)  {SAT \\ $a = \text{true},\quad b = \text{true}$};

        \draw[->]   (A) -- (B) node[midway, left=20pt] {$a = \text{false}$};
        \draw[->]   (A) -- (C) node[midway, right=20pt] {$a = \text{true}$};
        \draw[->]   (B) -- (D) node[midway, left=20pt] {Unit Propagation};
        \draw[->]   (C) -- (E) node[midway, right=20pt] {Unit Propagation};
    \end{tikzpicture}
    \caption{A run of DPLL on Equation \ref{eq:DP_example}}
    \label{fig:run_dpll}
\end{figure}

\subsection[CDCL]{Conflict Driven Clause Learning}
Recent competitive solvers are based on Conflict Driven Clause Learning (CDCL) \nomenclature[z-CDCL]{CDCL}{Conflict Driven CLause Learning}
\cite{SAT_Comp2017, sat2018descriptions}, such as Glucose.
CDCL builds on DPLL in a few ways, the key methods are as follows:

\begin{itemize}
    \item Derive new clauses when search yields a conflict \cite{marques1999grasp}.
    \item Use of efficient lazy data structures \cite{moskewicz2001chaff, ryan2004efficient}.
    \item Non-chronological backtracking \cite{marques1999grasp}.
    \item Restarting search \cite{gomes1998boosting, gomes2000heavy}
\end{itemize}

The structure of a CDCL solver without restarts can be seen in Algorithm \ref{alg:cdcl}.

\begin{algorithm}
\caption{Organisation of CDCL \cite{biere2009conflict, CDCL}}
\label{alg:cdcl}
\vspace{5pt}
\KwIn{CNF-SAT Formula $F$}
\KwOut{SAT or UNSAT}
\hrulefill\\

\nl $\alpha \gets \emptyset$ \\
\nl \If{unit\_propagation($F$, $\alpha$) == CONFLICT}
{
    \nl \Return{UNSAT} \\
}
\nl decision\_level $\gets 0$ \\
\nl \While{$\neg$ all\_variables\_assigned($F$, $\alpha$)}
{
    \nl $(x,v) \gets \text{pick\_branching\_variable}(F, \alpha)$ \\
    \nl decision\_level $\gets$ decision\_level $+ 1$ \\
    \nl $\alpha \gets \alpha \cup \{(x,v)\}$ \\
    \nl \If{unit\_propagation($F, \alpha$) == CONFLICT}
    {
        \nl $\beta \gets \text{conflict\_analysis}(F, \alpha)$ \\
        \nl \If{$\beta < 0$}
        {
            \nl \Return{UNSAT} \\
        }
        \nl \Else{
            \nl backtrack($F, \alpha, \beta$) \\
            \nl decision\_level $\gets \beta$ \\
        }
    }
}
\nl \Return{SAT}
\end{algorithm}

In Algorithm \ref{alg:cdcl} $\alpha$ refers to the set of assignments, where
each assignment is represented as a tuple $(x, v)$ where $x$ is a variable
and $v \in \{\text{false}, \text{true}\}$ is the value. The unit propagation
function call executes the one literal clause rule from Section \ref{sec:dpll}.
Pick branching variable chooses which variable to branch on and gives it a value.
Choice of heuristics for which variable to select is motivated mostly by which
features can be computed efficiently \cite{lewis2005speedup, moskewicz2001chaff}.

Backtrack takes in a parameter $\beta$, which indicates the decision level to
backtrack to. This means that search happens in a non-chronological fashion.
The level to backtrack to is decided by the conflict analysis function.
% This function looks at which clauses clauses the contradiction to occur
% and finds a clause implied by those clauses and adds it to the formula.
% The amount to backtrack by depends on what decision level the variables
% in the learned clause were assigned. One scheme involves always backtracking
% to the decision level of the most recently assigned variable \cite{biere2009conflict}.

\subsubsection{Conflict Analysis}
The most characteristic part of CDCL is the clause learning that results from the clause
analysis process. There are slight variants of this procedure, but we will cover the most
common variations popularised in GRASP \cite{marques1999grasp} and Chaff \cite{moskewicz2001chaff}.

For a particular partial assignment $\alpha$, we use decision level to mean the order that
a given variable was assigned by the \texttt{pick\_branching\_variable} routine. For
variables that were assigned via the \texttt{unit\_propagation} routine, they inherit the
highest decision level of the other variables in the clause. We consider a variable that
is assigned via \texttt{unit\_propagation} to be implied by the other variables in the unit clause.
Since \texttt{unit\_propagation} may lead to more clauses becoming unit, we can construct a directed
acyclic graph of implications. We call this graph the implication graph.

Consider the following boolean formula:
\begin{align}
    &(\neg x_1 \lor x_2 \lor x_4) \land (x_2 \lor \neg x_3 \lor \neg x_5) \land (\neg x_4 \lor x_5 \lor x_6) \nonumber \\
    \land &(\neg x_6 \lor x_7) \land (\neg x_3 \lor \neg x_6 \lor \neg x_8) \land (\neg x_7 \lor x_8) \label{eq:impl_ex}
\end{align}
For example, say that \texttt{pick\_branching\_variable} first assigns $x_1 = \text{true}$ at decision level 1, then
$x_3 = \text{true}$ at decision level 2 and then $x_2 = \text{false}$ at decision level 3. At this point, the first
two clauses in Equation \ref{eq:impl_ex} are unit clauses and applying \texttt{unit\_propagation} will lead to a conflict.
The resulting implication graph can be seen in Figure \ref{fig:implication_graph}. Each vertex represents an assignment
or a conflict and there is a directed edge from one vertex to another if that assignment implies the other via unit
propagation. For example, in Figure \ref{fig:implication_graph} there is an edge from $x_1 = \text{true}$ and $x_2 = \text{false}$
to $x_4 = \text{true}$ because the first two assignments make the clause $(\neg x_1 \lor x_2 \lor x_4)$ a unit clause which
implies $x_4 = \text{true}$. There are also edges from $x_7 = \text{true}$ and $x_8 = \text{false}$ to a conflict vertex since
these two assignments leave the clause $(\neg x_7 \lor x_8)$ unsatisfied. Each edge is labelled with the clause
that caused the implication.

\begin{figure}
    \centering
    \begin{tikzpicture}
        \node[draw] (x1) at (-5, 1) {$x_1 = \text{true}$};
        \node[draw] (x2) at (0, 0) {$x_2 = \text{false}$};
        \node[draw, fill=black!10] (x3) at (5, 1) {$x_3 = \text{true}$};
        \node[draw] (x4) at (-3, -4) {$x_4 = \text{true}$};
        \node[draw] (x5) at (3, -4) {$x_5 = \text{false}$};
        \node[draw, fill=black!10] (x6) at (0, -8) {$x_6 = \text{true}$};
        \node[draw] (x7) at (-3, -12) {$x_7 = \text{true}$};
        \node[draw] (x8) at (3, -12) {$x_8 = \text{false}$};
        \node[draw] (k) at (0, -16) {CONFLICT};

        \draw[gray] plot [smooth] coordinates {(-8, 0) (-4, 0.5) (-3, 2)};
        \draw[gray] plot [smooth] coordinates {(7, 0) (4, 0.5) (2.9, 2)};
        \node[] (dl1) at (-6, 2) {Decision Level 1};
        \node[] (dl2) at (6, 2) {Decision Level 2};
        \node[] (dl3) at (0, 1) {Decision Level 3};

        \path[->] (x1) edge node[below, sloped] {$(\neg x_1 \lor x_2 \lor x_4)$} (x4);
        \path[->] (x2) edge node[below, sloped] {$(\neg x_1 \lor x_2 \lor x_4)$} (x4);
        \path[->] (x2) edge node[below, sloped] {$(x_2 \lor \neg x_3 \lor \neg x_5)$} (x5);
        \path[->] (x3) edge node[above, sloped] {$(x_2 \lor \neg x_3 \lor \neg x_5)$} (x5);
        \path[->] (x3) edge node[below, sloped] {$(\neg x_3 \lor \neg x_6 \lor \neg x_8)$} (x8);
        \path[->] (x4) edge node[below, sloped] {$(\neg x_4 \lor x_5 \lor x_6)$} (x6);
        \path[->] (x5) edge node[below, sloped] {$(\neg x_4 \lor x_5 \lor x_6)$} (x6);
        \path[->] (x6) edge node[below, sloped] {$(\neg x_6 \lor x_7)$} (x7);
        \path[->] (x6) edge node[below, sloped] {$(\neg x_3 \lor \neg x_6 \lor \neg x_8)$} (x8);
        \path[->] (x7) edge node[below, sloped] {$(\neg x_7 \lor x_8)$} (k);
        \path[->] (x8) edge node[below, sloped] {$(\neg x_7 \lor x_8)$} (k);
    \end{tikzpicture}
    \caption{Implication graph for Equation \ref{eq:impl_ex} for a partial assignment to $x_1, x_2, x_3$.
    First UIP is shaded grey}
    \label{fig:implication_graph}
\end{figure}

The \texttt{conflict\_analysis} routine will construct this implication graph and then apply
a number of resolution steps to obtain the learned clause. Define a resolution operator $\odot$ for
two clauses $C_1$ and $C_2$ with a unique variable $x_r$ such that $\neg x_r \in C_1$ and $x_r \in C_2$ or
vice versa, then $C_1 \odot C_2 = (C_1 \cup C_2) \setminus \{x_r, \neg x_r\}$ i.e. a clause containing
all literals in $C_1$ and $C_2$ excluding $x_r$ and $\neg x_r$. \texttt{conflict\_analysis} then begins by
taking the clause that caused the conflict (in this example $(\neg x_7 \lor x_8)$) and applying the resolution
operator with this clause and a clause that implied one of the literals in the first clause. In our example
the first resolution step could be either $(\neg x_7 \lor x_8) \odot (\neg x_6 \lor x_7) = (\neg x_6 \lor x_8)$ or
$(\neg x_7 \lor x_8) \odot (\neg x_3 \lor \neg x_6 \neg x_8) = (\neg x_3 \lor \neg x_6 \lor \neg x_7)$.
A full set of resolution steps using the example from Equation \ref{eq:impl_ex} can be seen in Table \ref{tab:resolutions}.
Since $\forall C_1, C_2 \in F : C_1 \odot C_2 \approx C_1 \land C_2$ then $F \approx F \cup \{C_1 \odot C_2\}$. This means
that we can take any clause from Table \ref{tab:resolutions} and add it to Equation \ref{eq:impl_ex} without changing
its satisfiability.

\begin{table}[]
    \centering
    \begin{tabular}{l l}
    \toprule
        Step & Resolution \\
    \midrule
        1 & $(\neg x_7 \lor x_8) \odot (\neg x_6 \lor x_7) = (\neg x_6 \lor x_8)$ \\
        2 & $(\neg x_6 \lor x_8) \odot (\neg x_3 \lor \neg x_6 \lor \neg x_8) = (\neg x_3 \lor \neg x_6)$ \\
        3 & $(\neg x_3 \lor \neg x_6) \odot (\neg x_4 \lor x_5 \lor x_6) = (\neg x_3 \lor \neg x_4 \lor x_5)$ \\
        4 & $(\neg x_3 \lor \neg x_4 \lor x_5) \odot (x_2 \lor \neg x_3 \lor \neg x_5) = (x_2 \lor \neg x_3 \lor \neg x_4)$ \\
        5 & $(x_2 \lor \neg x_3 \lor \neg x_4) \odot (\neg x_1 \lor x_2 \lor x_4) = (\neg x_1 \lor x_2 \lor \neg x_3)$ \\
    \bottomrule
    \end{tabular}
    \caption{Resolution steps for Equation \ref{eq:impl_ex} with partial assignments to $x_1, x_2, x_3$}
    \label{tab:resolutions}
\end{table}

\nomenclature[z-UIP]{UIP}{Unit Implication Point}

There are different options for how many resolution steps to perform, however, a guiding principle is that the learned clause
should be as short as possible and should remain a unit clause after backtracking.
The most common approach is to stop at what is called the first Unit Implication Point (UIP) \cite{biere2009conflict}, which is the first point
at which there is only one literal in a learned clause that has the highest decision level. In Table \ref{tab:resolutions}
this occurs at step 2, since in the clause $(\neg x_3 \lor \neg x_6)$ only $x_6$ has decision level 3 while $x_3$
has decision level 2.
Note that if we look at the literals in a clause obtained by any number of resolution steps, the corresponding vertices
in the implication graph form a vertex separator, separating the side of the graph containing the conflict vertex
and the side containing the variables in the assignments made by \texttt{pick\_branching\_variable}.
Therefore, we can interpret a UIP as a clause where the variable with the highest decision level
is a dominator for the variable most recently assigned by \texttt{pick\_branching\_variable}
with respect to the conflict vertex. This can be seen in Figure \ref{fig:implication_graph} as all paths from $x_2 = \text{false}$
to the conflict vertex must pass through $x_6 = \text{true}$.

The amount to backtrack by (given by $\beta$ in Algorithm \ref{alg:cdcl}) also varies from solver to solver
and most often depends on how inexpensive backtracking is given the datastructures used by the solver.
However, most modern solvers now use lazy datastructures which make it cheap to backtrack. For these
solvers a common approach is to backtrack until just before the learned clause no longer is a unit clause.
In the example given in Equation \ref{eq:impl_ex}, the learned clause was $(\neg x_3 \lor \neg x_6)$. As long
as $x_3$ remains assigned then the learned clause will be a unit clause, and $x_3$ was assigned at decision level 2,
so we can backtrack to decision level 2. In general we can backtrack to the second highest decision level in the
learned clause. This is the backtracking method used by Chaff \cite{moskewicz2001chaff}.
We can guarantee completeness because, since the learned clauses prevent the solver from exploring
the same assignment twice.

\subsubsection{Performance}

\begin{figure}
    \centering
    \includegraphics[scale=1]{sat_performance2012.png}
    \caption{Number of sat problems taken from the
    2009 SAT competition that were solved by various solvers
    with a 20 minute timeout \cite{sat2009theory}}
    \label{fig:sat_perf}
\end{figure}

In figure \ref{fig:sat_perf} we can see the number of solved instances of practical
sat benchmarks submitted to the 2009 SAT competition by various different solvers
\cite{sat2009theory}. Notably
more recent solvers, whilst still demonstrating exponential scaling behaviour are
able to solve significantly more problems than prior solvers.

For the 2018 SAT competition the benchmarks were taken from various different domains:
\cite{sat2018descriptions}

\begin{itemize}
    \item Proving the existence of a nonce for blockchain mining algorithms.
    \item Searching for a unit-distance graph with chromatic number 6 and k-colourability of graphs.
    \item Verifying simple floating-point programs.
    \item Cryptanalysis.
    \item Time-Slot allocation problems with optimal preference satisfaction
\end{itemize}

\subsection{Glucose}
Glucose is an efficient implementation of CDCL that has featured prominently
in many SAT competitions \cite{audemard2009glucose} building on top of MiniSAT \cite{een2003extensible}.
Glucose implements a few additional
features on top of CDCL, most prominently it implements new clause deletion policies
which help control the memory usage of the learned clauses generated by
the conflict analysis. These deletion policies expand on those used in the BerkMin solver \cite{goldberg2007berkmin},
going beyond analysing clauses just by size and how close they are to being a unit clause \cite{dechter1990enhancement, bayardo1997using}.

These policies were established in work by Audemard et al. where it is observed that
the decision level at which conflicts are encountered
tends to decrease as the solver progresses \cite{audemard2009predicting},
although this effect was only observed for ``industrial'' SAT instances.
This means that the solver is able to find conflicts faster over time, i.e.
it is assigning many variables via unit propagation. Additionally, this allows
for a crude estimate of when the solver will terminate.
Audemard et al. then devise a metric called the ``Literal
Block Distance'' (LBD) \nomenclature[z-LBD]{LBD}{Literal Block Distance}
which is the number of distinct decision levels in a clause
for a given assignment and show that when keeping only clauses with low LBD
the observation that decision levels decrease still holds. This suggests
that these clauses are the most important for allowing the solver to
spend significant time executing unit propagation.
Applying this deletion policy, around 93\% of learned clauses were deleted \cite{audemard2009glucose}.
However, since the learned clauses are needed for guaranteeing completeness, Glucose
is not complete.

Of practical note is that the Glucose solver is particularly
amenable to paralellisation and remains one of the most
competitive parallel solvers \cite{audemard2018glucose}.

\section{Stochastic Local Search Algorithms}
A different family of algorithms to backtracking algorithms is local search algorithms
which are based on the idea of randomly choosing an assignment and then making
small adjustments to assignment in order to try and find a satisfying assignment.
We will cover an important algorithm,
namely Sch\"oning's algorithm \cite{schoning1999probabilistic}, and then consider
WalkSat, a solver based on similar principles.

\subsection{Sch\"oning's Algorithm}

An interesting $k$-SAT algorithm that is able to beat the $\mathcal{O}^*(2^n)$
complexity for bounded $k$ is Sch\"oning's algorithm seen in Algorithm \ref{alg:schoning}.
Given that a satisfying assignment exists it will be found in expectation after
$(2 - \frac{2}{k})^n$ runs of Sch\"oning's algorithm \cite{schoning1999probabilistic}.
Hence the expected running time is $\mathcal{O}^*((2 - \frac{2}{k})^n)$.

\begin{algorithm}
\caption{Sch\"oning's Algorithm for $k$-SAT}\label{alg:schoning}
\vspace{5pt}
\KwIn{$k$-SAT instance, $F$}
\KwOut{Instance is satisfiable or not}
\hrulefill \\
\nl Guess initial assignment \\
\nl \For{$i \in \{1, \dots, 3n\}$}{
    \nl \If{$F$ is Satisfied}{
        \nl \Return{Accept}
    }
    \nl \Else{
        \nl flip a random variable in an unsatisfied clause \\
    }
}
\end{algorithm}

\subsubsection{Analysis of Sch\"oning's Algorithm}
For the sake of brevity, we will only consider the case where $k=3$.
Since we require that there is at least one satisfying assignment let $\alpha^*$
be one such satisfying assignment. Let $\alpha$ be the assignment guessed in
the first line and let $d$ be the Hamming distance between $\alpha$ and $\alpha^*$,
i.e. $H(\alpha, \alpha^*) = d$ the number of variables in which they differ. The probability of guessing
an assignment $\alpha$ which differs by exactly $d$ variables from $\alpha^*$ is given by the following expression,
where $n$ is the number of variables.

\begin{equation}
    2^{-n}\binom{n}{d}
\end{equation}

If the algorithm flips a variable in which $\alpha$ and $\alpha^*$ differ, then
we call this a step towards the solution, i.e. $H(\alpha, \alpha^*)$ decreases by 1. Conversely, if they do not differ then
we call this a step away from the solution, i.e. $H(\alpha, \alpha^*)$ increases by 1.
%Let $T$ be the total number of steps,
%if a run of the algorithm where to find a satisfying assignment then we
%must have that the number of steps towards the solution must be
%$\frac{T + d}{2}$ and steps away must be $\frac{T - d}{2}$

Consider the probability of making $2d$ steps towards the solution
and $d$ steps away in the first $3d$ steps. It is clear that this will reach
$\alpha^*$ and since $d \leq n$, then the total number of steps
will not be greater than $3n$. Since, in an unsatisfied clause,
at least one variable is different between $\alpha$ and $\alpha^*$, then when
the algorithm flips a random variable in that clause the probability of it
being one where the assignments are different is $\frac{1}{3}$. Hence,
the probability of making $2d$ toward the solution and $d$ steps away in the
first $3d$ steps is given by the following expression.

\begin{equation}
    \binom{3d}{d} \Big( \frac{1}{3} \Big)^{2d} \Big( \frac{2}{3} \Big)^d
\end{equation}

Hence to find the probability that the algorithm finds a satisfying solution
we simply have combine this with the probability of guessing a first assignment
that has Hamming distance of exactly $d$, and sum over all possible values of $d$.

\begin{equation} \label{eq:p_find_assignment}
    \mathrm{Pr}[\text{Find satisfying assignment}] = \sum_{d=0}^{n} 2^{-n}\binom{n}{d}\binom{3d}{d} \Big( \frac{1}{3} \Big)^{2d} \Big( \frac{2}{3} \Big)^d
\end{equation}

To simplify Equation \ref{eq:p_find_assignment} it will be helpful to consider
a lower bound of $\binom{3d}{d}$. To do this we will use Stirling's approximation
of the factorial:

\begin{equation} \label{eq:stirling_fac}
    n! \sim \sqrt{2\pi n} \Big( \frac{n}{e} \Big)^n
\end{equation}

Using Equation \ref{eq:stirling_fac} we can derive an approximation as follows.

\begin{align*}
    \binom{3d}{d} &= \frac{(3d)!}{(2d)! \cdot d!} \\[5pt]
    &\sim \frac{\sqrt{6 \pi d}\Big(\frac{3d}{e}\Big)^{3d}}{\sqrt{4\pi d} \Big( \frac{2d}{e} \Big)^{2d}\sqrt{2\pi d} \Big( \frac{d}{e} \Big)^d} \\[5pt]
    &\sim \frac{\sqrt{3}}{2\sqrt{\pi d}} \cdot \frac{3^{3d}}{2^{2d}}
\end{align*}

Since $n \geq d$ we replace $\sqrt{d}$ with $\sqrt{n}$ to simplify the analysis.
Then since the binomial coefficient is always strictly positive then what
remains is to choose a sufficiently large constant $\delta$
such that we have for $d \geq 0$:

\begin{equation} \label{eq:binom_bound}
    \binom{3d}{d} \geq \frac{1}{\delta \sqrt{n}} \cdot \frac{3^{3d}}{2^{2d}}
\end{equation}

Combining Equations \ref{eq:p_find_assignment} and \ref{eq:binom_bound} we get
a lower bound on the probability of finding a satisfying assignment.

\begin{align*}
    \mathrm{Pr}[\text{Find satisfying assignment}] &\geq \sum_{d=0}^{n} 2^{-n}\binom{n}{d} \frac{1}{\delta \sqrt{n}} \cdot \frac{3^{3d}}{2^{2d}} \Big( \frac{1}{3} \Big)^{2d} \Big( \frac{2}{3} \Big)^d \\[5pt]
    %&\geq \frac{2^{-n}}{\delta \sqrt{n}}\sum_{d=0}^{n} \binom{n}{d}\frac{3^{3d}}{2^{2d}} \frac{1}{3^{2d}} \frac{2^d}{3^d} \\[5pt]
    &\geq \frac{2^{-n}}{\delta \sqrt{n}}\sum_{d=0}^{n} \binom{n}{d}\frac{1}{2^{d}} \\[5pt]
    &\geq \frac{2^{-n}}{\delta \sqrt{n}}\Big( 1 + \frac{1}{2} \Big)^n \\[5pt]
    &\geq \frac{1}{\delta \sqrt{n}}\Big( \frac{3}{4} \Big)^n \\[5pt]
\end{align*}

Repeatedly running Algorithm \ref{alg:schoning} defines a geometric sequence.
Therefore, the expectation of the number of runs required to find the satisfying
assignment is given by

\begin{equation}
    \mathbb{E}[\text{\#runs}] = \frac{1}{\mathrm{Pr}[\text{Find satisfying assignment}]} \leq \delta\sqrt{n}\Big( \frac{4}{3} \Big)^n
\end{equation}

Hence the time complexity can be written as $\mathcal{O}^*(1.334^n)$.

\subsection{WalkSat}
WalkSat is an solver similar to Sch\"oning's algorithm that is inspired
by GSAT \cite{selman1992new, selman1994noise, selman1993local} and by ideas laid forth by Papadimitriou \cite{papadimitriou1991selecting}. The main difference between
WalkSat and Sch\"oning's algorithm is that it employs a semi-greedy
strategy when deciding which variable to flip in an unsatisfied clause.

With some probability $p$ WalkSat will flip a random variable, but
with probability $1 - p$ WalkSat will flip the variable that minimises
the number of clauses that become unsatisfied. The parameter $p$
is often called the noise parameter and optimal choice of $p$ depends
on the structure of the formula being tested. Therefore, modern improvements
have attempted to adapt $p$ automatically to each instance \cite{hoos2002adaptive}.

\begin{table}
    \centering
    \begin{tabular}{l l c c c}
    \toprule
        \multicolumn{2}{l}{Formula} & DP Time & GSAT Time & WalkSat Time \\
        \#Vars & \#Clauses & & & \\
        \midrule
        708 & 1702 & * & 0.081 & 0.013 \\
        649 & 1562 & * & 0.058 & 0.014 \\
        8704 & 32316 & * & 94.1 & 1.0 \\
        8432 & 31310 & * & 456.6 & 0.7 \\
        300 & 730 & 23096 & 0.009 & 0.002 \\
        125 & 310 & 1.4 & 0.009 & 0.001 \\
        \bottomrule
    \end{tabular}
    \caption[Comparing DP, GSAT and WalkSat]
    {Comparing solution times of an implementation of
    the Davis Putnam procedure (DP) with GSAT and WalkSat
    (time in seconds) \cite{selman1993local}}
    \label{tab:walksat_times}
\end{table}

WalkSat is an efficient solver that is able to tackle SAT instances with a large
number of variables and clauses, and is able to find solutions to instances
where an efficient implementation of the Davis Putnam procedure was unable
to a find a solution. An overview of solution times for a few SAT benchmark
problems from 1993 is seen in Table \ref{tab:walksat_times}.


%%
%% Automatically generated file from DocOnce source
%% (https://github.com/hplgit/doconce/)
%%
%%
% #ifdef PTEX2TEX_EXPLANATION
%%
%% The file follows the ptex2tex extended LaTeX format, see
%% ptex2tex: http://code.google.com/p/ptex2tex/
%%
%% Run
%%      ptex2tex myfile
%% or
%%      doconce ptex2tex myfile
%%
%% to turn myfile.p.tex into an ordinary LaTeX file myfile.tex.
%% (The ptex2tex program: http://code.google.com/p/ptex2tex)
%% Many preprocess options can be added to ptex2tex or doconce ptex2tex
%%
%%      ptex2tex -DMINTED myfile
%%      doconce ptex2tex myfile envir=minted
%%
%% ptex2tex will typeset code environments according to a global or local
%% .ptex2tex.cfg configure file. doconce ptex2tex will typeset code
%% according to options on the command line (just type doconce ptex2tex to
%% see examples). If doconce ptex2tex has envir=minted, it enables the
%% minted style without needing -DMINTED.
% #endif

% #define PREAMBLE

% #ifdef PREAMBLE
%-------------------- begin preamble ----------------------

\documentclass[%
twoside,                 % oneside: electronic viewing, twoside: printing
final,                   % or draft (marks overfull hboxes, figures with paths)
10pt]{article}

\listfiles               % print all files needed to compile this document

\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts}
\usepackage[table]{xcolor}
\usepackage{bm,microtype}

\usepackage{ptex2tex}
% #ifdef MINTED
\usepackage{minted}
\usemintedstyle{default}
% #endif

\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

\usepackage{lmodern}         % Latin Modern fonts derived from Computer Modern

% Hyperlinks in PDF:
\definecolor{linkcolor}{rgb}{0,0,0.4}
\usepackage{hyperref}
\hypersetup{
    breaklinks=true,
    colorlinks=true,
    linkcolor=linkcolor,
    urlcolor=linkcolor,
    citecolor=black,
    filecolor=black,
    %filecolor=blue,
    pdfmenubar=true,
    pdftoolbar=true,
    bookmarksdepth=3   % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
    }
%\hyperbaseurl{}   % hyperlinks are relative to this root

\setcounter{tocdepth}{2}  % number chapter, section, subsection

\usepackage[framemethod=TikZ]{mdframed}

% --- begin definitions of admonition environments ---

% --- end of definitions of admonition environments ---

% prevent orhpans and widows
\clubpenalty = 10000
\widowpenalty = 10000

\newenvironment{doconceexercise}{}{}
\newcounter{doconceexercisecounter}

% ------ header in subexercises ------
%\newcommand{\subex}[1]{\paragraph{#1}}
%\newcommand{\subex}[1]{\par\vspace{1.7mm}\noindent{\bf #1}\ \ }
\makeatletter
% 1.5ex is the spacing above the header, 0.5em the spacing after subex title
\newcommand\subex{\@startsection{paragraph}{4}{\z@}%
                  {1.5ex\@plus1ex \@minus.2ex}%
                  {-0.5em}%
                  {\normalfont\normalsize\bfseries}}
\makeatother

% --- end of standard preamble for documents ---

% insert custom LaTeX commands...

\raggedbottom
\makeindex

%-------------------- end preamble ----------------------

\begin{document}

% endif for #ifdef PREAMBLE
% #endif

% ------------------- main content ----------------------

% ----------------- title -------------------------

\thispagestyle{empty}

\begin{center}
{\LARGE\bf
\begin{spacing}{1.25}
Conjugate gradient methods and other optimization methods
\end{spacing}
}
\end{center}

% ----------------- author(s) -------------------------

\begin{center}
{\bf Morten Hjorth-Jensen, National Superconducting Cyclotron Laboratory and Department of Physics and Astronomy, Michigan State University, East Lansing, MI 48824, USA {\&} Department of Physics, University of Oslo, Oslo, Norway${}^{}$} \\ [0mm]
\end{center}

    \begin{center}
% List of all institutions:
\end{center}

% ----------------- end author(s) -------------------------

\begin{center} % date
Spring 2015
\end{center}

\vspace{1cm}

% !split
\subsection{Motivation}

% --- begin paragraph admon ---
\paragraph{}
Our aim with this part of the project is to be able to
\begin{itemize}
\item find an optimal value for the variational parameters using only some few Monte Carlo cycles

\item use these optimal values for the variational parameters to perform a large-scale Monte Carlo calculation
\end{itemize}

\noindent
To achieve this will look at methods like \emph{Steepest descent} and the \emph{conjugate gradient method}. Both these methods allow us to find
the minima of a multivariable  function like our energy (function of several variational parameters).
Alternatively, you can always use Newton's method. In particular, since we will only have one variational parameter,
Newton's method can be easily used in finding the minimum of the local energy.
% --- end paragraph admon ---

% !split
\subsection{Simple example and demonstration}

% --- begin paragraph admon ---
\paragraph{}
Let us illustrate what is needed in our calculations using a simple example, the harmonic oscillator in one dimension.
For the harmonic oscillator in one-dimension we have a  trial wave function and probability
\begin{equation*}
\psi_T(x) = e^{-\alpha^2 x^2} \qquad P_T(x)dx = \frac{e^{-2\alpha^2 x^2}dx}{\int dx e^{-2\alpha^2 x^2}}
\end{equation*}
with $\alpha$ being the variational parameter.
We obtain then the following local energy
\begin{equation*}
E_L[\alpha] = \alpha^2+x^2\left(\frac{1}{2}-2\alpha^2\right),
\end{equation*}
which results in the expectation value for the local energy
\begin{equation*}
\langle  E_L[\alpha]\rangle = \frac{1}{2}\alpha^2+\frac{1}{8\alpha^2}
\end{equation*}
% --- end paragraph admon ---

% !split
\subsection{Simple example and demonstration}

% --- begin paragraph admon ---
\paragraph{}
The derivative of the energy with respect to $\alpha$ gives
\begin{equation*}
\frac{d\langle  E_L[\alpha]\rangle}{d\alpha} = \alpha-\frac{1}{4\alpha^3}
\end{equation*}
and a second derivative which is always positive (meaning that we find a minimum)
\begin{equation*}
\frac{d^2\langle  E_L[\alpha]\rangle}{d\alpha^2} = 1+\frac{3}{4\alpha^4}
\end{equation*}
The condition
\begin{equation*}
\frac{d\langle  E_L[\alpha]\rangle}{d\alpha} = 0,
\end{equation*}
gives the optimal $\alpha=1/\sqrt{2}$, as expected.
% --- end paragraph admon ---

% !split

% --- begin exercise ---
\begin{doconceexercise}
\refstepcounter{doconceexercisecounter}

\subsection*{Exercise \thedoconceexercisecounter: Find the local energy for the harmonic oscillator}

\subex{a)}
Derive the local energy for the harmonic oscillator in one dimension and find its expectation value.

\subex{b)}
Show also that the optimal value of optimal $\alpha=1/\sqrt{2}$.

\end{doconceexercise}
% --- end exercise ---

% !split
\subsection{Variance in the simple model}

% --- begin paragraph admon ---
\paragraph{}
We can also minimize the variance. In our simple model the variance is
\begin{equation*}
\sigma^2[\alpha] = \frac{1}{2}\alpha^4-\frac{1}{4}+\frac{1}{32\alpha^4},
\end{equation*}
with first derivative
\begin{equation*}
\frac{d \sigma^2[\alpha]}{d\alpha} = 2\alpha^3-\frac{1}{8\alpha^5}
\end{equation*}
and a second derivative which is always positive
\begin{equation*}
\frac{d^2\sigma^2[\alpha]}{d\alpha^2} = 6\alpha^2+\frac{5}{8\alpha^6}
\end{equation*}
% --- end paragraph admon ---

% !split
\subsection{Computing the derivatives}

% --- begin paragraph admon ---
\paragraph{}
In general we end up computing the expectation value of the energy in terms
of some parameters $\alpha_0,\alpha_1,\dots,\alpha_n$
and we search for a minimum in this multi-variable parameter space.
This leads to an energy minimization problem \emph{where we need the derivative of the energy as a function of the variational parameters}.

In the above example this was easy and we were able to find the expression for the derivative by simple derivations.
However, in our actual calculations the energy is represented by a multi-dimensional integral with several variational parameters.
How can we can then obtain the derivatives of the energy with respect to the variational parameters without having
to resort to expensive numerical derivations?
% --- end paragraph admon ---

% !split
\subsection{Expressions for finding the derivatives of the local energy}

% --- begin paragraph admon ---
\paragraph{}

To find the derivatives of the local energy expectation value as function of the variational parameters, we can use the chain rule and the hermiticity of the Hamiltonian.

Let us define
\[
\bar{E}_{\alpha}=\frac{d\langle  E_L[\alpha]\rangle}{d\alpha}.
\]
as the derivative of the energy with respect to the variational parameter $\alpha$ (we limit ourselves to one parameter only).
In the above example this was easy and we obtain a simple expression for the derivative.
We define also the derivative of the trial function (skipping the subindex $T$) as
\[
\bar{\psi}_{\alpha}=\frac{d\psi[\alpha]\rangle}{d\alpha}.
\]
% --- end paragraph admon ---

% !split
\subsection{Derivatives of the local energy}

% --- begin paragraph admon ---
\paragraph{}
The elements of the gradient of the local energy are then (using the chain rule and the hermiticity of the Hamiltonian)
\[
\bar{E}_{\alpha} = 2\left( \langle \frac{\bar{\psi}_{\alpha}}{\psi[\alpha]}E_L[\alpha]\rangle -\langle \frac{\bar{\psi}_{\alpha}}{\psi[\alpha]}\rangle\langle E_L[\alpha] \rangle\right).
\]
From a computational point of view it means that you need to compute the expectation values of
\[
\langle \frac{\bar{\psi}_{\alpha}}{\psi[\alpha]}E_L[\alpha]\rangle,
\]
and
\[
\langle \frac{\bar{\psi}_{\alpha}}{\psi[\alpha]}\rangle\langle E_L[\alpha]\rangle
\]
% --- end paragraph admon ---

% !split

% --- begin exercise ---
\begin{doconceexercise}
\refstepcounter{doconceexercisecounter}

\subsection*{Exercise \thedoconceexercisecounter: General expression for the derivative of the energy}

\subex{a)}
Show that
\[
\bar{E}_{\alpha} = 2\left( \langle \frac{\bar{\psi}_{\alpha}}{\psi[\alpha]}E_L[\alpha]\rangle -\langle \frac{\bar{\psi}_{\alpha}}{\psi[\alpha]}\rangle\langle E_L[\alpha] \rangle\right).
\]

\subex{b)}
Find the corresponding expression for the variance.

\end{doconceexercise}
% --- end exercise ---

% !split

% --- begin exercise ---
\begin{doconceexercise}
\refstepcounter{doconceexercisecounter}

\subsection*{Exercise \thedoconceexercisecounter: Helium atom and hydrogen-like functions}

\subex{a)}
The expectation value for the local energy of the Helium atom with a simple Slater determinant is given by (project 1)
\[
\langle E_{L} \rangle = \alpha^2-2\alpha\left(Z-\frac{5}{16}\right)
\]
Show that its derivative is
\begin{equation*}
\frac{d\langle E_{L}[\alpha] \rangle}{d\alpha} = 2\alpha-2\left(Z-\frac{5}{16}\right).
\end{equation*}

\subex{b)}
Use this result when you test your numerical derivative, using either the steepest descent method, Newton's method or the conjugate
gradient method.

\end{doconceexercise}
% --- end exercise ---

% !split
\subsection{Conjugate gradient (CG) method}

% --- begin paragraph admon ---
\paragraph{}
The success of the CG method  for finding solutions of non-linear problems is based
on the theory of conjugate gradients for linear systems of equations. It belongs
to the class of iterative methods for solving problems from linear algebra of the type
\begin{equation*}
  \hat{A}\hat{x} = \hat{b}.
\end{equation*}
In the iterative process we end up with a problem like

\begin{equation*}
  \hat{r}= \hat{b}-\hat{A}\hat{x},
\end{equation*}
where $\hat{r}$ is the so-called residual or error in the iterative process.

When we have found the exact solution, $\hat{r}=0$.
% --- end paragraph admon ---

% !split
\subsection{Conjugate gradient method}

% --- begin paragraph admon ---
\paragraph{}

The residual is zero when we reach the minimum of the quadratic equation
\begin{equation*}
  P(\hat{x})=\frac{1}{2}\hat{x}^T\hat{A}\hat{x} - \hat{x}^T\hat{b},
\end{equation*}
with the constraint that the matrix $\hat{A}$ is positive definite and symmetric.
If we search for a minimum of the quantum mechanical  variance, then the matrix
$\hat{A}$, which is called the Hessian, is given by the second-derivative of the function we want to minimize.  This quantity is always positive definite.  In our case this corresponds normally to the second derivative of the energy.
% --- end paragraph admon ---

% !split
\subsection{Conjugate gradient method, Newton's method first}

% --- begin paragraph admon ---
\paragraph{}
We seek the minimum of the energy or the variance as function of various variational parameters.
In our case we have thus a function $f$ whose minimum we are seeking.
In Newton's method we set $\nabla f = 0$ and we can thus compute the next iteration point
\begin{equation*}
\hat{x}-\hat{x}_i=\hat{A}^{-1}\nabla f(\hat{x}_i).
\end{equation*}
Subtracting this equation from that of $\hat{x}_{i+1}$ we have
\begin{equation*}
\hat{x}_{i+1}-\hat{x}_i=\hat{A}^{-1}(\nabla f(\hat{x}_{i+1})-\nabla f(\hat{x}_i)).
\end{equation*}
% --- end paragraph admon ---

% !split
\subsection{Simple example and demonstration}

% --- begin paragraph admon ---
\paragraph{}
The function $f$ can be either the energy or the variance.  If we choose the energy then we have
\begin{equation*}
\hat{\alpha}_{i+1}-\hat{\alpha}_i=\hat{A}^{-1}(\nabla E(\hat{\alpha}_{i+1})-\nabla E(\hat{\alpha}_i)).
\end{equation*}
In the simple harmonic oscillator model, the gradient and the Hessian $\hat{A}$ are
\begin{equation*}
\frac{d\langle  E_L[\alpha]\rangle}{d\alpha} = \alpha-\frac{1}{4\alpha^3}
\end{equation*}
and a second derivative which is always positive (meaning that we find a minimum)
\begin{equation*}
\hat{A}= \frac{d^2\langle  E_L[\alpha]\rangle}{d\alpha^2} = 1+\frac{3}{4\alpha^4}
\end{equation*}
% --- end paragraph admon ---

% !split
\subsection{Simple example and demonstration}

% --- begin paragraph admon ---
\paragraph{}
We get then
\begin{equation*}
\alpha_{i+1}=\frac{4}{3}\alpha_i-\frac{\alpha_i^4}{3\alpha_{i+1}^3},
\end{equation*}
which can be rewritten as
\begin{equation*}
\alpha_{i+1}^4-\frac{4}{3}\alpha_i\alpha_{i+1}^4+\frac{1}{3}\alpha_i^4.
\end{equation*}
% --- end paragraph admon ---

% !split
\subsection{Conjugate gradient method}

% --- begin paragraph admon ---
\paragraph{}
In the CG method we define so-called conjugate directions and two vectors
$\hat{s}$ and $\hat{t}$
are said to be
conjugate if
\begin{equation*}
\hat{s}^T\hat{A}\hat{t}= 0.
\end{equation*}
The philosophy of the CG method is to perform searches in various conjugate directions
of our vectors $\hat{x}_i$ obeying the above criterion, namely
\begin{equation*}
\hat{x}_i^T\hat{A}\hat{x}_j= 0.
\end{equation*}
Two vectors are conjugate if they are orthogonal with respect to
this inner product. Being conjugate is a symmetric relation: if $\hat{s}$ is conjugate to $\hat{t}$, then $\hat{t}$ is conjugate to $\hat{s}$.
% --- end paragraph admon ---

% !split
\subsection{Conjugate gradient method}

% --- begin paragraph admon ---
\paragraph{}
An example is given by the eigenvectors of the matrix
\begin{equation*}
\hat{v}_i^T\hat{A}\hat{v}_j= \lambda\hat{v}_i^T\hat{v}_j,
\end{equation*}
which is zero unless $i=j$.
% --- end paragraph admon ---

% !split
\subsection{Conjugate gradient method}

% --- begin paragraph admon ---
\paragraph{}
Assume now that we have a symmetric positive-definite matrix $\hat{A}$ of size
$n\times n$. At each iteration $i+1$ we obtain the conjugate direction of a vector
\begin{equation*}
\hat{x}_{i+1}=\hat{x}_{i}+\alpha_i\hat{p}_{i}.
\end{equation*}
We assume that $\hat{p}_{i}$ is a sequence of $n$ mutually conjugate directions.
Then the $\hat{p}_{i}$  form a basis of $R^n$ and we can expand the solution
$  \hat{A}\hat{x} = \hat{b}$ in this basis, namely

\begin{equation*}
  \hat{x}  = \sum^{n}_{i=1} \alpha_i \hat{p}_i.
\end{equation*}
% --- end paragraph admon ---

% !split
\subsection{Conjugate gradient method}

% --- begin paragraph admon ---
\paragraph{}
The coefficients are given by
\begin{equation*}
    \mathbf{A}\mathbf{x} = \sum^{n}_{i=1} \alpha_i \mathbf{A} \mathbf{p}_i = \mathbf{b}.
\end{equation*}
Multiplying with $\hat{p}_k^T$  from the left gives

\begin{equation*}
  \hat{p}_k^T \hat{A}\hat{x} = \sum^{n}_{i=1} \alpha_i\hat{p}_k^T \hat{A}\hat{p}_i= \hat{p}_k^T \hat{b},
\end{equation*}
and we can define the coefficients $\alpha_k$ as

\begin{equation*}
    \alpha_k = \frac{\hat{p}_k^T \hat{b}}{\hat{p}_k^T \hat{A} \hat{p}_k}
\end{equation*}
% --- end paragraph admon ---

% !split
\subsection{Conjugate gradient method and iterations}

% --- begin paragraph admon ---
\paragraph{}

If we choose the conjugate vectors $\hat{p}_k$ carefully,
then we may not need all of them to obtain a good approximation to the solution
$\hat{x}$.
We want to regard the conjugate gradient method as an iterative method.
This will us to solve systems where $n$ is so large that the direct
method would take too much time.

We denote the initial guess for $\hat{x}$ as $\hat{x}_0$.
We can assume without loss of generality that
\begin{equation*}
\hat{x}_0=0,
\end{equation*}
or consider the system
\begin{equation*}
\hat{A}\hat{z} = \hat{b}-\hat{A}\hat{x}_0,
\end{equation*}
instead.
% --- end paragraph admon ---

% !split
\subsection{Conjugate gradient method}

% --- begin paragraph admon ---
\paragraph{}
One can show that the solution $\hat{x}$ is also the unique minimizer of the quadratic form
\begin{equation*}
  f(\hat{x}) = \frac{1}{2}\hat{x}^T\hat{A}\hat{x} - \hat{x}^T \hat{x} , \quad \hat{x}\in\mathbf{R}^n.
\end{equation*}
This suggests taking the first basis vector $\hat{p}_1$
to be the gradient of $f$ at $\hat{x}=\hat{x}_0$,
which equals
\begin{equation*}
\hat{A}\hat{x}_0-\hat{b},
\end{equation*}
and
$\hat{x}_0=0$ it is equal $-\hat{b}$.
The other vectors in the basis will be conjugate to the gradient,
hence the name conjugate gradient method.
% --- end paragraph admon ---

% !split
\subsection{Conjugate gradient method}

% --- begin paragraph admon ---
\paragraph{}
Let  $\hat{r}_k$ be the residual at the $k$-th step:
\begin{equation*}
\hat{r}_k=\hat{b}-\hat{A}\hat{x}_k.
\end{equation*}
Note that $\hat{r}_k$ is the negative gradient of $f$ at
$\hat{x}=\hat{x}_k$,
so the gradient descent method would be to move in the direction $\hat{r}_k$.
Here, we insist that the directions $\hat{p}_k$ are conjugate to each other,
so we take the direction closest to the gradient $\hat{r}_k$
under the conjugacy constraint.
This gives the following expression
\begin{equation*}
\hat{p}_{k+1}=\hat{r}_k-\frac{\hat{p}_k^T \hat{A}\hat{r}_k}{\hat{p}_k^T\hat{A}\hat{p}_k} \hat{p}_k.
\end{equation*}
% --- end paragraph admon ---

% !split
\subsection{Conjugate gradient method}

% --- begin paragraph admon ---
\paragraph{}
We can also  compute the residual iteratively as
\begin{equation*}
\hat{r}_{k+1}=\hat{b}-\hat{A}\hat{x}_{k+1},
 \end{equation*}
which equals
\begin{equation*}
\hat{b}-\hat{A}(\hat{x}_k+\alpha_k\hat{p}_k),
 \end{equation*}
or
\begin{equation*}
(\hat{b}-\hat{A}\hat{x}_k)-\alpha_k\hat{A}\hat{p}_k,
 \end{equation*}
which gives

\begin{equation*}
\hat{r}_{k+1}=\hat{r}_k-\hat{A}\hat{p}_{k},
 \end{equation*}
% --- end paragraph admon ---

% !split
\subsection{Using the conjugate gradient method}

% --- begin paragraph admon ---
\paragraph{}
\begin{itemize}
  \item Start your program with calling a function which implements for example the  CG method  or the steepest descent method.

  \item This function needs a function for the expectation value of the local energy and the derivative of the local energy.

  \item Your function \textbf{func} is now the Metropolis part with a call to the local energy function. For every call to the function \textbf{func} many practitionser use 1000-10000 Monte Carlo cycles for the trial wave function.

  \item This gives an expectation value for the energy which is returned by the function \textbf{func}.

  \item When one calls the local energy one also computes the first derivative of the expectaction value of the local energy
\end{itemize}

\noindent
\begin{equation*} \frac{d\langle E_{L}[\alpha] \rangle}{d\alpha}= 2\langle \frac{\bar{\psi_T}_{\alpha}}{\psi_T[\alpha]}\left(E_L[\alpha]-\langle  E_L[\alpha]\rangle\right)\rangle.
\end{equation*}
The following two codes, to be found in the \href{{https://github.com/CompPhysics/ComputationalPhysics2/tree/gh-pages/doc/pub/cg/programs/c%2B%2B}}{programs folder} as well, implement first the conjugate gradient and steepest descent method to a simple $2\times 2$ problem. Finally, we include a program based on a more advanced version of the conjugate gradient method.
% --- end paragraph admon ---

% !split
\subsection{Simple codes for  steepest descent and conjugate gradient using a $2\times 2$ matrix}

% --- begin paragraph admon ---
\paragraph{}
\bcppcod
#include <cmath>
#include <iostream>
#include <fstream>
#include <iomanip>
#include "vectormatrixclass.h"
using namespace  std;
//   Main function begins here
int main(int  argc, char * argv[]){
  int dim = 2;
  Vector x(dim),xsd(dim), b(dim),x0(dim);
  Matrix A(dim,dim);

  // Set our initial guess
  x0(0) = x0(1) = 0;
  // Set the matrix
  A(0,0) =  3;    A(1,0) =  2;   A(0,1) =  2;   A(1,1) =  6;
  b(0) = 2; b(1) = -8;
  cout << "The Matrix A that we are using: " << endl;
  A.Print();
  cout << endl;
  x = ConjugateGradient(A,b,x0);
  xsd = SteepestDescent(A,b,x0);
  cout << "The approximate solution using Conjugate Gradient is: " << endl;
  x.Print();
  cout << endl;
  cout << "The approximate solution using Steepest Descent is: " << endl;
  xsd.Print();
  cout << endl;
}
\ecppcod
% --- end paragraph admon ---

% !split
\subsection{The routine for the steepest descent method}

% --- begin paragraph admon ---
\paragraph{}
\bcppcod
Vector SteepestDescent(Matrix A, Vector b, Vector x0){
  int IterMax, i;
  int dim = x0.Dimension();
  const double tolerance = 1.0e-14;
  Vector x(dim),f(dim),z(dim);
  double c,alpha,d;
  IterMax = 30;
  x = x0;
  f = A*x-b;
  i = 0;
  while (i <= IterMax){
    z = A*f;
    c = dot(f,f);
    alpha = c/dot(f,z);
    x = x - alpha*f;
    f =  A*x-b;
    if(sqrt(dot(f,f)) < tolerance) break;
    i++;
  }
  return x;
}
\ecppcod
% --- end paragraph admon ---

% !split
\subsection{Simple implementation of the Conjugate gradient algorithm}

% --- begin paragraph admon ---
\paragraph{}
\bcppcod
  Vector ConjugateGradient(Matrix A, Vector b, Vector x0){
  int dim = x0.Dimension();
  const double tolerance = 1.0e-14;
  Vector x(dim),r(dim),v(dim),z(dim);
  double c,t,d;

  x = x0;
  r = b - A*x;
  v = r;
  c = dot(r,r);
  int i = 0; IterMax = dim;
  while(i <= IterMax){
    z = A*v;
    t = c/dot(v,z);
    x = x + t*v;
    r = r - t*z;
    d = dot(r,r);
    if(sqrt(d) < tolerance)
      break;
    v = r + (d/c)*v;
    c = d;  i++;
  }
  return x;
}
\ecppcod
% --- end paragraph admon ---

% !split
\subsection{Codes from numerical recipes}

% --- begin paragraph admon ---
\paragraph{}
You can however use codes we have adapted from the text \href{{http://www.nr.com/}}{Numerical Recipes in C++}, see chapter 10.7.
Here we present a program, which you also can find at the webpage of the course we use the functions \textbf{dfpmin} and \textbf{lnsrch}.

\begin{itemize}
\item The program uses the harmonic oscillator in one dimensions as example.

\item The program does not use armadillo to handle vectors and matrices, but employs rather my own vector-matrix class. These auxiliary functions, and the main program \emph{model.cpp} can all be found under the \href{{https://github.com/CompPhysics/ComputationalPhysics2/tree/gh-pages/doc/pub/cg/programs/c%2B%2B}}{program link here}.
\end{itemize}

\noindent
Below we show only excerpts from the main program. For the full program, see the above link.
% --- end paragraph admon ---

% !split
\subsection{Finding the minimum of the harmonic oscillator model in one dimension}

% --- begin paragraph admon ---
\paragraph{}
\bcppcod
//   Main function begins here
int main()
{
     int n, iter;
     double gtol, fret;
     double alpha;
     n = 1;
//   reserve space in memory for vectors containing the variational
//   parameters
     Vector g(n), p(n);
     cout << "Read in guess for alpha" << endl;
     cin >> alpha;
     gtol = 1.0e-5;
//   now call dfmin and compute the minimum
     p(0) = alpha;
     dfpmin(p, n, gtol, &iter, &fret, Efunction, dEfunction);
     cout << "Value of energy minimum = " << fret << endl;
     cout << "Number of iterations = " << iter << endl;
     cout << "Value of alpha at minimum = " << p(0) << endl;
      return 0;
}  // end of main program

\ecppcod
% --- end paragraph admon ---

% !split
\subsection{Functions to observe}

% --- begin paragraph admon ---
\paragraph{}
The functions \textbf{Efunction} and \textbf{dEfunction} compute the expectation value of the energy and its derivative.
These functions need to be changed when you want to your own derivatives.
\bcppcod
//  this function defines the expectation value of the local energy
double Efunction(Vector  &x)
{
  double value = x(0)*x(0)*0.5+1.0/(8*x(0)*x(0));
  return value;
} // end of function to evaluate

//  this function defines the derivative of the energy
void dEfunction(Vector &x, Vector &g)
{
  g(0) = x(0)-1.0/(4*x(0)*x(0)*x(0));
} // end of function to evaluate
\ecppcod
You need to change these functions in order to compute the local energy for your system. I used 1000
cycles per call to get a new value of $\langle E_L[\alpha]\rangle$.
When I compute the local energy I also compute its derivative.
After roughly 10-20 iterations I got a converged result in terms of $\alpha$.
% --- end paragraph admon ---

% ------------------- end of main content ---------------

% #ifdef PREAMBLE
\printindex

\end{document}
% #endif


\documentclass[12pt]{article}
\usepackage[top=1in, bottom=1in, left=1in, right=1in]{geometry}

\usepackage{setspace}
\onehalfspacing

\newif\ifeqns
\eqnstrue

\usepackage{amssymb}
%% The amsthm package provides extended theorem environments
\usepackage{amsthm}
\usepackage{epsfig}
\usepackage{times}
\renewcommand{\ttdefault}{cmtt}
\usepackage{amsmath}
\usepackage{graphicx} % for graphics files

% Draw figures yourself
\usepackage{tikz}

% writing elements
\usepackage{mhchem}

% The float package HAS to load before hyperref
\usepackage{float} % for psuedocode formatting
\usepackage{xspace}

% from Denovo Methods Manual
\usepackage{mathrsfs}
\usepackage[mathcal]{euscript}
\usepackage{color}
\usepackage{array}

\usepackage[pdftex]{hyperref}
\usepackage[parfill]{parskip}

% math syntax
\newcommand{\nth}{n\ensuremath{^{\text{th}}} }
\newcommand{\ve}[1]{\ensuremath{\mathbf{#1}}}
\newcommand{\Macro}{\ensuremath{\Sigma}}
\newcommand{\rvec}{\ensuremath{\vec{r}}}
\newcommand{\vecr}{\ensuremath{\vec{r}}}
\newcommand{\omvec}{\ensuremath{\hat{\Omega}}}
\newcommand{\vOmega}{\ensuremath{\hat{\Omega}}}
\newcommand{\sigs}{\ensuremath{\Sigma_s(\rvec,E'\rightarrow E,\omvec'\rightarrow\omvec)}}
\newcommand{\el}{\ensuremath{\ell}}
\newcommand{\sigso}{\ensuremath{\Sigma_{s,0}}}
\newcommand{\sigsi}{\ensuremath{\Sigma_{s,1}}}
\newcommand{\cc}[1]{\ensuremath{\overline{#1}}}
\newcommand{\ccm}[1]{\ensuremath{\overline{\mathbf{#1}}}}
%---------------------------------------------------------------------------
%---------------------------------------------------------------------------
\begin{document}
\begin{center}
{\bf NE 250, F17\\
October 19, 2017
}
\end{center}

\subsection*{Reflected Slab}
Duderstadt and Hamilton Chp.\ 5.III.E

Before looking at MOC, we had looked at the criticality state for bare geometries using the one-group, steady state diffusion equation with fission
\begin{equation}
 - \nabla \cdot D(\vec{r}) \nabla	\phi(\vec{r}) + \Sigma_a(\vec{r}) \phi(\vec{r}) = \nu\Sigma_f (\vec{r})\phi(\vec{r})\:.
\end{equation}
%
We're going to take a moment to look at what happens when we add a reflector (in your homework you're getting to look at a case with moderator).

We'll look at a symmetric slab of fuel with thickness $a$ centered at zero, having a reflector with thickness $b$ on each side. We will have equations in two regions, recognizing symmetry (c = core and r = reflector):
\begin{align*}
-D_c\frac{d^2 \psi_c}{dx^2} &+ (\Sigma_a^c - \nu \Sigma_f^c) \psi^c(x) = 0  \qquad 0 \leq x \leq a/2\\
-D^r\frac{d^2 \psi^r}{dx^2} &+ \Sigma_a^r \psi^r(x) = 0  \qquad a/2 \leq x \leq a/2+b
\end{align*}
with boundary conditions
\begin{align*}
\frac{d \phi^c}{dx}|_{x=0} &= 0 \\
\phi^c(a/2) &= \phi^r(a/2)\\
J^c(a/2) &= J^r(a/2)\\
\phi^r(a/2+\tilde{b}) &= 0\:.
\end{align*}

We anticipate that we will need an exact balance of materials and geometry to get criticality. Let's see what that relationship turns out to be.
From previous experience and using symmetry, we know (or can solve for) the core region solution will look like:
\begin{align*}
\phi^c(x) &= C_1 \cos(B_m^c x)\\
\text{and }&(B_m^c)^2 = \frac{\nu \Sigma_f^c - \Sigma_a^c}{D^c}
\end{align*}

In the reflector, we know
\begin{align*}
\phi^r(x) &= C_3 \sinh(\frac{a/2+\tilde{b} - x}{L^r}) + C_4 \cosh(\frac{a/2+\tilde{b} - x}{L^r}) \\
C_4 &= 0 \text{ from the vacuum condition.}
\end{align*}

Now we need to apply the rest of the interface conditions:
\begin{align*}
C_1 \cos(\frac{B_m^c a}{2}) &= C_3 \sinh(\frac{\tilde{b}}{L^r})\\
C_1 D^c B_m^c \sin(\frac{B_m^c a}{2}) &= C_3 \frac{D^r}{L^r}\cosh(\frac{\tilde{b}}{L^r})
\end{align*}
We can manipulate these to see
\[
D^c B_m^c \tan(\frac{B_m^c a}{2}) = \frac{D^r}{L^r}\coth(\frac{\tilde{b}}{L^r})\:.
\]
This is the criticality relationship for a reflected slab. You can see it relates the materials ($B_m$ in the core and $L$ in the reflector) and the geometry ($a$ and $b$). These all have to match for criticality.

Duderstadt discusses some numerical and graphical approaches for solving this problem. In the graphical approach it can be seen that
\begin{align*}
\frac{B_m^c a}{2} &< \frac{\pi}{2}\\
(B_m^c)^2 &< (\frac{\pi}{a})^2
\end{align*}

Recall that in an \textit{unreflected core} $B_m = \frac{\pi}{\tilde{a}}$. This means that the width for a reflected core is smaller than the width for a bare core. The difference is what we call \textbf{reflector savings}:
\[
\delta = a_{\text{bare}} - a_{\text{reflected}}.
\]

\subsection*{Eigenvalue Solution}
Some from Duderstadt and Hamilton Chp.\ 5.IV (see textbook for rest of notes)

Transcendental solutions and graphing techniques are all well and good, but not idea. We'd rather have something that we can solve much more easily. That's where eigenvalue solution techniques come in.

\subsection*{In the Transport Equation}
We can do the same kind of thing in the transport equation when there is fission and no source:
\begin{align*}
\bigl[\vOmega \cdot \nabla + \Sigma_t\bigr] \psi(\vec{r}, E, \vOmega) &= \int_{4 \pi} d\vOmega' \int_0^{\infty} dE' \: \Sigma_s(E', \vOmega' \rightarrow E, \vOmega) \psi(\vec{r}, E', \vOmega')\\
 &+ \frac{\chi(E)}{4 \pi}\int_0^{\infty} dE' \: \nu(E') \Sigma_f(E') \int_{4 \pi} d\vOmega' \:\psi(\vec{r}, E', \vOmega')
\end{align*}
%
We again use a mathematical ``knob" to give us more flexibility. However, with the TE there are two ways we do this:
\begin{enumerate}
\item Alter the effective cross section by adding an $\alpha$  term (which I don't think anyone does in diffusion), or
\item Alter the effective fission yield, $\nu$, by scaling it with $k$ (which is what we just did with diffusion).
\end{enumerate}

\textbf{The $\alpha$ version}, which is used more frequently at LLNL and LANL, looks like this:
%
\begin{align*}
\bigl[\vOmega \cdot \nabla + \bigl(\Sigma_t + \underbrace{\frac{\alpha_0}{v}}_{\text{new}}\bigr)\bigr]\psi(\vec{r}, E, \vOmega) &= \int_{4 \pi} d\vOmega' \int_0^{\infty} dE' \: \Sigma_s(E', \vOmega' \rightarrow E, \vOmega) \psi(\vec{r}, E', \vOmega')\\
 +& \frac{\chi(E)}{4 \pi}\int_0^{\infty} dE' \: \nu(E') \Sigma_f(E') \int_{4 \pi} d\vOmega' \:\psi(\vec{r}, E', \vOmega')
\end{align*}
+ boundary conditions.\\
%
Some important notes about this
\begin{itemize}
\item In the $\alpha$-evaluation problem, the total cross section is modified by a 1/v absorber.
\item There is numerical difficulty if $\alpha_0 < 0$.\\
If $\frac{-\alpha_0}{v} > \Sigma_t$ then total interaction becomes a ``source" rather than a loss!
\item If $\alpha_0 > 0$, absorption of \textbf{slow} neutrons is enhanced by an $\alpha_0$/v absorber $\rightarrow$ harder spectrum.
\item If $\alpha_0 < 0$, absorption of \textbf{fast} neutrons is enhanced by an $\alpha_0$/v absorber $\rightarrow$ softer spectrum.
\item These spectral effects are important because reaction rate is $\int_0^{\infty} dE \:\Sigma_j(E) \phi(E)$
\end{itemize}

\textbf{The $k$-eigenvalue problem} is a little bit different:
%
\begin{align*}
\bigl[\vOmega \cdot \nabla + \Sigma_t \bigr]\psi(\vec{r}, E, \vOmega) &= \int_{4 \pi} d\vOmega' \int_0^{\infty} dE' \: \Sigma_s(E', \vOmega' \rightarrow E, \vOmega) \psi(\vec{r}, E', \vOmega')\\
 +& \underbrace{\frac{1}{k}}_{\text{new}}\frac{\chi(E)}{4 \pi}\int_0^{\infty} dE' \: \nu(E') \Sigma_f(E') \int_{4 \pi} d\vOmega' \:\psi(\vec{r}, E', \vOmega')
\end{align*}
\vspace*{-.5em}
+  boundary conditions.\\
%
Important notes about this version:
\begin{itemize}
\item we can see that $k=1$ means $\alpha_0 = 0$ and vice versa.
\item $k$ as a multiplication factor: the ratio of neutron production in one generation to the neutron production in the previous generation.
\item We often use the $k$ version because we don't have the mathematical difficulties associated with the $\alpha$ form, and the physical interpretation is helpful.
\end{itemize}

More on the physical meaning: Let's define $p$ as all phase space, then
  \[\int dp = \int_{V} d\rvec \int_{4 \pi} d\vOmega \int_0^{\infty} dE\]
compute neutron production in $p$:
  \[\int dp\:\frac{\chi(E)}{4 \pi}\int_0^{\infty} dE' \: \nu(E') \Sigma_f(E') \int_{4 \pi} d\vOmega' \:\psi(\vec{r}, E', \vOmega') \]
compute neutron losses in $p$:
  \[\int dp\:\biggl[\vOmega \cdot \nabla \psi + \Sigma_t \psi - \int_{4 \pi} d\vOmega' \int_0^{\infty} dE' \: \Sigma_s(E', \vOmega' \rightarrow E, \vOmega) \psi(\vec{r}, E', \vOmega') \biggr] \]
Steady state: losses in this generation = production in the immediate past generation.\\
$k$ is the ratio of these two equations, giving the multiplication factor over $p$.\\
Note that $p$ is arbitrary, it applies pointwise over the system (limit as $V$ goes to zero).\\
We move the scattering to the LHS since it involves neutrons from this generation while fission is the only term involving neutrons from the previous one.

\end{document}


\chapter{Conclusion and Future Extensions}
% Tony: "Conclusion is NOT a summary"
% e.g. result that you did not know about

\section{Analogy with Attachment Theory}

The thorough analysis of the attractor neural networks (Hopfield network and Boltzmann machine) has clearly proven their capabilities, being able to model complex attachment types. However, research done in linking the model with the real-life scenario is quite slim, and further work needs to be dedicated especially in this area.

\section{Future Extensions}

We have introduced the reader with only a few flavours of what Attractor neural networks can really do. Deep investigations should be done in order to further investigate their properties, and really push them to the limit. These might provide a good source of inspiration for developing even more biologically realistic models, that better simulate the human brain.

Another line of extension is about providing better visualisation tools for these highly abstract models, that work on N-dimensional spaces. Parallel coordinates can be used for easily visualising the N-dimensional attractors, or the convergence of the network to such an attractor. Furthermore, plots can be done on the available number of neurons that can be updated at each step in the convergence process.

The experiments we did on Hopfield networks should be reproduced using the Restricted Boltzmann machine, which is a more powerful neural network. Furthermore, biological plausibility also needs to be kept in mind, which would surely provide us with more efficient learning systems.

\subsection{Inconsistencies in the results}
\label{inconsistencies}

It is important mentioning a few apparent inconsistencies in the cluster experiments. For examples, the experiments that have been done, using 1 cluster, display inconsistencies in the results between methods T1 and T2. We believe the main reason for this inconsistency is the limitation of the methodology for sampling Gaussian-distributed patterns and the poor use of the state space. Another reason for this apparent inconsistency might also be the presence of spurious attractors, that can interfere with the real attractors and ``steal'' part of their basins.

Another reason for the inconsistencies might also be the chaotic shape of the basins. Since the Storkey-Valabregue measurement is only good for hyper-spherical basin shapes. A more powerful method, such as a Monte Carlo sampling in the neighbourhood, is necessary for measuring basins that don't have hyperspherical shapes.