\documentclass{article}

% these packages let you do math
\usepackage{amsmath}
\usepackage{amssymb}

% we need these packages for fancy R tables
\usepackage{booktabs}
\usepackage{float}
\usepackage{colortbl}
\usepackage{xcolor}

% these packages play with the spacing/margins of the document. Uncomment the commands on lines 16 and 17 to see what they do.
\usepackage{a4wide}
\usepackage{setspace}
\usepackage{geometry}
\usepackage{parskip}
%\doublespacing
%\geometry{margin=1.5in}

% this package helps us with including images. Setting the graphics path makes it easier to refer to things in the \includegraphics command.
\usepackage{graphicx}
\graphicspath{ {../figures/} }

% make some hyperlinks using the \href command
\usepackage{hyperref}
\hypersetup{
    colorlinks=true,
    linkcolor=black,
    urlcolor=blue
}

% set the author, title, and date of the document. \maketitle adds it to the document.
\author{David Woolley}
\title{My Paper on NLSY97 Data}
\date{Sping 2022}

\begin{document}
\maketitle

\section{The First Section}

This is where I talk about basic \LaTeX. Using the \texttt{parskip} package, I can create a new paragraph by using line breaks, which I will do now.

I can also make cool equations inline by using parentheses{\textemdash}like this: \(x + 2\){\textemdash}or by using single dollar signs{\textemdash}like this: $x+2$. Parentheses are preferred because the left and right delimiter are distinct.

I can make cool equations in a block style by using the \texttt{equation} environment like so:
\begin{equation*}
    y = x\beta + \varepsilon
\end{equation*}
or by using double dollar signs:
$$
    y = x\beta + \varepsilon
$$

Again, the \texttt{equation} environment is preferred because the begin and end delimiters are different.

I can also add a bibliography, but this is beyond the scope of our discussion right now. Overleaf has plenty of resources for this on their \href{https://www.overleaf.com/learn}{website}. Another good place to look for LaTeX help is the \href{https://en.wikibooks.org/wiki/LaTeX}{WikiBook} on it.

\newpage

\section{The Second Section}

Wherein we do tables and graphs. To include the graph we made in ggplot, we create the \texttt{figure} environment. The `H' option tells LaTeX to `hold' the position of the figure instead of positioning it somewhere else. I use the \texttt{caption} command to add a caption{\textemdash}although I also put a title on the plot in ggplot so you would typically choose one or the other. I use the \texttt{label} command after the caption to add a label. Then in my paper I can use the \texttt{ref} command and LaTeX knows I am referring to Figure \ref{fig:graph}.

\begin{figure}[H]
    \begin{center}
        \includegraphics[width=.85\textwidth]{incarceration_by_racegender}
    \end{center}
    \caption{Mean Number of Incarcerated in 2002 by Race and Gender (this is the LaTeX caption, not the ggplot title)}
    \label{fig:graph}
\end{figure}

Tables are somewhat easier, since \texttt{kableExtra} and \texttt{stargazer} generate LaTeX code that is ready to just ``copy-paste'' into our document. The \texttt{label} argument in the R code is the label that the table will have in the tex output, if you want to \texttt{ref} it.

\input{../tables/incarceration_by_racegender.tex}

\input{../tables/regress_incarceration_by_racegender.tex}

\section{Analysis of the Results}
As can clearly be seen from Figure 1, there is a much higher degree of incarceration among males than females with the possible exception of Mixed Race (Non-Hispanic); though the complete lack of any males in that category suggests to me that there may be some issue in the data collection or perhaps simply an anomaly resulting in the discrepancy.  We can also see that incarceration is far higher for black males than all others.\\
\\
This is further backed up by Table 1, providing more directly comparable numbers on each of the groups.  Table 2 solidifies this by showing that these differences are all statistically significant at the 5\% level.\\
\\
Overall the most noteworthy aspects of this appear to be the extremely high level of incarceration of black males, the generally higher incarceration among males, and the bizarrely high incarceration rate of Mixed Race (Non-Hispanic) females with the complete absence of incarceration among the males of that group.
\end{document}


\documentclass[9pt]{IEEEtran}

% basic
\usepackage[english]{babel}
\usepackage{graphicx,epstopdf,fancyhdr,amsmath,amsthm,amssymb,url,array,textcomp,svg,listings,hyperref,xcolor,colortbl,float,gensymb,longtable,supertabular,multicol,placeins}
\usepackage{diagbox}
 % `sumniki' in names
\usepackage[utf8x]{inputenc}
\usepackage{multirow}
\usepackage{makecell}

 % search and copy for `sumniki'
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage{placeins}
\input{glyphtounicode}
\pdfgentounicode=1

% tidy figures
\graphicspath{{./figures/}}
\DeclareGraphicsExtensions{.pdf,.png,.jpg,.eps}

% correct bad hyphenation here
\hyphenation{op-tical net-works semi-conduc-tor trig-gs}

% ============================================================================================

\title{\vspace{0ex} %
% TITLE IN HERE:
Detecting contours of human organs in CT images using the Canny edge detector
\\ \large{Exam Assignment}\\ \normalsize{Biometrical signal and image processing 2020/21, Faculty of Computer and Information Science, University of Ljubljana}}
\author{ %
% AUTHOR IN HERE:
Žiga~Kleine
\vspace{-4.0ex}
}

% ============================================================================================

\begin{document}

\maketitle

\begin{abstract}

This report describes the implementation and the theory behind the Canny edge detector for detecting edges of human organs in Computed Tomography images. The Canny detector consists of four main stages. First, we apply a gaussian smoothing filter on the image we want to detect edges on, second, we calculate the magnitude and angle image. After that, we apply nonmaxima suppression, and lastly, we use hysteresis thresholding and connectivity analysis to correctly find and link edges. The edge detector was implemented in the MATLAB environment and tested on the CTMRI database images \cite{ctmri}.

\end{abstract}

\section{Introduction}

Computed Tomography is a procedure where a computer controlled x-ray beam rotates quickly around the patient's body, producing signals that can be processed into images representing slices of the human body \cite{kalender2006x}. In this report, we will be tackling the problem of extracting edges, organ contour lines from the provided tomography images. We will be using an algorithm called the Canny detector \cite{mcilhagga2011canny}.

\section{Technical Background}

We can think of images as multi-dimensional signals. For example, a grayscale image can be thought of as a function that takes in two variables and produces a value. We can apply operations on the image signal.

Spatial convolution is an operation performed on the image by moving a smaller kernel window box through the image, and then in each step multiplying values from the kernel and the image at the same places, summing them together. This sum then represents the pixel under the centre of the kernel. The equation for convolution can be written as \ref{eq1}, where $g(x, y)$ represents the new convolved signal at the coordinate $(x, y)$, $f(x, y)$ represents the original image signal, and $w$ represents the kernel.

\begin{equation} \label{eq1}
g(x, y) = \dfrac{\Sigma_{s=-a}^a  \Sigma_{s=-b}^b w(s, t) * f(x + s, y + t)}{\Sigma_{s=-a}^a  \Sigma_{s=-b}^b w(s, t) }
 \end{equation}

 The process of convolution on images is mostly used for filtering. Here, we will focus on two types of filters: filters for blurring the image (low-pass filters), and filters for sharpening the image (high-pass filters).

 A popular type of kernel for low-pass filtering of an image is the Gaussian kernel. In can be calculated using this equation \ref{eq2}. The kernel is always symmetrical, and it has a square shape, with its size being an odd number. We usually calculate the $\sigma$ value with the equation $\sigma = 0.005*imgdim$, where $imgdim$ represents the smaller dimension of the two dimensions of the image we want to filter. The appropriate kernel size is then calculated as $ kernelsize = 6*\sigma $.

 \begin{equation} \label{eq2}
G[x, y] = \dfrac{e^{\dfrac{ -(x^2 + y^2)}{2\sigma^2}}}{2\pi\sigma^2}
 \end{equation}

\begin{figure}[!htb]
\centering
\includegraphics[width=1\columnwidth]{gaussian_kernel.png}
\caption[c1]{ 3D visualisation of a Gaussian kernel. }
\label{fig_1}
\end{figure}

 For image sharpening (high-pass filtering), we will present two types of kernels: the first derivative and second derivative kernels. First derivatives are usually used for detecting slopes in image values and second derivatives are used for detecting changes in slopes of values. For the first derivative, we usually use Roberts, Prewitt or Sobel kernels \ref{fig_10}, and for the second derivative, we use Laplacian kernels \ref{fig_11}.

 \begin{figure}[!htb]
\centering
\includegraphics[width=1\columnwidth]{first_derivative_kernels.png}
\caption[c1]{ Examples of first derivative kernels. }
\label{fig_10}
\end{figure}

\begin{figure}[!htb]
\centering
\includegraphics[width=1\columnwidth]{laplacian_kernels.png}
\caption[c1]{ Examples of Laplacian kernels. }
\label{fig_11}
\end{figure}

\section{Methodology}

In this assignment, we will be implementing the Canny edge detector in the MATLAB environment. To properly implement the Canny detector, we must implement the following steps:

\begin{enumerate}
\item Smooth the image with a Gaussian filter
\item Compute the gradient magnitude and angle images
\item Apply nonmaxima suppression to the gradient magnitude image
\item Use hysteresis thresholding and connectivity analysis to detect and link edges
\end{enumerate}

\subsection{Gaussian filter}
According to the list above, we first applied low-pass filtering to our input CT image using the Gaussian filter. We created a Gaussian kernel using the $fspecial()$ function in MATLAB, and we then filtered the image using the $imfilter()$ function.

\subsection{Gradient magnitude and angle images}
Then, we needed to calculate the magnitude and angle images from our source image. To achieve that, we first needed to derive the image on the x and y coordinates. To get the x and y derivatives of the image ($g_x and g_y$), we used Prewitt kernels \ref{fig_2}. If we put both derivatives in a vector together, they represent the gradient of the image at any given point. We would then like to represent the gradients of the image as vectors in polar coordinates, with each gradient being represented as a combination of the vector amplitude and the vector angle \ref{fig_3}. We computed the magnitude and angle values for each pixel of our image according to the following formulas \ref{eq3, eq4}. While computing the magnitude is pretty straightforward in MATLAB, it is important to note that we used the function $atan2()$ to calculate the arctan values for our gradient angles. This function is used to ensure that the values of arctan range from $-\pi$ to $\pi$, to ensure that we get correct angles for all 4 quadrants out of the equation as output.

\begin{figure}[!htb]
\centering
\includegraphics[width=1\columnwidth]{prewitt.png}
\caption[c1]{ Prewitt kernels to calculate the first derivative components of an image. }
\label{fig_2}
\end{figure}

\begin{figure}[!htb]
\centering
\includegraphics[width=1\columnwidth]{magnitude_angle.png}
\caption[c1]{ Edge gradient represented with a vector containing magnitude and angle attributes. }
\label{fig_3}
\end{figure}

\begin{equation} \label{eq3}
M[x, y] = mag(\nabla f) = \sqrt{g_x^2 + g_y^2}
 \end{equation}

 \begin{equation} \label{eq4}
\alpha[x, y] = tan^{-1}[\dfrac{g_y}{g_x}]
 \end{equation}

\subsection{Nonmaxima suppression}

The next step is nonmaxima suppression, a procedure applied to the magnitude image that removes unnecessary pixels in the gradient magnitudes image, based on the angle of the gradient. The procedure goes like this:

\begin{enumerate}
\item We define four basic directions (vertical, horizontal, -45 degree, 45 degree) that we get by dividing the circle into eight parts. For each pixel, we find the direction that most accurately describes the angle of the edge in the pixel. In our implementation, this is done by dividing the circle into eight equal parts, checking inside which of the parts is our angle, and assigning a direction based on the part of the circle that corresponds to the vector angle. It is important to note that the gradient angle represents the edge normal, the right angle of our edge, so we need to take that into account when deciding on the direction of our edge.
\item When we have the direction of our edge in each pixel, we calculate the new suppressed image values accordingly, by comparing magnitude values in the direction of the edge in a 3x3 grid around each pixel in the magnitude image. If the value of the magnitude pixel is less than any of the neighbouring values along the edge direction, we set the value of the suppressed image pixel at the same place as the centre magnitude pixel to 0. Otherwise, we set the new pixel value the same as the magnitude value at the same place.
\end{enumerate}

\begin{figure}[!htb]
\centering
\includegraphics[width=1\columnwidth]{nonmaxima_suppression.png}
\caption[c1]{ Nonmaxima suppression procedure visualized. }
\label{fig_4}
\end{figure}

\subsection{Hysteresis thresholding}

The last step of the Canny edge detector is the hysteresis thresholding. It performs additional edge thinning, and also links together edges that might be disconnected. The steps for hysteresis thresholding are these:

\begin{enumerate}
\item We will use two thresholds: $TH$ and $TL$. We select $TH$ empirically, and then set $TL$ as $TL = \dfrac{TH}{2}$.
\item We then calculate two images: $g_{NH}$ for strong pixels, which contains pixels in our image that are above the high threshold $TH$, and $g_{NL}$ for weak pixels, pixels that are above the threshold $TL$ but below the threshold $TH$. We calculate $g_{NH}$  with the equation $g_{NH}(x,y) = image(x,y) >= TH$, and  $g_{NL}$ with the equation $g_{NL}(x, y) = (image(x, y) >= TL) - g_{NH}$.
\item We can then calculate the thresholded image. All of the non-zero values in the $g_{NH}$ can be instantly classified as valid pixels for our thresholded image.
\item For the non-zero values in the $g_{NL}$, their validity is calculated a bit differently. We need to iterate through all of the non-zero values in $g_{NH}$, and then search the 3x3 area around every non-zero value in $g_{NH}$. If there is a non-zero value for  $g_{NH}$ in the area around, we can classify this pixel as valid in our thresholded image.
\end{enumerate}

After all these steps, we finally get the thresholded image, which represents our edges found in the input CT image with the Canny edge detector.

\section{Results}

To present the results of our implementation of the Canny edge detector, we will display the image after each step in the edge detection process for the image named $0090.png$ \ref{fig_5} in the CTMRI database.

\begin{figure}[!htb]
\centering
\includegraphics[width=0.8\columnwidth]{original_image.png}
\caption[c1]{ Our original input image. }
\label{fig_5}
\end{figure}

After the gaussian kernel filtering (with $\sigma = 2.5600$ and $kernelsize = 15$), we get the image  \ref{fig_6}.

\begin{figure}[!htb]
\centering
\includegraphics[width=0.8\columnwidth]{lp_filtered_image.png}
\caption[c1]{ Input image with the Gaussian filtering applied. }
\label{fig_6}
\end{figure}

Then we calculate the magnitude and angle images from our low-pass filtered image \ref{fig_7}.

\begin{figure}[!htb]
\centering
\includegraphics[width=0.8\columnwidth]{magnitudes_angles.png}
\caption[c1]{ Left part represents the magnitude image and the right part represents the angle image. }
\label{fig_7}
\end{figure}

We modify the magnitude image by performing nonmaxima supression \ref{fig_8}.

\begin{figure}[!t]
\centering
\includegraphics[width=0.8\columnwidth]{suppressed_image.png}
\caption[c1]{ The magnitudes image after nonmaxima supression. }
\label{fig_8}
\end{figure}

Finally, we perform hysteresis thresholding (with $TH = 0.1$ and $TL = 0.05$) on the nonmaxima supressed image, getting the final result of the Canny edge detector. \ref{fig_9}.

\begin{figure}[!t]
\centering
\includegraphics[width=0.8\columnwidth]{thresholded_image.png}
\caption[c1]{ The final image with edges detected. }
\label{fig_9}
\end{figure}

\section{Discussion}

While our Canny detector detects edges in CT images pretty well, there are many improvements that can be done to improve the performance. We could for example use better edge linking methods, or, instead of getting to $TH$ and $TL$ values empirically, using a more complex method for defining thresholds.

\FloatBarrier

\bibliographystyle{IEEEtran}
\bibliography{bibliography}

\end{document}


\documentclass[./Thesis.tex]{subfiles}
\begin{document}

\chapter{Introduction}
\label{chap:Introduction}
Imagine a world where teachers assigning proof filled math homework need not
grade that homework or a world where journals need not peer review their
submissions. This is a world \textit{proof assistants} attempt to bring about. A world
where computers can ensure the correctness of mathematics. This thesis is in
part an exploration of a specific proof assistant, \Agda{} \cite{agda}. As we
will come to see \Agda{} contains an impressive capability for mathematical
abstraction and can come to understand arbitrarily complex proofs. Unfortunately
it does not come without drawbacks. We find it useful to consider \Agda{} as an
infinitely intelligent toddler. An actor that knows nothing until it is taught
and requires convincing at every step. This is because \Agda{} does not have any
specific mathematics built into itself and that it does not perform any
\textit{proof search}. It simply accepts or rejects a given proof. As proof
assistants are useless without something to prove we turn to the other half of
this thesis, investigating prime numbers. \\

Prime numbers are of great interest to mathematicians and computer scientists
the world over. They form the backbone of a variety of fields of mathematics and
underpin the correctness of many real-world cryptographic protocols. In this
thesis we will explore formalizing primality tests, or algorithms that determine
if a given input $n$ is prime. These tests and their speed are critical to the
implementation of the cryptographic protocols mentioned above. Specifically the RSA
cryptosystem, which is a vital component of the world wide web, requires these
tests \cite{rsa}.
\section{Primality Tests}
There are many different primality tests and many authors choose to characterize
these tests along three dimensions. These dimensions are as follows
\textit{computational complexity} (how fast the algorithm runs),
\textit{determinism} (if the algorithm makes use of probabilistic primitive
operations), and \textit{conditionality} (if the algorithm requires unproven
conjectures in its proof of correctness or speed). Let us first consider the
brute force primality test. The algorithm determines the primality of a given
input $n$ by testing every number $m$ less than $n$ to see if $m$ divides $n$.
If no numbers divides $n$ then $n$ is prime. This test is both deterministic and
unconditional but suffers one major flaw, its computational complexity is
$\BigO{n}$. At first glance this does not seem bad but for any real world prime
this becomes untenable. For instance say we tried to test the primality of any
prime used in RSA. These primes hover around $2^{4096}$ in size and even at a
rate of 10 billion divisibility checks a second it would take $100$ times the
length of the universe to finish. Thus a computational complexity that is
polynomial in the size of the description of the input $\BigO{\Lg{n}^c}$ is desired. \\

In 1975, a deterministic polynomial time primality test was designed by Miller.
Unfortunately this test required the
\textit{Extended Riemann Hypothesis} a famous unproven conjecture \cite{miller}.
Thus the Miller primality test is considered conditional. A similar
unconditional primality test was later developed by Rabin but the algorithm was
probabilistic \cite{rabin}. Interestingly the probability of failure for this
algorithm is quite low so many still use this test for real world primality
testing. In 1983, the authors Adleman, Pomerance, and Rumely created the first
unconditional deterministic sub-exponential primality test with a run-time of
$\BigO{\Lg{n}^{c \, \text{lg} \, \text{lg} \, \text{lg} \, n}}$ \cite{apr}. \\

This string of accomplished authors shows both the difficulty and importance of this
problem. Thus it was big news when Manindra Agrawal, Neeraj Kayal,and Nitin
Saxena announced that they had created a unconditional deterministic
primality test that runs in polynomial time $\BigO{\Lg{n}^{15/2}}$ \cite{aks}.
This accomplishment is especially impressive considering that the last two
authors where undergraduate researchers at the time of publication (2002). The
last names of these authors are abbreviated to yield the name of this algorithm,
AKS. The correctness proof of this algorithm is comparably simpler to all the
previous algorithms listed above, requiring only undergraduate levels of number
theory. This property makes AKS a great candidate for a ``replication'' in
\Agda{}. Replications are common in various scientific fields and amount to
recreating a previously reported experiment. Usually the goal of a replication
is to verify the correctness of the data and conclusions of the previous
experiment. In a similar vein, if we implement AKS in \Agda{} and \Agda{}
accepts our implementation then we will have verified the correctness of the AKS
paper. Unfortunately this is no small task as \Agda{} does not know any number theory.
We will then have to formulate every piece of number theory and more generally
mathematics used in the AKS paper. Specifically we will have to formalize these
concepts in the \Agda{} proof assistant. \\

In this thesis we will illustrate only
a selection of these concepts as this is a undergraduate time limited thesis.
This being said we estimate to have completed formalizing 75\% of the required
concepts and this work can be found in the following git repository
\url{https://github.com/mckeankylej/thesis}. The required and completed concepts
include but are not limited to a GCD function that works over an arbitrary
Euclidean Domain, a polynomial datatype that forms a Euclidean Domain,
a modular ring datatype, a proven correct exponentiation by squaring algorithm,
and a brute force primality test. In this thesis we will specifically cover the
last two concepts. \\

In order to explain the intricacies of these constructions we must first
introduce the reader to the \Agda{} proof assistant and we accomplish this in
Chapter \ref{chap:agda-briefly}. Then we move on to describe an exponentiation
by squaring algorithm written \Agda{}. This algorithm is required multiple times
in AKS. It constitutes the very last step of the algorithm in which the input
must be tested to ensure it is not a perfect power \cite{smid}. After this in
Chapter \ref{chap:termination}, we take a brief aside in order to discuss how
\Agda{} ensures the correctness of recursive programs. Finally in Chapter
\ref{chap:primality} we create a brute force primality test in \Agda{}. The AKS
algorithm requires a list of ``small'' primes so this algorithm will be used to
generate that list.
\section{Type Theory}
In the rest of this Chapter we provide an informal motivation of Type Theory for a reader
familiar with the standard foundations of mathematics, Set Theory. Our proof
assistant \Agda{} is a proof assistant based on Type Theory. For readers that
desire a more formal specification of Type Theory we recommend the following
sources: a topological perspective \cite{hott-book}, a categorical perspective
\cite{nlab}, and a programming language theory perspective \cite{harper}.
\subsection{History}
\label{sec:history}
Type Theory was created by Bertrand Russell as an alternate foundation for
mathematics in order to quash the plethora of
paradoxes plaguing Set Theory at the turn of the century. The paradoxes
where eventually resolved and type theory faded into obscurity. It resurfaced in
the 1940's when Alonzo Church applied it to his \textit{lambda calculus}. This
application began a resurgence in which multiple authors steadily improved the
expressivity of Type Theory. This culminated with Per Martin-L\"{o}f's
dependent, intuitionistic type theory often called
\textit{Martin-L\"{o}f Type Theory} \cite{martin-lof}.
This theory is considered by many as the foundations of the field. The
programming language \Agda{} that we will come to know in this thesis is a
variant of his Type Theory. This being said the field is vast and there are many
type theories created with wildly different axioms and properties in mind.
\subsection{The Typing Relation}
\label{sec:typing-relation}
The primitive concept underlying Type Theory is the \textit{typing relation},
namely a \textit{term} $a$ has \textit{type} $A$ written $a : A$. In the
language of Set Theory this would represent ``a is an element of the set A''
denoted $a \in A$. Despite these similarities types differ from sets in many
ways. For instance the perfectly valid set
$\{ \, \text{true}, \, 0, \, \emptyset \, \}$
is inexpressible in Type Theory as types are ``homogeneous''. More importantly
there is not an equivalent concept of a non-membership ($\notin$) in most Type
Theories. Instead we say $2$ does not have type boolean or we say $2$ is not
well typed as a boolean. Thus while it is sensible to ask the set theoretic question
``Is $a$ a member of $A$ for some pre-existing objects $a$ and $A$?''. Type
theoretically we are not permitted to consider objects without their type and
every object has a type from creation. For instance, we can define the booleans
($𝔹$) as two constants $\text{true} : 𝔹$ and $\text{false} : 𝔹$.
\subsection{Propositions as Types}
In reality Set Theory is actually a dual theory. There is a base theory called
First Order Logic in which a collection of axioms are developed that describe
the structure of sets. So Set Theory is really the study of these two
interleaved theories. Type Theory is its own logic which collapses the two
levels into one. The two basic elements of Set Theory, logical propositions and
sets thus also collapse together. They collapse together into types and
propositions become manipulatable objects. For instance
a function $f$ is injective if the following proposition is true.
\begin{align}
  \text{for any } x, y \text{ if } f(x) = f(y) \text{ then } x = y
\end{align}
We can represent this proposition type theoretically with the following
expression.
\begin{equation}
  \label{eqn:injective}
  \text{Injective}(f : A \to B) :\equiv \prod_{x : A} \prod_{y : A} \, f(x) \equiv_B f(y) \to x \equiv_A y
\end{equation}
Now in order to prove some function $f$ is injective, we simply construct a proof
that has the type specified above, for instance $p : \text{Injective}(f)$.
Thus proving a proposition is simply finding an element of some type.
Also note that proofs are values and so they can be
manipulated like any other value. This concept is called
\textit{proof relevance} and it is essential to Type Theory and this thesis.
\subsection{Intuitionistic Logic}
In this section we investigate the deep consequences of proof relevance.
Consider the Collatz conjecture, concerning the sequence that follows, start with some
integer $n$ and add $n$ to the sequence. If $n$ is even then divide by $2$
otherwise multiply $n$ by $3$ and add $1$. The result of this computation is the
next element of the sequence. Repeat the process above. The conjecture is
that for any starting value $n$ the sequence eventually reaches $1$. This
conjecture is a well known unsolved problem in mathematics and as such the truth
value of the proposition is unknown. Now consider the following proposition
often called the law of the excluded middle.
\begin{align}
  \label{eqn:set-LEM}
  \text{for any proposition } p \text{ either } p \text{ or } \neg p
\end{align}
We can transform this proposition into Type Theory just as we have before. Note
that propositions are types so we can easily quantify over every proposition by
quantifying over every type.
\begin{align}
  \label{eqn:type-LEM}
  \text{LEM} :\equiv \prod_{p  : \text{Type}} p \vee \neg p
\end{align}
In Set Theory the law of the excluded middle is assumed as an axiom of first
order logic. Presumably Type Theorists could assume it as well and there would
be some value $\text{lem} : \text{LEM}$ representing the axiom. This axiom behaves weirdly
with proof relevance in mind. Consider using this axiom to construct a proof
that the Collatz conjecture is true or false. We simply instantiate $p$ to our
type theoretic encoding of the Collatz conjecture. Thus if we inspect this proof
term we can learn if the Collatz conjecture is true or false. As solving math
problems is not this easy, we must not assume the law of the excluded middle.
This variant of logic is called \textit{intuitionistic logic}. \\

This being said the law of the excluded middle appears often
and is a useful axiom. Thankfully this axiom is often provable when you choose a
specific $p$. Imagine instantiating $p$ with the proposition
$n \equiv_{\mathbb{N}} m$ for some $n$ and $m$.
The resulting type is described below.
\begin{align}
  \label{eqn:nat-dec}
  \text{Equals?} :\equiv (n \equiv_{\mathbb{N}} m) \vee \neg (n \equiv_{\mathbb{N}} m)
\end{align}
This is the type of our first decision procedure. A value of this type would
prove that the two numbers are equal or prove that the two numbers are unequal.
Finding a value of this type is as simple as describing a decision procedure.
Simply count $n$ and $m$ down by $1$. If both numbers reach $0$ at the same time
then they are equal otherwise they are unequal. \\

Keep in mind that assuming the law of the excluded middle also invalidates the
entire purpose of this thesis. We can define a proposition that represents
primality (and in fact we do in Chapter \ref{chap:primality}) apply the law of
the excluded middle to this type and now we have a term that ``computes''
primality.
\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% UMB-CS110-2015S: Introduction to Computing
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/UMB-CS110-2015S
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\def \topDirectory {.}
\def \texDirectory {\topDirectory/src/main/tex}

\documentclass[12pt,letterpaper,twoside]{article}
\usepackage{\texDirectory/template/style/directives}
\usepackage{\texDirectory/template/style/assignment}
\input{\texDirectory/template/config}

\begin{document}

\doc{title}{Solution to Assignment 4}
\doc{date-pub}{Apr 06, 2015 at 00:00 AM}
\doc{date-due}{Apr 16, 2015 at 5:30 PM}
\doc{points}{8}

\prepare{header}

\section*{Question 1}

Back to Dalton Brothers! Develop a simple class \texttt{Brother.java} and its data memebrs and methods as you think best fits this program. Neglecting their original heights, write a program \texttt{Daltons3.java} that uses the \texttt{Brother.java} class to sort Joe, William, Jack and Averell after getting their heights from the user. The program should output names of the brothers in ascending order of their heights as specified.

\begin{figure}[H]\centering
\includegraphics{\texDirectory/template/images/daltons.png}
\end{figure}

\subsection*{Solution}

\begin{enumerate}
\item Class \texttt{Brother.java}

\lstset{language=java,tabsize=2}
\begin{lstlisting}
public class Brother {
	public double height;
	public String name;
	public Brother(String hisName, double hisHeight) {
		height = hisHeight;
		name = hisName;
	}
	public Brother() {
	}
	public double getHeight() {
		return height;
	}
	public void setHeight(double height) {
		this.height = height;
	}
	public String getName() {
		return name;
	}
	public void setName(String name) {
		this.name = name;
	}
}
\end{lstlisting}

\item Class \texttt{DaltonsSort.java}
\lstset{language=java,tabsize=2}
\begin{lstlisting}
import java.util.Scanner;
public class DaltonsSort {
	public static void main(String[] args) {
		Scanner input = new Scanner(System.in);
		// Let's start with what we know: Names of brothers.
		String[] broNames = {"Joe", "William", "Jack", "Averell"};
		// Now that we can treat these brothers as humans (objects) not just variables,
		// Let's declare an array of objects and put all brothers in it.
		Brother[] bros = new Brother[broNames.length];
		// Now we can give identity to each brother by setting its name
		for (int i = 0; i < bros.length; i++){
			System.out.print("What is the height of " + broNames[i] + "? ");
			int height = input.nextInt();
			bros[i] = new Brother(broNames[i], height);
		}
		// So far, we have four brothers with different names and different heights
		// Let's ask user the sorting order
		System.out.print("In which order do you want to sort the Daltons [ASC, DESC]? ");
		String orderName = input.next();
		boolean orderAsc = orderName.equals("ASC") ? true : false;
		// Now we have all we want. Let's close the Scanner object then.
		input.close();
		// Time for sorting.
		for (int j = 0; j < bros.length; j++) {
			int minIndex = j;
			double min = Double.MAX_VALUE;
			for (int i = j; i < bros.length; i++) {
				if (bros[i].height < min) {
					min = bros[i].height;
					minIndex = i;
				}
			}
			if (j != minIndex) {
				Brother temp = new Brother();
				temp = bros[j];
				bros[j] = bros[minIndex];
				bros[minIndex] = temp;
			}
		}
		// Print names
		for (int i = 0; i < bros.length; i++) {
			System.out.println(orderAsc ? bros[i].getName() : bros[bros.length-1-i].getName());
		}
	}
}
\end{lstlisting}
\end{enumerate}

\section*{Question 2}

A student recently found a scrap of paper in MBTA Red line train on which the following code was written. Without modifying the code, develop a class \texttt{Elevator.java} to be used by this program.
\lstset{language=java,tabsize=2}
\begin{lstlisting}
public class ElevatorTest {
	public static void main(String[] args) {
		Elevator.currentLevel(); //outputs current floor
		Elevator.go(2); //moves the elevator two floors up
		Elevator.currentLevel();
	}
}
\end{lstlisting}

\subsection*{Solution}

Class \texttt{Elevator.java}
\lstset{language=java}
\begin{lstlisting}
public class Elevator {
	private static int floor;
	private final static int limitUp = 10;
	private final static int limitDown = -2;
	private Elevator() { floor = 0; }
	protected static void go(int level) {
		int dest = floor + level;
		if (dest <= limitUp && dest >= limitDown) {
			floor += level;
		}
		else {
			System.out.println("Impossible!");
		}
	}
	protected static void currentLevel() {
		System.out.println("Elevator is at floor "+floor);
	}
}
\end{lstlisting}

\section*{Question 3}

Objective is to simulate a circle-shaped pond with 100 meters radius, in which there are initially 100 salmons and one shark. As sharks and salmons know swimming, they continually wander around the pond. It's all about survival! To avoid starving to death, the shark must eat. To avoid extinction, salmons reproduce. Contrary to what it seems however, the shark has a very poor fate. If it doesn't eat for too long it will sadly die. If it manages to eat all the salmons and there are no more to eat, it won't be able to escape his destiny either. It's not about survival then. It's all about time!
\begin{figure}[H]\centering
\includegraphics[width=8cm,height=5cm]{\texDirectory/template/images/pond.png}
\end{figure}

Write a program \texttt{Simulation.java} that determines how many days the shark will live. Develop classes \texttt{Fish.java}, \texttt{Salmon.java}, \texttt{Shark.java} and \texttt{Pond.java} with their class members as you think best fits this program. Use the following assumptions.
\begin{itemize}[itemsep=1mm] \parskip=0pt \parsep=0pt
\item[] Fish are initially positioned randomly in the pond.
\item[] Fish move randomly in the pond.
\item[] Salmons move 10 meter per hour.
\item[] Salmons reproduce every day.
\item[] Salmons don't age and don't need to eat.
\item[] Salmons won't die unless they are eaten.
\item[] The shark moves 40 meters per hour in one direction.
\item[] The shark cannot eat while it's moving.
\item[] The shark is always hungry and never loses interest in eating.
\item[] The shark swallows all salmons within its 20 meters of proximity in a blink of an eye.
\item[] The shark dies if it does not eat in three days.
\item[] The shark is initially well fed.
\end{itemize}

\subsection*{Solution}

\begin{enumerate}
\item Class \texttt{Simulation.java}
\lstset{language=java, tabsize=2}
\begin{lstlisting}
import java.util.ArrayList;
public class Simulation {
	public static void main(String[] args) {
		initialize();
		while (shark.getHunger() < 3 * 24) {
			nextHour();
			eat();
			if (time % 24 == 0) {
				reproduce();
			}
			showStatistics();
		}
	}
	private static ArrayList<Salmon> salmons;
	private static Shark shark;
	private static int time = 0;
	// setup 100 salmons and 1 shark in Pond
	private static void initialize() {
		System.out.println("Initalizing Pond...");
		salmons = new ArrayList<Salmon>(101);
		for (int i = 0; i < 100; i++) {
			Salmon salmon = new Salmon();
			salmons.add(salmon);
		}
		shark = new Shark();
		System.out.println("Pond Initialized.");
	}
	// move all salmons and the shark
	private static void nextHour() {
		for (int i = 0; i < salmons.size(); i++) {
			salmons.get(i).move();
		}
		shark.move();
		time++;
	}
	// check if it's time to eat
	private static void eat() {
		double[] posShark = shark.getPosition();
		double[] posSalmon = new double[2];
		double dist;
		for (int i = salmons.size() - 1; i >= 0; i--) {
			posSalmon = salmons.get(i).getPosition();
			dist = Math.sqrt(Math.pow(posShark[0] - posSalmon[0], 2) + Math.pow(posShark[1] - posSalmon[1], 2));
			if (dist < shark.getRange()) {
				salmons.remove(i);
				shark.eat();
			}
		}
	}
	private static void reproduce() {
		int num = salmons.size();
		for (int i = 0; i < num; i++) {
			Salmon salmon = new Salmon(salmons.get(i).getPosition());
			salmons.add(salmon);
		}
	}
	private static void showStatistics() {
		System.out.printf("Time: %d\n", time);
		System.out.printf("Fish: %d\n", salmons.size());
		System.out.println();
	}
}
\end{lstlisting}
\item Class \texttt{Pond.java}
\lstset{language=java, tabsize=2}
\begin{lstlisting}
public class Pond {
	// Fields
	private static double radius = 100;
	// Instantiation should be forbidden
	private Pond() {}
	// Since radius of pond is fixed, setter for radius is not needed
	public static double getRadius() {
		return radius;
	}
}
\end{lstlisting}
\item Class \texttt{Fish.java}
\lstset{language=java, tabsize=2}
\begin{lstlisting}
public class Fish {
	// fields
	private static int numFish = 0;
	// attributes
	private double posRho;
	private double posAng;
	private double speed;
	// methods
	public void move() {
		double oldX = posRho * Math.cos(posAng);
		double oldY = posRho * Math.sin(posAng);
		while (true) {
			double theta = Math.random() * 2 * Math.PI;
			double newX = oldX + speed*Math.cos(theta);
			double newY = oldY + speed*Math.sin(theta);
			double newRho = Math.sqrt(Math.pow(newX,2) + Math.pow(newY,2));
			if (newRho < Pond.getRadius()) {
				posRho = newRho;
				posAng = Math.atan2(newY, newX);
				break;
			}
		}
	}
	// constructor
	public Fish() {
		posRho = Math.random() * Pond.getRadius();
		posAng = Math.random() * 2 * Math.PI;
		numFish++;
	}
	public Fish(double[] position) {
		posRho = Math.sqrt(Math.pow(position[0], 2) + Math.pow(position[1], 2));
		posAng = Math.atan2(position[1], position[0]);
		numFish++;
	}
	// getter and setter for position
	public void setSpeed(double speed) {
		this.speed = speed;
	}
	public static int getNumFish() {
		return numFish;
	}
	public double[] getPosition() {
		double[] position = new double[2];
		position[0] = posRho * Math.cos(posAng);
		position[1] = posRho * Math.sin(posAng);
		return position;
	}
}
\end{lstlisting}
\item Class \texttt{Salmon.java}
\lstset{language=java, tabsize=2}
\begin{lstlisting}
public class Salmon extends Fish {
	private static int numSalmons = 0;
	public Salmon() {
		super();
		setSpeed(10);
		numSalmons++;
	}
	public Salmon(double[] position) {
		super(position);
		setSpeed(10);
		numSalmons++;
	}
	public static int getNumSalmons() {
		return numSalmons;
	}
}
\end{lstlisting}
\item Class \texttt{Shark.java}
\lstset{language=java, tabsize=2}
\begin{lstlisting}
public class Shark extends Fish{
	private static int numSharks = 0;
	private int hunger = 0;
	private double range = 20;
	public Shark() {
		super();
		setSpeed(40);
		numSharks++;
	}
	public void move() {
		super.move();
		hunger++;
	}
	public static int getNumSharks() {
		return numSharks;
	}
	public int getHunger() {
		return hunger;
	}
	public void eat() {
		hunger = 0;
	}
	public double getRange() {
		return range;
	}
}
\end{lstlisting}
\end{enumerate}

\end{document}


\documentclass[a4paper,12pt]{article}
\usepackage{graphicx}
\usepackage{enumerate}
\usepackage{geometry}
\usepackage{times}
\geometry{legalpaper, portrait, margin=.75in}

\begin{document}

\begin{flushright}

\vspace{1.1cm}

{\bf\Huge Lab 2}

\rule{0.25\linewidth}{0.5pt}

\vspace{0.5cm}
%Put Authors
Justin Ely
\linebreak
\newline
%Put Author's affiliations
\footnotesize{615.202.81.FA15 Data Structures \\}
\vspace{0.5cm}
% Date here below
27 October, 2015
\end{flushright}

\noindent\rule{\linewidth}{1.0pt}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Comments}
In many senses, a matrix is just a specialized array with different (and additional) mathematical operations.  Thinking this way,
it's a seamless transition to implement a matrix type with an underlying array.  Random access to the array gives the power to quickly,
$O(1)$, access elements as is necessary in many of the matrix functions.  Additionally, the inherent multi-dimensional shape of the
matrix can be mirrored in the array, something not available in other data structures.  In other 1D structures, the 2D indices would need
to be stored as metadata to the actual data values themselves instead of being inherent in the array indices.

However, beyond the natural extension of array to matrix, the core array data structure lends little benefit to some of the most costly
methods implemented.  The Minor method, used as part of the Determinate, requires iterating over every element of the input array
and thus gains no benefit from the random-access of the array.  Such an implementation would see similar performance from a
linked implemenation, where iterating is a necessity.

\subsection{Iterative}
An iterative solution isn't very well suited to this problem as the solution is well-expressed (and given) recursively.  This means
an iterative solution needs to be worked out and tested from the initial recursive definition, which takes additional time and effort.

However, in this case an iterative solution could be done without too much trouble: iterating from the initial order down to 1, removing
rows/columns and continuing to add terms to the determinate calculation.  In this case, I would strongly consider a linked-list implementation.
A linked structure would allow deletion of all elements in a row/column on each iteration, whereas this would be much more difficult in an array of fixed size.
This would decrease the space complexity, as the space required would actually decrease with each successive call, but increase the
time complexity as some terms in the determinate calculation benefit from random access and would now require iteration.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Design}
Much of the design was limited by the supplied requirements: an array data structure using a recursive call for calculating the determinate.  However, limitations and enhancements to the specific design employed are discussed below.

\subsection{Limitations}
The Matrix class is limited to only integer types of the standard int precision.  A matrix, in abstract, is not limited to integers and works just as
well with floating-point numbers.
For the current purposes, the required input was limited to integers, so only an integer constructor and methods were added.

The Determinate method, however, is limited by long-integer precision.  For orders beyond 6, development quickly showed that
the standard int precision wasn't sufficient.  Therefore a long-int precision was implemented instead.  However, this still limits the final
determinate calculation, though to a higher value than int.

As discussed in the Efficiency section, the current algorithms employed are a major limiting factor in the actual usefulness of the
current implementation.  The determinate for matrices of order 13 take $\sim$30 minutes to calculate, which severely limits the usefulness
of the code.

\subsection{Enhancements}
Though not entirely an enhancement, the given requirements only specify a maximum matrix order of 6, and the design employed
allows an arbitrary matrix size.  This arbitrary size is only realistically useful up to order $\sim$10, after which the evaluation time
becomes prohibitive.

Arguably an enhancement, the main driver's file parsing is able to handle very poorly formatted input matrices.  The only requirement
is for a first integer $(n)$ followed by $n^2$ integers.  The first integer is evaluated as the matrix order, and the following values are
inserted in row-major order.  This means they could be input all as a single line, each on newlines, or anything in-between so long as
the number of values is equal to $o^2$.  This is only {\it arguably} an enhancement as, in certain circumstances, this could be seen as
a limitation in not checking for properly formatted row/column data.  For the current requirements, this was deemed not a concern, and in
many implementations such a flexible input format could be a benefit.

An additional enhancement was the implementation of a Print() method for the matrix.  Very useful for debugging and
verification, the simple method was perhaps the most useful addition during the development. phase

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Efficiency}
The efficiency of the determinate calculation is limited most by the method employed to find the minor matrix, as the
random access of the array implementation allows O(1) operations for other two terms of the Determinate calculation.
The Minor method needs to iterate over every element of the input matrix, copying it's value into the output matrix (while
skipping the row, column for which the matrix is to be determinate).  Thus, a single minor call is of $O(n^2)$, but a
recursive call to Determinate will need to calculation the minor N times.

To test the real-world complexity of the determinate calculation, timing functionality was added to test the actual time required
to calculate the determinate of each order. The results, as well as a fit to the data, can be seen in Figure 1.  From this analysis, we
can see that the time-complexity is exponential, $O(10^n)$, and grows significantly which each successive order.
 For the system used for testing (2.8Ghz Intel Core i7), the determinate of the matrix order 13 took $\sim$30 minutes to evaluate.

\begin{figure}[h]
    \centering
    \includegraphics[width=.9\textwidth]{time_complexity.pdf}
    \caption{Time complexity of the determinate method.  Top subplot shows the data and fit on a linear y-scale, where the bottom plot shows
    the same data on a logarithmic y-scale.  Note the variation in behavior at low-order.}
    \label{fig:mesh1}
\end{figure}

Real-world evaluation of space complexity was difficult to evaluate directly, so a theoretical calculation was done instead.  Each call to the
minor method created a new array of $(n-1) * (n-1)$ elements, which leads to a growth in required size with each step.
This could potentially be made more efficient by implementing a strategy that doesn't require creating a new copy of the array for
each recursive call.  Something that masked-out the deleted rows/columns could perform the same calculation with a constant space
requirement.

\begin{figure}[h]
    \centering
    \includegraphics[width=.9\textwidth]{space_complexity.pdf}
    \caption{Space complexity of the determinate method.  The behavior shown is calculated, not observed, and is well fit by
     an $O(n^3)$ polynomial.}
    \label{fig:mesh1}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{What I learned}
In this lab, the most impactful thing I learned was the true, real-world, cost of exponential growth algorithms.  A small matrix
determinate can be evaluated in nanoseconds, but increase just a single order of magnitude and nanoseconds quickly becomes
minutes, hours, and days.  Thus, the algorithm employed here quickly becomes useless in any real-world setting beyond a moderate
sized matrix.  Spending time on finding the most efficient algorithm can save orders of magnitude more time down the road.  A corollary
of this is that small-number tests can hide the inefficiency of algorithms, and special attention must be paid to fully exercise an implementation.
If my tests never went beyond order 6, this cost would not have been fully understood.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Next time}
In a future implementation, I'd like to add more methods to the matrix object to support common operations.  Operations such
as transpose, add, subtract, multiply, and divide are commonly used methods that would make the matrix much more useful
to an end-user.  Additionally, allowing floating-point numbers in the matrix would be a considerable advantage to real-word use.

Perhaps the most useful change I'd like to make would be to the Minor method itself.  The algorithm implemented here is
very brute-force, with no attempt at optimization, and is very slow for even moderately sized matrices.  Next time, I'd devote
more effort into researching and implementing alternate Minor algorithms that have better time efficiency (should one exist).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\end{document}


\documentclass[main.tex]{subfiles}
\begin{document}

\section*{Thu Jan 09 2020}

Many of these topics will be discussed in more details in the course by Nicola Bartolo on cosmological perturbations and in the course by Giacomo Ciani on gravitational waves.

Today we will focus on the LISA detector, which works for frequencies \(f \sim \num{e-5} \divisionsymbol \SI{e-1}{Hz}\), and on the cosmological stochastic background of GWs (CSGWB).

We can make an analogy with the CMB: we expect to see a background of gravitational radiation coming from all directions.

The stochastic background can have different origins:
\begin{enumerate}
  \item astrophysical origin: a coherent superposition of a large number of astrophysical sources, which are too weak to be detected separately;
  \item cosmological origin: it is generated in the early universe by a variety of mechanisms:
  \begin{enumerate}
    \item amplification of primordial tensor fluctuations via inflation;
    \item GWs from phase transition around the \SI{}{TeV} scale (this is not the only energy scale at which they can be emitted, but it is the energy we'd need in order to detect them with LISA);
    \item GWs from topological defects.
  \end{enumerate}
\end{enumerate}

A stochastic background of cosmological origin is expected to be
\begin{enumerate}
  \item isotropic;
  \item stationary;
  \item unpolarized, which means that the cross and plus polarizations will have the same amplitude.
\end{enumerate}

Let us discuss the main properties of the frequency spectrum: it is characterized
\begin{enumerate}
  \item in terms of (normalized) energy density per unit logarithmic interval of frequency: this is called \(h_0^2 \Omega_{GW} (f)\);
  \item in terms of the spectral density of the ensemble average of the Fourier component of the metric \(S_{h}(f)\);
  \item more on the experimental side: in terms of a characteristic amplitude of the stochastic background \(h_{c} (f)\).
\end{enumerate}

Let us define these three quantities. The energy density is given by
%
\begin{align}
\Omega_{GW} = \frac{1}{\rho_{c}} \dv{\rho_{GW} }{\log f}
\,,
\end{align}
%
where \(\rho_{GW}\) is the energy density of SGWB, \(f\) is the frequency, while \(\rho_{c}\) is the present value of the critical energy density, defined as
%
\begin{align}
\rho_{c} = \frac{3 H_0^2}{8 \pi G }
\,,
\end{align}
%
where we usually write \(H_0 = h_0 \times \SI{100}{km / (s Mpc)}\).

So, usually we plot \(h_0^2 \Omega_{GW}\) in order to ignore the uncertainties on the measurements of \(H_0 \).

We write stochastic GW at a given point \(\vec{x} = 0 \) in the transverse traceless (TT) gauge: \(h_{ii} = 0\) and \(\partial_{i} h^{ij}\): we get
%
\begin{align}
h_{ab}(t) = \sum _{A = +, \times } \int_{\mathbb{R}} \dd{f} \int_{S^{2}} \dd{\Omega } \hat{h}_A (f, \Omega ) \exp( - 2 \pi i f t ) e^{A}_{ab} (\hat{\Omega})
\,,
\end{align}
%
where we must have the condition \(\hat{h}_A (f, \Omega ) = \hat{h}^{*}_{A} (-f, \Omega )\). Here \(\hat{\Omega}\) is a unit vector representing the direction of propagation of the wave, and \(\dd{\hat{\Omega}} = \dd{\cos(\theta )} \dd{\phi }\), \(e^{A}_{ab}\) are the polarization tensors:
\begin{enumerate}
  \item \(e^{+}_{ab} (\hat{\Omega}) = 2\hat{m}_{[a} \hat{m}_{b]}\) ;
  \item \(e^{ \times }_{ab} (\hat{\Omega}) = 2\hat{m}_{(a} \hat{m}_{b)}\);
\end{enumerate}
where \(\hat{m}_{a, b}\) are unit vectors orthogonal to each other and to the propagation direction.

We have the condition \(e^{A}_{ab} e^{B ab} = 2 \delta^{AB}\).

Then, assuming a SGWB which is isotropic, unpolarized and stationary we will have
%
\begin{align}
\expval{\hat{h}^{*}_{A} (f, \hat{\Omega}) \hat{h}_{A'} (f',  \hat{\Omega }' )} = \delta (f- f') \frac{1}{4 \pi } \delta^{(2)}( \hat{\Omega}, \hat{\Omega }' ) \delta_{A A'} \frac{1}{2} S_{h}(f)
\,,
\end{align}
%
where
%
\begin{align}
\delta^{(2)} (\hat{\Omega}, \hat{\Omega}')= \delta(\phi - \phi') \delta(\cos(\theta ) - \cos(\theta' ))
\,.
\end{align}

The factors of \(1 / 4 \pi \) and \(1 / 2\) are for convention, for normalization, so that \(\int S_{h} (f) \dd{f} = 1\).

Then the remaining bit, \(S_{h}(f)\), is called the spectral density.

So, using the equaitions we found so far, we get
%
\begin{subequations}
\begin{align}
\expval{h_{ab}(t) h^{ab}(t)} &= 2 \int_{\mathbb{R}} \dd{f} S_h (f)  \\
&= 4 \int_{f=0}^{f= \infty } \dd{\log f} f S_h (f)
\,.
\end{align}
\end{subequations}

We define the characteristic amplitude \(h_{c}(f)\) as
%
\begin{align}
\expval{h_{ab} (t) h^{ab } (t)} = 2 \int_{f=0}^{f = \infty} \dd{\log f} h_c^2 (f)
\,,
\end{align}
%
which means that \(h_c^2(f) = 2 f S_h(f)\).

The last step is to relate \(h_c(f)\) and \(h_0^2 \Omega_{GW}(f)\).

The energy density is defined as:
%
\begin{align}
\rho_{GW} =  \frac{1}{32 \pi G} \expval{\dot{h}_{ab} \dot{h}^{ab} }
\,,
\end{align}
%
where the average is performed over a wavelength, but by the ergodic theorem it can also be performed over a period.
So,
%
\begin{align}
\rho_{GW} = \frac{4}{32 \pi G} \int_{f=0}^{f= \infty } \dd{(\log f)} f (2 \pi f)^2 S_h (f)
\,,
\end{align}
%
whicm means that
%
\begin{subequations}
\begin{align}
\dv{\rho_{GW}}{\log f} &= \frac{\pi }{4 G} f^2 h_c^2(f)  \\
&= \frac{\pi}{2G} f^3 S_h (f)
\,,
\end{align}
\end{subequations}
%
so in the end we have
%
\begin{align}
\Omega_{GW} (f) = \frac{2 \pi^2}{3 H_0^2} f^2 h_c^2 (f)
\,,
\end{align}
%
therefore
%
\begin{align}
\Omega_{GW}(f) = \frac{4 \pi^2}{3 H_0^2} f^3 S_h (f)
\,.
\end{align}

The \emph{strain} is the quantity \(S_h\).

An advantage of GWs is the fact that they decouple right after emission: they maintain the spectral shape.

If we start from the Einstein-Hilbert action
%
\begin{align}
S = \int \dd[4]{x} F g \frac{M_P^2}{2} R
\,,
\end{align}
%
and plug in a tensor-perturbed FRLW metric:
%
\begin{align}
\dd{s^2} = - a^2 \qty(\dd{\eta^2  } + (\delta_{ij} + h_{ij}) \dd{\vec{x}}^2)
\,,
\end{align}
%
we get the equations of motion for gravitational waves.
The scenario of slow-roll inflation gives rise to an energy spectrum which is unobservable with LISA as well as ground-based detectors.

If we add an axion to the inflaton Lagrangian:
%
\begin{align}
\mathcal{L} = \frac{1}{2} \partial_{\mu } \phi \partial^{\mu }\phi + V(\phi ) + \underbrace{\frac{\phi}{\Lambda } F_{\mu \nu } \widetilde{F}^{\mu \nu }}_{\text{Additional term}}
\,,
\end{align}
%
we can see enhanced gravitational waves, which we hope would be detectable with LISA! However the enhancement is very small, we cannot see it.

Chaotic preheating models predict higher amplitudes, however they are at very high frequencies.

We can get observable signals by fine-tuning the parameters, unlikely scenario.

What about phase transition, the collisions of primordial vacuum bubbles? We can get numerical estimates on the spectral shape of this signal, and they seem promising and detectable.

What about cosmic defects? We can have Domain Walls, Cosmic Strings and Cosmic Monopoles.

The prediction here is a flat spectrum at observable frequencies, at possibly observable amplitudes depending on the emission time.

\end{document}


As a systems language, Rust operates at a low level. If you're coming from a high-level language, there are some aspects of systems
programming that you may not be familiar with. The most important one is how memory works, with a stack and a heap. If you're familiar
with how C-like languages use stack allocation, this chapter will be a refresher. If you're not, you'll learn about this more general
concept, but with a Rust-y focus.

\blank

As with most things, when learning about them, we'll use a simplified model to start. This lets you get a handle on the basics, without
getting bogged down with details which are, for now, irrelevant. The examples we'll use aren't 100\% accurate, but are representative for
the level we're trying to learn at right now. Once you have the basics down, learning more about how allocators are implemented, virtual
memory, and other advanced topics will reveal the leaks in this particular abstraction.

\subsection*{Memory management}

These two terms are about memory management. The stack and the heap are abstractions that help you determine when to allocate and
deallocate memory.

\blank

Here's a high-level comparison:

\blank

The stack is very fast, and is where memory is allocated in Rust by default. But the allocation is local to a function call, and
is limited in size. The heap, on the other hand, is slower, and is explicitly allocated by your program. But it's effectively
unlimited in size, and is globally accessible.

\subsection*{The Stack}

Let's talk about this Rust program:

\begin{rustc}
fn main() {
    let x = 42;
}
\end{rustc}

This program has one variable binding, \x. This memory needs to be allocated from somewhere. Rust 'stack allocates' by default, which means that basic values 'go on the stack'. What does that mean?

\blank

Well, when a function gets called, some memory gets allocated for all of its local variables and some other information. This is called a
'stack frame', and for the purpose of this tutorial, we're going to ignore the extra information and only consider the local variables
we're allocating. So in this case, when \code{main()} is run, we'll allocate a single 32-bit integer for our stack frame. This is
automatically handled for you, as you can see; we didn't have to write any special Rust code or anything.

\blank

When the function exits, its stack frame gets deallocated. This happens automatically as well.

\blank

That's all there is for this simple program. The key thing to understand here is that stack allocation is very, very fast. Since we know
all the local variables we have ahead of time, we can grab the memory all at once. And since we'll throw them all away at the same time as
well, we can get rid of it very fast too.

\blank

The downside is that we can't keep values around if we need them for longer than a single function. We also haven't talked about what
the word, 'stack', means. To do that, we need a slightly more complicated example:

\begin{rustc}
fn foo() {
    let y = 5;
    let z = 100;
}

fn main() {
    let x = 42;

    foo();
}
\end{rustc}

This program has three variables total: two in \code{foo()}, one in \code{main()}. Just as before, when \code{main()} is called, a single
integer is allocated for its stack frame. But before we can show what happens when \code{foo()} is called, we need to visualize what's
going on with memory. Your operating system presents a view of memory to your program that's pretty simple: a huge list of addresses, from
0 to a large number, representing how much RAM your computer has. For example, if you have a gigabyte of RAM, your addresses go from \code{0}
to \code{1,073,741,823}. That number comes from $2^{30}$, the number of bytes in a gigabyte.\footnote{'Gigabyte' can mean two things:
$10^{9}$, or $2^{30}$. The SI standard resolved this by stating that 'gigabyte' is $10^{9}$, and 'gibibyte' is $2^{30}$. However, very
few people use this terminology, and rely on context to differentiate. We follow in that tradition here.}

\blank

This memory is kind of like a giant array: addresses start at zero and go up to the final number. So here's a diagram of our first
stack frame:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    0 & x & 42 \\
    \hline
  \end{tabular}
\end{table}

We've got \x\ located at address \code{0}, with the value \code{42}.

When \code{foo()} is called, a new stack frame is allocated:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    2 & z & 100 \\
    \hline
    1 & y & 5 \\
    \hline
    0 & x & 42 \\
    \hline
  \end{tabular}
\end{table}

Because \code{0} was taken by the first frame, \code{1} and \code{2} are used for \code{foo()}'s stack frame. It grows upward, the more
functions we call.

\blank

There are some important things we have to take note of here. The numbers 0, 1, and 2 are all solely for illustrative purposes, and
bear no relationship to the address values the computer will use in reality. In particular, the series of addresses are in reality
going to be separated by some number of bytes that separate each address, and that separation may even exceed the size of the value
being stored.

\blank

After \code{foo()} is over, its frame is deallocated:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    0 & x & 42 \\
    \hline
  \end{tabular}
\end{table}

And then, after \code{main()}, even this last value goes away. Easy!

\blank

It's called a 'stack' because it works like a stack of dinner plates: the first plate you put down is the last plate to pick back up.
Stacks are sometimes called 'last in, first out queues' for this reason, as the last value you put on the stack is the first one you
retrieve from it.

\blank

Let's try a three-deep example:

\begin{rustc}
fn italic() {
    let i = 6;
}

fn bold() {
    let a = 5;
    let b = 100;
    let c = 1;

    italic();
}

fn main() {
    let x = 42;

    bold();
}
\end{rustc}

We have some kooky function names to make the diagrams clearer.

\blank

Okay, first, we call \code{main()}:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    0 & x & 42 \\
    \hline
  \end{tabular}
\end{table}

Next up, \code{main()} calls \code{bold()}:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    \textbf{3} & \textbf{c} & \textbf{1} \\
    \hline
    \textbf{2} & \textbf{b} & \textbf{100} \\
    \hline
    \textbf{1} & \textbf{a} & \textbf{5} \\
    \hline
    0 & x & 42 \\
    \hline
  \end{tabular}
\end{table}

And then \code{bold()} calls \code{italic()}:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    \textit{4} & \textit{i} & \textit{6} \\
    \hline
    \textbf{3} & \textbf{c} & \textbf{1} \\
    \hline
    \textbf{2} & \textbf{b} & \textbf{100} \\
    \hline
    \textbf{1} & \textbf{a} & \textbf{5} \\
    \hline
    0 & x & 42 \\
    \hline
  \end{tabular}
\end{table}

Whew! Our stack is growing tall.

\blank

After \code{italic()} is over, its frame is deallocated, leaving only \code{bold()} and \code{main()}:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    \textbf{3} & \textbf{c} & \textbf{1} \\
    \hline
    \textbf{2} & \textbf{b} & \textbf{100} \\
    \hline
    \textbf{1} & \textbf{a} & \textbf{5} \\
    \hline
    0 & x & 42 \\
    \hline
  \end{tabular}
\end{table}

And then \code{bold()} ends, leaving only \code{main()}:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    0 & x & 42 \\
    \hline
  \end{tabular}
\end{table}

And then we're done. Getting the hang of it? It's like piling up dishes: you add to the top, you take away from the top.

\subsection*{The Heap}

Now, this works pretty well, but not everything can work like this. Sometimes, you need to pass some memory between different
functions, or keep it alive for longer than a single function's execution. For this, we can use the heap.

\blank

In Rust, you can allocate memory on the heap with the \href{https://doc.rust-lang.org/std/boxed/}{Box<T> type}. Here's an example:

\begin{rustc}
fn main() {
    let x = Box::new(5);
    let y = 42;
}
\end{rustc}

Here's what happens in memory when \code{main()} is called:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    1 & y & 42 \\
    \hline
    0 & x & ?????? \\
    \hline
  \end{tabular}
\end{table}

We allocate space for two variables on the stack. \y\ is \code{42}, as it always has been, but what about \x? Well, \x\ is a
\code{Box<i32>}, and boxes allocate memory on the heap. The actual value of the box is a structure which has a pointer to 'the heap'.
When we start executing the function, and \code{Box::new()} is called, it allocates some memory for the heap, and puts \code{5} there.
The memory now looks like this:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 5 \\
    \hline
    \ldots & \ldots & \ldots \\
    \hline
    1 & y & 42 \\
    \hline
    0 & x & 42 \\
    \hline
  \end{tabular}
\end{table}

We have ($2^{30}$) - 1 addresses in our hypothetical computer with 1GB of RAM. And since our stack grows from zero, the easiest place
to allocate memory is from the other end. So our first value is at the highest place in memory. And the value of the struct at \x\ has a
raw pointer (see \nameref{sec:syntax_rawPointers}) to the place we've allocated on the heap, so the value of \x\ is ($2^{30}$) - 1, the
memory location we've asked for.

\blank

We haven't really talked too much about what it actually means to allocate and deallocate memory in these contexts. Getting into very
deep detail is out of the scope of this tutorial, but what's important to point out here is that the heap isn't a stack that grows from
the opposite end. We'll have an example of this later in the book, but because the heap can be allocated and freed in any order, it can
end up with 'holes'. Here's a diagram of the memory layout of a program which has been running for a while now:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 5 \\
    \hline
    ($2^{30}$) - 2 & & \\
    \hline
    ($2^{30}$) - 3 & & \\
    \hline
    ($2^{30}$) - 4 & & 40 \\
    \hline
    \ldots & \ldots & \ldots \\
    \hline
    3 & y & $\rightarrow(2^{30})$-4 \\
    \hline
    2 & y & 42 \\
    \hline
    1 & y & 42 \\
    \hline
    0 & x & $\rightarrow(2^{30})$-1 \\
    \hline
  \end{tabular}
\end{table}

In this case, we've allocated four things on the heap, but deallocated two of them. There's a gap between ($2^{30}$) - 1 and ($2^{30}$) - 4
which isn't currently being used. The specific details of how and why this happens depends on what kind of strategy you use to manage the
heap. Different programs can use different 'memory allocators', which are libraries that manage this for you. Rust programs use
\href{http://www.canonware.com/jemalloc/}{jemalloc} for this purpose.

\blank

Anyway, back to our example. Since this memory is on the heap, it can stay alive longer than the function which allocates the box. In
this case, however, it doesn't.\footnote{We can make the memory live longer by transferring ownership, sometimes called 'moving out of
the box'. More complex examples will be covered later.} When the function is over, we need to free the stack frame for \code{main()}.
\code{Box<T>}, though, has a trick up its sleeve: Drop (see \nameref{sec:syntax_drop}). The implementation of \code{Drop} for \code{Box}
deallocates the memory that was allocated when it was created. Great! So when \x\ goes away, it first frees the memory allocated on the
heap:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    1 & y & 42 \\
    \hline
    0 & x & ?????? \\
    \hline
  \end{tabular}
\end{table}

And then the stack frame goes away, freeing all of our memory.

\subsection*{Arguments and borrowing}

We've got some basic examples with the stack and the heap going, but what about function arguments and borrowing? Here's a small
Rust program:

\begin{rustc}
fn foo(i: &i32) {
    let z = 42;
}

fn main() {
    let x = 5;
    let y = &x;

    foo(y);
}
\end{rustc}

When we enter \code{main()}, memory looks like this:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    1 & y & $\rightarrow$ 0 \\
    \hline
    0 & x & 5 \\
    \hline
  \end{tabular}
\end{table}

\x\ is a plain old \code{5}, and \y\ is a reference to \x. So its value is the memory location that \x\ lives at, which in this
case is \code{0}.

\blank

What about when we call \code{foo()}, passing \y\ as an argument?

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    3 & z & 42 \\
    \hline
    2 & i & $\rightarrow$ 0 \\
    \hline
    1 & y & $\rightarrow$ 0 \\
    \hline
    0 & x & 5 \\
    \hline
  \end{tabular}
\end{table}

Stack frames aren't only for local bindings, they're for arguments too. So in this case, we need to have both \code{i}, our argument,
and \z, our local variable binding. \code{i} is a copy of the argument, \y. Since \code{y}'s value is \code{0}, so is \code{i}'s.

\blank

This is one reason why borrowing a variable doesn't deallocate any memory: the value of a reference is a pointer to a memory location.
If we got rid of the underlying memory, things wouldn't work very well.

\subsection*{A complex example}

Okay, let's go through this complex program step-by-step:

\begin{rustc}
fn foo(x: &i32) {
    let y = 10;
    let z = &y;

    baz(z);
    bar(x, z);
}

fn bar(a: &i32, b: &i32) {
    let c = 5;
    let d = Box::new(5);
    let e = &d;

    baz(e);
}

fn baz(f: &i32) {
    let g = 100;
}

fn main() {
    let h = 3;
    let i = Box::new(20);
    let j = &h;

    foo(j);
}
\end{rustc}

First, we call \code{main()}:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 20 \\
    \hline
    \ldots & \ldots & \ldots \\
    \hline
    2 & j & $\rightarrow$ 0 \\
    \hline
    1 & i & $\rightarrow$ ($2^{30}$)-1 \\
    \hline
    0 & h & 3 \\
    \hline
  \end{tabular}
\end{table}

We allocate memory for \code{j}, \code{i}, and \code{h}. \code{i} is on the heap, and so has a value pointing there.

\blank

Next, at the end of \code{main()}, \code{foo()} gets called:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 20 \\
    \hline
    \ldots & \ldots & \ldots \\
    \hline
    5 & z & $\rightarrow$ 4 \\
    \hline
    4 & y & 10 \\
    \hline
    3 & x & $\rightarrow$ 0 \\
    \hline
    2 & j & $\rightarrow$ 0 \\
    \hline
    1 & i & $\rightarrow$ ($2^{30}$)-1 \\
    \hline
    0 & h & 3 \\
    \hline
  \end{tabular}
\end{table}

Space gets allocated for \x, \y, and \z. The argument \x\ has the same value as \code{j}, since that's what we passed it in. It's
a pointer to the \code{0} address, since \code{j} points at \code{h}.

\blank

Next, \code{foo()} calls \code{baz()}, passing \z:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 20 \\
    \hline
    \ldots & \ldots & \ldots \\
    \hline
    7 & g & 100 \\
    \hline
    6 & f & $\rightarrow$ 4 \\
    \hline
    5 & z & $\rightarrow$ 4 \\
    \hline
    4 & y & 10 \\
    \hline
    3 & x & $\rightarrow$ 0 \\
    \hline
    2 & j & $\rightarrow$ 0 \\
    \hline
    1 & i & $\rightarrow$ ($2^{30}$)-1 \\
    \hline
    0 & h & 3 \\
    \hline
  \end{tabular}
\end{table}

We've allocated memory for \code{f} and \code{g}. \code{baz()} is very short, so when it's over, we get rid of its stack frame:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 20 \\
    \hline
    \ldots & \ldots & \ldots \\
    \hline
    5 & z & $\rightarrow$ 4 \\
    \hline
    4 & y & 10 \\
    \hline
    3 & x & $\rightarrow$ 0 \\
    \hline
    2 & j & $\rightarrow$ 0 \\
    \hline
    1 & i & $\rightarrow$ ($2^{30}$)-1 \\
    \hline
    0 & h & 3 \\
    \hline
  \end{tabular}
\end{table}

Next, \code{foo()} calls \code{bar()} with \x\ and \z:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 20 \\
    \hline
    ($2^{30}$) - 2 & & 5 \\
    \hline
    \hline
    \ldots & \ldots & \ldots \\
    \hline
    10 & e & $\rightarrow$ 9 \\
    \hline
    9 & d & $\rightarrow$ ($2^{30}$)-2 \\
    \hline
    8 & c & 5 \\
    \hline
    7 & b & $\rightarrow$ 4 \\
    \hline
    6 & a & $\rightarrow$ 0 \\
    \hline
    5 & z & $\rightarrow$ 4 \\
    \hline
    4 & y & 10 \\
    \hline
    3 & x & $\rightarrow$ 0 \\
    \hline
    2 & j & $\rightarrow$ 0 \\
    \hline
    1 & i & $\rightarrow$ ($2^{30}$)-1 \\
    \hline
    0 & h & 3 \\
    \hline
  \end{tabular}
\end{table}

We end up allocating another value on the heap, and so we have to subtract one from ($2^{30}$) - 1. It's easier to write that than
\code{1,073,741,822}. In any case, we set up the variables as usual.

At the end of \code{bar()}, it calls \code{baz()}:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 20 \\
    \hline
    ($2^{30}$) - 2 & & 5 \\
    \hline
    \ldots & \ldots & \ldots \\
    12 & g & 100 \\
    \hline
    11 & f & $\rightarrow$ ($2^{30}$)-2\\
    \hline
    10 & e & $\rightarrow$ 9 \\
    \hline
    9 & d & $\rightarrow$ ($2^{30}$)-2 \\
    \hline
    8 & c & 5 \\
    \hline
    7 & b & $\rightarrow$ 4 \\
    \hline
    6 & a & $\rightarrow$ 0 \\
    \hline
    5 & z & $\rightarrow$ 4 \\
    \hline
    4 & y & 10 \\
    \hline
    3 & x & $\rightarrow$ 0 \\
    \hline
    2 & j & $\rightarrow$ 0 \\
    \hline
    1 & i & $\rightarrow$ ($2^{30}$)-1 \\
    \hline
    0 & h & 3 \\
    \hline
  \end{tabular}
\end{table}

With this, we're at our deepest point! Whew! Congrats for following along this far.

\blank

After \code{baz()} is over, we get rid of \code{f} and \code{g}:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 20 \\
    \hline
    ($2^{30}$) - 2 & & 5 \\
    \hline
    \ldots & \ldots & \ldots \\
    \hline
    10 & e & $\rightarrow$ 9 \\
    \hline
    9 & d & $\rightarrow$ ($2^{30}$)-2 \\
    \hline
    8 & c & 5 \\
    \hline
    7 & b & $\rightarrow$ 4 \\
    \hline
    6 & a & $\rightarrow$ 0 \\
    \hline
    5 & z & $\rightarrow$ 4 \\
    \hline
    4 & y & 10 \\
    \hline
    3 & x & $\rightarrow$ 0 \\
    \hline
    2 & j & $\rightarrow$ 0 \\
    \hline
    1 & i & $\rightarrow$ ($2^{30}$)-1 \\
    \hline
    0 & h & 3 \\
    \hline
  \end{tabular}
\end{table}

Next, we return from \code{bar()}. \code{d} in this case is a \code{Box<T>}, so it also frees what it points to: ($2^{30}$) - 2.

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 20 \\
    \hline
    \ldots & \ldots & \ldots \\
    \hline
    5 & z & $\rightarrow$ 4 \\
    \hline
    4 & y & 10 \\
    \hline
    3 & x & $\rightarrow$ 0 \\
    \hline
    2 & j & $\rightarrow$ 0 \\
    \hline
    1 & i & $\rightarrow$ ($2^{30}$)-1 \\
    \hline
    0 & h & 3 \\
    \hline
  \end{tabular}
\end{table}

And after that, \code{foo()} returns:

\begin{table}[H]
  \begin{tabular}{|l|l|l|}
    \hline
    \textbf{Address} & \textbf{Name} & \textbf{Value} \\
    \hline
    ($2^{30}$) - 1 & & 20 \\
    \hline
    \ldots & \ldots & \ldots \\
    \hline
    2 & j & $\rightarrow$ 0 \\
    \hline
    1 & i & $\rightarrow$ ($2^{30}$)-1 \\
    \hline
    0 & h & 3 \\
    \hline
  \end{tabular}
\end{table}

And then, finally, \code{main()}, which cleans the rest up. When \code{i} is \code{Drop}ped, it will clean up the last of the heap too.

\subsection*{What do other languages do?}

Most languages with a garbage collector heap-allocate by default. This means that every value is boxed. There are a number of reasons
why this is done, but they're out of scope for this tutorial. There are some possible optimizations that don't make it true 100\% of
the time, too. Rather than relying on the stack and \code{Drop} to clean up memory, the garbage collector deals with the heap instead.

\subsection*{Which to use?}

So if the stack is faster and easier to manage, why do we need the heap? A big reason is that Stack-allocation alone means you only
have 'Last In First Out (LIFO)' semantics for reclaiming storage. Heap-allocation is strictly more general, allowing storage to be
taken from and returned to the pool in arbitrary order, but at a complexity cost.

Generally, you should prefer stack allocation, and so, Rust stack-allocates by default. The LIFO model of the stack is simpler,
at a fundamental level. This has two big impacts: runtime efficiency and semantic impact.

\myparagraph{Runtime Efficiency}

Managing the memory for the stack is trivial: The machine increments or decrements a single value, the so-called \enquote{stack pointer}.
Managing memory for the heap is non-trivial: heap-allocated memory is freed at arbitrary points, and each block of heap-allocated memory
can be of arbitrary size, so the memory manager must generally work much harder to identify memory for reuse.

\blank

If you'd like to dive into this topic in greater detail, \href{http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.143.4688}{this paper}
is a great introduction.

\myparagraph{Semantic impact}

Stack-allocation impacts the Rust language itself, and thus the developer's mental model. The LIFO semantics is what drives how the
Rust language handles automatic memory management. Even the deallocation of a uniquely-owned heap-allocated box can be driven by the
stack-based LIFO semantics, as discussed throughout this chapter. The flexibility (i.e. expressiveness) of non LIFO-semantics means that
in general the compiler cannot automatically infer at compile-time where memory should be freed; it has to rely on dynamic protocols,
potentially from outside the language itself, to drive deallocation (reference counting, as used by \code{Rc<T>} and \code{Arc<T>}, is
one example of this).

\blank

When taken to the extreme, the increased expressive power of heap allocation comes at the cost of either significant runtime support
(e.g. in the form of a garbage collector) or significant programmer effort (in the form of explicit memory management calls that
require verification not provided by the Rust compiler).


\input{preamble}

\begin{document}
    \section{Subjects}
    \begin{itemize}
        \item What is an evolutionary tree?
        \item Computing an evolutionary tree from distance data.
    \end{itemize}

    \section{Notes}

    \subsection{Phylogenetic trees}
    There is strong evidence that all life on earth is decended from a single
    common ancestor. Over the course of at least 3.8 billion years, that life
    form has changed and split itself into new and independent lineages. The
    evolutionary relationships among these species is referred to as their
    phylogeny, phylogenetic reconstruction is concerned with inferring the
    phylogeny of groups of organisms.

    We call these groups \textit{taxa} (or singular taxon), the splitting of
    lineages is called \textit{speciation} (think of splitting it into
    different species). Usually speciation happens if one population is split
    into two that can no longer interbreed (e.g. if a river splits them apart).
    We can use methods of phylogenetic to contemplate over the tree of life, or
    simply to inferr the phylogeny of different populations within a species.

    In a rooted phylogenetic tree $T$, the root node $r$ corresponds to the
    last common ancestor of all species in $T$, we then call a path from the
    root to a leaf an \textit{evolutionary path}. If an equal amount of change
    occurs on every evolutionary path (i.e. each species have the same amount
    of ``change'') then the evolutionary change occur in a more-or-less
    clocklike fashion and we then say that this tree satisfies the
    \textit{molecular clock hypothesis}, we can then assign a time $t(v)$ to
    every internal node $v$ in the tree and a length of $t(v)-t(w)$ to an edge
    $(v,w)$ in the tree.

    Every extant (still alive) species corresponds to time $0$, and a
    speciation event (internal node $v$) occured in the tree $t(v)$ time ago.
    We then have that the length of an edge represents the amount of time that
    lies between two speciation events.

    Furthermore, the last common ancestor (root) lived $t(r)$ and all
    evolutionary paths have the same length $t(r)$ where the length of a path
    is the sum of the lengths of all edges along the path.

    Once the \textit{molecular clock hypotheses} was widely accepted, but has
    since been disproven so now time instead refers to the expected amount of
    evolution.

    \subsubsection{Distance methods}
    Distance methods construct a phylogenetic tree from a distance matrix that
    contains the evolutionary distances between all pairs of taxa (groups of
    organisms). If we ignore edge weights, we speak of the topology (shape) of
    a tree, and it turns out that we have methods that \textit{provably} find
    the correct tree if the distance matrix is ultrametric, and some that works
    if the distance matrix is additive. However the data we can record is
    usually approximation of the true (additive) data, and thus we can't rely
    on the distance matrix being additive. However, it turns out that the
    methods can still find the correct \textit{topology} of the tree under
    certain criteria.

    If we just have the topology, then we must assign weights to the edges of
    the tree that best fit the data, which can be done with the least squares
    methods.

    \subsubsection{Basic definitions}
    Phylogenies are usually represented as binary trees, because generially
    speciation happens when one lineage splits into two independent lineages.
    This is not entirely correct, sometimes horizontal gene-transfer happens
    and hybrid speciation but it is rare. So for simplicity we just look deal
    with phylogenetic trees, however let's not insist they have to be binary
    for the moment:

    Let $S=\{s_1,\dots,s_n\}$ be a set of taxa. A phylogenetic tree on $S$ is a
    triple $T=(V,E,\alpha)$ where:
    \begin{itemize}
        \item $V$ is the set of nodes, $E$ is the set of undirected edges
        \item $(V,E)$ is a an acyclic connected graph, in which there might be
        a distinguished root node of degree $\geq 2$ and all other internal
        nodes have a degree $\geq 3$. So either rooted or unrooted. We will
        denote the set of leaves by $V_L$ and the set of internal nodes by $V_I$
        \item $\alpha$ is a bijection $\alpha: S \rightarrow V_L$ between the
        set of taxa and the set of leaves
    \end{itemize}
    An edge $(v,w) \in E$ is an external edge if either $v$ or $w$ is a leaf.
    Otherwise it is an internal edge.

    We haven't included edge-weights here, they will be introduced later.

    A semimetric on $S$ is a function $d:S\times S\rightarrow \Re_{\geq 0}$
    that satisfies, for all $x,y,z \in S$:
    \begin{align*}
        d(x,y) &= 0 \iff x=y\\
        d(x,y) &= d(y,x)
    \end{align*}
    A metric, or distance function on $S$ is a semimetric that satisfies the
    triangle inequality:
    \begin{equation*}
        d(x,y) \leq d(x,z) + d(z,y)
    \end{equation*}
    A metric $d:S\times S\rightarrow \Re_{\geq 0}$ is called \textit{additive}
    if it satisfies the \textit{additive inequality}:
    \begin{equation*}
        d(w,x)+d(y,z) \leq \max\{d(x,y)+d(w,z), d(x,z)+d(w,y)\}
    \end{equation*}
    An \textit{ultrametric} on $S$ is an additive metric that satisfies the
    \textit{ultrametric inequality}:
    \begin{equation*}
        d(x,y) \leq \max\{d(x,z),d(y,z)\}
    \end{equation*}

    A symmetric $n \times n$ matrix $D=(d_{ij})$ satisfying $d_{ii}=0$ and
    $d_{ij}>0$ for all $i\neq j$ with $i,j \in \{1,\dots,n\}$ is called a
    dissimilarity matrix. That is, the matrix is symmetric and the diagonal is
    all $0$s. We will now assume that the input to a phylogenetic
    reconstruction algorithm is such a dissimilarity matrix. We can now say
    that the dissimilarity matrix, induces a function $d:S \times S\rightarrow
    R_{\geq 0}$ defined by $d(x,y)=d_{xy}$. We see that this definition of a
    dissimilarity matrix implies that $d$ is semimetric.

    An $n \times n$ dissimilarity matrix $D=(d_{ij})$ is called a
    \textit{distance matrix} if the induced function $d$ is a \textit{distance
    function}. Furthermore, we say that the matrix $D$ is additive if the
    induced function $d$ is additive, and the matrix is ultrametrix if the
    induced function $d$ is ultrametric.

    \subsubsection{Least square methods}
    The simplest way to find the edge lengths for a tree topology $T$ is by
    selecting the edge lengths such that:
    \begin{equation*}
        Q(T)=\sum_{i=1}^{n}\sum_{j=1}^{n}(D_{ij}-d_{ij})^2
    \end{equation*}
    Where $d_{ij}$ is from the matrix and $D_{ij}$ is induced by the tree. We
    can solve this in \bigO{n^3} if we know the topology. If we don't know the
    topology, then it is NP-complete.

    \subsubsection{Ultrametric distance matrices and trees}
    Let $T=(V,E,\alpha)$ be a rooted phylogenetic tree on a set $S$ of taxa.
    then $T$ with a marking of the internal nodes with positive numbers,
    $\mu:V_I \rightarrow \Re_{>0}$, is an \textit{ultrametric tree} provided
    that for each path from the root $r$ to a leaf $k$ the sequence of marks
    $\mu(r), \cdots , \mu(k)$ is strictly decreasing.

    We define the \textit{lowest common ancestor} ($\LCA(v,w)$) as the first
    node $u$ where $v$ is in one subtree of $u$ and $w$ is in the other.

    Now let $S=\{1,\dots,n\}$ be a set of taxa, let $D$ be an $n \times n$
    dissimilarity matrix, and let $T=(V,E,\alpha,\mu)$ be an ultrametric tree
    on $S$. We say that $D$ and $T$ are consistent if $\mu(\LCA(i,j))=d_{ij}$
    holds true for any two leaves (taxa) $i$ and $j$ of $T$.

    An $n \times n$ dissimilarity matrix $D$ satisfies the \textit{3-point
    condition} if for all $i,j,k\in \{1,\dots,n\}$ the two largest values out
    of $d_{ik},d_{ij},d_{jk}$ are equal. Then we have that the following
    statements are equivalent:
    \begin{align*}
        &D \text{ is ultrametric} \\
        \iff &D \text{ satisfies the 3-point condition} \\
        \iff &D \text{ is consistent with an unique ultrametric tree}
    \end{align*}

    \subsubsection{The UPGMA-algorithm}
    UPGMA stands for \textit{unweigthed pair group method using arithmetic
    averages}. Clustering is the assigmenet of a set of observations into
    subsets (or clusters). There are in general two ways of performing
    clustering:
    \begin{itemize}
        \item Agglomerative, each observation starts in its own cluster, and we
        then merge the clusters that have the shortest distance (or highest
        similarity)
        \item Divisive, all observations starts in a single cluster, and at
        each step we divide a cluster into two siblings, in order to maximize
        the distance (minimize the similarity) between each cluster.
    \end{itemize}
    Usually, the way we measure the distance between two cluster $C_i,C_j$ is
    with one of the following methods:
    \begin{description}
        \item[Complete linkage] The maximum distance between elements of each
        cluster:
        \begin{equation*}
            d(i,j)=\max\{d_{xy} : x \in C_i, y\in C_j\}
        \end{equation*}
        \item[Single linkage] The minimum distance between elements of each
        cluster:
        \begin{equation*}
            d(i,j)=\min\{d_{xy} : x \in C_i, y\in C_j\}
        \end{equation*}
        \item[Average linkage] The mean distance between elements of each
        cluster (used in UPGMA):
        \begin{equation*}
            d(i,j) = \frac{1}{|C_i| \cdot |C_j|}\sum_{x\in C_i, y \in C_j}d_{xy}
        \end{equation*}
    \end{description}

    We now have the tools we need to describe the UPGMA algorithm which
    agglomerative:

    \begin{algorithm}
        \caption{UPGMA algorithm}\label{alg:upgma}
        \begin{algorithmic}
            \Procedure{UPGMA}{$D: n \times n$ matrix}
            \State \textit{// Initialization}
            \State Let $S=\{1,\dots,n\}$ be the set of taxa
            \State Each taxon $i\in S$ is a leaf in the tree $T$
            \State Each taxon $i\in S$ defines a cluster $C_i=\{i\}$ of size
            $|C_i|=1$.
            \State For $i,j\in S$ define $d(i,j)=d_{ij}$
            \State \textit{// Agglomeration}
            \While{$|S| \geq 2$}
                \State Determine $i,j \in S$ with $i \neq j$ such that $d(i,j)$
                is minimal
                \State Let $C_k=C_i\cup C_j$ be a new cluster of size $|C_k| =
                |C_i| + |C_j|$
                \State For each $l \in S \setminus \{i,j\}$ define:
                \begin{equation*}
                    d(k,l) = \frac{|C_i|\cdot d(i,l) + |C_j| + d(j,l)}{|C_i| +
                    |C_j|}
                \end{equation*}
                \State Set $S=(S \setminus \{i,j\}) \cup \{k\}$
                \State Add a new node $k$ to $T$ with mark $d(i,j)$
                \State Add edges from $k$ to $i$ and from $k$ to $j$ to the
                tree $T$
            \EndWhile
            \EndProcedure
            \State Output the tree $T$
        \end{algorithmic}
        Running time: \bigO{n^3}
    \end{algorithm}
    It is important to note here that the distance $d(k,l)$ is simply the
    average linkage between the newly merged cluster $C_i,C_j$ and the rest of
    the clusters.

    The sequence of marks generated in the $n-1$ iterations of the UPGMA
    algorithm is increasing, since (intuitionally) we start with finding the
    smallest distance and then, as we work our way up from the internal nodes
    that connect the leafs 'till we eventually hit the root, the distance will
    keep increasing, as the smallest distance becme bigger and bigger.

    Now the tree generated by the UPGMA algorithm produces the correct tree if
    the input is an ultrametric dissimilarity matrix $D$, however the algorithm
    described here only produces binary trees, so if the trees that is
    consistent with $D$ are all non-binary then we have to modify the
    algorithm. I will not explain this here.

    So this algorithm works for ultrametric trees, and if the molecular clock
    theorem holds then the distance matrix will be ultrametric (although the
    opposite does not hold)

    The UPGMA algorithm runs in \bigO{n^3}, however if we use quad-trees to
    find the minimum we can do it in \bigO{n^2}. Quad-trees recursively split
    the matrix into $4$ squares and constructs a tree where each node
    represents the minimum element of the $4$ subtrees. The quad-tree can be
    constructed in \bigO{n^2} and we can update it in \bigO{n}.

    \subsubsection{Neighbour joining}
    Neighbour joining is divisive in that we select two neighbour leaves, and
    join them by adding a parent to them. So in the beginning, all leaves are
    connected, and then we then select neighbours and join them. The generic NJ
    algorithm is the following:

    \begin{algorithm}
        \caption{Generic neighbour-joining algorithm}
        \begin{algorithmic}
            \Procedure{Neighbour-Join}{$D: n \times n$ where $n \geq 3$}
            \State \textit{// Initialization}
            \State Let $S = \{1,\dots,n\}$ be the set of taxa
            \State Each taxon $i$ is a leaf in the tree $T$
            \While{$|S|>3$}
                \State Using a specific neighbour selection criterion, select
                two taxa $i$ and $j$
                \State \textit{// $i$ and $j$ will be leaf neighbours in the
                (yet unknown) tree $T$}
                \State Add a new node $k$ to the tree $T$
                \State Choose an $m \in S \setminus \{i,j\}$ and add edges
                $(k,i)$ and $(k,j)$
                \State Compute the weight
                $\gamma(k,i)=\frac{1}{2}(d_{im}-d_{jm}+d_{ij})$
                \State Compute the weight
                $\gamma(k,j)=d_{ij}-\gamma(k,i)=\frac{1}{2}(d_{jm}-d_{im}+d_{ij})$
                \State
                \State Add the edges with the weights to the tree $T$
                \State Update the dissimilarity matrix by deleting the rows and
                columns
                \State corresponding to $i$ and $j$ and adding a new
                row and column for the
                \State new taxon $k$ with
                $d_{km}=\frac{1}{2}(d_{im}+d_{jm}-d_{ij})$ for all $m \in S
                \setminus \{i,j,k\}$
            \EndWhile
            \State Let $i,j,m$ be the remaining three taxa. Add a new internal
            node $v$ to the tree $T$ and add edges $(v,i), (v,j)$ and $(v,m)$
            to the tree $T$ with weights:
            \begin{align*}
                \gamma(v,i)&=\frac{d_{ij}+d_{im}-d_{jm}}{2}\\
                \gamma(v,j)&=\frac{d_{ij}+d_{jm}-d_{im}}{2}\\
                \gamma(v,i)&=\frac{d_{im}+d_{jm}-d_{ij}}{2}
            \end{align*}
            \State Output: The tree $T$
            \EndProcedure
        \end{algorithmic}
    \end{algorithm}

    The key point here is that if the selection criterion truly identifies leaf
    neighbours, then the generic NJ algorithm applied to the dissimilarity
    matrix $D$ will construct the additive binary tree which $D$ is consistent
    with.

    Neighbour joining works for both additive distance matrices, but for many
    others as well. It does not assume the existence of a molecular clock, and
    ensures that the clusters that are merged are both close to each other, and
    far apart from the rest.

    \subsubsection{Saitou and Nei}
    The most popular NJ algorithm, is due to Saitou and Nei, its neighbour
    selection crierion defines the matrix $N=(n_{ij})_{i,j\in S}$ as:
    \begin{equation*}
        n_{ij}=d_{ij}-(r_i+r_j)
    \end{equation*}
    where
    \begin{equation*}
        r_i = \frac{1}{|S|-2}\sum_{m \in S}d_{im}
    \end{equation*}
    If a dissimilarity matrix $D$ is consistent with an additive binary tree
    $T$ and $n_{ij}$ is a minimum entry in the corresponding $N$ matrix, then
    $i$ and $j$ are leaf neighbours in $T$.

    \begin{itemize}
        \item Computing the row sum $r_i$ for every $i$ takes \bigO{n^2} time
        \item Computing the matrix $N$ takes \bigO{n^2} time
        \item Finding the minimum $n_{ij}$ takes \bigO{n^2} time
        \item Adding node $k$ takes \bigO{1} time
        \item Adding two edges takes \bigO{1} time
        \item Updating the distance matrix takes \bigO{n} time
        \item Total running time: \bigO{n^3}
    \end{itemize}
\end{document}