%*****************************************
% Justin Ragatz
%
% 19 April 2016
%
% OnRamp Module for Monte Carlo Methods
%*****************************************

% Document options
\documentclass[a4paper, 11pt]{article}

\makeatletter
% Change the square brackets for each bibliography item from '[1]' to '1.'
\renewcommand\@biblabel[1]{\textbf{#1.}}

% Reduce the space between items in the itemize and enumerate environments and the bibliography
\renewcommand{\@listI}{\itemsep=0pt}

% Decimal section numbers
\renewcommand*\thesection{\arabic{section}.0}
\renewcommand*\thesubsection{\arabic{section}.\arabic{subsection}}

% Custom title
\renewcommand{\maketitle}
{
	\begin{flushright}
	{\LARGE\@title}

	\vspace{20pt}

	{\large\@author}

	\end{flushright}
}
\makeatother

% Header options
\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhf{}
\rhead{\small Monte Carlo Methods}
\lhead{\small OnRamp Module}
\rfoot{\small Page \thepage}

% Image options
\usepackage{graphicx}
\usepackage{float}
\usepackage{wrapfig}
\graphicspath{{images/}}

% Font options
\usepackage{mathpazo}
\usepackage[scaled]{helvet}
\usepackage{courier}
\normalfont
\usepackage[T1]{fontenc}
\usepackage{microtype}
\usepackage{url}
\urlstyle{same}
\usepackage{algpseudocode}
\usepackage{algorithm}
\usepackage{listings}
\usepackage{color}

% Title
\title{\textbf{Monte Carlo Methods}}
\author{\textsc{Justin Ragatz}}

\lstdefinestyle{customc}{
	belowcaptionskip=1\baselineskip,
	breaklines=true,
	frame=lines,
	xleftmargin=\parindent,
	language=C,
	showstringspaces=false,
	basicstyle=\footnotesize\ttfamily
}

\lstdefinestyle{customasm}{
	belowcaptionskip=1\baselineskip,
	frame=L,
	xleftmargin=\parindent,
	language=[x86masm]Assembler,
	basicstyle=\footnotesize\ttfamily
}

\lstset{escapechar=@,style=customc}

\begin{document}

% Title
\maketitle
\bigskip

% Module summary
\begin{description}
  \item[Summary] Use Monte Carlo methods to approximate the probabilities of various outcomes in card games and the roulette wheel.
  \item[Duration] 5 hours
  \item[Prerequisites] Familiarity with for-loops, rand(), and 1D arrays.
\end{description}

% Overview
\section{Overview}

Monte Carlo methods are a class of computational algorithms that use repeated random sampling to obtain numerical results. Typically, a single workhorse for-loop is used to generate the repeated and independent simulations. Because the numerous simulations are independent of each other, they can be easily split among multiple processing units. A problem such as this, where little effort is required to separate the problem into parallel tasks is known as "embarrassingly parallel". Thus, Monte Carlo methods are a convenient vehicle to introduce parallel concepts at the CS1/CS2 level.

By exposing students to parallel solutions at the introductory level, they come to expect parallel solutions throughout the curriculum. In particular, students are not overwhelmed by the sudden introduction of concurrency and similar topics in senior-level coursework. Upon completion of this module, students should be able to: 1) Identify embarrassingly parallel problems 2.) Understand the real-life applications of stochastic methods 3.) Explain at a high level how OpenMP works, and 4.) Measure the scalability of a parallel application.

Much of the material in this module has been adapted from the Monte Carlo Methods module written by Devin Bjelland and Libby Shoop\cite{website:shoop:2015}.

% Introducation to the problem
\section{Introduction to Monte Carlo Methods}

Monte Carlo methods are often employed when there is not a closed form or deterministic solution to the underlying problem. As this sort of problem is quite common, Monte Carlo methods are used in a wide variety of fields from computational chemistry to finance. While these topics are important, the typical CS1/CS2 student may be more interested in the more intriguing application of Monte Carlo methods to gambling and card games. The well known games of roulette and poker will both be used as the basis for the exercises in this module.

\subsection{How Computers Generate Random Numbers}

Algorithms that implement Monte Carlo methods require a source of randomness. When writing a serial algorithm, we can simply use the standard random number generator, for example, rand() in C. Computers (unless specifically fitted with a chip which gathers randomness from some other source) cannot produce true random numbers. Rather, the standard library random number generator takes some relatively random 32-bit or 64-bit integer input, called the seed, and transforms it sequentially to produce a stream of pseudo-random numbers. A random number generator is created with the seed as input to a function called srand(), and each time a new random number is requested using the rand() function, the integer pattern is changed to produce a new number. The sequence of numbers produced by such a pseudo-random number generation algorithm try to approximate a uniform distribution of numbers, each one statistically independent from the others.

However, it is important to realize that for a given input seed a pseudo-random number generator function such as rand() creates the same sequence of numbers every time. In C and C++, the function time() is often used to create the seed integer, because it returns the number of seconds since January 1, 1970. When running a sequential program multiple times, this seed would be different each time the program was run and the pattern of random numbers generated would be different.

\subsection{Testing out random number generators}

A simple way to test a random number generator is to simulate flipping a coin for a large number of trials.

We will use srand() to seed our random number generator with a large integer and then make numerous calls to rand\_r() to obtain a series of random integers. If the integer is even, we call it a "head", otherwise it is a "tail". This code sets up trials of coin flips with ever increasing numbers of flips. It also calculates the Chi Square statistic using the number of heads and tails. Generally, a Chi-Square value of 3.8 or less is considered a good random distribution of "heads" and "tails". We want to verify that the random number generator provides such an independent distribution.

\paragraph{Coding Exercise:}
Please refer to the sequential coin-flipping program in the accompanying files to run this test for yourself.

% Introduction to Parallelism
\section{Introduction to Parallelism}

We can make our coin flipping simulation run much faster by making use of the multiple cores available in modern CPUs. This type of code is known as parallel code, because we can designate portions of our program to run concurrently in parallel on different cores, computing part of the overall solution. In the case of flipping a coin, we can intuitively sense that it might be simple enough to designate that each core we have available could carry out some portion of the flips independently while other cores were taking care of the rest of the needed flips.

\subsection{Parallel Code with Threads}

A common mechanism for running code on multiple cores simultaneously is to start threads that can execute part of our code independently and in parallel on separate cores, sharing data values in memory if needed. When a program using threads begins execution, it is always running on a single main thread, which we conceptually label as thread 0. Then within the code we can designate that more threads should start executing in parallel along with thread 0. We call a point in the code where multiple threads are executing concurrently a fork of those threads. Then when they are done executing, we think of them as joining back with the main thread. Conceptually, this looks like this:

\begin{figure}[h]
	\begin{center}
		\includegraphics[width=0.9\textwidth]{figure1.pdf}
		\caption{Fork and Join}
	\end{center}
\end{figure}

\subsection{Introduction to OpenMP}

While there are many thread libraries available, pthreads is the basic library for threading in C. A more convenient way to get started using threads is to use OpenMP, which is built into several popular C compilers as a means to compile high-level directives into threaded code using an underlying threads library. Let's take a look at a very simple example of how this works:

\pagebreak
%
% Illustration of OpenMP thread forking
%
\begin{lstlisting}[language=C]
01. #include<stdio.h>
02. #include<omp.h>
03.
04. int main (int argc, char* argv[]) {
05.     #pragma omp parallel
06.     {
07.         int id = omp_get_thread_num();
08.         int n_threads = omp_get_num_threads();
09.         printf("Hello from thread %d of %d\n",
10.                id, n_threads);
11.     }
12.     return 0;
13. }
\end{lstlisting}

Line 5 of this code illustrates how we can designate that the main thread 0 should fork and start multiple threads simultaneously. The code within the block following that line and between the curly braces will execute independently on each thread. Lines 7 and 8 illustrate functions that are available as part of the OpenMP library, which was included on line 2. There are several other functions available, most notably one that lets you set the number of threads to use, omp\_set\_num\_threads(), and one that lets you time your threaded code, omp\_get\_wtime(), to see how much faster it performs.

\pagebreak

\subsection{For-loop parallelization}

In a great deal of code examples, much of the work being performed can be found within for loops that are performing a large number of iterations, such as the coin-flipping example in the previous section. A well-used pattern in parallel programming is to split the work being done in these loops across multiple forked threads. OpenMP has a pragma for designating this in the code. Here is a simple example:

\begin{lstlisting}[language=C]
01. #include<stdio.h>
02. #include<omp.h>
03.
04. int main (int argc, char* argv[]) {
05.     int n_threads = 16;
06.     int i;
07.
08.     #pragma omp parallel for num_threads(n_threads)
09.     for (i = 0; i < n_threads; i++) {
10.         int id = omp_get_thread_num();
11.         printf("Thread %d performed iteration %d \n", id, i);
12.     }
13.     return 0;
14. }
\end{lstlisting}

In this example, we set up a very small number of repetitions of the loop, simply to illustrate how forking threads and running the loop iterations works. The OpenMP pragma on line 8 is asking the compiler to set up an equal distribution of work for each thread, which will take place like this for the 4 threads and 16 repetitions of the for loop.

\pagebreak
\subsection{Coin-flipping Revisted}

Now that we know a bit about how OpenMP works to provide threads that run code in parallel, let's look at how we can update our coin-flipping example.

\begin{lstlisting}[language=C]
01. #pragma omp parallel for
02.     num_threads(n_threads)
03.     default(none)
04.     private(n_flips, seed)
05.     shared(trial_flips)
06.     reduction(+:n_heads, n_tails)
07.     for (n_flips = 0; n_flips < trial_flips; n_flips++) {
08.         #pragma omp parallel for num_threads(n_threads)
09.             for (i = 0; i < n_threads; i++) {
10.                 // Flip coin and increment the appropriate
11.                 // counter, heads or tails.
12.             }
\end{lstlisting}

Similar to the last example we are using the $parallel for$ directive; however, we have also introduced many other useful directives including:

\begin{itemize}
\item $default()$ designates that all variables in the loop will be defined as either private within each thread or shared between the threads by the next three directives.

\item $private()$ designates that each thread will keep its own private copy of a variable and update it independently.

\item $shared()$ designates that a variable is shared by all of the threads.

\item $reduction()$ OpenMP will have each thread keep a private copy of these variables while they execute their portion of the loop. Then, when they join back after they have finished, each thread's private sum is added to an overall sum and stored in thread 0's copy of the variable.
\end{itemize}

\paragraph{Coding Exercise:}
Please refer to the parallel coin-flipping program in the accompanying files to run this program for yourself.

\pagebreak
% Experimenting with the code
\section{Experimenting with the code}

\subsection{Roulette Simulation}

An American Roulette wheel has 38 slots: 18 are red, 18 are black, and 2 are green, which the house always wins. When a person bets on either red or black, the odds of winning are 18/38, or 47.37\% of the time.

Our next is example is a simulation of spinning the Roulette wheel. We have a main simulation loop that is similar to the coin-flipping example. The code for determining a win on each spin is more involved than flipping a coin, and the sequential version, roulette\_simulation\_seq.c is decomposed into several functions. Look at this original code file to see how we run the simulations using increasing numbers of random spins of the wheel.

The function that actually runs a single simulation of the roulette wheel, called spin\_red(), is quite simple. It generates a random number to represent the slot that the ball ends up in and gives a payout according to the rules of Roulette.

\lstset{language=C}
\begin{lstlisting}[frame=single]
int spin_red(int bet, unsigned int *seed) {
    int payout;
    int slot = rand_int_between(1, 38, seed);

    if (slot <= 18)      /*spin was red   - win      */
        payout = bet;
    else if (slot <= 36) /*spin was black - lose all */
        payout = -bet;
    else                 /*spin was green - lose half*/
        payout = -(bet/2);

    return payout;
}
\end{lstlisting}

\pagebreak

\subsection{Roulette Simulation with Parallelism}

We add OpenMP parallelism as in the coin flip example, by running the loop of random spins for each trial on several threads. This code is located in the roulette\_simu\_omp.c file.

\lstset{language=C}
\begin{lstlisting}[frame=single]
int getNumWins(int numSpins, unsigned int seed) {
    int wins = 0;   /* our counter      */
    int spin;       /* loop control var */
    int myBet = 10; /* bet per spin     */

    #pragma omp parallel for num_threads(nThreads)
        default(none)
        shared(numSpins, myBet)
        private(spin, seed)
        reduction(+:wins)
        for (spin=0; spin < numSpins; spin++)
            if (spinRed(myBet, &seed) > 0) /* winner */
                wins++;
    return wins;
}
\end{lstlisting}

Both num\_spins and my\_bet are shared between threads while spin is the loop index and unique to each thread. When using rand\_r() as the thread-safe random number generator in linux, the seed should be private to each thread also. Like the previous example, we combine the partial results from each thread with reduction(+:wins).

\subsection{Drawing Four Cards of the Same Suit}

Now let's turn our attention to the card game of Poker. In the next example, we will examine the probability that given a random hand of 5 cards, four cards will each have a different suit.

We represent the deck of cards as an array of integers. The function for simulating deck shuffling is not the most efficient, but it mimics how a traditional "fan" shuffle actually works. We also have helper functions for initializing a deck, drawing a hand, and checking if the hand has four cards of the same suit. The accompanying code is located in the draw\_four\_suits\_seq.c file.

The function test\_one\_Hand initializes a deck, shuffles it, draws a hand, and then checks if all four suits are represented.

\pagebreak

\lstset{language=C}
\begin{lstlisting}[frame=single]
int testOneHand(unsigned int *seed_ptr){
    int deck[MAX_CARDS];     /* standard deck  */
    int hand[CARDS_IN_HAND]; /* hand of cards  */

    init_deck(deck, seed_ptr); /* create and shuffle */
    draw_hand(deck, hand, seed_ptr); /* pick four */

    return isFourSuits(hand); /* test if same suits */
}
\end{lstlisting}

\subsection{Drawing Four with Parallelism}

Converting our sequential code to use OpenMP is quite simple. We add the pragma compiler directive to the main simulation loop to run the loop simultaneously on multiple CPUs. The directive tells OpenMP to give each thread a different copy of the variable $i$ since each thread needs to keep track of its own loop iterations. The variable num\_tests is shared because the total number of tests to run is doubled only once per iteration of the out while loop. Finally, the directive reduction (+:total) tells OpenMP to combine each of the threads' partial results by summing to find the total number of hands that contained all four suits.

\lstset{language=C}
\begin{lstlisting}[frame=single]
while (numTests < MAX) {
    #pragma omp parallel for num_threads(nThreads)
        default(none)
        private (i, seed)
        shared (numTests)
        reduction (+:total)
        schedule(dynamic)
        for (i = 0; i < numTests; i++)
            if (testOneHand(&seed))
                total ++;

    /* calculate percentage and print results */

    numTests += numTests;
}
\end{lstlisting}

\section{Running Simulations with OnRamp}
\subsection{N and NP}
The values of these parameters are ignored.

\pagebreak
\subsection{Modes}
Use the modes parameter to select which program and version to run.

\begin{tabular}{ c l l }
	\textbf{mode} & \textbf{program} & \textbf{version} \\
	1s & coin\_flip & sequential \\
	1p & coin\_flip & parallel \\
	2s & draw\_four\_suits & sequential \\
	2p & draw\_four\_suits & parallel \\
	3s & roulette\_sim & sequential \\
	3p & roulette\_sim & parallel \\
\end{tabular}

\subsection{Threads}
Use the threads parameter to select how many threads to use in the parallel version. The default setting is 1 and the maximum setting is 32.

% Diving deeper
\section{Diving deeper}

\subsection{Project Ideas}
\begin{enumerate}
\item We know that the sequence of numbers from a random generator is always the same when it starts with the same seed. How should the random number generator be seeded for multi-threaded programs?

\item Use the helper functions from the draw four example, to write a new program that determines on average, how many cards a person needs to draw before all four suits are represented.

\item Suppose that a dart board is inscribed inside a square outer target. Since the sides of the square are twice the radius of the circular dart board, we have that the ratio, $\rho$ of the area of the circle to the area of the square is $\rho = \frac{\pi {r}^2}{{\left( 2r \right)^2}}$. With some rearrange we find that $\pi = 4\rho$. If we can determine the ratio of the area of the circle to the total area of the square inscribing it, we can approximate $\pi$. Calculating this ratio with a Monte Carlo simulation is quite simple. Simulate a random distribution of darts thrown at the square and compare the ratio of darts that land inside the circle and outside the circle. Multiple this result by $4$ to estimate $\pi$. How many darts do we need to estimate $\pi$ to 3 digits?

\end{enumerate}
\subsection{Other Modules:}

\begin{enumerate}
\item Refer to the Area Under the Curve Module for a different approach to estimating $\pi$.
\end{enumerate}

% References
\bibliographystyle{ieeetr}

\bibliography{monte_carlo}

%----------------------------------------------------------------------------------------

\end{document}


% $Header: /cvsroot/latex-beamer/latex-beamer/solutions/conference-talks/conference-ornate-20min.en.tex,v 1.6 2004/10/07 20:53:08 tantau Exp $

\documentclass{beamer}
%\documentclass[handout]{beamer}
%\usepackage{pgfpages}
%\pgfpagesuselayout{2 on 1}[a4paper,border shrink=5mm]

% This file is a solution template for:

% - Talk at a conference/colloquium.
% - Talk length is about 20min.
% - Style is ornate.

% Copyright 2004 by Till Tantau <tantau@users.sourceforge.net>.
%
% In principle, this file can be redistributed and/or modified under
% the terms of the GNU Public License, version 2.
%
% However, this file is supposed to be a template to be modified
% for your own needs. For this reason, if you use this file as a
% template and not specifically distribute it as part of a another
% package/program, I grant the extra permission to freely copy and
% modify this file as you see fit and even to delete this copyright
% notice.

\mode<presentation>
{
%  \usetheme{Warsaw}
%  \usetheme{Boadilla}
%  \usetheme{Goettingen}
%  \usetheme{Hannover}
%  \usetheme{Madrid}
%  \usetheme{Marburg}
%  \usetheme{Montpellier}
%  \usetheme{Pittsburgh}
  \usetheme{Hawke}
  % or ...

  \setbeamercovered{transparent}
  % or whatever (possibly just delete it)
}

\usepackage[english]{babel}
% or whatever

\usepackage[latin1]{inputenc}
% or whatever

\usepackage{times}
\usepackage[T1]{fontenc}
% Or whatever. Note that the encoding and the font should match. If T1
% does not look nice, try deleting the line with the fontenc.

\usepackage{multimedia}

%%%%%%
% My Commands
%%%%%%

\newcommand{\ml}{{\sc matlab}}
\newcommand{\bb}{{\boldsymbol{b}}}
\newcommand{\bx}{{\boldsymbol{x}}}
\newcommand{\by}{{\boldsymbol{y}}}
\newcommand{\bfm}[1]{{\boldsymbol{#1}}}

%%%%

\title[Lecture 20] % (optional, use only with long paper titles)
{Lecture 20 - Stability of Multistep Methods}

% \subtitle
% {Include Only If Paper Has a Subtitle}

\author[I. Hawke] % (optional, use only with lots of authors)
{I.~Hawke}
% - Give the names in the same order as the appear in the paper.
% - Use the \inst{?} command only if the authors have different
%   affiliation.

\institute[University of Southampton] % (optional, but mostly needed)
{
%  \inst{1}%
  School of Mathematics, \\
  University of Southampton, UK
}
% - Use the \inst command only if there are several affiliations.
% - Keep it simple, no one is interested in your street address.

\date[Semester 1] % (optional, should be abbreviation of conference name)
{MATH3018/6141, Semester 1}
% - Either use conference name or its abbreviation.
% - Not really informative to the audience, more for people (including
%   yourself) who are reading the slides online

\subject{Numerical methods}
% This is only inserted into the PDF information catalog. Can be left
% out.

% If you have a file called "university-logo-filename.xxx", where xxx
% is a graphic format that can be processed by latex or pdflatex,
% resp., then you can add a logo as follows:

\pgfdeclareimage[height=0.5cm]{university-logo}{mathematics_7469}
\logo{\pgfuseimage{university-logo}}

% Delete this, if you do not want the table of contents to pop up at
% the beginning of each subsection:
%  \AtBeginSubsection[]
%  {
%    \begin{frame}<beamer>
%      \frametitle{Outline}
%      \tableofcontents[currentsection,currentsubsection]
%    \end{frame}
%  }
\AtBeginSection[]
{
  \begin{frame}<beamer>
    \frametitle{Outline}
    \tableofcontents[currentsection]
  \end{frame}
}

% If you wish to uncover everything in a step-wise fashion, uncomment
% the following command:

%\beamerdefaultoverlayspecification{<+->}

\begin{document}

\begin{frame}
  \titlepage
\end{frame}

\section{Stability of Multistep Methods}

\subsection{Stability of Multistep Methods}

\begin{frame}
  \frametitle{Multistep Methods}

  Looking at IVPs of the form
  \begin{equation*}
    \by'(x) = \bfm{f}(x, \by(x)).
  \end{equation*} \pause

  Looked at the multistep methods using the $k$-step method formula
  \begin{equation*}
    a_k y_{n+1} + a_{k-1} y_n + \dots + a_0 y_{n+1-k} = h \left[ b_k
      f_{n+1} + b_{k-1} f_n + \dots + b_0 f_{n+1-k} \right];
  \end{equation*}
  explicit when $b_k = 0$, implicit otherwise. All multistep methods
  need a different method to start. \pause

  \vspace{1ex}

  Under what circumstances do we get the right results?

\end{frame}

\begin{frame}
  \frametitle{Example}

  \begin{columns}
    \begin{column}{0.5\textwidth}
      Integrate
      \begin{equation*}
        y'(x) = -y, \quad y(0) = 1
      \end{equation*}
      with solution $y = y(0)\exp(-x)$. \pause

      \vspace{1ex}

      Using Milne's method the result diverges in an oscillatory
      fashion. \pause

      \vspace{1ex}

      Improve step size ($h = 0.01 \rightarrow 0.001$) just delays the
      oscillations. \pause

      \vspace{1ex}

      Contrast with Adams-Bashforth 5, which is fine.
    \end{column}
    \begin{column}{0.5\textwidth}
      \begin{overlayarea}{\textwidth}{0.6\textheight}
        \only<2>
        {
          \begin{center}
            \includegraphics[height=0.5\textheight]{figures/MilneStability1}
          \end{center}
        }
        \only<3|handout:0>
        {
          \begin{center}
            \includegraphics[height=0.5\textheight]{figures/MilneStability2}
          \end{center}
        }
        \only<4|handout:0>
        {
          \begin{center}
            \includegraphics[height=0.5\textheight]{figures/AB5Stability}
          \end{center}
        }
      \end{overlayarea}
    \end{column}
 \end{columns}

\end{frame}

\begin{frame}
  \frametitle{Consistency}

  \emph{Consistency}: the numerical algorithm faithfully represents
  the differential equation. \pause

  \vspace{1ex}

  In other words, in the limit of infinite resolution, $h \rightarrow
  0$, the algorithm should match the differential equation. \pause

  \vspace{1ex}

  To prove consistency use the Taylor expansion of the algorithm. If
  the algorithm is consistent then the expansion at order $h^0 = 1$
  should be the original differential equation. \pause

  \vspace{1ex}

  In practice inconsistent algorithms are useless and will not be seen.

\end{frame}

\begin{frame}
  \frametitle{Stability}

  \emph{Stability}: the method gives bounded results over a
  finite interval. \pause

  \vspace{1ex}

  In other words, for \emph{any} step size $h$ the iteration $\by_n$
  at $x = X$ is \emph{finite}. \pause

  \vspace{1ex}

  Note that we only apply stability criteria to problems where the
  exact solution is finite; the problem
  \begin{equation*}
    y' = \frac{1}{(1-x)^2}
  \end{equation*}
  with solution
  \begin{equation*}
    y(x) = \frac{1}{1-x}
  \end{equation*}
  cannot be considered on intervals including $x = 1$. \pause

  \vspace{1ex}

  Stability is not the only property of an algorithm that we want to
  show, but combined with consistency it does give us everything we
  need.

\end{frame}

\begin{frame}
  \frametitle{Convergence}

  If $y_e(x)$ is the exact solution to the IVP, and $y_h(x)$ the
  numerical solution computed with step size $h$, then a method is
  \emph{convergent} if
  \begin{equation*}
    \lim_{h \rightarrow 0} y_h(x) = y_e(x).
  \end{equation*}
  Convergence is what we want to require of any algorithm. \pause

  \vspace{1ex}

  Convergence is difficult to prove directly. Instead we rely on:

  \vspace{1ex}

  {\bf Theorem}: A multistep method is convergent if and only if it is
  consistent and stable.

\end{frame}

\begin{frame}
  \frametitle{Polynomials for multistep methods}

  \begin{overlayarea}{\textwidth}{0.9\textheight}
    \only<1|handout:1>
    {
      From the $k$-step formula
      \begin{equation*}
        a_k y_{n+1} + a_{k-1} y_n + \dots + a_0 y_{n+1-k} = h \left[ b_k
          f_{n+1} + b_{k-1} f_n + \dots + b_0 f_{n+1-k} \right]
      \end{equation*}
      for multistep methods, define two polynomials
      \begin{align*}
        p(z) & = a_k z^k + \dots + a_1 z + a_0, \\
        q(z) & = b_k z^k + \dots + b_1 z + b_0.
      \end{align*}
    }
    \only<2|handout:2>
    {
      \begin{align*}
        p(z) & = a_k z^k + \dots + a_1 z + a_0, \\
        q(z) & = b_k z^k + \dots + b_1 z + b_0.
      \end{align*}
      Evaluate the \emph{stability} polynomial $p$ and its
      derivative $p'(z)$ at $z=1$:
      \begin{align*}
        p(1) & = \sum_{j=0}^k a_j, \\
        p'(1) & = \sum_{j=0}^k j a_j.
      \end{align*}
    }
    \only<3|handout:3>
    {
      \begin{align*}
        p(z) & = a_k z^k + \dots + a_1 z + a_0, \\
        q(z) & = b_k z^k + \dots + b_1 z + b_0.
      \end{align*}
      \begin{align*}
        p(1) & = \sum_{j=0}^k a_j, \\
        p'(1) & = \sum_{j=0}^k j a_j.
      \end{align*}
      Evaluate the second polynomial $q$ at $z=1$:
      \begin{align*}
        q(1) & = \sum_{j=0}^k b_j.
      \end{align*}
    }
    \only<4|handout:4>
    {
      \begin{align*}
        p(z) & = a_k z^k + \dots + a_1 z + a_0, \\
        q(z) & = b_k z^k + \dots + b_1 z + b_0.
      \end{align*}
      \begin{equation*}
        p(1) = \sum_{j=0}^k a_j, \quad p'(1)  = \sum_{j=0}^k j a_j,
        \quad q(1) = \sum_{j=0}^k b_j.
      \end{equation*}
      In the previous lecture we computed the Taylor expansion using
      the linear functional, showing that for consistency we need
      \begin{align*}
        \sum_{j=0}^k a_j & = p(1) = 0, \\
        \sum_{j=0}^k (j a_j - b_j) & = p'(1) - q(1) = 0.
      \end{align*}
    }
  \end{overlayarea}

\end{frame}

\begin{frame}
  \frametitle{How many solutions...}

  A differential equation has many solutions; initial conditions pick
  out one unique solution.  \pause With non-zero $h$ and finite
  precision, we introduce some error: numerical solution picks up a
  small contribution from other solutions. \pause

  \vspace{1ex}

  Two types of stability are
  \begin{enumerate}
  \item Over a finite interval, say $x \in [0, s]$, the difference
    between the numerical solution and the true solution is
    bounded. \pause
  \item As the number of steps taken is increased the contribution of
    the ``spurious'' solutions tends to zero.
  \end{enumerate} \pause

  Check stability by looking at the roots of the stability polynomial
  \begin{equation*}
    p(z) = \sum_{j=0}^k a_j z^j.
  \end{equation*}

\end{frame}

\begin{frame}
  \frametitle{Why the roots of the polynomial?}

  For stability the numerical solution must remain finite. \pause
  Consider the case $f(x, y(x)) = 0$.  Algorithm becomes
  \begin{equation*}
    \sum_{j=0}^k a_j y_{n+1-j} = 0.
  \end{equation*} \pause

  \vspace{1ex}

  Difference equation has $k$ solutions of form $A z^n$: $A$
  depends on initial data, $z$ depends on algorithm. \pause To
  converge, require $|z| < 1$. \pause

  \vspace{1ex}

  To find solutions substitute in $A z^n$, finding
  \begin{equation*}
    A z^{n+1-k} \sum_{j=0}^k a_j z^j \propto p(z).
  \end{equation*} \pause
  Therefore the size of the roots of $p(z)$ are crucial for stability.

\end{frame}

\begin{frame}
  \frametitle{Stability conditions}

  Write roots of stability polynomial $p$ as $\{r_j\}$. Two
  conditions:
  \begin{enumerate}
  \item If $ | r_j | \leq 1$ for all of the $k$ roots, \emph{and} any
    root with $|r_j| = 1$ is simple (i.e.\ is not repeated), then the
    \emph{root condition} is satisfied. \pause
  \item If $r_0 = 1$ and $ | r_j | < 1$ for all of the other $k-1$
    roots then the \emph{strong} root condition is satisfied.
  \end{enumerate} \pause

  \vspace{1ex}

  Either condition is sufficient for \emph{weak} stability. To ensure
  correct \emph{long term} integrations, need both stability
  (guaranteed by root condition) and \emph{relative} stability (given
  by strong root condition).

\end{frame}

\begin{frame}
  \frametitle{Example}

  \begin{overlayarea}{\textwidth}{0.9\textheight}
    \only<1|handout:1>
    {
      The Milne method
      \begin{equation*}
        y_{n+1} - y_{n-1} = \frac{h}{3} \left( f_{n+1} + 4 f_{n} + f_{n-1} \right)
      \end{equation*}
      is a $k$-step method with coefficients
    }
    \begin{align*}
      a_0 & = -1, & a_1 & = 0, & a_2 & = 1, \\
      b_0 & = 1/3, & b_1 & = 4/3, & b_2 & = 1/3.
    \end{align*}
    \only<2-3|handout:1>
    {
      Consistency shown when it was proved to have
      order 4. Can check:
    }
    \only<2-5|handout:1-2>
    {
      \begin{align*}
        &&    p(z) & = \sum_{j=0}^k a_j z^j & q(z) & = \sum_{j=0}^k b_j z^j \\
        && & = z^2 - 1 && = \tfrac{1}{3} \left( z^2 + 4 z + 1 \right)
        \\
        \Rightarrow && p'(z) & = 2 z
      \end{align*}
    }
    \only<3|handout:0>
    {
      and hence
      \begin{equation*}
        p(1)  = 0, \quad
        p'(1)  = 2 = q(1).
      \end{equation*}
    }
    \only<4-5|handout:2>
    {
      Roots of the stability polynomial are $r_j = \pm 1$: Milne's
      method is only weakly stable.
    }
    \only<5|handout:2>
    {
      Expect the method to converge, but spurious solutions may
      dominate for long integrations.
    }
  \end{overlayarea}
\end{frame}

\begin{frame}
  \frametitle{Example: 2}

  \begin{columns}
    \begin{column}{0.5\textwidth}
      Integrate
      \begin{equation*}
        y'(x) = -y, \quad y(0) = 1
      \end{equation*}
      with solution $y = y(0)\exp(-x)$. \pause

      \vspace{1ex}

      As the Milne method is not strongly stable we see
      divergence. \pause

      \vspace{1ex}

      As the method is weakly stable, we see that the time the
      divergence sets in increases with better resolution.
    \end{column}
    \begin{column}{0.5\textwidth}
      \begin{overlayarea}{\textwidth}{0.6\textheight}
        \only<2|handout:0>
        {
          \begin{center}
            \includegraphics[height=0.5\textheight]{figures/MilneStability1}
          \end{center}
        }
        \only<3>
        {
          \begin{center}
            \includegraphics[height=0.5\textheight]{figures/MilneStability2}
          \end{center}
        }
      \end{overlayarea}
    \end{column}
 \end{columns}

\end{frame}

\section{Summary}

\subsection{Summary}

\begin{frame}
  \frametitle{Summary}

  \begin{itemize}
  \item Convergence to the correct solution is the requirement for any
    algorithm.
  \item For differential equations the algorithm must be
    \emph{consistent} and \emph{stable}; in many cases this is
    sufficient for convergence.
  \item For multistep methods we define two polynomials $p$ (the
    stability polynomial) and $q$ depending on the coefficients in the
    $k$-step formula.
  \item Consistency is given by two simple relations between the
    polynomials (and derivatives) evaluated at $z=1$; it is equivalent
    to proving that the lowest order local truncation error terms
    vanish.
  \item Stability is shown by checking the modulus of the roots of $p$:
    \begin{enumerate}
    \item If all roots are $|r_j|<1$ (except $r_0=1$) then we have
      relative stability.
    \item If we do not have relative stability, but all $|r_j| \leq
      1$, then we have \emph{weak} stability which may not be
      sufficient for good long term integrations.
    \end{enumerate}
  \end{itemize}

\end{frame}

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


\chapter{Early Cryptography} % Main chapter title
\label{Early Cryptography} % For referencing the chapter elsewhere, use \ref{Chapter1}

%----------------------------------------------------------------------------------------

As discussed in the \hyperref[Introduction]{Introduction} chapter, cryptography has a very long and interesting history.
This chapter will dive further and explore the various cryptography methods and inventions which paved the
way for modern-day cryptography.

\section{Greek Cryptography}

\subsection{The Scytale}

You may have heard of the \textit{Spartan} warrior society, famed for it's military-focused lifestyle featuring austerity,
strength, and fitness \cite{wiki:spartan_army}. The Spartan warrior society also utilized a cryptography device called the
\textit{Scytale}. The Scytale is a cryptography device in the shape of cylinder which would be distributed to both the sender
and receiver. In order to encrypt a message, a narrow strip of parchment paper would be wrapped around and then the message
written on the strip of parchment paper. Once unwrapped, the paper displays seemingly useless and confusing letters. The
strip of paper containing the encrypted message would then be sent to whom it is intended for. To decrypt, the receiver would
wrap the strip of paper around a Scytale with the same diameter as the one used to encrypt the message
\cite{history_of_cryptography}.

This type of cryptography technique is commonly referred to as a \textit{transposition} cipher because the letters
in the message remain the same while the position of the letters change \cite{wiki:scytale}. In mathematics, to encrypt
(using the transposition cipher) you would use a \textit{bijective} function and to decrypt you would use an
\textit{inverse function}. The Scytale allowed the ancient Greeks (and Spartans in particular) to send messages
securely, although it should be said that this method of encryption is very easy to break by simply brute-forcing the different
diameters.

\subsection{Caesar Cipher}

Named after the Greek emperor \textit{Julius Caesar}, this cryptography method works by substituting the $ith$ letter in the
message by the letter that is $x$ positions to the right. Then to decrypt, you substitute the $ith$ letter in the message by
the letter that is $x$ positions to the left. Caesar Cipher is commonly referred to as an \textit{addition} or \textit{substitution}
cipher.

\section{Cryptography in the Medieval and Renaissance Periods}

For a long time, advancements in cryptography were put to a halt. Even though the Romans were aware and utilized Greek
cryptography techniques, there was no focus on inventing new techniques, that is until the \textit{Medieval and Renaissance}
periods.

\subsection{Arabic Cryptography}

Modern cryptography techniques originated from the Arabic people, the first to document cryptanalysis techniques
in a systematic fashion. During 800 AD, the Arab mathematician \textit{al-Kindi} invented the \textit{frequency-analysis}
technique. This technique had so much impact on the field of cryptography that it was the most significant advance in
cryptography until World War 2. In al-Kindi's \textit{Risalah fi Istikhraj al-Mu'amma}
(\textit{Manuscript for the Deciphering Cryptographic Messages}) he described the first techniques of cryptanalysis,
classification of ciphers, and most importantly gave the first descriptions of frequency-analysis.
Frequency-analysis is the study of the frequency of letters within ciphertext based on the fact that in any
written language there are trends of occurrence among certain letters and combinations of letters \cite{wiki:frequency_analysis}. For example, in the
English language the letters \textit{E}, \textit{T}, and \textit{A} are the most common while \textit{Z}, \textit{Q}, and \textit{X} are the least common.
Frequency-analysis is used in cryptanalysis to aid in breaking a cipher. In addition, The \textit{Subh al-a 'sha}, a 14-volume encyclopedia written by
\textit{Ahmad al-Qalqashandi} included a dedicated section on cryptography \cite{wiki:history_of_cryptography}.

\subsection{Cryptography in Europe}

In Europe, cryptography became a more integral part of defense due to political and religious change. During and after the Renaissance period,
many contributions to the field of cryptography can be attributed to the citizens of Italian states \cite{wiki:history_of_cryptography}.
Furthermore, during the reign of Queen Elizabeth I cryptanalysts were able to decrypt messages regarding a plot to assassinate the Queen.
This plot is known as the \textit{Babington Plot}.\footnote{In 1586, Anthony Babington and John Ballard colluded to assassinate Queen Elizabeth I
and put \textit{Mary, Queen of Scots} on the throne. After cryptanalysts decrypted a message sent by Queen Mary herself regarding the assassination, Queen
Elizabeth I ordered the execution of Queen Mary and shortly after, she was executed \cite{wiki:babington_plot}. }

\section{Conclusion}

As you can see, early cryptography laid out the fundamentals for core techniques used in modern-day cryptography.
Both the \textit{transposition} and \textit{substitution} ciphers remain fundamental concepts in complex cryptography
algorithms used today. In addition, the everlasting contributions made by mathematician al-Kindi in the field
of cryptography were so important that no other invention exceeded al-Kindi's until World War 2. Consequently, his descriptions of
frequency-analysis tremendously aided cryptographers in cryptanalysis; such as preventing the assassination of Queen Elizabeth I.


% standard LaTeX packages
\documentclass[11pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[english]{babel}
% \usepackage[margin=3cm]{geometry}
\usepackage[hidelinks]{hyperref}
\usepackage{parskip}
\usepackage{xifthen,xparse}

% math packages
\usepackage{mathtools,amsfonts,amssymb,mathdots}
\usepackage{siunitx}
\usepackage{nicefrac}
\mathtoolsset{showonlyrefs}
\DeclareSIUnit{\year}{yr}
\DeclareSIUnit{\astronomicalunit}{AU}
\DeclareSIUnit{\solarmass}{M_\odot}

% plotting and tables
\usepackage{tikz}
\usepackage{pgfplots}
\usepackage{pgfplotstable}
\usepackage{caption}
\usepgfplotslibrary{external}
\tikzexternalize[prefix=tikz/,mode=list and make]
\usetikzlibrary{spy}
\usepgfplotslibrary{units}
\usepgfplotslibrary{groupplots}
\pgfplotsset{
  compat=1.13,
  % scaled x ticks = false,
  % x tick label style={
  %   /pgf/number format/fixed,
  %   /pgf/number format/1000 sep = \thinspace
  % },
  % scaled y ticks = false,
  % y tick label style={
  %   /pgf/number format/fixed,
  %   /pgf/number format/1000 sep = \thinspace
  % },
  legend cell align=left,
  % unit
  unit markings=slash space,
  unit code/.code 2 args=
    \expandafter\expandafter\expandafter\expandafter
    \expandafter\expandafter\expandafter\si
    \expandafter\expandafter\expandafter\expandafter
    \expandafter\expandafter\expandafter{#2},
  % legend style={
  %   at={(1.02,1)},
  %   anchor=north west,
  % },
  filter discard warning=false,
  discard if not/.style 2 args={
    x filter/.code={
      \edef\tempa{\thisrow{#1}}
      \edef\tempb{#2}
      \ifx\tempa\tempb
      \else
        \def\pgfmathresult{inf}
      \fi
    }
  },
}

% bibliography packages
% \usepackage[backend=bibtex8,style=numeric,autocite=inline,sorting=ynt]{biblatex}

% other packages
% \usepackage{filecontents}
\usepackage{marginnote}

% bibliography
% \begin{filecontents}{bibliography.bib}
% \end{filecontents}
% \addbibresource{bibliography.bib}
% \nocite{*}

% custom math commands
\newcommand\V[1]{\mathbf{#1}}                  % vector notation
\newcommand\M[1]{\begin{bmatrix} #1 \end{bmatrix}} % matrix shorthand

% commands for derivatives/integral dx expressions
\newcommand\D[1]{{\,\mathrm{d}{#1}}}
\newcommand\dd[2][]{{%
  \ifthenelse{\isempty{#1}}%
    {\mathrm{d}{#2}}%
    {\mathrm{d}^{#1}{#2}}%
}}
\newcommand\df[3][]{{%
  \ifthenelse{\isempty{#1}}%
    {\frac{\mathrm{d}{#2}}{\mathrm{d}{#3}}}%
    {\frac{\mathrm{d}^{#1}{#2}}{{\mathrm{d}{#3}^{#1}}}}%
}}
\newcommand\pD[1]{{\,\mathrm{d}{#1}}}
\newcommand\pdd[2][]{{%
  \ifthenelse{\isempty{#1}}%
    {\parital{#2}}%
    {\partial^{#1}{#2}}%
}}
\newcommand\pdf[3][]{{%
  \ifthenelse{\isempty{#1}}%
    {\frac{\partial{#2}}{\partial{#3}}}%
    {\frac{\partial^{#1}{#2}}{{\partial{#3}^{#1}}}}%
}}
\let\epsilon\varepsilon

\def\sun{\odot}

\DeclareDocumentCommand{\reversemarginnote}{ m O{} }{\reversemarginpar\marginnote{#1}[#2]\reversemarginpar}
\newcommand\oppg[1]{\reversemarginnote{\textcolor{black!40}{#1)}}[2mm]}

\begin{document}

\title{Project 3 - FYS3150 Computational Physics}
\author{Fredrik Østrem (\texttt{fredost})}
\date{\today}

\maketitle

\begin{abstract}
  In this project, we will study different simulations of the solar system. We will discuss the differences between Euler's forward algorithm and the Verlet algorithm
  in such simulations, and we will build up a program which can simulate the gravitational interaction between the Sun and all the planets. Finally, we will use this program to
  study the perihelion precession of Mercury due to general relativity.
\end{abstract}

\tableofcontents

\clearpage
\section{Introduction}
In this project, we will build a simple object-oriented program that simulates the gravitational interaction between the sun and planets in the solar system, by using the velocity Verlet
algorithm. We will start writing a simple program for simulating the Earth-Sun system with the Sun fixed at the origin, and use this to compare the Verlet algorithm to an algorithm
that we're already very familiar with: Euler's forward algorithm. We will expand this program to simulate three bodies (Earth, Sun and Jupiter) and then the entire solar system (Sun, Moon
and all the eight planets). We will also use our code to explore two specific problems: the escape velocity of Earth in the gravity of the Sun, and the perihelion precession of Mercury due to
general relativity.

\subsection{Units and constants}

We will throughout this assignment assume that the base units are expressed in \emph{astronomical units} (\si{\astronomicalunit}) for distance, \emph{years} (\si{\year}) for time, and \emph{solar masses} (\si{\solarmass}) for mass. All other units will be derived from these, such as energy (\si{\solarmass \astronomicalunit^2 \per \year^2}) and angular momentum (\si{\solarmass \astronomicalunit^2 \per \year}).

When using these units, we get nice expressions for relevant constants:
\begin{itemize}
  \item The distance between the Earth and the Sun is $r = \SI{1}{\astronomicalunit}$.
  \item The velocity of the Earth is $v = 2 \pi \, \si{\astronomicalunit . \year^{-1}}$.
  \item The universal gravitational constant is $G = 4 \pi^2 \, \si{\astronomicalunit^3 . \solarmass^{-1} . \year^{-2}}$.
\end{itemize}

\clearpage
\section{Methods}

\subsection{Discretization of a two-body system}
\oppg{a}

We start by only considering a simple two-body gravitational system, such as the Earth-Sun system. We can express the gravitational interaction of the Sun on the Earth by the differential equation
\begin{equation}
  \V{a} = \df[2]{\V{x}}{t} = - \frac{F_G}{M} \cdot \frac{\V{r}}{r} = - \frac{G M_\sun \V{r}}{r^3}
\end{equation}
where $\V{x}$ is the position of the Earth, $\V{r}$ is the position of the Earth relative to the Sun, and $M$ and $M_\sun$ are the masses of the Earth and the Sun.

To be able to solve this differential equation numerically, we want to discretize it. If we use Euler's forward algorithm to discretize this with time steps $\Delta{t}$, we get
the equations
\begin{alignat}{4}
  \V{x}_{n+1} &= \V{x}_n + \V{v}_n \cdot \Delta{t} \\
  \V{v}_{n+1} &= \V{v}_n + \V{a}_n \cdot \Delta{t} &&= \V{v}_n - \frac{G M_\sun \V{r}_n}{r_n^3} \cdot \Delta{t} \\
\intertext{where}
  \V{r}_n &= \V{x}_n - \V{x}_{\sun,n}
\end{alignat}

We can also use the Verlet algorithm, which is based on Netwon's laws of motion. If we use that, we get the equations
\begin{alignat}{4}
  \V{x}_{n+1} &= \V{x}_n + \V{v}_n \cdot \Delta{t} + \frac{1}{2} \V{a}_n \cdot \Delta{t}^2 \\
  \V{v}_{n+1} &= \V{v}_n + \frac{ \V{a}_n + \V{a}_{n+1} }{2} \cdot \Delta{t} \\
  \V{a}_{n}   &= - \frac{G M_\sun \V{r}_n}{r^3}
\end{alignat}

Since the mass of the Sun is about \num{300000} times larger than that of the Earth, we can assume that the Sun is position in the center of mass, and we will assume that it lies in the origin.

As we will see later, Euler's forward algorithm is not numerically stable with respect to total energy, while Verlet keeps the total energy of the system constant.

\subsection{Implementation of the Earth-Sun system}
\oppg{b}

When implementing the simulation of the Earth-Sun system, we may want to create and generalize classes to be able to reuse functionality. In particular, we may want to write:
\begin{itemize}
  \item A \texttt{vec3} class, that stores a 3D vector and handles vector operations.
  \item A \texttt{CelestialBody} class, that stores the properties of some celestial body (in this case, the Earth and the Sun).
  \item A \texttt{SolarSystem} class, that stores all the celestial bodies of the simulation (in this case, the Earth and the Sun).
  \item A \texttt{Solver} class, that solves the differential equations for the movement of the celestial bodies, and that can be subclassed to provide different algorithms (such as Euler's forward algorithm or the Verlet algorithm).
\end{itemize}

Writing classes like this can have many advantages. In particular, we keep the information about an object and the functions that act on it bundled together, and we can easily replace or extend the functionality of a class by creating a subclass (like we do with the \texttt{Solver} class).

\clearpage
\section{Results}

\subsection{Earth-Sun system}
\oppg{c}

When we run the program \texttt{code/3b}, we will get two data files \texttt{3b\_euler.dat} and \texttt{3b\_verlet.dat}, that contain the data for the simulation over a five-year period with time steps at $\nicefrac{1}{1000}$ of a year.

In figure \ref{fig:3c}, we see the $x$ and $y$ coordinates of the Earth's orbit around the sun (fixed at the origin). In figure \ref{fig:3c-E}, we can see the kinetic, potential and total energies over time, and in figure \ref{fig:3c-L}, we can see the total angular momentum of the system over time.

In all three plots, we can see that the Verlet algorithm (in blue) is stable; the Earth moves in a near perfect circle around the sun, total energy is constant and so is total angular momentum. This reflects what we already know about the conservation of energy and angular momentum in a closed system: that these quantities are preserved.

On the other hand, Euler's forward algorithm (in red) is numerically unstable: instead of moving in a circular orbit, the Earth's distance from the sun gradually increases, and its total energy and angular momentum increase significantly, despite being a closed system.

The Verlet algorithm is specifically designed for calculations of forces, being based on Newton's laws of motion. Because of this, it ensures that quantities such as total energy or total angular momentum is preserved when we have conservative forces, such as gravity. However, Euler's forward algorithm is a generic algorithms for all differential equations, and pretty much the simplest algorithm you could use.

The Verlet algorithm uses three times more FLOPs than Euler's forward algorithm: while Euler uses 4 FLOPs\footnote{\textbf{F}loating \textbf{p}oint \textbf{op}erations.} per calculation per dimension, Verlet uses 12. (For a three-dimensional system such as ours, this means Euler will use 12 FLOPs while Verlet will use 36.) The naïvely implemented Verlet algorithm also has to recalculate the forces twice: once for the previous position, and once for the new position. However, this can be reduced by simply only recalculating when the position has changed; when it has not, you can just reuse the forces calculated for the previous Verlet step. It will often be the calculation of the forces that is the most expensive, especially in systems with more than one moving body, since the number of forces to be calculated is approximately $N^2$ for $N$ bodies.

\begin{figure}[!ht]
  \centering

  \tikzpicturedependsonfile{data/3b_verlet.dat}
  \tikzpicturedependsonfile{data/3b_euler.dat}
  \begin{tikzpicture}
    \begin{axis}[
        width=0.8\textwidth,
        height=0.8\textwidth,
        xlabel={$x$}, x unit={\astronomicalunit},
        xmin=-1.5, xmax=1.5,
        xtick={-1,0,1},
        ylabel={$y$}, y unit={\astronomicalunit},
        ymin=-1.5, ymax=1.5,
        ytick={-1,0,1},
      ]

      \draw[fill=yellow] (axis cs:0,0) circle (2mm);
      \addlegendimage{only marks,mark=*,black,mark options={scale=2, fill=yellow}}
      \addlegendentry{Sun}

      \addplot[blue,thick,smooth,->] table[x=earth_x, y=earth_y] {data/3b_verlet.dat};
      \addlegendentry{Verlet}

      \addplot[red,thick,smooth,->] table[x=earth_x, y=earth_y] {data/3b_euler.dat};
      \addlegendentry{Euler}
    \end{axis}
  \end{tikzpicture}

  \caption{Five-year simulation of the Earth-Sun system, using the Verlet algorithm and Euler's forward algorithm. Sun is not to scale.} \label{fig:3c}
\end{figure}

\begin{figure}[!ht]
  \centering

  \tikzpicturedependsonfile{data/3b_verlet.dat}
  \tikzpicturedependsonfile{data/3b_euler.dat}
  \begin{tikzpicture}
    \begin{axis}[
        width=\textwidth,
        height=0.6\textwidth,
        xlabel={Time}, x unit={\year},
        xmin=0, xmax=5,
        ylabel={Energy}, y unit={\solarmass . \astronomicalunit^2 . \year^{-2}},
        ymin=-1.5e-4, ymax=1.5e-4,
        grid=both,
      ]

      \addplot[blue,dashed,thick,smooth,forget plot] table[x=t, y=K] {data/3b_verlet.dat};
      \addplot[blue,dotted,thick,smooth,forget plot] table[x=t, y=U] {data/3b_verlet.dat};
      \addplot[blue,thick,smooth,forget plot] table[x=t, y expr={\thisrow{U}+\thisrow{K}}] {data/3b_verlet.dat};

      \addplot[red,dashed,thick,smooth,forget plot] table[x=t, y=K] {data/3b_euler.dat};
      \addplot[red,dotted,thick,smooth,forget plot] table[x=t, y=U] {data/3b_euler.dat};
      \addplot[red,thick,smooth,forget plot] table[x=t, y expr={\thisrow{U}+\thisrow{K}}] {data/3b_euler.dat};

      \addlegendimage{black,dashed,thick} \addlegendentry{Kinetic energy $K$}
      \addlegendimage{black,dotted,thick} \addlegendentry{Potential energy $U$}
      \addlegendimage{black,thick} \addlegendentry{Total energy $E$}
    \end{axis}
  \end{tikzpicture}

  \caption{Kinetic, potential and total energy of the five-year simulation from figure \ref{fig:3c}. Blue is with Verlet algorithm, red is with Euler's forward algorithm.} \label{fig:3c-E}
\end{figure}

\begin{figure}[!ht]
  \centering

  \tikzpicturedependsonfile{data/3b_verlet.dat}
  \tikzpicturedependsonfile{data/3b_euler.dat}
  \begin{tikzpicture}
    \begin{axis}[
        width=\textwidth,
        height=0.6\textwidth,
        xlabel={Time}, x unit={\year},
        xmin=0, xmax=5,
        ylabel={Total angular momentum}, y unit={\solarmass . \astronomicalunit^2 . \year^{-1}},
        ymin=0, ymax=4e-5,
        grid=both,
      ]
      \addplot[blue,thick,smooth] table[x=t, y expr={sqrt((\thisrow{Lx})^2+(\thisrow{Ly})^2+(\thisrow{Lz})^2)}] {data/3b_verlet.dat};
      \addplot[red,thick,smooth] table[x=t, y expr={sqrt((\thisrow{Lx})^2+(\thisrow{Ly})^2+(\thisrow{Lz})^2)}] {data/3b_euler.dat};

      \legend{Verlet,Euler}
    \end{axis}
  \end{tikzpicture}

  \caption{Total angular momentum $L$ of the five-year simulation from figure \ref{fig:3c}. Blue is with Verlet algorithm, red is with Euler's forward algorithm.} \label{fig:3c-L}
\end{figure}

\clearpage
\subsection{Escape velocity}
\oppg{d}

Through trial and error (as shown in figure \ref{fig:3d}), I found the best approximation of the escape velocity be
\begin{align}
  v_{\mathrm{escape}}
    &\approx 2\pi \cdot \SI{1.42}{\astronomicalunit . \year^{-1}} \\
    &\approx \SI{8.92}{\astronomicalunit . \year^{-1}}
\end{align}
where $\SI{1}{\astronomicalunit \per \year} \cdot 2 \pi$ is the speed at which a circular orbit would be made.

We know that the gravitational potential energy of Earth, $U(r)$, is
\begin{equation}
  U(r) = - \frac{G M M_\sun}{r}
\end{equation}

If we let $U_\infty = \lim_{r \to \infty} U(r)$ be the potential energy at ``infinity'', we easily see that $U_\infty = 0$. This means that to escape the gravitational well of the sun, we need to have a total energy $E_{\mathrm{tot}} \ge U_\infty$; the escape velocity occurs exactly where $E_{\mathrm{tot}} = U_\infty = 0$.

This means that the escape velocity $v_{\mathrm{escape}}$ is given by the equation:
\begin{align}
  \frac{1}{2} M v_{\mathrm{escape}}^2 &- \frac{G M M_\sun}{r} = 0 \\
\intertext{which when rearranged gives}
  v_{\mathrm{escape}} &= \sqrt{\frac{2 G M_\sun}{r}} \\
\intertext{When we insert the quantities $G$, $M_\sun$ and $r$ for the Earth-Sun system, we get}
  v_{\mathrm{escape}} &= 2 \sqrt{2} \pi \, \si{\astronomicalunit . \year^{-1}}
    \approx \SI{8.89}{\astronomicalunit . \year^{-1}}
\end{align}
which is not far off our earlier approximation.

\begin{figure}[!ht]
  \centering

  \tikzpicturedependsonfile{data/3d.dat}
  \begin{tikzpicture}
    \begin{axis}[
        width=0.8\textwidth,
        height=0.8\textwidth,
        xlabel={$x$}, x unit={\astronomicalunit},
        xmin=-600, xmax=100,
        xtick={-600,-450,...,150},
        minor x tick num=2,
        ylabel={$y$}, y unit={\astronomicalunit},
        ymin=-100, ymax=600,
        ytick={-150,0,...,600},
        minor y tick num=2,
        grid=both,
      ]
      \addplot[red,thick,smooth,discard if not={v0}{1.40}] table[x=earth_x, y=earth_y] {data/3d.dat};
      \addplot[purple,thick,smooth,discard if not={v0}{1.41}] table[x=earth_x, y=earth_y] {data/3d.dat};
      \addplot[blue,thick,smooth,discard if not={v0}{1.42}] table[x=earth_x, y=earth_y] {data/3d.dat};
      \addplot[cyan,thick,smooth,discard if not={v0}{1.43}] table[x=earth_x, y=earth_y] {data/3d.dat};
      \addplot[green,thick,smooth,discard if not={v0}{1.44}] table[x=earth_x, y=earth_y] {data/3d.dat};

      \legend{$k = 1.40$,$k = 1.41$,$k = 1.42$,$k = 1.43$,$k = 1.44$}
    \end{axis}
  \end{tikzpicture}

  \caption{Plot of a planet's orbit at different initial velocities ${v_0 = 2 \pi k \cdot \SI{1}{\astronomicalunit \per \year}}$.} \label{fig:3d}
\end{figure}

\clearpage
\subsection{Earth-Jupiter-Sun system}
\oppg{e}

We now add a third body, Jupiter, to our simulation. Because of the way we designed the program, this change only needs one or two more lines of code. We try three different ``Jupiters'': one with the realistic mass (\SI{1e-3}{\solarmass}), one with ten times this (\SI{1e-2}{\solarmass}) and one with 1000 times this (\SI{1}{\solarmass}).

As we see in figure \ref{fig:3e}, adding the realistic Jupiter does not significantly affect the orbit of the Earth. If we change the mass of Jupiter to 10 times its real mass, we see that it will disturb Earth's orbit slightly, making it slowly lose speed and move towards the Sun. This will likely cause the Earth to crash into the Sun after a number of years. When we use a mass 1000 times that of Jupiter, Earth's orbit becomes completely unstable, and the Earth will crash into the sun after only a few years.

\begin{figure}[!ht]
  \centering

  \tikzpicturedependsonfile{data/3b_verlet.dat}
  \tikzpicturedependsonfile{data/3e.dat}
  \tikzpicturedependsonfile{data/3e-10.dat}
  \tikzpicturedependsonfile{data/3e-1000.dat}
  \begin{tikzpicture}
    \newsavebox{\pipboxA}
    \savebox{\pipboxA}{%
      \begin{tikzpicture}
        \begin{axis}[
            axis background/.style={fill=white},
            width=1.5cm,
            height=1.5cm,
            xmin=0.97, xmax=1.03,
            ymin=-0.03, ymax=0.03,
            ticks=none,
            enlargelimits=false,
            scale only axis,
          ]
          \addplot[blue,smooth,->] table[x=earth_x, y=earth_y] {data/3b_verlet.dat};
        \end{axis}
      \end{tikzpicture}
    }

    \newsavebox{\pipboxB}
    \savebox{\pipboxB}{%
      \begin{tikzpicture}
        \begin{axis}[
            axis background/.style={fill=white},
            width=1.5cm,
            height=1.5cm,
            xmin=0.97, xmax=1.03,
            ymin=-0.03, ymax=0.03,
            ticks=none,
            enlargelimits=false,
            scale only axis,
          ]
          \addplot[blue,smooth,->] table[x=earth_x, y=earth_y] {data/3e.dat};
        \end{axis}
      \end{tikzpicture}
    }

    \newsavebox{\pipboxC}
    \savebox{\pipboxC}{%
      \begin{tikzpicture}
        \begin{axis}[
            axis background/.style={fill=white},
            width=1.5cm,
            height=1.5cm,
            xmin=0.97, xmax=1.03,
            ymin=-0.03, ymax=0.03,
            ticks=none,
            enlargelimits=false,
            scale only axis,
          ]
          \addplot[blue,smooth,->] table[x=earth_x, y=earth_y] {data/3e-10.dat};
        \end{axis}
      \end{tikzpicture}
    }

    \newsavebox{\pipboxD}
    \savebox{\pipboxD}{%
      \begin{tikzpicture}
        \begin{axis}[
            axis background/.style={fill=white},
            width=1.5cm,
            height=1.5cm,
            xmin=0.97, xmax=1.03,
            ymin=-0.03, ymax=0.03,
            ticks=none,
            enlargelimits=false,
            scale only axis,
          ]
          \addplot[blue,smooth,->] table[x=earth_x, y=earth_y] {data/3e-1000.dat};
        \end{axis}
      \end{tikzpicture}
    }

    \begin{groupplot}[
        group style={
          group size=2 by 2,
          xlabels at=edge bottom,
          xticklabels at=edge bottom,
          ylabels at=edge left,
          yticklabels at=edge left,
          % vertical sep=1pt,
          % horizontal sep=0pt,
        },
        width=0.5\textwidth,
        height=0.5\textwidth,
        xlabel={$x$ \; / \; \si{\astronomicalunit}},
        xmin=-2, xmax=2,
        xtick={-2,-1,...,2},
        ylabel={$y$ \; / \; \si{\astronomicalunit}},
        ymin=-2, ymax=2,
        ytick={-2,-1,...,2},
      ]

      \nextgroupplot[title={No Jupiter}]
        \draw[fill=yellow] (axis cs:0,0) circle (2mm);
        \addlegendimage{only marks,mark=*,black,mark options={scale=2, fill=yellow}}
        \addlegendentry{Sun}

        \addplot[blue,smooth] table[x=earth_x, y=earth_y] {data/3b_verlet.dat};
        \addlegendentry{Earth}

        \draw (axis cs:0.97,-0.03) rectangle (axis cs:1.03,0.03);
        \draw (axis cs:1.03,-0.03) -- (axis cs:1.23,-1.4);
        \draw (axis cs:1.23,-1.4) node[inner sep=0,outer sep=0] {\usebox{\pipboxA}};

      \nextgroupplot[title={Realistic Jupiter}]
        \draw[fill=yellow] (axis cs:0,0) circle (2mm);
        \addlegendimage{only marks,mark=*,black,mark options={scale=2, fill=yellow}}
        \addlegendentry{Sun}

        \addplot[blue,smooth] table[x=earth_x, y=earth_y] {data/3e.dat};
        \addlegendentry{Earth}

        \draw (axis cs:0.97,-0.03) rectangle (axis cs:1.03,0.03);
        \draw (axis cs:1.03,-0.03) -- (axis cs:1.23,-1.4);
        \draw (axis cs:1.23,-1.4) node[inner sep=0,outer sep=0] {\usebox{\pipboxB}};

      \nextgroupplot[title={$\text{Jupiter} \times 10$}]
        \draw[fill=yellow] (axis cs:0,0) circle (2mm);
        \addlegendimage{only marks,mark=*,black,mark options={scale=2, fill=yellow}}
        \addlegendentry{Sun}

        \addplot[blue,smooth] table[x=earth_x, y=earth_y] {data/3e-10.dat};
        \addlegendentry{Earth}

        \draw (axis cs:0.97,-0.03) rectangle (axis cs:1.03,0.03);
        \draw (axis cs:1.03,-0.03) -- (axis cs:1.23,-1.4);
        \draw (axis cs:1.23,-1.4) node[inner sep=0,outer sep=0] {\usebox{\pipboxC}};

      \nextgroupplot[title={$\text{Jupiter} \times 1000$}]
        \draw[fill=yellow] (axis cs:0,0) circle (2mm);
        \addlegendimage{only marks,mark=*,black,mark options={scale=2, fill=yellow}}
        \addlegendentry{Sun}

        \addplot[blue,smooth] table[x=earth_x, y=earth_y] {data/3e-1000.dat};
        \addlegendentry{Earth}

        \draw (axis cs:0.97,-0.03) rectangle (axis cs:1.03,0.03);
        \draw (axis cs:1.03,-0.03) -- (axis cs:1.23,-1.4);
        \draw (axis cs:1.23,-1.4) node[inner sep=0,outer sep=0] {\usebox{\pipboxD}};
    \end{groupplot}
  \end{tikzpicture}

  \caption{Five-year simulation of the Earth-Jupiter-Sun system, with different Jupiter masses. Sun is not to scale.} \label{fig:3e}
\end{figure}

\clearpage
\subsection{Complete solar system}
\oppg{f}

We add all the remaining planets of the solar system, in addition to our own Moon, to the program, and run a simulation for 25 years.
The paths of all of the planets are visualized in figures \ref{fig:3f-inner} and \ref{fig:3f-outer}.

\begin{figure}[!ht]
  \centering

  \tikzpicturedependsonfile{data/3f.dat}
  \begin{tikzpicture}
    \begin{axis}[
        width=0.8\textwidth,
        height=0.8\textwidth,
        xlabel={$x$}, x unit={\astronomicalunit},
        xmin=-2, xmax=2,
        xtick={-2,-1,0,1,2},
        ylabel={$y$}, y unit={\astronomicalunit},
        ymin=-2, ymax=2,
        ytick={-2,-1,0,1,2},
      ]

      \addplot[orange,ultra thick,smooth,discard if not={body}{0}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Sun}

      \addplot[red!50!black,smooth,discard if not={body}{1}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Mercury}

      \addplot[green!50!black,smooth,discard if not={body}{2}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Venus}

      \addplot[blue,smooth,discard if not={body}{3}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Earth}

      \addplot[red,smooth,discard if not={body}{5}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Mars}
    \end{axis}
  \end{tikzpicture}

  \caption{25-year simulation of the inner solar system (Sun, Mercury, Venus, Earth, Mars).} \label{fig:3f-inner}
\end{figure}

\begin{figure}[!ht]
  \centering

  \tikzpicturedependsonfile{data/3f.dat}
  \begin{tikzpicture}
    \begin{axis}[
        width=0.8\textwidth,
        height=0.8\textwidth,
        xlabel={$x$}, x unit={\astronomicalunit},
        xmin=-35, xmax=35,
        xtick={-30,-20,...,30},
        ylabel={$y$}, y unit={\astronomicalunit},
        ymin=-35, ymax=35,
        ytick={-30,-20,...,30},
        legend style={
          at={(1.02,1)},
          anchor=north west,
        },
      ]

      \addplot[orange,ultra thick,smooth,discard if not={body}{0}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Sun}

      \addplot[red!50!black,thick,smooth,discard if not={body}{1}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Mercury}

      \addplot[green!50!black,thick,smooth,discard if not={body}{2}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Venus}

      \addplot[blue!50!green,thick,smooth,discard if not={body}{3}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Earth}

      \addplot[red,thick,smooth,discard if not={body}{5}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Mars}

      \addplot[purple,thick,smooth,discard if not={body}{6}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Jupiter}

      \addplot[orange,thick,smooth,discard if not={body}{7}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Saturn}

      \addplot[blue!50,thick,smooth,discard if not={body}{8}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Uranus}

      \addplot[blue,thick,smooth,discard if not={body}{9}] table[x=x, y=y] {data/3f.dat};
      \addlegendentry{Neptune}
    \end{axis}
  \end{tikzpicture}

  \caption{25-year simulation of the outer solar system (Jupiter, Saturn, Uranus, Neptune).} \label{fig:3f-outer}
\end{figure}

\clearpage
\subsection{Perihelion precession of Mercury}
\oppg{g}

We rewrite our simulation to only handle the Mercury-Sun two-body system, with the Sun fixed at the origin. However,
we modify the gravitational force wih a corrective factor for relativistic effects, so that it becomes
\begin{equation}
  F_G = \frac{G M_{\mathrm{mercury}} M_\sun}{r^2} \cdot \left( 1 + \frac{3 \cdot |\vec{r} \times \vec{v}|^2}{r^2 c^2} \right)
\end{equation}

We then run the simulation and note where Mercury is when it is at perihelion (at speed \SI{12.44}{\astronomicalunit . \year^{-1}}).
When we do this for normal Newtonian gravity and gravity with relativistic corrections, we see that Mercury is at perihelion at the
angles shown in figure \ref{fig:3g}.

We see that perihelion with the normal Netwonian gravity barely moves at all, keeping within $\approx \nicefrac{1}{1000}$ of a degree from where it started.
Even this difference is most likely a result of errors in the numeric calculations we do, and so we can assume that perihelion occurs at the same position every time.

On the other hand, with the relativistically corrected gravity, the angle changes at a rate equal to approximately 0.011 degrees per century; this is very close to
the 43 arc seconds (0.01194 degrees) per century we already know that Mercury's precession due to general relativity is. This precession can be explain by the fact that general relativity predicts a
slightly stronger gravitational pull than Newtonian gravity when two bodies are close to each other, which in this case will alter the shape of the orbit slightly, causing a precession of the perihelion.

\begin{figure}[!ht]
  \centering

  \tikzpicturedependsonfile{data/3g.dat}
  \begin{tikzpicture}
    \begin{groupplot}[
        group style={
          group size=1 by 2,
          xlabels at=edge bottom,
          xticklabels at=edge bottom,
          ylabels at=edge left,
          yticklabels at=edge left,
          vertical sep=1.5cm,
          % horizontal sep=0pt,
        },
        width=\textwidth,
        height=0.5\textwidth,
        xmin=0, xmax=100,
        xlabel={Time $t$ \; / \; \si{\year}},
        ymin=-1e-2, ymax=3e-2,
        ylabel={Angle $\theta_p$ at perihelion (degrees)},
        scaled y ticks = false,
        y tick label style={
          /pgf/number format/fixed,
          /pgf/number format/1000 sep = \thinspace
        },
        grid=both,
      ]

      \nextgroupplot[title={Newtonian gravity}]
        \addplot[red] table[x=t, y=angle] {data/3g-normal.dat};
        \addlegendentry{Angle at perihelion}

        \addplot[black,dashed,samples=2,domain=0:100] {0};
        \addlegendentry{Expected angle at perihelion}

      \nextgroupplot[title={Gravity with general relativistic correction}]
        \addplot[red] table[x=t, y=angle] {data/3g.dat};
        \addlegendentry{Angle at perihelion}

        \addplot[black,dashed,samples=2,domain=0:100] {0.0119444*(x/100)};
        \addlegendentry{Expected angle at perihelion}
    \end{groupplot}
  \end{tikzpicture}

  \caption{Angle of Mercury at perihelion as it changes over time, in the case of normal Newtonian gravity (top) and Newtonian gravity with relativistic corrections  (bottom).} \label{fig:3g}
\end{figure}

\clearpage
\appendix
\section{Appendix}

All files used in this project can be found at \url{https://github.com/frxstrem/fys3150/tree/master/project3}.
The following code files are used:
\begin{enumerate}
  \item[b)]
    \begin{itemize}
      \item \href{https://github.com/frxstrem/fys3150/tree/master/project3/code/3b.cc}{\tt code/3b.cc}
    \end{itemize}

  \item[d)]
    \begin{itemize}
      \item \href{https://github.com/frxstrem/fys3150/tree/master/project3/code/3b.cc}{\tt code/3d.cc}
    \end{itemize}

  \item[e)]
    \begin{itemize}
      \item \href{https://github.com/frxstrem/fys3150/tree/master/project3/code/3b.cc}{\tt code/3e.cc}
    \end{itemize}

  \item[f)]
    \begin{itemize}
      \item \href{https://github.com/frxstrem/fys3150/tree/master/project3/code/3b.cc}{\tt code/3f.cc}
    \end{itemize}

  \item[g)]
    \begin{itemize}
      \item \href{https://github.com/frxstrem/fys3150/tree/master/project3/code/3b.cc}{\tt code/3g.cc}
    \end{itemize}
\end{enumerate}

In addition, the following files contains classes used in all of (or most of) the above programs:
\begin{itemize}
  \item \href{https://github.com/frxstrem/fys3150/tree/master/project3/code/CelestialBody.cc}{\tt code/CelestialBody.cc}
    \begin{itemize}
      \item \texttt{CelestialBody}
    \end{itemize}
  \item \href{https://github.com/frxstrem/fys3150/tree/master/project3/code/SolarSystem.cc}{\tt code/SolarSystem.cc}
    \begin{itemize}
      \item \texttt{SolarSystem}
      \item \texttt{RelativisticSolarSystem}
    \end{itemize}
  \item \href{https://github.com/frxstrem/fys3150/tree/master/project3/code/Solver.cc}{\tt code/Solver.cc}
    \begin{itemize}
      \item \texttt{EulerSolver}
      \item \texttt{VerletSolver}
    \end{itemize}
\end{itemize}

% \clearpage
% \printbibliography[heading=bibnumbered,title=Bibliography]

\end{document}