\documentclass[aspectratio=169]{beamer}

\input{preamble.tex}

\title{OpenMP for Computational Scientists}
\subtitle{3: Vectorisation and NUMA}

\begin{document}

\frame{\titlepage}

%-------------------------------------------------------------------------------
\begin{frame}[fragile]
\frametitle{Previous exercise}

Take your parallel 5-point stencil, and implement a reduction:
\begin{minted}[frame=single,breaklines,fontsize=\small]{fortran}
total = 0.0
!$omp parallel do collapse(2) reduction(+:total)
do i = 1, nx
    do j = 1, ny
    Atmp(i,j) = (A(i-1,j) + A(i+1,j) + A(i,j) + A(i,j-1) + A(i,j+1)) / 5.0
    total = total + Atmp(i,j)
    end do
end do
!$omp end parallel do
\end{minted}

\begin{itemize}
    \item Well done if you managed this!
    \item 5-point stencil is simple, but captures the \emph{essence} of more complicated codes.
    \item Extension: did anyone try the parallelising the Jacobi solver?
\end{itemize}

\end{frame}

%-------------------------------------------------------------------------------
%-------------------------------------------------------------------------------
\section{Vectorisation}
\begin{frame}
\frametitle{Vectorisation}
$$C=A+B$$
\begin{columns}
\begin{column}{0.5\textwidth}
Scalar operations \\
\begin{adjustbox}{max totalsize={\textwidth}{0.6\textheight},center}

\begin{tikzpicture}
  \draw (-0.5,2) rectangle (0.5,3);
  \draw (1,2) rectangle (2,3);
  \draw[->] (0,2) -- (.74,1.2);
  \draw[->] (1.5,2) -- (.76,1.2);
  \draw (.75,.75) circle (.4cm);
  \draw (.75,.75) node {$+$};
  \draw[->] (.75,0.3) -- (.75,-0.5);
  \draw (.25,-1.5) rectangle (1.25,-0.5);
\end{tikzpicture}
\end{adjustbox}
\end{column}

\begin{column}{0.5\textwidth}
Vector operations \\
\begin{adjustbox}{max totalsize={\textwidth}{0.6\textheight},center}
\begin{tikzpicture}
  \draw[step=1cm] (0,2) grid (4,3);
  \draw[step=1cm] (0,0) grid (4,1);
  \draw[->] (2,0) -- (2,-0.6);
  \draw[->] (0,2.5) -- (-0.5,2.5) -- (-0.5,-1) -- (1.6,-1);
  \draw (2,-1) circle (.4cm);
  \draw (2,-1) node {$+$};
  \draw[->] (2,-1.4) -- (2,-1.9);
  \draw[step=1cm] (0,-3) grid (4,-2);
\end{tikzpicture}
\end{adjustbox}
\end{column}
\end{columns}

\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
\frametitle{Why vectorise?}
\begin{itemize}
  \item Vectorisation gives you more compute per cycle.
  \item Hence may increase the FLOP/s rate of the processor.
  \item Also results in fewer instructions to process (less pressure on instruction decode units).
  \item Vectors help make good use of the memory hierarchy (often the main benefit).
  \item Vectorisation helps you write code which has good access patterns to maximise bandwidth.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
\frametitle{Auto-vectorisation}
\begin{itemize}
  \item Modern compilers are very good at automatically vectorising your loops.
  \item Fortran helps as arrays can not alias (overlap), unlike C.
  \item But compiler needs to be sure it's safe to vectorise.
  \item Read compiler reports to see if it's already vectorising.
    \begin{itemize}
      \item Intel: \mintinline{bash}|-qopt-report=5|
      \item Cray: \mintinline{bash}|-hlist=a|
      \item GNU (old): \mintinline{bash}|-ftree-vectorizer-verbose=2|
      \item GNU (new): \mintinline{bash}|-fopt-info-vec|
      \item Clang: \mintinline{bash}|-Rpass=loop-vectorize| \mintinline{bash}|-Rpass-missed=loop-vectorize| \mintinline{bash}|-Rpass-analysis=loop-vectorize|
    \end{itemize}
  \item Often the memory access pattern prevents (efficient) auto-vectorisation.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\subsection{OpenMP SIMD}
\begin{frame}[fragile]
\frametitle{OpenMP SIMD}
\begin{itemize}
  \item Sometimes the compiler needs help in confirming loops are vectorisable.
  \item OpenMP \mintinline{fortran}|simd| constructs give this information.
  \item Can combine with \mintinline{fortran}|parallel do| construct to ensure a parallel vector loop: \mintinline{fortran}|omp parallel do simd|
  \item Generally want to vectorise inner loops and parallelise outer loops.
\end{itemize}

\begin{minted}[frame=single]{fortran}
!$omp simd
do i = 1, N
  C(i) = A(i)+B(i)
end do
!$omp end simd
\end{minted}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}[fragile]
\frametitle{SIMD functions}
Say you've written an update function to update values in the loop:
\begin{minted}[frame=single]{fortran}
do i = 1, N
  A(i) = magic_maths(A(i))
end do
\end{minted}

\begin{itemize}
  \item The situation gets complicated.
  \item If the function is small, then likely inlined and loop will auto-vectorise.
  \item Otherwise need to use the \mintinline{fortran}|simd| construct, but need compiler to create a vector version of the function.
\end{itemize}

\begin{minted}[frame=single]{fortran}
function magic_maths(value) result(r)
!$omp declare simd(magic_maths)
  implicit none
  real(kind=8) :: value, r
  r = value * value
end function
\end{minted}

\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}[fragile]
\frametitle{SIMD clauses}
\begin{itemize}
  \item All the usual data-sharing and reduction clauses can be applied.
  \item \mintinline{fortran}|safelen(4)|: distance between iterations where its safe to vectorise.
  \begin{minted}[frame=single]{fortran}
  !$omp simd safelen(4)
  do i = 1, N-4
    A(i) = A(i) + A(i+4)
  end do
  !$omp end simd
  \end{minted}
  \item \mintinline{fortran}|simdlen(4)|: preferred iterations to be performed concurrently as a vector.
  Specifying explicit vector lengths builds in obsolescence to the code as hardware vector lenghts continually change --- don't recommend using this clause.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}[fragile]
\frametitle{SIMD clauses}
\begin{itemize}
  \item \mintinline{fortran}|linear(var)|: variable is private and linear to the loop iterator.
  \begin{minted}[frame=single]{fortran}
  !$omp simd linear(j)
  do i = 1, N
    j = j + 1
    A(j) = B(i)
  end do
  !$omp end simd
  \end{minted}
  \item \mintinline{fortran}|aligned(var)|: says the array is aligned.
  \item \mintinline{fortran}|uniform(var)|: for \mintinline{fortran}|declare simd| construct, the variable is the same in all vector lanes.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
\frametitle{SIMD summary}

\begin{itemize}
  \item Sometimes need to force the compiler to auto-vectorise (the correct) loop with the \mintinline{fortran}|simd| construct.
  \item As with \mintinline{fortran}|parallel|, you are telling the compiler it is safe to vectorise and to ignore its data dependancy analysis.
  \item Check the compiler report before and after the check it did the right thing!
  \item Use \mintinline{fortran}|declare simd| and appropriate clauses if you need to create vectorised versions of functions.
  \begin{itemize}
    \item The clauses can give more information to the compiler so it does a better job.
  \end{itemize}
\end{itemize}

\end{frame}

%-------------------------------------------------------------------------------
\section{Derived types}
\begin{frame}[fragile]
\frametitle{Derived types}
2D grid of cells, each cell containing 4 different values.
\begin{minted}[frame=single,linenos,fontsize=\footnotesize]{fortran}
type cell
  real(kind=8) :: property1
  real(kind=8) :: property2
  real(kind=8) :: property3
  real(kind=8) :: property4
end type

type(cell), allocatable :: grid(:,:)

do j = 1, ny
  do i = 1, nx
    grid(i,j)%property1 = update_1()
    grid(i,j)%property2 = update_2()
    grid(i,j)%property3 = update_3()
    grid(i,j)%property4 = update_4()
  end do
end do
\end{minted}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
\frametitle{Derived types}
\begin{itemize}
  \item What do Fortran derived types look like in memory?
  \item Organised as an array of structures.
  \item<2-> What happens when we vectorise our loop over cells?
\end{itemize}

\begin{adjustbox}{max width={\textwidth}}
\begin{tikzpicture}
  \draw[step=1cm] (0,0) grid (13,1);
  \foreach \i in {0,4,8,12} {
    \draw (\i+.5,.5) node {P1};
  }
  \foreach \i in {0,4,8} {
    \draw (\i+1.5,.5) node {P2};
    \draw (\i+2.5,.5) node {P3};
    \draw (\i+3.5,.5) node {P4};
  }

  \foreach \i in {0,4,8,12} {
    \draw<3->[->] (\i+.5,-1) -- (\i+.5,0);
  }
\end{tikzpicture}
\end{adjustbox}

\begin{itemize}
  \item<4-> The \mintinline{fortran}|property1| values are gathered into a vector register.
  \item<5-> After the computation, the results are scattered back into memory.
  \item<6-> A cache line is 64 bytes, so only the first two values are on the first cache line.
  \item<6-> Must read two cache lines to fill the vector up.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}[fragile]
\frametitle{Structure of arrays}
Switch type around to have an array per property.
\begin{minted}[frame=single,linenos,fontsize=\small]{fortran}
type grid
  real(kind=8), allocatable :: property1(:,:)
  real(kind=8), allocatable :: property2(:,:)
  real(kind=8), allocatable :: property3(:,:)
  real(kind=8), allocatable :: property4(:,:)
end type

do j = 1, ny
  do i = 1, nx
    grid%property1(i,j) = update_1()
    grid%property2(i,j) = update_2()
    grid%property3(i,j) = update_3()
    grid%property4(i,j) = update_4()
  end do
end do
\end{minted}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
\frametitle{Structure of arrays}
\begin{itemize}
  \item Order of data in memory has changed.
  \item<2-> What happens when we vectorise?
\end{itemize}

\begin{adjustbox}{max width={\textwidth}}
\begin{tikzpicture}
  \draw[step=1cm] (0,0) grid (13,1);
  \foreach \i in {0,...,4} {
    \draw (\i+.5,.5) node {P1};
  }
  \draw (5.5,.5) node {\dots};

  \foreach \i in {5,...,9} {
    \draw (\i+1.5,.5) node {P2};
  }
  \draw (11.5,.5) node {\dots};

  \foreach \i in {10} {
    \draw (\i+2.5,.5) node {P3};
  }

  \foreach \i in {0,...,3} {
    \draw<3->[->] (\i+.5,-1) -- (\i+.5,0);
  }
\end{tikzpicture}
\end{adjustbox}

\onslide<4->{
\begin{itemize}
  \item Coalesced memory accesses are key for high performance code.
  \item Adjacent vector lanes read adjacent memory locations.
  \item A cache line is 64 bytes, so can fill the vector from a single cache line.
  \item More efficient vectorisation.
\end{itemize}
}
\end{frame}

%-------------------------------------------------------------------------------
\section{Memory access patterns}
\begin{frame}[fragile]
\frametitle{Memory access patterns}
\begin{minted}{fortran}
do i = 1, N
  val = A(i)
end do
\end{minted}
\begin{adjustbox}{max width={\textwidth}}
\begin{tikzpicture}
  \draw[step=1cm] (-3,0) grid (11,1);
  \draw[dashed] (0,-.5) -- (0,1.5);
  \draw[dashed] (8,-.5) -- (8,1.5);
  \draw (0,-1) node {64 byte boundary};
  \foreach \i in {0,...,7} {
    \draw[->] (\i+.5,2) -- (\i+.5,1.2);
  }
\end{tikzpicture}
\end{adjustbox}
\begin{itemize}
  \item Ideal memory access pattern.
  \item All access is coalesced.
  \item Vectors are aligned to cache line boundary.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}[fragile]
\frametitle{Memory access patterns}
\begin{minted}{fortran}
do i = 1, N
  val = A(i+3)
end do
\end{minted}
\begin{adjustbox}{max width={\textwidth}}
\begin{tikzpicture}
  \draw[step=1cm] (-3,0) grid (11,1);
  \draw[dashed] (0,-.5) -- (0,1.5);
  \draw[dashed] (8,-.5) -- (8,1.5);
  \draw (0,-1) node {64 byte boundary};
  \foreach \i in {0,...,7} {
    \draw[->] (\i+.5,2) -- (3+\i+.5,1.2);
  }
\end{tikzpicture}
\end{adjustbox}
\begin{itemize}
  \item OK memory access pattern.
  \item All access is coalesced, but split across cache lines.
  \item Still get good use of cache lines, but not as efficient as aligned version.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}[fragile]
\frametitle{Memory access patterns}
\begin{minted}{fortran}
do i = 1, N
  val = A(j,i) ! equiv. A(j+3*i)
end do
\end{minted}
\begin{adjustbox}{max width={\textwidth}}
\begin{tikzpicture}
  \draw[step=1cm] (-3,0) grid (11,1);
  \draw[dashed] (0,-.5) -- (0,1.5);
  \draw[dashed] (8,-.5) -- (8,1.5);
  \draw (0,-1) node {64 byte boundary};
  \foreach \i in {0,...,3} {
    \draw[->] (\i+.5,2) -- (3*\i+.5,1.2);
  }
\end{tikzpicture}
\end{adjustbox}
\begin{itemize}
  \item Strided access results in multiple memory transactions.
  \item Kills throughput due to poor reuse of cached data.
  \item Very easy to fall into this trap with multi-dimensional arrays.
  \item Check your strides!
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}[fragile]
\frametitle{Memory access patterns}
\begin{minted}{fortran}
do i = 1, N
  val = A(B(i))
end do
\end{minted}
\begin{adjustbox}{max width={\textwidth}}
\begin{tikzpicture}
  \draw[step=1cm] (-3,0) grid (11,1);
  \draw[dashed] (0,-.5) -- (0,1.5);
  \draw[dashed] (8,-.5) -- (8,1.5);
  \draw (0,-1) node {64 byte boundary};
  \draw[->] (0.5,2) -- (-3.5,1.2);
  \draw[->] (1.5,2) -- (3.5,1.2);
  \draw[->] (2.5,2) -- (0.5,1.2);
  \draw[->] (3.5,2) -- (8.5,1.2);
  \draw[->] (4.5,2) -- (-1.5,1.2);
  \draw[->] (5.5,2) -- (7.5,1.2);
  \draw[->] (6.5,2) -- (1.5,1.2);
  \draw[->] (7.5,2) -- (-2.5,1.2);
\end{tikzpicture}
\end{adjustbox}
\begin{itemize}
  \item Essentially random access to memory.
  \item Little reuse of cache lines.
  \item Unpredictable pattern, so hardware prefetchers won't work efficiently.
  \item Very challenging!
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\section{NUMA}
\begin{frame}
\frametitle{NUMA Architecture}

Recall this cartoon of a dual-socket, shared memory system:
\begin{center}
\begin{tikzpicture}
  % Draw 4 cores for socket 0
  \draw (0,0) rectangle (1,1);
  \draw (1,0) rectangle (2,1);
  \draw (0,1) rectangle (1,2);
  \draw (1,1) rectangle (2,2);

  % Draw 4 cores for socket 1
  \draw (3,0) rectangle (4,1);
  \draw (4,0) rectangle (5,1);
  \draw (3,1) rectangle (4,2);
  \draw (4,1) rectangle (5,2);

  % Draw large memory
  \draw (-0.5,3) rectangle (5.5,4);
  \draw (2.5,3.5) node {Memory};

  % Connect sockets to memory
  \draw (1,2) -- (1,3);
  \draw (4,2) -- (4,3);
  \draw[dashed] (2,1) -- (3,1); % QPI

\end{tikzpicture}
\end{center}

\emph{All} threads (each running on a core) can access the same memory.

\end{frame}
%-------------------------------------------------------------------------------

\begin{frame}
\frametitle{NUMA Architecture}
\begin{itemize}
  \item In reality on a dual-socket system each \emph{socket} is physically connected to half of the memory.
  \item Still shared memory: all cores can access all the memory.
  \item A core in the first socket wanting memory attached to the other socket must:
    \begin{itemize}
      \item Go via the socket-to-socket interconnect.
      \item Access memory via the other socket's memory controllers.
    \end{itemize}
  \item Accessing memory from other socket is slower than access from own socket.
\end{itemize}
\begin{center}
\resizebox{!}{3.5cm}{
\begin{tikzpicture}
  % Draw 4 cores for socket 0
  \foreach \i in {0,1,3,4} {
    \foreach \j in {0, 1} {
      \draw (\i,\j) rectangle (\i+1,\j+1);
    }
  }

  % Draw sockets around cores
  \draw (-0.2, -0.2) rectangle (2.2, 2.2);
  \draw (2.8, -0.2) rectangle (5.2, 2.2);

  % Draw large memory
  \draw (-0.5,3) rectangle (2.3,4);
  \draw (2.7,3) rectangle (5.5,4);
  \draw[dashed] (-0.7,2.8) rectangle (5.7,4.2);

  % Connect sockets to memory
  \draw (1,2.2) -- (1,3);
  \draw (4,2.2) -- (4,3);
  \draw[dashed] (2.2,1) -- (2.8,1); % QPI

  % Show memory shared
  \pause
  \draw[fill=red] (0.5,3.2) rectangle (1,3.7);
  \draw (3.5,1.5) node {Read};
  \pause
  \draw[->,red,thick] (0.7,3.2) -- (0.7,2.1) -- (2.1,2.1) -- (2.1,1.1) -- (2.9,1.1) -- (3.5,1.2);

\end{tikzpicture}
}
\end{center}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
\frametitle{Memory allocation}
\begin{itemize}
  \item What happens when you run \mintinline{fortran}|allocate(A(1:N))|?
  \pause
  \item Allocating memory does not necessarily allocate memory!
  \item Memory is allocated when it's first used (i.e. \mintinline{fortran}|A(i) = 1.0|), one \emph{page} at a time.
  \item OS tends to use a \emph{first touch policy}.
  \item Memory is allocated in the closest NUMA region to the thread that first touches the data.
  \item Ideally want threads to use data in local NUMA region to reduce socket-to-socket interconnect transfers.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\subsection{First touch}
\begin{frame}[fragile]
\frametitle{Taking advantage of first touch}
Parallelising your data initialisation routine might mean your main loops go faster!

\begin{minted}[fontsize=\small,linenos,frame=single]{fortran}
! Allocate and initialise vectors
allocate(A(N), B(N), C(N))
!$omp parallel do
do i = 1, N
  A(i) = 1.0
  B(i) = 2.0
  C(i) = 0.0
end do
!$omp end parallel do

! Vector add
!$omp parallel do
do i = 1, N
  C(i) = A(i) + B(i)
end do
!$omp end parallel do
\end{minted}

\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
\frametitle{NUMA-aware}
\begin{itemize}
  \item Parallelise your initialisation routines the same way you parallelise the main loops.
  \item This means each thread touches the same data in initialisation and compute.
  \item Should reduce the number of remote memory accesses needed and improve run times.
  \item But, OS is allowed to move threads around cores, and between sockets.
  \item This will mess up your NUMA aware code!
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\section{Thread affinity}
\begin{frame}
\frametitle{Pinning threads}
\begin{itemize}
  \item OpenMP gives you the controls to pin threads to specific cores.
  \item Exposed as \emph{places} and \emph{thread pinning policy} to those places.
  \item By default there is one place consisting of all the cores.
  \item Use the \mintinline{bash}|OMP_PROC_BIND| environment variable to set pinning for all \mintinline{fortran}|parallel| regions.
  \item Can use the \mintinline{bash}|proc_bind| clause for control of specific regions, but advise against this.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
\frametitle{OMP\_PROC\_BIND}
\begin{itemize}
  \item \mintinline{bash}|OMP_PROC_BIND=false|: Often the default; threads may move! \mintinline{fortran}|proc_bind| clauses ignored.
  \item \mintinline{bash}|OMP_PROC_BIND=true|: Threads won't move, and follow \mintinline{fortran}|proc_bind| clauses or else the implementation default pinning.
  \item \mintinline{bash}|OMP_PROC_BIND=master|: Threads pinned to same place as master thread.
  \item \mintinline{bash}|OMP_PROC_BIND=close|: Threads are assigned to places close to the master thread.
  If \mintinline{bash}|OMP_NUM_THREADS.eq.ncores|: thread 0 will pin to core 0; thread 1 will pin to core 1; etc
  \item \mintinline{bash}|OMP_PROC_BIND=spread|: Threads are assigned to places ``sparsely''.
  If \mintinline{bash}|OMP_NUM_THREADS.eq.ncores|: thread 0 will pin to socket 0 core 0; thread 1 will pin to socket 1 core 0; thread 2 will pin to socket 0 core 1; etc.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
\frametitle{Places}
\begin{itemize}
  \item The affinity (policy) defines how threads are assigned to places.
  \item Places allow you to divide up the hardware resource, so that threads can be assigned to them.
  \item Default: one place with all cores.
  \item Use \mintinline{bash}|OMP_PLACES| environment variable to control.
  \item \mintinline{bash}|OMP_PLACES=thread|: each place is a single hardware thread.
  \item \mintinline{bash}|OMP_PLACES=cores|: each place is a single core (containing one or more hardware threads).
  \item \mintinline{bash}|OMP_PLACES=sockets|: each place contains the cores of a single socket.
  \item Can also use list notation: \mintinline{bash}|OMP_PLACES="{0:4},{4:4},{8:4},{12:4}"|
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
\frametitle{Thread pinning summary}
\begin{itemize}
  \item In general, going to want to just use \mintinline{bash}|OMP_PROC_BIND=true|.
  \item Sometimes \mintinline{bash}|spread| or \mintinline{bash}|close| gets better performance.
  \item Pinning rules can get complicated when there are multiple places, so prefer to use the predefined values.
  \item Most effective with a NUMA-aware implementation.
  \item Also helps reduce run-to-run timing variability.
  \item But must be careful with MPI+OpenMP pinning.
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\section{Hybrid MPI and OpenMP}
\begin{frame}
\frametitle{Why combine MPI+OpenMP}
\begin{itemize}
  \item Supercomputers are often constructed with a hierarchical structure:
    \begin{itemize}
      \item Shared memory nodes connected with a network.
    \end{itemize}
  \item Need MPI (or similar) to communicate between distributed nodes.
  \item With multi-core, could just run MPI everywhere (flat MPI).
  \item But there are advantages to running \emph{hybrid} MPI and OpenMP:
    \begin{itemize}
      \item Larger fewer messages to take advantage of network bandwidth.
      \item Fewer MPI ranks to manage (fewer to synchronise and for collectives).
      \item Can avoid memory copies for intra-node communication.
      \item Reduced memory footprint.
      \item Parallelise other problem dimensions not decomposed with MPI.
    \end{itemize}
\end{itemize}
\end{frame}

%-------------------------------------------------------------------------------
\begin{frame}
  \frametitle{Thread support levels}
  \begin{itemize}
    \item \mintinline{fortran}|MPI_THREAD_SINGLE| \\
    Only one thread will execute (no threads allowed).

    \item \mintinline{fortran}|MPI_THREAD_FUNNELED| \\
    May spawn threads, but only the original process may call MPI routines: the one that called \mintinline{fortran}|MPI_Init|.

    \item \mintinline{fortran}|MPI_THREAD_SERIALIZED| \\
    May spawn threads and any thread can make MPI calls, but only one at a time. \emph{Your} responsibility to synchronise.

    \item \mintinline{fortran}|MPI_THREAD_MULTIPLE| \\
    May spawn threads and any thread can make MPI calls. The MPI library has to deal with being called in parallel.
  \end{itemize}

  Remember to make sure ranks still match the MPI communications to avoid deadlock.

  \end{frame}

%-------------------------------------------------------------------------------
\section{Exercise}
\begin{frame}
\frametitle{Exercise}
\begin{itemize}
  \item Take your parallel 5-point stencil code and optimise it.
  \item Think about:
    \begin{itemize}
      \item Memory access patterns
      \item Vectorisation
      \item NUMA
    \end{itemize}
  \item Note down the performance differences your optimisations make.
  \item Calculate the achieved memory bandwidth of your stencil code.
  \item Extension: consider these optimisaions for the Jacobi solver.
\end{itemize}
\end{frame}

\end{document}


\documentclass[12pt,presentation]{beamer}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{fixltx2e}
\usepackage{graphicx}
\usepackage{longtable}
\usepackage{float}
\usepackage{wrapfig}
\usepackage{soul}
\usepackage{textcomp}
\usepackage{marvosym}
\usepackage{wasysym}
\usepackage{latexsym}
\usepackage{amssymb}
\usepackage{hyperref}
\tolerance=1000
\usepackage[english]{babel} \usepackage{ae,aecompl}
\usepackage{mathpazo,courier,euler} \usepackage[scaled=.95]{helvet}
\usepackage{listings}
\lstset{language=Python, basicstyle=\ttfamily\bfseries,
commentstyle=\color{red}\itshape, stringstyle=\color{green},
showstringspaces=false, keywordstyle=\color{blue}\bfseries}
\providecommand{\alert}[1]{\textbf{#1}}

\title{SEES: Test Driven Development}
\author{FOSSEE}

\usetheme{Warsaw}\usecolortheme{default}\useoutertheme{infolines}\setbeamercovered{transparent}

\AtBeginSection[]
{
  \begin{frame}<beamer>
    \frametitle{Outline}
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\maketitle

\begin{frame}
\frametitle{Outline}
\setcounter{tocdepth}{3}
\tableofcontents
\end{frame}

\section{Introduction}

\begin{frame}
  \frametitle{Objectives}
  At the end of this section, you will be able to:
  \begin{itemize}
  \item Write your code using the TDD paradigm.
  \item Use the nose module to test your code.
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{What is TDD?}
  The basic steps of TDD are roughly as follows --
  \begin{enumerate}
  \item<1-> Decide upon feature to implement and methodology of
    testing
  \item<2-> Write tests for feature decided upon
  \item<3-> Just write enough code, so that the test can be run, but it fails.
  \item<4-> Improve the code, to just pass the test and at the same time
    passing all previous tests.
  \item<5-> Run the tests to see, that all of them run successfully.
  \item<6-> Refactor the code you've just written -- optimize the algorithm,
    remove duplication, add documentation, etc.
  \item<7-> Run the tests again, to see that all the tests still pass.
  \item<8-> Go back to 1.
  \end{enumerate}
\end{frame}

\section{First Test}

\begin{frame}[fragile]
  \frametitle{First Test -- GCD}
  \begin{itemize}
  \item simple program -- GCD of two numbers
  \item What are our code units?
    \begin{itemize}
    \item Only one function \texttt{gcd}
    \item Takes two numbers as arguments
    \item Returns one number, which is their GCD
    \end{itemize}
\begin{lstlisting}
c = gcd(44, 23)
\end{lstlisting}
  \item c will contain the GCD of the two numbers.
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Test Cases}
  \begin{itemize}
  \item Important to have test cases and expected outputs even before
    writing the first test!
  \item $a=48$, $b=48$, $GCD=48$
  \item $a=44$, $b=19$, $GCD=1$
  \item Tests are just a series of assertions
  \item True or False, depending on expected and actual behavior
  \end{itemize}

\end{frame}

\begin{frame}[fragile]
  \frametitle{Test Cases -- general idea}
\begin{lstlisting}
tc1 = gcd(48, 64)
if tc1 != 16:
    print "Failed for a=48, b=64. Expected 16. \
    Obtained %d instead." % tc1
    exit(1)

tc2 = gcd(44, 19)
if tc2 != 1:
    print "Failed for a=44, b=19. Expected 1. \
    Obtained %d instead." % tc2
    exit(1)

print "All tests passed!"
\end{lstlisting}
\begin{itemize}
\item The function \texttt{gcd} doesn't even exist!
\end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Test Cases -- code}
  \begin{itemize}
      \item Let us make it a function!
      \item Use assert!
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Test Cases -- code}
\begin{lstlisting}
# gcd.py
def test_gcd():
    assert gcd(48, 64) == 16
    assert gcd(44, 19) == 1

test_gcd()
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Stubs}
  \begin{itemize}
  \item First write a very minimal definition of \texttt{gcd}
    \begin{lstlisting}
def gcd(a, b):
    pass
    \end{lstlisting}
  \item Written just, so that the tests can run
  \item Obviously, the tests are going to fail
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{\texttt{gcd.py}}
\begin{lstlisting}
def gcd(a, b):
    pass

def test_gcd():
    assert gcd(48, 64) == 16
    assert gcd(44, 19) == 1

if __name__ == '__main__':
    test_gcd()
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{First run}
\begin{lstlisting}
$ python gcd.py
Traceback (most recent call last):
  File "gcd.py", line 9, in <module>
    test_gcd()
  File "gcd.py", line 5, in test_gcd
    assert gcd(48, 64) == 16
AssertionError
\end{lstlisting} %$

  \begin{itemize}
  \item We have our code unit stub, and a failing test.
  \item The next step is to write code, so that the test just passes.
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Euclidean Algorithm}
  \begin{itemize}
  \item Modify the \texttt{gcd} stub function
  \item Then, run the script to see if the tests pass.
  \end{itemize}
\begin{lstlisting}
def gcd(a, b):
    if a == 0:
        return b
    while b != 0:
        if a > b:
            a = a - b
        else:
            b = b - a
    return a
\end{lstlisting}
\begin{lstlisting}
$ python gcd.py
All tests passed!
\end{lstlisting} %$
  \begin{itemize}
  \item \alert{Success!}
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Euclidean Algorithm -- Modulo}
  \begin{itemize}
  \item Repeated subtraction can be replaced by a modulo
  \item modulo of \texttt{a\%b} is always less than b
  \item when \texttt{a < b}, \texttt{a\%b} equals \texttt{a}
  \item Combine these two observations, and modify the code
\begin{lstlisting}
def gcd(a, b):
    while b != 0:
        a, b = b, a % b
    return a
\end{lstlisting}
  \item Check that the tests pass again
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Euclidean Algorithm -- Recursive}
  \begin{itemize}
  \item Final improvement -- make \texttt{gcd} recursive
  \item More readable and easier to understand
\begin{lstlisting}
def gcd(a, b):
    if b == 0:
        return a
    return gcd(b, a%b)
\end{lstlisting}
  \item Check that the tests pass again
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Document \texttt{gcd}}
  \begin{itemize}
  \item Undocumented function is as good as unusable
  \item Let's add a docstring \& We have our first test!
  \end{itemize}
\begin{lstlisting}
def gcd(a, b):
    """Returns the Greatest Common Divisor of the
    two integers passed as arguments.

    Args:
      a: an integer
      b: another integer

    Returns: Greatest Common Divisor of a and b
    """
    if b == 0:
        return a
    return gcd(b, a%b)
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Persistent Test Cases}
  \begin{itemize}
  \item Tests should be pre-determined and written, before the code
  \item The file shall have multiple lines of test data
  \item Separates the code from the tests
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
    \frametitle{Separate \texttt{test\_gcd.py}}
\begin{lstlisting}
from gcd import gcd

def test_gcd():
    assert gcd(48, 64) == 16
    assert gcd(44, 19) == 1

if __name__ == '__main__':
    test_gcd()
\end{lstlisting}
\end{frame}

\section{Python Testing Frameworks}

\begin{frame}[fragile]
  \frametitle{Python Testing Frameworks}
  \begin{itemize}
  \item Testing frameworks essentially, ease the job of the user
  \item Python provides two frameworks for testing code
    \begin{itemize}
    \item \texttt{unittest} framework
    \item \texttt{doctest} module
    \end{itemize}
\item \texttt{nose} is a package to help test
  \end{itemize}
\end{frame}

\subsection{\texttt{nose}}

\begin{frame}[fragile]
  \frametitle{\texttt{nose} tests}
  \begin{itemize}
  \item It is not easy to organize, choose and run tests scattered
    across multiple files.
  \item \texttt{nose} module aggregate these tests automatically
  \item Can aggregate \texttt{unittests} and \texttt{doctests}
  \item Allows us to pick and choose which tests to run
  \item Helps output the test-results and aggregate them in various
    formats
  \item Not part of the Python distribution itself
\begin{lstlisting}
$ apt-get install python-nose
\end{lstlisting} %$
  \item Run the following command in the top level directory
\begin{lstlisting}
$ nosetests
\end{lstlisting} %$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{\texttt{py.test}}
  \begin{itemize}
  \item Another test runner with different features
  \end{itemize}
\end{frame}
\end{document}


\documentclass[a4paper,12pt]{article} % This defines the style of your paper

\usepackage[top = 2.5cm, bottom = 2.5cm, left = 2.5cm, right = 2.5cm]{geometry}
\usepackage[utf8]{inputenc} %utf8 % lettere accentate da tastiera
\usepackage[english]{babel} % lingua del documento
\usepackage[T1]{fontenc} % codifica dei font

\usepackage{multirow} % Multirow is for tables with multiple rows within one
%cell.
\usepackage{booktabs} % For even nicer tables.

\usepackage{graphicx}

\usepackage{setspace}
\setlength{\parindent}{0in}

\usepackage{float}

\usepackage{fancyhdr}

\usepackage{caption}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{mathtools}
\usepackage{color}

\usepackage[hidelinks]{hyperref}
\usepackage{csquotes}
\usepackage{subfigure}

\pagestyle{fancy}

\setlength\parindent{24pt}

\fancyhf{}

\lhead{\footnotesize Deep Learning Lab: Assignment 1}

\rhead{\footnotesize Giorgia Adorni}

\cfoot{\footnotesize \thepage}

\begin{document}

	\thispagestyle{empty}
	\noindent{
	\begin{tabular}{p{15cm}}
		{\large \bf Deep Learning Lab} \\
		Università della Svizzera Italiana \\ Faculty of Informatics \\ \today  \\
		\hline
		\\
	\end{tabular}

	\vspace*{0.3cm}

	\begin{center}
		{\Large \bf Assignment 1: Polynomial Regression}
		\vspace{2mm}

		{\bf Giorgia Adorni (giorgia.adorni@usi.ch)}

	\end{center}
}
	\vspace{0.4cm}

	%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
	%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

	\noindent {Consider the polynomial $p$ given by}
	\begin{equation*}
	p(x)=x^3+2x^2-4x-8=\sum_{i=1}^4 w_i^*x^{i-1} \mbox{,}
	\end{equation*}
	where $\textbf{w}^*=[-8,-4,2,1]^T$.

	Consider also an iid dataset $\mathcal{D} = \{(x_i, y_i)\}^N_{i=1}$, where
	$y_i = p(x_i)+\epsilon_i$, and each $\epsilon_i$ is drawn from a normal
	distribution with mean zero and standard deviation $\sigma = \frac{1}{2}$.

	If the vector $\textbf{w}^*$ were unknown, linear regression could estimate it given the dataset $\mathcal{D}$. This would require applying a feature map to transform the
	original dataset $\mathcal{D}$ into an expanded dataset $\mathcal{D}'= \{(x_i, y_i)\}^N_{i=1}$ , where $x_i = [1,x_i,x_i^2,x_i^3]$.

	%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

	\section{Introduction}
	The scope of this {project} is to perform polynomial regression using a dataset $\mathcal{D}'$, in particular finding an estimate of $\textbf{w}^*=[-8,-4,2,1]^T$ supposing that such vector is unknown.\\
	An interval $[-3, 2]$ for $x_i$, a sample of size $100$ created with a seed of $0$ for training, and a sample of size $100$ created with a seed of $1$ for validation, and $\sigma = \frac{1}{2}$ were assumed.

	\section{Tuning the Learning Rate}
	The learning rate is a configurable hyper-parameter that represents the
	scaling factor of the gradient by which weights are updated during gradient
	descent.\\
	I tried to discover a suitable learning rate via trial and error, setting
	the initial number of iterations to $2000$, a relatively high value for the
	amount of parameters present.
	In the first test, I set the learning rate to a traditional default value
	of $0.1$. In this case, the value is so high that the algorithm diverges.\\
	Hence, I chose to decrease the value to $0.01$ obtaining a validation loss
	of $0.22$, that is a good enough result for the purpose of this assignment.

	\section{Iterations and Early Stopping}
	Fixed the learning rate, I decided to reduce the number of iterations using
	the \textit{early stopping} technique. This rule can to abort the
	training procedure when the performance on the validation set stops
	improving and therefore it avoids overfitting. \\
	In particular, I measured the validation loss after each iteration, keeping
	track of the lowest one, and I stopped the training when the validation
	loss did not improve, compared to the best, after $10$ steps.\\
	In this case, after $1248$ iterations the model reached the best loss of
	$0.2177$.

	\section{Loss}
	I used {TensorBoard} to display the loss curve as a function of the
	gradient
	descent iterations, for both the training and validation set, which are
	shown in Figure \ref{fig:model1-loss}.

	\begin{figure}[htb]
		\centering
		\includegraphics[width=0.65\linewidth]{../src/img/loss/model1-es-loss.jpg}
		\captionof{figure}{Training and validation loss}
		\label{fig:model1-loss}
	\end{figure}

	The two training and validation curves start at $29.69$ and $29.10$
	respectively and go down to $0.24$ and $0.22$ at the $1248\mathrm{th}$
	iteration.
	It is therefore seen that the validation loss is lower than the training
	loss. {\color{red}The reason why this happens is that the training loss is
	measured during each epoch while validation loss is computed at the end of
	the learning phase of the same epoch.}

	\section{Polynomial Regression}
	Considered the polynomial defined by $\textbf{w}^*=[-8,-4,2,1]^T$, Figure
	\ref{fig:model1-dataset} visualises the training and the validation
	datasets generated, while Figure \ref{fig:model1-polynomial}, shows
	the polynomial defined by $\textbf{w}^*$ and the polynomial estimated
	$\hat{\textbf{w}}$.

	\begin{figure}[H]
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=1\linewidth]{../src/img/model1-dataset.png}
			\caption{Training and validation datasets}
			\label{fig:model1-dataset}
		\end{minipage}
		~
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=1\linewidth]{../src/img/model1-polynomial.png}
			\caption{True and estimated polynomials}
			\label{fig:model1-polynomial}
		\end{minipage}
	\end{figure}

 	The true polynomial defined by the coefficients
 	$\textbf{w}^*=[-8,-4,2,1]^T$ and the one that has been estimated
 	$\hat{\textbf{w}}=[-7.86,-4.08,1.98,1.01]^T$ are very close.
 	From the plot in Figure \ref{fig:model1-polynomial+dataset} it is clear
 	that the estimated polynomial curve is fitting well enough the data.

	\begin{figure}[H]
		\centering
		\includegraphics[width=0.8\linewidth]{../src/img/model1-polynomial+dataset.png}
		\caption{Polynomial curves and datasets}
		\label{fig:model1-polynomial+dataset}
	\end{figure}

	\section{Training Set Reduction}
	It is a fact that the behaviour of the model depends on the number of
	samples in te training set.

	From the below pictures we can observe that reducing the training dataset
	from $100$ to $50$, $10$, and finally $5$ observations, the model is not
	able to understand the pattern in the set anymore.  \\
	While with $50$ observations the loss curve shows that the model continues
	performs well. \\
	By reducing the number of samples down to $10$, the validation loss starts
	fluctuating but with a tendency to decrease.\\
	Finally, reducing the number of observations to $5$ the model overfits.

	\begin{figure}[H]
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=0.85\linewidth]{../src/img/loss/model2-loss.jpg}
			%\caption*{model2-loss}
			%\label{fig:model2-loss}
		\end{minipage}
		~
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=1\linewidth]{../src/img/model2-polynomial+dataset.png}
			%\caption*{model2-polynomial+dataset}
			%\label{fig:model2-polynomial+dataset}
		\end{minipage}
	\caption{Loss and polynomial curve for training dataset with $50$
	observations}
	\end{figure}

	\begin{figure}[H]
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=0.85\linewidth]{../src/img/loss/model3-loss.jpg}
			%\caption*{model3-loss}
			%\label{fig:model3-loss}
		\end{minipage}
		~
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=1\linewidth]{../src/img/model3-polynomial+dataset.png}
			%\caption*{model3-polynomial+dataset}
			%\label{fig:model3-polynomial+dataset}
		\end{minipage}
	\caption{Loss and polynomial curve for training dataset with $10$
		observations}
	\end{figure}

	\begin{figure}[H]
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=0.85\linewidth]{../src/img/loss/model4-loss.jpg}
			%\caption*{model4-loss}
			%\label{fig:model4-loss}
		\end{minipage}
		~
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=1\linewidth]{../src/img/model4-polynomial+dataset.png}
			%\caption*{model4-polynomial+dataset}
			%\label{fig:model4-polynomial+dataset}
		\end{minipage}
	\caption{Loss and polynomial curve for training dataset with $5$
	observations}
	\end{figure}

	\section{Sigma Increase}
	From the below pictures we can observe that if we further increase the
	standard deviation, during the creation of the dataset, from $0.5$ to $2$,
	$4$, and finally $8$, the model performance worsen a lot. The losses
	increase respectively to $3.48$, $13.93$ and $55.74$. Moreover, the
	gap between training and validation loss growth.
	One solution to improve the performance could be to increase the
	observations in the training set.

	\begin{figure}[H]
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=0.85\linewidth]{../src/img/loss/model5-loss.jpg}
			%\caption*{model5-loss}
			%\label{fig:model5-loss}
		\end{minipage}
		~
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=1\linewidth]{../src/img/model5-polynomial+dataset.png}
			%\caption*{model5-polynomial+dataset}
			%\label{fig:model5-polynomial+dataset}
		\end{minipage}
		\caption{Loss and polynomial curves for training dataset with sigma $2$}
	\end{figure}

	\begin{figure}[H]
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=0.85\linewidth]{../src/img/loss/model6-loss.jpg}
			%\caption*{model6-loss}
			%\label{fig:model6-loss}
		\end{minipage}
		~
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=1\linewidth]{../src/img/model6-polynomial+dataset.png}
			%\caption*{model6-polynomial+dataset}
			%\label{fig:model6-polynomial+dataset}
		\end{minipage}
	\caption{Loss and polynomial curves for training dataset with sigma $4$}
	\end{figure}

	\begin{figure}[H]
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=0.85\linewidth]{../src/img/loss/model7-loss.jpg}
			%\caption*{model7-loss}
			%\label{fig:model7-loss}
		\end{minipage}
		~
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=1\linewidth]{../src/img/model7-polynomial+dataset.png}
			%\caption*{model7-polynomial+dataset}
			%\label{fig:model7-polynomial+dataset}
		\end{minipage}
	\caption{Loss and polynomial curves for training dataset with sigma $8$}
	\end{figure}

	\section{Higher-degree Polynomial}
	The last experiment consists in reducing the training dataset to $10$
	observations and comparing a fitted polynomial of degree three with
	one of degree four.
	The results are visualised in Figure \ref*{fig:model8}.

	\begin{figure}[H]
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=0.85\linewidth]{../src/img/loss/model8-loss.jpg}
			%\caption{Loss of a polynomial of $4\mathrm{th}$ degree}
			%\label{fig:model8-loss}
		\end{minipage}
		~
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=1\linewidth]{../src/img/model8-polynomial+dataset.png}
			%\caption{Polynomial curves of $4\mathrm{th}$ degree}
			%\label{fig:model8-polynomial+dataset}
		\end{minipage}
	\caption{Polynomial of 4th degree}
	\label{fig:model8}
	\end{figure}

	For what concerns the loss, the validation loss continues to grow in the
	first $400$ iterations and after that starts going down, while the
	training loss is constantly decreasing.
	In Figure \ref*{fig:model8-test} is clear that the estimated
	polynomial of $4\mathrm{th}$ degree is capable of fit well the data in
	the interval $[-1,2]$ while the polynomial of $3\mathrm{rd}$ degree
	in the interval $[-1.5,1]$.\bigskip

	Increasing the number of observations in the training set up to $8000$, it
	is possible to have a good understanding of the validation loss behaviour
	over time.

	\begin{figure}[H]
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=0.85\linewidth]{../src/img/loss/model8-loss-test.jpg}
			%\caption*{model8-loss-test}
			%\label{fig:model8-loss-test}
		\end{minipage}
		~
		\begin{minipage}[c]{.49\textwidth}
			\centering
			\includegraphics[width=1\linewidth]{../src/img/model8-test-polynomial+dataset.png}
			%\caption*{model8-test-polynomial+dataset}
			%\label{fig:model8-test-polynomial+dataset}
		\end{minipage}
	\caption{Polynomial of 4th degree over $8000$ iterations}
	\label{fig:model8-test}
	\end{figure}

\end{document}


\documentclass[12]{scrartcl}
\usepackage{amssymb,amsmath,gensymb,dsfont,calc,multicol,fullpage,dsfont}
\makeatletter
\newcommand\Aboxed[1]{
   \@Aboxed#1\ENDDNE}
\def\@Aboxed#1&#2\ENDDNE{%
   &
   \settowidth\@tempdima{$\displaystyle#1{}$}
   \setlength\@tempdima{\@tempdima+\fboxsep+\fboxrule}
   \kern-\@tempdima
   \boxed{#1#2}
}
\makeatother

\begin{document}

\title{Homework 3, Section 1.4: 6, 8, 9, 12, 13, 14 }
\author{Alex Gordon}
\date{\today}
\maketitle
\section*{Homework}
\subsection*{6.}
\begin{multicols}{2}
\begin{align*}
1) false\\2) true\\3) false\\4) false\\5) true\\6) true\\7) false\\8) true\\
\end{align*}
\begin{align*}
\\9) false\\10) false\\11) true \\12) false\\13) true\\14) false\\15) true\\16) true
\end{align*}
\end{multicols}\

\subsection*{8. a}
D= all biology majors. \\
B(n) = geometry\\
$\exists_n \in D, B(n)$
\subsection*{8. b}
D = all computer science majors. \\
B(n) = Math Minors\\
$\exists_n \in D, \neg B(n)$
\subsection*{8. c}
D = Math Majors \\
B(n) = n is required to take a business course\\
$\forall_n \in D, \neg B(n)$
\subsection*{8. d}
D = puzzles \\
B(n) = all puzzle solutions\\
$\forall_n \in D, \neg B(n)$

\subsection*{9. a}
D = All all real numbers \\
B(n): 2n = 9\\
$\forall_n \in \mathds{R}, \neg B(n)$

\subsection*{9. b}
D = all triangles\\
B(n) = equilateral and perimeter of 10\\
$\exists_n \in D, B(n)$

\subsection*{9. c}
D = Every Circle\\
B(n) = Has an integer diameter or an integer area\\
$\forall_n \in D, B(n)$\

\subsection*{9. d}
D = Real Numbers\\
B(n, m) = an integer between\\
$\forall_n \in \mathds{R}, \forall_m  \in \mathds{R}, B(n, m )$

\subsection*{12. a)}
D= even numbers\\
B(n) = prime numbers\\
$\forall_n \in D, \neg B(n)$

\subsection*{12. b)}
D = All Triangles\\
B(n) = has 4 sides\\
$\forall_n \in D, \neg B(n)$

\subsection*{12. c)}
D = all integers\\
$B(a, b) = \frac{a^2}{b^2} = 2$\\
$\forall_a \ \in D,\ \forall_b \in D, \neg B(a, b)$

\subsection*{12. d)}
D = all square numbers\\
B(n) = prime number plus one\\
$\forall_n \in D, \neg B(n)$

\subsection*{13. a) I)}
-12
\subsection*{13. a) II)}
-2, -4
\subsection*{13. a) III)}
The pattern is even numbers!

\subsection*{13. b) I)}
i = -3, 23, 3, -31
\subsection*{13. b) II)}
use the value y in the equation and solve it.

\subsection*{14. a) }

$\exists_a \in \mathds{R}, \ \exists_b \ \mathds{Z}, a^2 + b \notin \mathds{Z} $

\subsection*{14. b) }
$\exists_y \in \mathds{R}, \ \exists_x \ \mathds{R}, x + y \neq x$

\subsection*{14. c) }
$\exists_x \in \mathds{Z}, \ \forall_y \ \mathds{R}, x \neq 2y$

\subsection*{14. d) }
$\exists_x \in \mathds{Z}, \ \forall_y \ \mathds{R}, \frac{x}{y} \neq 2$

\end{document}


% standard LaTeX packages
\documentclass[12pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[english]{babel}
\usepackage[margin=3cm]{geometry}
\usepackage[hidelinks]{hyperref}
\usepackage{parskip}
\usepackage{xifthen,xparse}

% math packages
\usepackage{mathtools,amsfonts,amssymb,mathdots}
\usepackage{mathrsfs}
\usepackage{siunitx}
\usepackage{physics}
\usepackage{nicefrac}
\mathtoolsset{showonlyrefs}
\DeclareSIUnit{\year}{yr}
\DeclareSIUnit{\astronomicalunit}{AU}
\DeclareSIUnit{\solarmass}{M_\odot}

% plotting and tables
\usepackage{tikz}
\usepackage{pgfplots}
\usepackage{pgfplotstable}
\usepackage{caption}
\usepgfplotslibrary{external}
\tikzexternalize[prefix=tikz-,mode=list and make]
\usepgfplotslibrary{units}
\usepgfplotslibrary{groupplots}
\pgfplotsset{
  compat=1.13,
  scaled x ticks = false,
  x tick label style={
    /pgf/number format/fixed,
    /pgf/number format/1000 sep = \thinspace
  },
  scaled y ticks = false,
  y tick label style={
    /pgf/number format/fixed,
    /pgf/number format/1000 sep = \thinspace
  },
  legend cell align=left,
  % unit
  unit markings=slash space,
  unit code/.code 2 args=
    \expandafter\expandafter\expandafter\expandafter
    \expandafter\expandafter\expandafter\si
    \expandafter\expandafter\expandafter\expandafter
    \expandafter\expandafter\expandafter{#2},
  % legend style={
  %   at={(1.02,1)},
  %   anchor=north west,
  % },
  filter discard warning=false,
  unbounded coords=discard,
  discard if not/.style 2 args={
    x filter/.code={
      \edef\tempa{\thisrow{#1}}
      \edef\tempb{#2}
      \ifx\tempa\tempb
      \else
        \def\pgfmathresult{inf}
      \fi
    }
  },
}

% bibliography packages
% \usepackage[backend=bibtex8,style=numeric,autocite=inline,sorting=ynt]{biblatex}

% other packages
% \usepackage{filecontents}
\usepackage{marginnote}
\usepackage{booktabs}

% bibliography
% \begin{filecontents}{bibliography.bib}
% \end{filecontents}
% \addbibresource{bibliography.bib}
% \nocite{*}

% custom math commands
\newcommand\V[1]{\mathbf{#1}}                  % vector notation
\newcommand\M[1]{\begin{bmatrix} #1 \end{bmatrix}} % matrix shorthand

% commands for derivatives/integral dx expressions
\newcommand\D[1]{{\,\mathrm{d}{#1}}}
\renewcommand\dd[2][]{{%
  \ifthenelse{\isempty{#1}}%
    {\mathrm{d}{#2}}%
    {\mathrm{d}^{#1}{#2}}%
}}
\newcommand\df[3][]{{%
  \ifthenelse{\isempty{#1}}%
    {\frac{\mathrm{d}{#2}}{\mathrm{d}{#3}}}%
    {\frac{\mathrm{d}^{#1}{#2}}{{\mathrm{d}{#3}^{#1}}}}%
}}
\newcommand\pD[1]{{\,\mathrm{d}{#1}}}
\newcommand\pdd[2][]{{%
  \ifthenelse{\isempty{#1}}%
    {\parital{#2}}%
    {\partial^{#1}{#2}}%
}}
\newcommand\pdf[3][]{{%
  \ifthenelse{\isempty{#1}}%
    {\frac{\partial{#2}}{\partial{#3}}}%
    {\frac{\partial^{#1}{#2}}{{\partial{#3}^{#1}}}}%
}}
\let\epsilon\varepsilon

\def\M{\mathscr{M}}

\DeclareDocumentCommand{\reversemarginnote}{ m O{} }{\reversemarginpar\marginnote{#1}[#2]\reversemarginpar}
\newcommand\oppg[1]{\reversemarginnote{\textcolor{black!40}{#1)}}[2mm]}

\begin{document}

\title{Project 3 - FYS3150 Computational Physics}
\author{Fredrik Østrem (\texttt{fredost})}
\date{\today}

\maketitle

% \begin{abstract}
% \end{abstract}

\tableofcontents

\clearpage
\section{Introduction}
In this project, we will look at the Ising model and use it to study phase transistions in magnetic systems. We will implement the Metropolis algorithm to simulate the Ising model of a two-dimensional $L \times L$ spin lattice with periodic boundary conditions and compare our simulated values to analytic expressions for the theoretical values. We will also consider the specific temperatures $t = 1.0$ and $t = 2.4$ (the latter of which is very close to the critical temperature), and look at how the simulations behave at these temperatures and others near the critical temperature.

\section{Methods}

\oppg{a}

We want to find analytical solutions for the expected energy, expected absolute magnetization, specific heat capacity and susceptibility of the $2 \times 2$ lattice Ising model with periodic boundary conditions.

For convenience, we define the ``dimensionless temperature'' $t = \dfrac{k T}{J} = \dfrac{1}{\beta J}$ and $\beta = \dfrac{1}{k T} = \frac{1}{J t}$.

\begin{table}[b]
  \centering

  \begin{tabular}{llll}
    \toprule
    \textbf{Number of spins up} & \textbf{Degeneracy} & \textbf{Energy} & \textbf{Magnetiation} \\
    \toprule
    4 & 1 & $-8J$ &  4 \\
    3 & 4 &    0  &  2 \\
    2 & 4 &    0  &  0 \\
    2 & 2 & $ 8J$ &  0 \\
    1 & 4 &    0  & -2 \\
    0 & 1 & $-8J$ & -4 \\
    \hline
  \end{tabular}

  \caption{Energy and magnetization for the two-dimensional Ising model with $N = 2 \times 2$ spins with periodic boundary conditions. (Table 13.4 in lecture notes.)} \label{tbl:ising2x2p}
\end{table}

In table \ref{tbl:ising2x2p}, we see the possible states we can have. Looking at this, we can see that the partition function $Z$ is
\begin{alignat}{4}
  Z &= \sum_i g(i) e^{-\beta E_i} \\
    &= 2 e^{8 \beta J} + 12 e^{- 0 \cdot \beta J} + 2 e^{- 8 \beta J} \\
    &= 2 e^{8 \beta J} + 12 + 2 e^{- 8 \beta J} \\
    &= 4 \cosh(8 \beta J) + 12 \\
    &= 4 \cosh(8 / t) + 12
\end{alignat}

The expected energy is then
\begin{align}
  \expval{E} &= \frac{1}{Z} \sum_{i} g(i) E_i e^{- \beta E_i} \\
    &= - \frac{1}{Z} \df{Z}{\beta} \\
    &= - \frac{1}{Z} \cdot 32 J \sinh(8 \beta J) \\
    &= - 8 J \cdot \frac{4 \sinh(8 \beta J)}{4 \cosh(8 \beta J) + 12} \\
    &= - 8 J \cdot \frac{\sinh(8 \beta J)}{\cosh(8 \beta J) + 3} \\
    &= - 8 J \cdot \frac{\sinh(8 / t)}{\cosh(8 / t) + 3}
\end{align}

The expected absolute magnetization is
\begin{align}
  \expval{|\M|} &= \frac{1}{Z} \sum_{i} g(i) |\M_i| e^{- \beta E_i} \\
    &= \frac{1}{Z} \qty( 2 \cdot 4 e^{8 \beta J} + 8 \cdot 2 e^{- 0 \cdot \beta J } + 4 \cdot 0 e^{-0 \cdot \beta J} + 2 \cdot 0 e^{-8 \beta J} ) \\
    &= \frac{8 e^{8 \beta J} + 16}{4 \cosh( 8 \beta J) + 12} \\
    % &= \frac{2 \cosh(8 \beta J) + 2 \sinh(8 \beta J) + 4}{\cosh(8 \beta J) + 3} \\
    &= 2 \qty( 1 + \frac{\sinh(8 \beta J) - 1}{\cosh(8 \beta J) + 3} ) \\
    &= 2 \qty( 1 + \frac{\sinh(8/t) - 1}{\cosh(8/t) + 3} ) \\
\end{align}

The specific heat capacity is
\begin{align}
  C_V &= \pdf{\expval{E}}{T} \\
    &= - 8 J \cdot \pdf{}{T} \qty( \frac{\sinh(8 \beta J)}{\cosh(8 \beta J) + 3} ) \\
    &= - 8 J \cdot \df{\beta}{T} \pdf{}{\beta} \qty( \frac{\sinh(8 \beta J)}{\cosh(8 \beta J) + 3} ) \\
    &= \frac{64 J^2}{kT^2} \qty( \frac{3 \cosh(8 \beta J) + 1}{(\cosh(8 \beta J) + 3)^2} ) \\
    &= \frac{64 k}{t^2} \qty( \frac{3 \cosh(8/t) + 1}{(\cosh(8/t) + 3)^2} )
\end{align}
(This can also be expressed as $\qty( \expval{E^2} - \expval{E}^2 ) / k T^2$.)

We can calculate the susceptibility with $\chi = \qty( \expval{\M^2} - \expval{\M}^2 ) / k T$.
\begin{align}
  \expval{\M} &= 0 \\
  \expval{\M^2} &= \sum_i g(i) \M_i^2 e^{- \beta E_i} \\
    &= \frac{1}{Z} \qty( 2 \cdot 4^2 e^{8 \beta J} + 8 \cdot 2^2 e^{- 0 \cdot \beta J } + 4 \cdot 0^2 e^{-0 \cdot \beta J} + 2 \cdot 0^2 e^{-8 \beta J} ) \\
    &= \frac{32 e^{8 \beta J} + 32}{4 \cosh(8 \beta J) + 12} \\
    &= \frac{8 e^{8 \beta J} + 8}{\cosh(8 \beta J) + 3} \\
    % &= 8 \cdot \qty( \frac{\cosh(8 \beta J) +\sinh(8 \beta J) + 1}{\cosh(8 \beta J) + 3} ) \\
    &= 8 \cdot \qty( 1 + \frac{\sinh(8 \beta J) - 2}{\cosh(8 \beta J) + 3} ) \\
    &= 8 \cdot \qty( 1 + \frac{\sinh(8/t) - 2}{\cosh(8/t) + 3} ) \\
\intertext{With this, we see that the susceptibility is}
  \chi &= \frac{1}{k T} \qty( \expval{\M^2} - \underbrace{\expval{\M}^2}_{= \, 0} ) = \frac{1}{Jt} \expval{\M^2} \\
    &= \frac{8}{J t} \qty( 1 + \frac{\sinh(8/t) - 2}{\cosh(8/t) + 3} )
\end{align}

\clearpage
\section{Results}

\subsection{Simulation of $2 \times 2$ lattice Ising model}

\oppg{b}
We want to compare our analytical results to results we get from a Monte Carlo simulation of the model.
For this, we implement the following algorithm:

\begin{enumerate}
  \item Start with a random state.
  \item For each Monte Carlo cylce:
    \begin{enumerate}
      \item Flip a single spin at random.
      \item Calculate the difference in energy $\Delta{E}$ from the flip.
      \item If $\Delta{E} \le 0$, accept state.
      \item If $\Delta{E} \ge 0$, accept state with probability $w = e^{-\beta \Delta{E}}$.
    \end{enumerate}
\end{enumerate}

We calculate the average energy, absolute magnetization, specific heat capacity and susceptibility at the end of each Monte Carlo cycle. We see these plotted as a function of temperature $t$, along with the analytic results we got earlier, in figure \ref{fig:b}. When we adjust the total number of Monte Carlo cycles, we see that we need approximately \num{100000} cycles to get results that are close to the analytic results.

% Note: \texttt{default\_random\_engine} is much faster than \texttt{random\_engine}.

\begin{figure}[!ht]
  \centering

  \begin{tikzpicture}
    \begin{groupplot}[
        group style={
          group size=1 by 4,
          xlabels at=edge bottom,
          xticklabels at=edge bottom,
          ylabels at=edge left,
          yticklabels at=edge left,
          vertical sep=5pt,
        },
        width=\textwidth,
        height=0.45\textwidth,
        xlabel={Temperature $t$},
        xlabel style={at={(axis description cs:0.5,-0.1)},anchor=north},
        ylabel style={at={(axis description cs:-0.05,0.5)},anchor=south,align=center},
        xmin=0, xmax=10,
        grid=both,
      ]

      \nextgroupplot[
        ylabel={Expected \\ energy \\ $\expval{E} \; / \; J$},
        ymin=-10, ymax=10,
      ]
        \addplot[red,domain=0:10,samples=100] ({x},{- 8 * sinh(8 / x) / (cosh(8 / x) + 3)});
        \addplot[blue,only marks,mark=*,mark size=1] table[x=t,y=E] {data/4b.dat};
        % \addplot[yellow,only marks,mark=*,mark size=1] table[x=t,y=E] {data/4e-out};

        \legend{Theoretical,Simulated}

      \nextgroupplot[
        ylabel={Expected \\ magnetization \\ $\expval{|\M|}$},
        ymin=0, ymax=5,
      ]
        \addplot[red,domain=0:10,samples=100] ({x},{2 * (1 + (sinh(8 / x) - 1) / (cosh(8 / x) + 3))});
        \addplot[blue,only marks,mark=*,mark size=1] table[x=t,y=absM] {data/4b.dat};
        % \addplot[yellow,only marks,mark=*,mark size=1] table[x=t,y=absM] {data/4e-out};

      \nextgroupplot[
        ylabel={Heat capacity \\ $C_V \; / \; k$},
        ymin=0, ymax=2.5,
      ]
        \addplot[red,domain=0:10,samples=100] ({x},{64/x^2 * ((3 * cosh(8/x) + 1) / (cosh(8/x)+3)^2)});
        \addplot[blue,only marks,mark=*,mark size=1] table[x=t,y=Cv] {data/4b.dat};
        % \addplot[yellow,only marks,mark=*,mark size=1] table[x=t,y=Cv] {data/4e-out};

      \nextgroupplot[
        ylabel={Susceptibility \\ $\chi \; / \; (1/J)$},
        ymin=0, ymax=23,
      ]
        \addplot[red,domain=0:10,samples=100] ({x},{8/x * (1 + (sinh(8/x)-2) / (cosh(8/x)+3)});
        \addplot[blue,only marks,mark=*,mark size=1] table[x=t,y=chi] {data/4b.dat};
        % \addplot[yellow,only marks,mark=*,mark size=1] table[x=t,y=chi] {data/4e-out};
    \end{groupplot}
  \end{tikzpicture}

  \caption{Plots that show theoretical and simulated values of $\expval{E}$, $\expval{|\M|}$, $C_V$ and $\chi$ for the Ising model of a $2 \times 2$ square spin lattice, from \textbf{a)} and \textbf{b)}.} \label{fig:b}
\end{figure}

\subsection{Evolution by number of Monte Carlo cycles}
\oppg{c}
After this, we study how $\expval{E}$, $\expval{|\M|}$ and the number of accepted states behave as a function of the number of Monte Carlo cycles $n$ in a $20 \times 20$ Ising model, for ordered and random initial states, as we see in figures \ref{fig:c1} and \ref{fig:c2}.

It's worth noting that at temperature $t = 1.0$, the equalibrium point lies almost at the ordered states already, so the model will use many orders of magnitude fewer cycles to achieve equalibrium than by starting at a random initial state. We can also note that the number of accepted state increases linearly with $n$, while the random states only approach the equalibrium and approximately linear number of accepted states after $n \approx \num{e6}$.

At temperature $t = 2.4$, the picture is a bit different, however: the equalibrium energy is about midway between the random inital state (at energy $\approx 0$) and the ordered initial state (at energy $-800J$). This means that both the initial state use an approximately equal number of cycles to reach eqalibrium, after $n \approx \num{e7}$.

\begin{figure}[!ht]
  \centering
  \begin{tikzpicture}
    \begin{groupplot}[
        group style={
          group size=1 by 3,
          xlabels at=edge bottom,
          xticklabels at=edge bottom,
          ylabels at=edge left,
          yticklabels at=edge left,
          vertical sep=5pt,
        },
        width=\textwidth,
        height=0.45\textwidth,
        xlabel={Monte Carlo cycles $n$},
        xlabel style={at={(axis description cs:0.5,-0.15)},anchor=north},
        ylabel style={at={(axis description cs:-0.1,0.5)},anchor=south,align=center},
        xmin=1, xmax=1e9,
        xmode=log,
        grid=both,
      ]

      \nextgroupplot[
        ylabel={Mean energy \\ $E \; / \; J$},
        % ymin=-8.02, ymax=-7.9,
      ]
        \addplot[red,only marks,mark=*,mark size=1] table[x=n,y=E] {data/4c-1.0-ordered.dat};
        \addplot[blue,only marks,mark=*,mark size=1] table[x=n,y=E] {data/4c-1.0-random.dat};

        \legend{Ordered initial state,Randomized initial state}

      \nextgroupplot[
        ylabel={Mean abs. magnetization \\ $|\M|$},
        % ymin=3.95, ymax=4.01,
      ]
        \addplot[red,only marks,mark=*,mark size=1] table[x=n,y=absM] {data/4c-1.0-ordered.dat};
        \addplot[blue,only marks,mark=*,mark size=1] table[x=n,y=absM] {data/4c-1.0-random.dat};

      \nextgroupplot[
        ylabel={Accepted states},
        ymin=1, ymax=1e9, ymode=log,
        ytick={1e0,1e2,1e4,1e6,1e8},
      ]
        \addplot[red,only marks,mark=*,mark size=1] table[x=n,y=accepted] {data/4c-1.0-ordered.dat};
        \addplot[blue,only marks,mark=*,mark size=1] table[x=n,y=accepted] {data/4c-1.0-random.dat};
    \end{groupplot}
  \end{tikzpicture}

  \caption{Mean energy, mean absolute magnetization and number of accepted states as a function of the number of Monte Carlo cycles for $t = 1.0$.} \label{fig:c1}
\end{figure}

\begin{figure}[!ht]
  \centering
  \begin{tikzpicture}
    \begin{groupplot}[
        group style={
          group size=1 by 3,
          xlabels at=edge bottom,
          xticklabels at=edge bottom,
          ylabels at=edge left,
          yticklabels at=edge left,
          vertical sep=5pt,
        },
        width=\textwidth,
        height=0.45\textwidth,
        xlabel={Monte Carlo cycles $n$},
        xlabel style={at={(axis description cs:0.5,-0.15)},anchor=north},
        ylabel style={at={(axis description cs:-0.1,0.5)},anchor=south,align=center},
        xmin=1, xmax=1e9,
        xmode=log,
        grid=both,
      ]

      \nextgroupplot[
        ylabel={Mean energy \\ $E \; / \; J$},
        % ymin=-8.02, ymax=-7.9,
      ]
        \addplot[red,only marks,mark=*,mark size=1] table[x=n,y=E] {data/4c-2.4-ordered.dat};
        \addplot[blue,only marks,mark=*,mark size=1] table[x=n,y=E] {data/4c-2.4-random.dat};

        \legend{Ordered initial state,Randomized initial state}

      \nextgroupplot[
        ylabel={Mean abs. magnetization \\ $|\M|$},
        % ymin=3.95, ymax=4.01,
      ]
        \addplot[red,only marks,mark=*,mark size=1] table[x=n,y=absM] {data/4c-2.4-ordered.dat};
        \addplot[blue,only marks,mark=*,mark size=1] table[x=n,y=absM] {data/4c-2.4-random.dat};

      \nextgroupplot[
        ylabel={Accepted states},
        ymin=1, ymax=1e9, ymode=log,
        ytick={1e0,1e2,1e4,1e6,1e8},
      ]
        \addplot[red,only marks,mark=*,mark size=1] table[x=n,y=accepted] {data/4c-2.4-ordered.dat};
        \addplot[blue,only marks,mark=*,mark size=1] table[x=n,y=accepted] {data/4c-2.4-random.dat};
    \end{groupplot}
  \end{tikzpicture}

  \caption{Mean energy, mean absolute magnetization and number of accepted states as a function of the number of Monte Carlo cycles for $t = 2.4$.} \label{fig:c2}
\end{figure}

\clearpage
\section{Probability distributions}
\oppg{d}
We can then look at how the probability distribution of the energy states for the same two temperatures ($t = 1.0$ and $t = 2.4$) for the same $20 \times 20$ Ising model. For this, we simply skip ahead until we get a steady state, and then count the number of times each energy level occur during the simulation.

First of all, we note that for $t = 1.0$, the most likely energy level is $-800J$, which is the same as the equalibrium energy level we got for this temperature earlier (see figure \ref{fig:c1}). For $t = 2.4$, the most likely energy level is about $-480J$, which is about the same as the equalibrium level we got for this temperature as well (see figure \ref{fig:c2}).

We can also tell that the probability distribution for $t = 2.4$ is much wider, with a standard deviation that we calculate to be $\sigma_E = 57.04$, while the probability distribution for $t = 1.0$ only has a standard deviation of $\sigma_E = 3.063$. A quick comparision with the plots in figure \ref{fig:c1} and \ref{fig:c2} confirm that these are reasonable numbers.

\begin{figure}[!ht]
  \centering
  \begin{tikzpicture}
    \begin{groupplot}[
        group style={
          group size=1 by 2,
          % vertical sep=5pt,
        },
        width=\textwidth,
        height=0.7\textwidth,
        xlabel={Energy $E \; / \; J$},
        xlabel style={at={(axis description cs:0.5,-0.15)},anchor=north},
        ylabel={Probability $P(E)$},
        ylabel style={at={(axis description cs:-0.1,0.5)},anchor=south,align=center},
        grid=both,
      ]

      \nextgroupplot[
        ymin=0, ymax=1,
      ]
        \addplot[blue,mark=*,ycomb] table[x=E,y=P] {data/4d-1.0.dat};
        \legend{$t = 1.0$}

      \nextgroupplot[
        ymin=0, ymax=0.05,
      ]
        \addplot[red,mark=*,ycomb] table[x=E,y=P] {data/4d-2.4.dat};
        \legend{$t = 2.4$}
    \end{groupplot}
  \end{tikzpicture}

  \centering{Relative frequencies of the different energy levels after a steady state is reached, for temperatures $t = 1.0$ and $t = 2.4$ on a $20 \times 20$ Ising model.} \label{fig:d}
\end{figure}

\subsection{Numerical studies of phase transitions}
\oppg{e}
We run the Metropolis algorithm for the Ising model at temperatures in the range $t \in [2.0,2.3]$ in temperature steps of $\Delta{t} = 0.005$, for lattices of sizes $L = 40, 60, 100, 140$. For each lattice, we run $\num{100000} \cdot L^2$ Monte Carlo cycles. For the latter half of these cycles (after we assume we have reach a somewhat stable state) we calculate the mean energy $\expval{E}$ and mean absolute magnetization $\expval{|\M|}$, along with the specific heat capacity $C_V$ and susceptibility $\chi$, for each time step. (This took about 2-3 hours to run on 4 processor cores.)

From these results which are shown in figure \ref{fig:e}, we can tell that as the temperature approaches the critical temperature, the number of cycles we used is no longer sufficient to get a stable state, and instead we get big fluctuations from one time step to another. For some reason I can't explain, the heat capacity $C_V$ \emph{decreases} in this interval, although we expect that it increase in this interval; this is unexpected since we know that
\begin{equation}
  C_V = \pdf{\expval{E}}{T}
\end{equation}
and $\expval{E}$ is definitely increasing more with higher temperature on this interval, and we also expect it to have a maximum close to the critical temperature (at $t = 2.269$ if we assume $\nu = 1$), which we don't see.

\begin{figure}[!ht]
  \centering

  \begin{tikzpicture}
    \begin{groupplot}[
        group style={
          group size=1 by 4,
          xlabels at=edge bottom,
          xticklabels at=edge bottom,
          ylabels at=edge left,
          yticklabels at=edge left,
          vertical sep=5pt,
        },
        width=\textwidth,
        height=0.45\textwidth,
        xmin=2.0, xmax=2.3,
        xlabel={Temperature $t$},
        xlabel style={at={(axis description cs:0.5,-0.1)},anchor=north},
        ylabel style={at={(axis description cs:-0.1,0.5)},anchor=south,align=center},
        legend pos=north west,
        % grid=both,
      ]

      \nextgroupplot[
        ylabel={Expected \\ energy per spin \\ $\expval{E} \; / \; J L^2$},
        % ymin=-10, ymax=10,
      ]
        \addplot[blue,mark=*,mark size=1] table[x=t,y expr={\thisrow{E} / 40^2}] {data/4e-40x40.dat};
        \addplot[red,mark=*,mark size=1] table[x=t,y expr={\thisrow{E} / 60^2}] {data/4e-60x60.dat};
        \addplot[purple,mark=*,mark size=1] table[x=t,y expr={\thisrow{E} / 100^2}] {data/4e-100x100.dat};
        \addplot[green!50!black,mark=*,mark size=1] table[x=t,y expr={\thisrow{E} / 140^2}] {data/4e-140x140.dat};

        \legend{$L=40$,$L=60$,$L=100$,$L=140$}

      \nextgroupplot[
        ylabel={Expected \\ magnetization per spin \\ $\expval{|\M|} \; / \; L^2$},
        % ymin=0, ymax=5,
      ]
        \addplot[blue,mark=*,mark size=1] table[x=t,y expr={\thisrow{absM} / 40^2}] {data/4e-40x40.dat};
        \addplot[red,mark=*,mark size=1] table[x=t,y expr={\thisrow{absM} / 60^2}] {data/4e-60x60.dat};
        \addplot[purple,mark=*,mark size=1] table[x=t,y expr={\thisrow{absM} / 100^2}] {data/4e-100x100.dat};
        \addplot[green!50!black,mark=*,mark size=1] table[x=t,y expr={\thisrow{absM} / 140^2}] {data/4e-140x140.dat};

      \nextgroupplot[
        ylabel={Heat capacity \\ per spin squared \\ $C_V \; / \; k L^4$},
        % ymin=0, ymax=2.5,
      ]
        \addplot[blue,mark=*,mark size=1] table[x=t,y expr={\thisrow{Cv} / 40^4}] {data/4e-40x40.dat};
        \addplot[red,mark=*,mark size=1] table[x=t,y expr={\thisrow{Cv} / 60^4}] {data/4e-60x60.dat};
        \addplot[purple,mark=*,mark size=1] table[x=t,y expr={\thisrow{Cv} / 100^4}] {data/4e-100x100.dat};
        \addplot[green!50!black,mark=*,mark size=1] table[x=t,y expr={\thisrow{Cv} / 140^4}] {data/4e-140x140.dat};

      \nextgroupplot[
        ylabel={Susceptibility \\ per spin squared \\ $\chi \; / \; (L^4/J)$},
        % ymin=0, ymax=23,
      ]
        \addplot[blue,mark=*,mark size=1] table[x=t,y expr={\thisrow{chi} / 40^4}] {data/4e-40x40.dat};
        \addplot[red,mark=*,mark size=1] table[x=t,y expr={\thisrow{chi} / 60^4}] {data/4e-60x60.dat};
        \addplot[purple,mark=*,mark size=1] table[x=t,y expr={\thisrow{chi} / 100^4}] {data/4e-100x100.dat};
        \addplot[green!50!black,mark=*,mark size=1] table[x=t,y expr={\thisrow{chi} / 140^4}] {data/4e-140x140.dat};
    \end{groupplot}
  \end{tikzpicture}

  \caption{Plots that show the simulated values of $\expval{E}$, $\expval{|\M|}$, $C_V$ and $\chi$ for the Ising model of a $L \times L$ square spin lattice, where $L = 40,60,100,140$.} \label{fig:e}
\end{figure}

\oppg{f}
At this point, if we got the expected curves for $C_V$ in figure \ref{fig:e}, we should have seen spikes in the heat capacity around the critical temperature. We should then have been example to use the equation
\begin{equation}
  T_C(L) = a L^{-1/\nu} + T_C(L = \infty) = a L^{-1/\nu} + b
\end{equation}
and find a best-fit for the three parameters $a,b,\nu$ (or just $a,b$ if we are assuming $\nu = 1$). The parameter $b$ would then be the critical temperature in the thermodynamical limit $L \to \infty$.

\clearpage
\section{Conclusion}
In this project, we have tried to model phase transitions in a magnetic material by using the Ising model. The initial results for the $2 \times 2$ lattice were promising and matched the analytic, but when we later tried to use the same code for lattices, we didn't get the expected results, especially with respect to the heat capacity and susceptibility, so that we're unable to use this to calculate the critical temperature. I suspect this is because of some error in the implementation of the algorithm, but I cannot find what it is that causes this. Regardless of this, we can still see that the algorithm becomes much less accurate around the critical temperature.

\appendix
\section{Appendix}

All the code and data files used for this project can be found at \url{https://github.com/frxstrem/fys3150/tree/master/project4/code} and \url{https://github.com/frxstrem/fys3150/tree/master/project4/data}, respectively.

\end{document}


\documentclass[12pt, a4paper]{article} % book, report, article, letter, slides
                                       % letterpaper/a4paper, 10pt/11pt/12pt, twocolumn/twoside/landscape/draft

%%%%%%%%%%%%%%%% PACKAGES %%%%%%%%%%%%%%%%%%%%%

\usepackage[utf8]{inputenc} % encoding

\usepackage[english]{babel} % use special characters and also translates some elements within the document.

\usepackage{amsmath}        % Math
\usepackage{amsthm}         % Math, \newtheorem, \proof, etc
\usepackage{amssymb}        % Math, extended collection
\newtheorem{theorem}{Theorem}[section]
\newtheorem{corollary}{Corollary}[theorem]
\newtheorem{lemma}[theorem]{Lemma}

\usepackage{hyperref}       % Hyperlinks \url{url} or \href{url}{name}

\usepackage{parskip}        % \par starts on left (not idented)

\usepackage{abstract}       % Abstract

\usepackage{tocbibind}      % Adds the bibliography to the table of contents (automatically)

\usepackage{graphicx}       % Images
\graphicspath{ {./images/} }

% \usepackage[document]{ragged2e}  % Left-aligned (whole document)
% \begin{...} ... \end{...}   flushleft, flushright, center

%%%%%%%%%%%%%%%% CODE %%%%%%%%%%%%%%%%%%%%%

\usepackage{minted}         % Code listing
% \mint{html}|
Something here
|
% \inputminted{octave}{BitXorMatrix.m}

%\begin{listing}[H]
  %\begin{minted}[xleftmargin=20pt,linenos,bgcolor=codegray]{haskell}
  %\end{minted}
  %\caption{Example of a listing.}
  %\label{lst:example} % You can reference it by \ref{lst:example}
%\end{listing}

\newcommand{\code}[1]{\texttt{#1}} % Define \code{foo.hs} environment

%%%%%%%%%%%%%%%% COLOURS %%%%%%%%%%%%%%%%%%%%%

\usepackage{xcolor}         % Colours \definecolor, \color{codegray}
\definecolor{codegray}{rgb}{0.9, 0.9, 0.9}
% \color{codegray} ... ...
% \textcolor{red}{easily}

%%%%%%%%%%%%%%%% CONFIG %%%%%%%%%%%%%%%%%%%%%

\renewcommand{\absnamepos}{flushleft}
\setlength{\absleftindent}{0pt}
\setlength{\absrightindent}{0pt}

%%%%%%%%%%%%%%%% GLOSSARIES %%%%%%%%%%%%%%%%%%%%%

%\usepackage{glossaries}

%\makeglossaries % before entries

%\newglossaryentry{latex}{
    %name=latex,
    %description={Is a mark up language specially suited
    %for scientific documents}
%}

% Referene to a glossary \gls{latex}
% Print glossaries \printglossaries

%%%%%%%%%%%%%%%% HEADER %%%%%%%%%%%%%%%%%%%%%

\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhf{}
\rhead{Arnau Abella - MIRI}
\lhead{ADS - First Delivery}
\rfoot{Page \thepage}

%%%%%%%%%%%%%%%% TITLE %%%%%%%%%%%%%%%%%%%%%

\title{%
  Bloom Filter \\
  \large A Simple Implementation In Haskell}
\author{Arnau Abella}
\date{\today}

%%%%%%%%%%%%%%%% DOCUMENT %%%%%%%%%%%%%%%%%%%%%

\begin{document}

\maketitle

\section{Introduction}\label{s:introduction}

A \textbf{Bloom filter} is a space-efficient probabilistic data structure that guarantees constant string comparison with allowable errors \cite{bloom}.

The na\"ive algorithm for string comparison has a computational cost of \\ $\mathcal{O}(m \cdot (n-m+1))$ in time. In contrast, a Bloom filter, performs exact string search in $\mathcal{O}(m+n)$ time by allowing a small probabilistic error. Apart from an excelent computational time, it is also space-efficient and this can be exploited in applications such as dictionaries.

The Bloom filter has a wide range of forms such as Counting Bloom filter or Distributed Bloom filter. And each one has several different applications that we are not going to get in detail.

\begin{figure}[H]
  \includegraphics[scale=0.3]{bloom-filter}
  \centering
  \caption{Example of a Bloom filter.}
  \label{fig:bloom}
\end{figure}

A Bloom filter is implemented using an array-like data structure of size $m$ and $H := \{h_1, \dots, h_k\}$ hashing functions, where $h_i := U \to \{0, m-1\}$. Depending on the value of $m$ (size of the array) and $k$ (number of hashing functions), the $\varepsilon$ (expected error rate) will be different. The algorithm has the following steps:

\begin{enumerate}
  \item Initialize an array of size $m$ to 0.
  \item On each new string, hash it using $h_{\in H}$ and add one to the corresponding index in the array.
  \item Given a new word, hash it again using $h_{\in H}$ and check if their corresponding indexes are 1 in the array. If all indexes' positions are set to 1, you have a high probability that the word has been already searched. Otherwise, you are guaranteed that the word has not been searched and you can add it to the array.
  \item (Optional) At some point, $70-80\%$ of the positions of the array will be one, so the bloom filter will not work properly. You can implement a \code{reset} method to restart the data structure setting all bits to zero.
  \item (Optional) You can have multiple layers to check if the word has been searches $n$ times.
\end{enumerate}

This process is illustrated in the figure \ref{fig:bloom}.

\subsection{Probability of the error rate}\label{s:probability}

This section explains formally the $\varepsilon$ (expected error rate) $wrt.$ the independent variables $m$ (array size), $n$ (number of strings present in the filter), and $k$ (number of hashing functions).

\begin{itemize}
  \item The probability of setting a bit is $1/m$. This comes from the assumption that $H$ belongs to the class of Universal Hashing Functions.
  \item The probability of not setting a bit to 1 is $1 - 1/m$.
  \item The probability of not setting a bit to 1 after the $k$ hashing functions is $(1 - 1/m)^k$.
  \item The probability of not setting a bit to 1 after $k$ hashing functions and $n$ strings is $(1 - 1/m)^{kn}$.
  \item The probability of mistaken a bit is $1 - (1 - 1/m)^{kn}$
  \item And error is produced when we mistake the $k$ indexes of a string and the probability is $(1 - (1 - 1/m)^{kn})^k$.
  \item This can be approximated by $1 - e^{-kn/m}$
\end{itemize}

From the formula of the error rate

\[
  \varepsilon = 1 - e^{-kn/m}
\]

it is easy to appreciate that when $m \to \infty^+$, $\varepsilon \to 0$. So, $m$ should be big. But, how big ?

Before answering this question, we should note that there is an optimal $k$ value where the function $p = e^{-kn/m}$ is minimal. This can be computed differentiating the equation and equating it to zero. The optimal $k$ can be found by the formula:

\[
  k = \frac{m}{n}\log_{} 2
\]

As we said, the size of $m$ depends on $n$ (but this is fixed by the problem), $k$ (but we found that there is an optimal one) and $\varepsilon$. Now, the size of $m$ only depends on the error rate and it can be computed this way:

\[
  m = \lceil \frac{\log \frac{1}{\varepsilon} \cdot n}{\log^2 2} \rceil
\]

Notice that the error rate is only influence by the FPR (false positives rate) as it is guarantee that there are no false negatives.

\section{Experiment}\label{s:experiment}

The experiment for this delivery consist of empirically proving that the implementation proposed in this delivery of a Bloom filter in Haskell, which uses the previous formulas, has the expected theoretically error rates.

The experiment is quite simple and it is implemented in the following steps:

\begin{itemize}
  \item For $n = \langle 100, 1000, 10000\rangle$:
  \item For $\varepsilon = \langle 0.05, 0.01, 0.001\rangle$:
  \item For $i = \{0..100\}$:
  \item Generate $n$ strings,
  \item construct a Bloom filter using the $n$ strings,
  \item then, test it against $m \sim n$ new strings,
  \item and collect if the test was a true positive, a false positive, a true negative or a false negative.
  \item This process is repeated $i$ times to improve the quality of the test and it is a probabilistic data structure.
  \item Finally, output the results for each $n$ and $\varepsilon$.
\end{itemize}

\begin{listing}[H]
  \begin{minted}[xleftmargin=0pt]{haskell}
-- | Runs an error rate test
-- and writes the result at the given file path.
errorRateTest :: FilePath -> IO ()
errorRateTest file =
  withFile file WriteMode $ \h -> do
    forM_ [100, 1000, 10000] $ \n -> do
      forM_ [0.05, 0.01, 0.001] $ \e -> do
        (fp, fn) <- fmap unzip $ forM [0..100] $
            \(_ :: Int) -> do
              samples' <- generate (genSamples n)
              let (trainingSet, testingSet) =
                coerce @_ @([String], [String]) samples'
              let !filt = BF.create e trainingSet
              let wrds = Set.fromList trainingSet
              let (l, r) = foldr (testFilt filt wrds) (0,0) testingSet
              let overN x = (fromIntegral x) / (fromIntegral n) :: Double
              return (overN l, overN r)
        let (fpr, fnr) = (mean fp, mean fn)
        let (sfp, sfn) = (stdDev fp, stdDev fn)
        _ <- traverse (hPutStrLn h) [ "False Positive Rate"
                                    , "mean      = " ++ show fpr
                                    , "std. dev. = " ++ show sfp
                                    , "False Negative Rate"
                                    , "mean      = " ++ show fnr
                                    , "std. dev. = " ++ show sfn
                                    ]
  \end{minted}
  \caption{Error rate test implementation}
  \label{lst:error-rate-test}
\end{listing}

\newpage

The output of the experiment for $\varepsilon = 0.05$ and $n = \{100, 1000, 10000\}$ is the following:

\begin{listing}[H]
  \begin{minted}[xleftmargin=0pt]{haskell}
N = 100
Expected e = 5.0e-2

False Positive Rate
	mean = 6.0e-3
	std. dev. = 1.5e-4

False Negative Rate
	mean = 0.0
	std. dev. = NaN

-----------------------------

N = 1000
Expected e = 5.0e-2

False Positive Rate
	mean = 1.4e-3
	std. dev. = 1.65e-4

False Negative Rate
	mean = 0.0
	std. dev. = NaN

-----------------------------

N = 10000
Expected e = 1.0e-2

False Positive Rate
	mean = 5.4e-3
	std. dev. = 1.2e-4

False Negative Rate
	mean = 0.0
	std. dev. = NaN
  \end{minted}
  \caption{Output of the experiment for $\varepsilon = 0.05$}
  \label{lst:test-output}
\end{listing}

\section{Conclusion}\label{s:conclusion}

We presented an implementation of the most simple version of a Bloom filter written in Haskell with an efficient-space representation and reasonable performance. We proved by experimentation that, in practice, the theoretical error rate holds.

In the future, this data structure can be extended to support concurrent and distributed computation.

%%%%%%%%%%%%%%%% BIBLIOGRAPHY %%%%%%%%%%%%%%%%%%%%%

\bibliographystyle{unsrt} % abbrv, aplha, plain, abstract, apa, unsrt,
\bibliography{refs}

\end{document}


\documentclass[12pt]{article}
\input{physics1}
\begin{document}

\section*{NYU Physics I---friction}

In lecture we did a problem of a block on an inclined plane.
Here we consider the same ``block on plane'' problem but with
friction.  When we ``switch on'' friction, the contact force between
the block and the plane is no longer purely normal but has both normal
and transverse (frictional) components.  For definiteness, imagine a
plane inclined at about $20\,\deg$ to the horizontal in what
follows. Work with a partner!

\paragraph{\theproblem}\refstepcounter{problem}%
Following lecture, or whatever method you like, work out the problem
of a block on an inclined plane in the \emph{absence of friction}.
That is, compute the magnitude of the normal force and the magnitude
and direction of the acceleration of the block.

\paragraph{\theproblem}\refstepcounter{problem}%
Make dimensional and scaling arguments that it makes sense for the
frictional force magnitude to be some dimensionless constant $\mu$
times the normal force magnitude. What typical values of $\mu$ do you
find in the world? Are there limits on the possible values of $\mu$?
(Hint: Cars used in motorsports can pull 6 lateral gs. What does that
imply about friction?)

\paragraph{\theproblem}\refstepcounter{problem}%
There are two kinds of friction, sliding (or kinetic) and static. What
is the difference between them? You might want to use the internet or
a book to find this out.

\paragraph{\theproblem}\refstepcounter{problem}%
Imagine that between the block and the plane there is a coefficient of
friction $\mu=0.05$.  Draw the forces on the block, separating the
contact force into its normal and transverse components.

\paragraph{\theproblem}\refstepcounter{problem}%
Write down the kinematic constraint, and solve for the magnitude of
the normal force and the magnitude of the frictional (transverse)
component of the contact force.  Which way does the frictional force
point, and why?

\paragraph{\theproblem}\refstepcounter{problem}%
What is the acceleration of the block in this case?

\paragraph{\theproblem}\refstepcounter{problem}%
Describe a situation in which the frictional force would point
\emph{down} the plane.  When you answered the previous questions, you
made an assumption.  What was it?

\paragraph{\theproblem}\refstepcounter{problem}%
Now imagine that $\mu=0.9$.  What is the magnitude of the frictional
force? Give a simple argument that it \emph{cannot} be
$\mu\,m\,g\,\cos\theta$.

\end{document}


%\documentclass[12pt]{article}
\documentclass[12pt,landscape]{article}

\include{preamble}

\newcommand{\instr}{\small Your answer will consist of a lowercase string (e.g. \texttt{aebgd}) where the order of the letters does not matter. \normalsize}

\title{Math 369 / 690 Fall \the\year{} \\ Midterm Examination One}
\author{Professor Adam Kapelner}

\date{Thursday, October 7, \the\year{}}

\begin{document}
\maketitle

%\noindent Full Name \line(1,0){410}

\thispagestyle{empty}

\section*{Code of Academic Integrity}

\footnotesize
Since the college is an academic community, its fundamental purpose is the pursuit of knowledge. Essential to the success of this educational mission is a commitment to the principles of academic integrity. Every member of the college community is responsible for upholding the highest standards of honesty at all times. Students, as members of the community, are also responsible for adhering to the principles and spirit of the following Code of Academic Integrity.

Activities that have the effect or intention of interfering with education, pursuit of knowledge, or fair evaluation of a student's performance are prohibited. Examples of such activities include but are not limited to the following definitions:

\paragraph{Cheating} Using or attempting to use unauthorized assistance, material, or study aids in examinations or other academic work or preventing, or attempting to prevent, another from using authorized assistance, material, or study aids. Example: using an unauthorized cheat sheet in a quiz or exam, altering a graded exam and resubmitting it for a better grade, etc.
\\

\noindent By taking this exam, you acknowledge and agree to uphold this Code of Academic Integrity. \\

%\begin{center}
%\line(1,0){250} ~~~ \line(1,0){100}\\
%~~~~~~~~~~~~~~~~~~~~~signature~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ date
%\end{center}

\normalsize

\section*{Instructions}
This exam is 70 minutes (variable time per question) and closed-book. You are allowed \textbf{one} page (front and back) of a \qu{cheat sheet}, blank scrap paper and a graphing calculator. Please read the questions carefully. I recommend answering all questions that are easy first and then circling back to work on the harder ones. No food is allowed, only drinks. %If the question reads \qu{compute,} this means the solution will be a number otherwise you can leave the answer in \textit{any} widely accepted mathematical notation which could be resolved to an exact or approximate number with the use of a computer. I advise you to skip problems marked \qu{[Extra Credit]} until you have finished the other questions on the exam, then loop back and plug in all the holes. I also advise you to use pencil. The exam is 100 points total plus extra credit. Partial credit will be granted for incomplete answers on most of the questions. \fbox{Box} in your final answers. Good luck!

\pagebreak

\problem\timedsection{6} These are conceptual questions about statistical inference and sampling.

\vspace{-0.2cm}\benum\truefalsesubquestionwithpoints{14}

\begin{enumerate}[(a)]
\item Beginning with a sample of data, it may be difficult to clearly define the population.
\item Beginning with a well-defined population, it may be difficult to sample uniformly from this population.
\item In a simple random sample, the probability of each item in the population being in the sample is the same.
\item Samples from a real-world population are never truly independent.
\item If you had access to the entire population, population parameter values would be known exactly.
\item Samples can sometimes be thought of realizations from a process defined by random variables, the DGP.
\item The data $\xoneton$ can be sampled only after assuming a DGP explicitly.
\item The goal of point estimation is to find an approximation of the true value of $\theta$.
\item The goal of theory testing is to find an approximation of the true value of $\theta$.
\item The goal of statistical inference (in general) is to learn the true values of $\xoneton$.
\item The goal of statistical inference (in general) is to learn the true values of $x_1, \ldots, x_N$.
\item Statistical estimates can only be computed after the data $\xoneton$ is sampled.
\item The sampling distribution can only be known after the data's values $\xoneton$ are known.
\item The estimator $\Xbar$ is unbiased for all DGP's (as long as the DGP's expectation exists).
\end{enumerate}
\eenum\instr\pagebreak

\problem\timedsection{15} Consider the DGP $\Xoneton \iid \bernoulli{\theta}$ and the estimators\\ $\thetahat_B = \Xbar + 0.1$ and $\thetahat_A = \Xbar + 0.1 / n$.

\vspace{-0.2cm}\benum\truefalsesubquestionwithpoints{19}

\begin{enumerate}[(a)]
\item The estimator $\thetahat_B$ is unbiased for all $\theta \in \Theta$.
\item The estimator $\thetahat_B$ is unbiased for at least one $\theta \in \Theta$.
\item The estimator $\thetahat_B$ is asymptotically unbiased for all $\theta \in \Theta$.
\item Every possible estimate $\thetahathat_B$ is in $\Theta$.
\item At least one estimate $\thetahathat_B$ is in $\Theta$.
\item The L1 loss for $\thetahat_B$ is always $\geq 0.1$.
\item The L2 loss for $\thetahat_B$ is always  $\geq 0.1$.
\item The L1 loss for $\thetahat_B$ is always  $\geq 0$.
\item The L2 loss for $\thetahat_B$ is always  $\geq 0$.
\item The risk under L1 loss for $\thetahat_B$ is 0.1.
\item The risk under L2 loss for $\thetahat_B$ is $\expe{(\Xbar - \theta)^2 + 0.2 (\Xbar - \theta)}$ + 0.01.
\item The risk under L2 loss for $\thetahat_B$ is $\theta(1-\theta)  / n$ + 0.01.
\item $\text{MSE}[\thetahat_B] = \theta(1-\theta)  / n$ + 0.01.
\item The sup risk under L2 loss for $\thetahat_B$ is $\inverse{4n}$.
\item The estimator $\thetahat_A$ is unbiased for all $\theta \in \Theta$.
\item The estimator $\thetahat_A$ is asymptotically unbiased for all $\theta \in \Theta$.
\item $\text{MSE}[\thetahat_A] = \theta(1-\theta)  / n + 0.01 /n^2$.
\item The sup risk under L2 loss for $\thetahat_A$ is $\inverse{4n}$.
\item The loss function $\ell(\thetahathat, \theta) = |\thetahathat - \theta|^{0.5}$ is a valid loss function for these both $\thetahat_B$ and $\thetahat_A$.
\end{enumerate}
\eenum\instr\pagebreak

\problem\timedsection{10} The following are conceptual question about hypothesis testing.

\vspace{-0.2cm}\benum\truefalsesubquestionwithpoints{17}

\begin{enumerate}[(a)]
\item If you wish to prove a theory to the wide world, it is best to let detractors prove your theory wrong and if they cannot, your theory will be widely accepted.
\item If the value of $\theta$ were to be known, there would be no reason to do statistical hypothesis testing.
\item Using a sample of data, it is possible to absolutely prove (or disprove) a theory.
\item If you are trying to prove that your investment idea is correct, the null hypothesis should be that your investment idea is incorrect.
\item Type I errors are only possible if $H_0$ is true.
\item When doing a power calculation, we assume $H_0$ is true.
\item The type I error rate is unaffected by the sample size.
\item $\prob{\thetahat \in \text{RET}} = \alpha$.
\item If you retain $H_0$ it is impossible to know if you made a Type I error.
\item If you retain $H_0$ it is impossible to know if you made a Type II error.
%\item Level and size are always the same for any test.
%\item Fisher's p-value is unaffected by the value of $\alpha$.
\item The lower Fisher's p-value is, the more \qu{statistically significant} the result of the test becomes.
\item If $\alpha = 0$, power is always 1.
\item In the one-sample z test, if $\alpha$ decreases, power decreases.
\item In the one-sample z test, if $n$ decreases, power decreases.
\item In the one-sample z test, if $\sigsq$ decreases, power decreases.
\item In the one-sample z test, if $H_0: \theta \leq 3.6$, $H_a: \theta > 3.6$, this test would be \qu{one-sided} and \qu{right-tailed}.
\item In the one-sample z test, if $H_0: \theta \leq 3.6$, $H_a: \theta > 3.6$, $\alpha = 5\%$, $n=500$ and $\sigsq = 1.3$, power can be computed.
%\item In the one-sample z test, if $H_0: \theta \leq 3.6$, $H_a: \theta > 3.6$, $\alpha = 5\%$ and $\sigsq = 1.3$, power approaches 100\% as $n$ increases.
\end{enumerate}
\eenum\instr\pagebreak

\problem\timedsection{12} We flip a fair coin 7 times and record the proportion of heads $\thetahathat$. Let $\theta$ denote the probability of flipping heads. Below is a table of the PMF of the $\binomial{7}{0.5}$ rounded to three digits.

\begin{table}[h]
\centering
\begin{tabular}{c|cccccccc} %round(dbinom(0:7,7,0.5), 3)
$x$ & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ \hline
$p(x)$ & 0.008 & 0.055 & 0.164 & 0.273 & 0.273 & 0.164 & 0.055 & 0.008
\end{tabular}
\end{table}

\vspace{-0.2cm}\benum\truefalsesubquestionwithpoints{13}

\begin{enumerate}[(a)]
\item The sample size is $n=7$.\\

Assume for the following questions that we are trying to prove that the coin is unfair.
\item $H_0: \theta = 0.5$
\item The size 5\% is not attainable for our test scenario.
\item If the level was 1\%, the rejection region for $\thetahathat$ would be $\braces{0, 1}$.
\item If the level was 7.5\%, the rejection region for $\thetahathat$ would be $\braces{0, \oneover{7}, \frac{6}{7}, 1}$.
\item If the level was 5\%, and $\thetahathat  = 4/7$, the null hypothesis would be retained.
\item The test in (f) is called a \qu{Binomial Exact Test}.\\

Assume for the following questions that we are trying to prove that the coin is biased towards heads.
\item $H_0: \theta = 0.5$
\item If the level was 5\%, the rejection region for $\thetahathat$ would be $\braces{0, \oneover{7}, \frac{6}{7}, 1}$.
\item If the level was 5\%, and $\thetahathat  = 4/7$, the null hypothesis would be retained.
\item If the level was 5\%, and $\thetahathat  = 4/7$, Fisher's $p$-val would be 22.7\%.
\item If the level was 1\%, and $\thetahathat  = 5/7$, Fisher's $p$-val would be larger than Fisher's $p$-val when $\thetahathat  = 4/7$.
\item If the level was 1\%, and $\thetahathat  = 5/7$, you have enough information to compute power of this test.
\end{enumerate}
\eenum\instr\pagebreak

%set.seed(1)
%x = round(rnorm(7, 27, 2), 1)
%sum((x - mean(x))^2) / length(x)
%(27.1 - 29) / (sd(x) / sqrt(7))
%(27.1 - 29) / (sqrt(7/6*2.71) / sqrt(7))
\problem\timedsection{13} We suspect that pollution in a certain river is affecting the nutrition for a Atlantic Salmon and thereby making their adult size less than what is historically expected for salmon. Salmon are known to have mean length 29in with standard deviation 0.95in and for the distribution to be normal. However, we are not sure what the standard deviation for malnourished salmon will be. We collect the following lengths of salmon from the river: $\bv{x} = \angbraces{25.7, 27.4, 25.3, 30.2, 27.7, 25.4, 28.0}$ where $\xbar = 27.1$ and $\hat{\sigma}^2 = 2.71$. We would like to prove that $\theta$, the length of fish from the polluted river, are smaller than regular salmon via a 1-sample t-test. Below is a table of CDF values for the T distribution for many different degrees of freedom (df):

\begin{table}[h]
\centering
\begin{tabular}{c|ccccccc} %round(dbinom(0:7,7,0.5), 3)
df & 0.1\% & 0.5\% & 1\% & 5\% & 10\% & 15\% & 20\% \\ \hline
6 & -5.21 & -3.71 & -3.14 &-1.94 &-1.44 &-1.13 &-0.91 \\
7 & -4.79 & -3.50 &-3.00 &-1.89 &-1.41 &-1.12 &-0.90 \\
8 & -4.50 & -3.36 &-2.90 &-1.86 &-1.40 &-1.11 &-0.89 \\
\end{tabular}
\end{table}

\vspace{-0.2cm}\benum\truefalsesubquestionwithpoints{11}

\begin{enumerate}[(a)]
\item The value $\xbar = 27.1$ is a estimate realized from an unbiased estimator.
\item The value $\hat{\sigma}^2 = 2.71$ is a estimate realized from an unbiased estimator.
\item The reason why we use a 1-sample t-test is because we know $\sigma = 0.95$.
\item The value of $\xbar$ standardized is -1.855 rounded to the nearest 3 decimals.
\item The value of $\xbar$ standardized is -2.827 rounded to the nearest 3 decimals.
\item The value of $\xbar$ standardized is -3.298 rounded to the nearest 3 decimals.
\item The value of $\xbar$ standardized is -3.054 rounded to the nearest 3 decimals.
\item The null hypothesis is rejected at $\alpha = 1\%$.
\item The null hypothesis is rejected at $\alpha = 5\%$.
\item Fisher's $p$-val for this test is likely closest to 1\% among all the options in the table of CDF values for the T distribution.
\item Fisher's $p$-val for this test is likely closest to 5\% among all the options in the table of CDF values for the T distribution.
\end{enumerate}
\eenum\instr\pagebreak

\problem\timedsection{14} Chevalier de Mere was a French nobleman writer / philosopher who lived from 1607-1684. But he is far better known for his contribution to probability theory likely due to his compulsive gambling habit. Before notions of probabilitistic independence, complement rule, random variable theory (all the familiar 241 topics) were discovered, he wrote a letter to Blaise Pascal where he accurately conjectured that the probability of winning a certain dice game. In this dice game, you \qu{win} if you get at least one double 6 in a roll of two dice among 24 rolls of two dice. He conjectured that the probability of winning was less than 50\% and the actual probability is $\theta = 49.14\%$. Assuming each of these entire 24 rolls defines one game and he plays many games, then the DGP for winning of each game truly is $\iid \bernoulli{0.4914}$. Assuming he kept a flawless mental record of wins and losses, how many games would he need to observe to reject the null hypothesis that $\theta \geq 50\%$ at $\alpha = 5\%$ with power of 80\%? The two CDF values for the standard normal distribution that are needed are: $\Phi(-1.645)  = 5\%$ and $\Phi(0.84)  = 80\%$.

\vspace{-0.2cm}\benum\multchoicewithpoints{12}{Select the answer choice below whose number is \emph{closest} to your answer.}
\begin{multicols}{2}
\begin{enumerate}[(a)]
\item 1,000
\item 5,000
\item 7,500
\item 10,000
\item 15,000
\item 20,000
\item 30,000
\item 40,000
\item 50,000
\item 75,000
\item 100,000
\item 200,000

\end{enumerate}
\end{multicols}
\eenum\instr\pagebreak

\end{document}