\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[legalpaper, margin=1in]{geometry}

\usepackage[english]{babel}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{physics}

\usepackage{graphicx}
\usepackage{booktabs}

\usepackage{xcolor}
\usepackage{listings}
\usepackage{makecell}

\usepackage{hyperref}
\usepackage{cleveref}

\usepackage{marginnote}
\usepackage{csquotes}
\usepackage{todonotes}

\usepackage{listings}
\usepackage{xcolor}
\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{codepurple}{rgb}{0.58,0,0.82}
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}
\lstdefinestyle{mystyle}{
    backgroundcolor=\color{backcolour},
    commentstyle=\color{codegreen},
    keywordstyle=\color{magenta},
    numberstyle=\tiny\color{codegray},
    stringstyle=\color{codepurple},
    basicstyle=\ttfamily\footnotesize,
    breakatwhitespace=false,
    breaklines=true,
    captionpos=b,
    keepspaces=true,
    numbers=left,
    numbersep=5pt,
    showspaces=false,
    showstringspaces=false,
    showtabs=false,
    tabsize=2
}
\lstset{style=mystyle}

\newtheorem{definition}{Definition}
\newtheorem{theorem}{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{corollary}[lemma]{Corollary}
\newtheorem{property}{Property}

% Start every section on new page
% \usepackage{titlesec}
% \newcommand{\sectionbreak}{\clearpage}

\title{CSC413: Programming Assignment 3: Attention-Based Neural Machine Translation}
\author{Anzhe Dong, 1002608163}
\date{March 2021}

\begin{document}

\maketitle

\section*{Part 1: LSTMs}

\subsection*{1. LSTM Training}

\begin{quote}
A screenshot of your \texttt{fullMyLSTMCell} implementation
\end{quote}

\begin{lstlisting}[language=Python]
class MyLSTMCell(nn.Module):
    def __init__(self, input_size, hidden_size):
        super(MyLSTMCell, self).__init__()

        self.input_size = input_size
        self.hidden_size = hidden_size

        # ------------
        # FILL THIS IN
        # ------------
        self.Wii = nn.Linear(input_size, hidden_size)
        self.Whi = nn.Linear(hidden_size, hidden_size)

        self.Wif = nn.Linear(input_size, hidden_size)
        self.Whf = nn.Linear(hidden_size, hidden_size)

        self.Wig = nn.Linear(input_size, hidden_size)
        self.Whg = nn.Linear(hidden_size, hidden_size)

        self.Wio = nn.Linear(input_size, hidden_size)
        self.Who = nn.Linear(hidden_size, hidden_size)

    def forward(self, x, h_prev, c_prev):
        """Forward pass of the LSTM computation for one time step.

        Arguments
            x: batch_size x input_size
            h_prev: batch_size x hidden_size
            c_prev: batch_size x hidden_size

        Returns:
            h_new: batch_size x hidden_size
            c_new: batch_size x hidden_size
        """

        # ------------
        # FILL THIS IN
        # ------------
        i = torch.sigmoid(self.Wii(x) + self.Whi(h_prev))
        f = torch.sigmoid(self.Wif(x) + self.Whf(h_prev))
        g = torch.tanh(self.Wig(x) + self.Whg(h_prev))
        o = torch.sigmoid(self.Wio(x) + self.Who(h_prev))
        c_new = f * c_prev + i * g
        h_new = o * torch.tanh(c_new)
        return h_new, c_new
\end{lstlisting}

\begin{quote}
... the loss plots output by \texttt{save\_loss\_comparison\_lstm}
\end{quote}

\includegraphics[width=\linewidth]{loss_plot_lstm}

\begin{quote}
... and your analysis

Does  either  model perform significantly better?  Why might this be the case?
\end{quote}

The \texttt{pig\_latin\_small} model performed better with lower validation loss. The small model was trained with smaller batch sizes (\texttt{'batch\_size':64}), which tend to have better generalization ability than ones trained with larger batch sizes (\texttt{'batch\_size':512}). [Reference: Keskar NS, Mudigere D, Nocedal J, Smelyanskiy M, Tang PT. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836. 2016 Sep 15.]

\subsection*{Model Failure}

\begin{quote}
Identify a distinct failure mode and briefly describe it.
\end{quote}

\begin{lstlisting}
source:		the air conditioning is working
translated:	ethay airway oniningbay-intway isway orkingway

source:		this has a lot of errors
translated:	isthay ashay away otlay ofay errorsway

source:		the princess listened peacefully to what the frogs had to sing
translated:	ethay insecspray istenedlay eaffulecay otay athay ethay ogsgay adhay otay ingsay
\end{lstlisting}

The model fails apparently at (1) when the leading character is an vowel, (2) when the leading consonant pairs such as "sh" and "wh", (3) long words. % and often fails at words that start with vowel, with which it often correctly adds ``-ay'' at the end but scrambles the other letters.

\subsection*{Model size}

\begin{quote}
Write down the number of neurons and connections of this encoder model as a function of H, K, and D.  For simplicity, you may ignore the bias units.
\end{quote}

Number of neurons:

Number of connections:

\section*{Part 2: Additive Attention}

\subsection*{1}

\begin{align*}
\tilde{\alpha}^{(t)}_i &= W_2^\intercal \textrm{ReLU} ( W_1^\intercal
\begin{bmatrix}
Q_t \\ K_i
\end{bmatrix}
)
\\
\alpha^{(t)}_i &= \textrm{softmax}(\tilde{\alpha}^{(t)})_i = \frac
{\exp(\tilde{\alpha}^{(t)}_i)}
{\sum_{j=1}^{\textrm{seq\_len}} \exp(\tilde{\alpha}^{(t)}_j) }
\\
c_t &= {\alpha^{(t)}}^\intercal K = \sum_{i=1}^{\textrm{seq\_len}} \alpha^{(t)}_i K_i
\end{align*}

\subsection*{2}

\subsection*{3}
The training speed is faster with the attention model which reached less than $1.0$ validation loss within 7 epochs, whereas previously it took the ``small'' non-attention model 20 epochs to reach this level of validation loss.

\subsection*{4}

\section*{Part 3: Scaled Dot Product Attention}

\subsection*{1. Implement the scaled dot-product attention mechanism.}

\begin{quote}
A screenshot of your \texttt{ScaledDotAttention} implementation
\end{quote}

\begin{lstlisting}[language=Python]
class ScaledDotAttention(nn.Module):
    def __init__(self, hidden_size):
        super(ScaledDotAttention, self).__init__()

        self.hidden_size = hidden_size

        self.Q = nn.Linear(hidden_size, hidden_size)
        self.K = nn.Linear(hidden_size, hidden_size)
        self.V = nn.Linear(hidden_size, hidden_size)
        self.softmax = nn.Softmax(dim=1)
        self.scaling_factor = torch.rsqrt(torch.tensor(self.hidden_size, dtype= torch.float))

    def forward(self, queries, keys, values):
        """The forward pass of the scaled dot attention mechanism.

        Arguments:
            queries: The current decoder hidden state, 2D or 3D tensor. (batch_size x (k) x hidden_size)
            keys: The encoder hidden states for each step of the input sequence. (batch_size x seq_len x hidden_size)
            values: The encoder hidden states for each step of the input sequence. (batch_size x seq_len x hidden_size)

        Returns:
            context: weighted average of the values (batch_size x k x hidden_size)
            attention_weights: Normalized attention weights for each encoder hidden state. (batch_size x seq_len x 1)

            The output must be a softmax weighting over the seq_len annotations.
        """

        # ------------
        # FILL THIS IN
        # ------------
        batch_size, _i, _j = queries.size()
        q = self.Q(queries).view(batch_size, -1, self.hidden_size)
        k = self.K(keys).view(batch_size, -1, self.hidden_size)
        v = self.V(values).view(batch_size, -1, self.hidden_size)
        unnormalized_attention = torch.bmm(k, q.transpose(1,2)) * self.scaling_factor
        attention_weights = self.softmax(unnormalized_attention)
        context = torch.bmm(attention_weights.transpose(1,2),v)
        return context, attention_weights
\end{lstlisting}

\subsection*{2. Implement the causal scaled dot-product attention mechanism.}

\begin{quote}
A screenshot of your \texttt{CausalScaledDotAttention} implementation
\end{quote}

\begin{lstlisting}[language=Python]
class CausalScaledDotAttention(nn.Module):
    def __init__(self, hidden_size):
        super(CausalScaledDotAttention, self).__init__()

        self.hidden_size = hidden_size
        self.neg_inf = torch.tensor(-1e7)

        self.Q = nn.Linear(hidden_size, hidden_size)
        self.K = nn.Linear(hidden_size, hidden_size)
        self.V = nn.Linear(hidden_size, hidden_size)
        self.softmax = nn.Softmax(dim=1)
        self.scaling_factor = torch.rsqrt(torch.tensor(self.hidden_size, dtype= torch.float))

    def forward(self, queries, keys, values):
        """The forward pass of the scaled dot attention mechanism.

        Arguments:
            queries: The current decoder hidden state, 2D or 3D tensor. (batch_size x (k) x hidden_size)
            keys: The encoder hidden states for each step of the input sequence. (batch_size x seq_len x hidden_size)
            values: The encoder hidden states for each step of the input sequence. (batch_size x seq_len x hidden_size)

        Returns:
            context: weighted average of the values (batch_size x k x hidden_size)
            attention_weights: Normalized attention weights for each encoder hidden state. (batch_size x seq_len x 1)

            The output must be a softmax weighting over the seq_len annotations.
        """

        # ------------
        # FILL THIS IN
        # ------------
        batch_size, _i, _j = queries.size()
        q = self.Q(queries).view(batch_size, -1, self.hidden_size)
        k = self.K(keys).view(batch_size, -1, self.hidden_size)
        v = self.V(values).view(batch_size, -1, self.hidden_size)
        unnormalized_attention = torch.bmm(k, q.transpose(1,2)) * self.scaling_factor
        mask = self.neg_inf * torch.tril(torch.ones_like(unnormalized_attention), diagonal=-1)
        attention_weights = self.softmax(unnormalized_attention + mask)
        context = torch.bmm(attention_weights.transpose(1,2),v)
        return context, attention_weights
\end{lstlisting}

\subsection*{3. }

\subsection*{4. }

\subsection*{5. }

\end{document}


\section{Torsion or Dihedral Angle Coherency}\label{coherency}
\index{Dihedral angles!coherency}\index{Chirality}\index{Enantiomers}
MOPAC  calculations  do  not   distinguish   between   enantiomers,
consequently  the  sign of the dihedrals can be multiplied by $-1$ and the
calculations will be unaffected.  However, if chirality is important,  a user
should be aware of the sign convention used.

The dihedral angle convention used in  MOPAC  is  that  defined  by Klyne and
Prelog~\cite{klyne}. \index{klyne@{\bf Klyne and Prelog}}  In this convention,
four atoms, AXYB, with a dihedral angle of 90 degrees, will have atom  B
rotated  by 90 degrees clockwise relative to A when X and Y are lined up in the
direction of sight, X being nearer to the eye.  In  their  words, ``To
distinguish between enantiomeric types the angle $\tau$ is considered as
positive when it is measured clockwise from the front  substituent  A to
the   rear   substituent  B,  and  negative  when  it  is  measured
anticlockwise.'' The alternative  convention  was  used  in  programs which
preceded MOPAC.


\documentclass[simplex.tex]{subfiles}
% NO NEED TO INPUT PREAMBLES HERE
% packages are inherited; you can compile this on its own
\providecommand{\mb}[1]{\boldsymbol{#1}}
\providecommand{\mv}[1]{\vec{#1}}
\providecommand{\ve}[1]{\boldsymbol{#1}}
\newcommand{\bpsi}{\ve{\psi}}
\newcommand{\bv}{\mb{v}}
\newcommand{\bx}{\mb{x}}

\onlyinsubfile{
\title{NeuroData SIMPLEX Report: Subfile}
}

\begin{document}
\onlyinsubfile{
\maketitle
\thispagestyle{empty}

The following report documents the progress made by the labs of Randal~Burns and Joshua~T.~Vogelstein at Johns Hopkins University towards goals set by the DARPA SIMPLEX grant.

%%%% Table of Contents
\tableofcontents

%%%% Publications
\bibliographystyle{IEEEtran}
\begin{spacing}{0.5}
\section*{Publications, Presentations, and Talks}
%\vspace{-20pt}
\nocite{*}
{\footnotesize	\bibliography{simplex}}
\end{spacing}
%%%% End Publications
}

\subsection{Discriminability}

We develop a measure of discriminability (or reliability).  It
is intuitive to understand and easy to implement.
Discriminability is defined to be the probability of within
subject distances being smaller than the cross subject
distances. If we let $x_{i,t}$ denote the t$^{\text{th}}$ trial of subject
$i$ and $\Delta(,)$ be the metric, the (population) discriminability $D$
is: $D= P (\Delta(x_{i,t} , x_{i,t’}) \leq  \Delta(x_{i,t} , x_{i’,t’’}))$.
We want to search for the optimal processing pipeline which has the
maximal discriminability.

During the last month, we have been trying to finalize the paper. Specifically, we formalize three algorithms and write the pseudocode for the algorithms. The three algorithms are computing sample discriminability from test and retest data set \ref{alg:dhat}, testing whether the data is better than random \ref{alg:ost}, and testing whether one processing pipeline is better than another \ref{alg:tst}. The pseudocode is provided below. For the R code and functions, please see \href{http://github.com/neurodata/discriminability}{neurodata/discriminability}.

Furthermore, we carry out the Algorithm \ref{alg:tst} on 13 fMRI data sets. Previously, we process 13 fMRI data sets with 64 pipelines, and show some pipelines yields data
sets with high sample discriminability. However, we do not have a valid test to compare different pipelines other than rank them by mean discriminability. We use the algorithm \ref{alg:tst} to carry out a two sample test to compare different pipelines for each data set. Specifically, all pipelines are compared to pipeline CFXSG. Then, a single p-values is calculated by applying Fisher's method to combine p-values from 13 sets. We group the pipelines by p-values and within each group the pipelines are ordered by mean discriminability. The result of rank graphs is shown in Figure \ref{fig:pipes}. The result of raw graphs can also be found \href{http://github.com/neurodata/discriminability}{here}. The new two sample test provide a better ordering of pipelines compared to the previous ordering by Wilcoxon signed-rank test.

\begin{figure}[h!]
	\begin{cframed}
		\centering
		\includegraphics[width=\textwidth]{../../figs/fmri_rank_pv_v2.png}
		\caption{
			{\bf Discriminability of rank fmri graphs from 13 data sets processed 64 ways.}  Discriminability of BNU1, BNU2, DC1, HNU1, IACAS, IBATRT, IPCAS, JHNU, NYU1, SWU1, UM, UWM and XHCUMS pre-processed by 64 pipelines are shown in the plot. Color of each dot indicates data set and size indicates the number of measurements in data set. The black square indicates the weighted mean discriminability across 13 data sets.  For each data set, all pipelines are compared to the pipeline CFXSG using the two sample test, and a single p-value is calculated by Fisher's method. The pipelines are grouped by p-values. The number at the top indicates the range of the p-values. Within each group, the pipelines are ordered by the mean discriminability. CFXSG pipeline has the best mean discriminability across data sets.}
		\label{fig:pipes}
	\end{cframed}
\end{figure}

\subsubsection*{Discriminability Algorithms}
\begin{algorithm}
	\caption{Compute discriminability estimate $\hat{D}$.  }
	\label{alg:dhat}
	\begin{algorithmic}
		\Require Test-retest samples $\{\bx_{i,t}\}$.
		\Ensure Sample discriminability $\hat{D}$.
		\Function{ComputeDiscriminability}{}
		\For{$i,t,i',t'$} \Comment{compute pairwise distances}
		\State $PD[i,t,i',t'] = \delta(\bx_{i,t},\bx_{i',t'})$
		\EndFor
		\State Dsum = 0;
		\State Count = 0;
		\For{$i = 1,...,n$}
		\For{$t = 1,...,s$}
		\State $d =$ across subject distances to $\bx_{i,t}$
		\For{$t' = 1,...,s$ and $t' \neq t$}
		\State $\hat{D}_{i,t,t'} =\sum_j {\bf I}\{PD[i,t,i,t'] < d[j]\}/Length(d)$ \Comment{compare within and across distances}
		\State $Dsum = Dsum + \hat{D}_{i,t,t'}$
		\State $Count = Count + 1$
		\EndFor
		\EndFor
		\EndFor
		\State $\hat{D} = Dsum / Count$ \Comment{Sample Discrminability}
		\EndFunction
	\end{algorithmic}
\end{algorithm}

\begin{algorithm}
	\caption{The function returns a p-value for testing the null hypothesis that $D = 0.5$.  }
	\label{alg:ost}
	\begin{algorithmic}
		\Require Test-retest samples $\{\bx_{i,t}\}$, the number of permutations $np$.
		\Ensure The p-value $p \in [0,1]$ for testing the hypothesis that $D = 0.5$.
		\Function{OneSampleTest}{}
		\State $\hat{D}= ComputeDiscriminability(\{\bx_{i,t}\})$ \Comment{compute true sample discriminability}
		\For{$j = 1,...,np$}
		\State $\{\bx^{(j)}_{i,t}\} = Permute(\{\bx_{i,t}\})$ \Comment{permute the subject labels}
		\State $d[j] = ComputeDiscriminability(\{\bx^{(j)}_{i,t}\})$ \Comment{compute discriminability of permuted samples}
		\EndFor
		\State $p = \sum_j {\bf I}\{\hat{D} < d[j]\} / np$ \Comment{compute p-value}
		\EndFunction
	\end{algorithmic}
\end{algorithm}

\begin{algorithm}
	\caption{The function returns a p-value for testing the null hypothesis that $D(\bpsi_1)=D(\bpsi_2)$.  }
	\label{alg:tst}
	\begin{algorithmic}
		\Require Raw data $\{f_\phi(\bv_i)\}$ , pipeline $\bpsi_1$, pipeline $\bpsi_2$, the number of bootstrapped samples $nb$.
		\Ensure The p-value $p \in [0,1]$ for testing the hypothesis that $D(\bpsi_1)=D(\bpsi_2)$.
		\Function{TwoSampleTest}{}
		\State $\{^1\bx_{i,t}\}= g_{\bpsi_1}(\{f_\phi(\bv_i)\})$
		\State $\{^2\bx_{i,t}\}= g_{\bpsi_1}(\{f_\phi(\bv_i)\})$ \Comment{process the raw data with two pipelines}
		\State $\hat{D}(\bpsi_1)=ComputeDiscriminability(\{^1\bx_{i,t}\})$
		\State $\hat{D}(\bpsi_2)=ComputeDiscriminability(\{^1\bx_{i,t}\})$ \Comment{compute sample discriminability estimates}
		\For{$j = 1,...,nb$}
		\For{$i = 1,...,n$}
		\State $i_1, i_2 = Sample(n)$ \Comment{randomly select two subjects}
		\State $\lambda = SampleUniform$
		\For{t = 1,...,s}
		\State $^1\bx^{(j)}_{i,t} =  ^1\bx_{i_1,t} \lambda +   ^1\bx_{i_2,t}(1-\lambda)$ \Comment{Linearly combine two observations}
		\EndFor
		\EndFor
		\State $\hat{D}^{(j)}(\bpsi_1)=ComputeDiscriminability(\{^1\bx^{(j)}_{i,t}\})$
		\State $\hat{D}^{(j)}(\bpsi_2)=ComputeDiscriminability(\{^2\bx^{(j)}_{i,t}\})$
		\EndFor
		\State $ind = 0$
		\For{$j = 1,...,nb$} \Comment{generate the null distribution}
		\For{$j' = i+1,...,nb$}
		\State $d[ind] =  \hat{D}^{(j)}(\bpsi_1)-\hat{D}^{(j')}(\bpsi_1)$
		\State $ind = ind + 1$
		\State $d[ind] =  \hat{D}^{(j)}(\bpsi_2)-\hat{D}^{(j')}(\bpsi_2)$
		\State $ind = ind + 1$
		\EndFor
		\EndFor
		\State $p = \sum_j {\bf I}\{\hat{D}(\bpsi_1)-\hat{D}(\bpsi_2) < d[j]\} / Length(d)$ \Comment{compute p-value}
		\EndFunction
	\end{algorithmic}
\end{algorithm}

\end{document}


\chapter{Stratified sampling}\label{chap:stratified_sampling_chapter}

In the previous chapter we introduced the GNK value and recognised that it could be approximated by a stratified sampling process (section \ref{sec:sampling_techniques}).
Accordingly, in this chapter we investigate the different possible rules for choosing samples from strata in the context of stratified sampling.

We review the orthodox solution to the problem which is called Neyman allocation, and show how it is equivalent to minimising a concentration inequality called Chebyshev's ineqality.
We then turn our attention to two different approaches to developing novel concentration inequalities for stratified sampling, whose minimisation yields new stratified sampling methods.
These new sampling methodologies are then tested for their performance in the context of synthetic sample data sets, and in the context of sampling the Shapley Value of cooperative games.

The structure of this Chapter is as follows:
\begin{enumerate}
\item	In section \ref{sec:sampling_background}, we give a review of relevant background information and techniques, particularly addressing the recent innovation of Empirical Bernstein Bounds (EBBs).
\item	In section \ref{section:old_statistics}, of this chapter we show how EBBs can be bound together to create concentration inequalities appropriate for Stratified Sampling and give an algorithm for choosing samples to minimise these bounds.
\item	In section \ref{section:new_EBB} of this chapter we derive a unique and stronger EBB, for the purposes of evaluation in the context of stratified sampling.
\item	In section \ref{section:SEBB}, we provide multi-part derivations of complicated concentration inequalities specifically tailored for the purposes of stratified sampling, and give an algorithm for choosing samples to minimise these inequalities.
\item	In section \ref{section:statistics_results}, we give the performance results of various methods of stratified sampling in the context of synthetic data sets
\item	In section \ref{sec:shapley}, we give performance results of approximating the Shapley Value for various cooperative games.
\item	In section \ref{sec:discussion} we give discussion to the derivations and results,
\item	in section \ref{sec:multi} and talk briefly about future work about a multidimensional stratified sampling EBB, and conclude.
\end{enumerate}

The work and development from the second and third section most directly reflects published material: ``An Engineered Empirical Bernstein Bound'',\\ European Conference on Machine Learning (ECML-PKDD) 2019\\(accessible: \href{http://ecmlpkdd2019.org/downloads/paper/435.pdf}{ecmlpkdd2019.org/downloads/paper/435.pdf})\\
The work and development from the fourth and fifth section most directly reflect published material: ``Approximating the Shapley Value Using Stratified Empirical Bernstein Sampling'', IJCAI 2021\\
(accessible: \href{https://www.ijcai.org/proceedings/2021/0011.pdf}{ijcai.org/proceedings/2021/0011.pdf})\\

%In the same way as the average of a statistic over the population of a country is expressible as the weighted average over the averages of its regions, so to the Shapley Value and GNK value are expressible as a weighted average over the averages of coalitions by different sizes.
%Hence component of our investigation turns fundamentally into a question about stratified sampling.

%Particularly, when we have a population decomposed into strata of known sizes which we can selectively sample from, how do we choose samples from the strata to get the most accuracy in the final population estimate?

%This question is directly relevant to sampling the GNK and Shapley value, since they are both immediately expressible as a weighted average of strata averages, where the strata and their sizes are immediately given. And in the context of sampling the Shapley Value, the question of which strata should sampled is equivalent to a question about what sized coalitions should be sampled in consideration of each players marginal contributions to them.

\section{Introduction and background}\label{sec:sampling_background}

%In the previous chapter, we defined the GNK value and saw that there is a significant computational difficulties in solving the GNK value at scale.
%Particularly that the GNK value and its Shapley Value formulation are definable as being an average over a combinatorial number of averages (via equations \ref{da_value_eq} or \ref{eq:shapley_value2} by conversion identified by equation \ref{convert1}).

%It was realised that these averages might be approximated by sample averages, and hence that one potential avenue of approximating the GNK value at scale was by sampling.
%And the process of replacing analytic averages with sample averages leads to the question of how that sampling process could be done most effectively.

Stratified sampling is a well known example of a process of selecting samples to most accurately estimate an average over weighted sample averages.
Particularly stratified random sampling is a process of estimating the average over a population by breaking a population into mutually-exclusive subgroups and sampling them randomly.
Such stratified sampling has been identified to lead to improved reliability in estimation over simple random sampling of the population by \cite{1938.10503378,10.2307/23339498} particularly when:
\begin{itemize}
\item The population is divisible into strata, in which there is less variance in each stratum than across them all
\item When the size of the strata are known or reasonably estimated
\item When sampling selectively from each strata is possible
\end{itemize}

The concept of stratified random sampling is easily illustrated, for instance, to poll the population of a country's support for a particular government policy, it is possible to sample the different demographic regions within the country.
For instance, if regions $A$,$B$ and $C$ contain 10\%, 40\% and 50\% of a population, and sampling of these regions reliably show support levels of 2\%, 70\% and 30\%, respectively, then it is possible to discern that 43.2\% of the total population supports the policy.

Our primary investigative question of this chapter is this: when we have a population decomposed into strata of known sizes which we can selectively sample from, how do we choose samples from the strata to get the most accuracy in the final population estimate?

For instance, if we sample primarily from a single stratum, we would likely have a very good estimate of the average for that one stratum, but no accuracy in estimate for the others, leading to a weak estimate of the population average.
Conversely, if we choose to take the same number of samples from all the strata, some of the strata might be far smaller or have far less variance than others, resulting in those strata being oversampled.

%The core question we face is, how do we sample optimally between the strata in stratified sampling?
This question has been considered before, and the most direct historical answer to the question is called \textit{Neyman allocation} or \textit{sampling} \citep{1938.10503378}.
The principle of Neyman sampling is that it seeks the minimise the weighted variance of the population estimate assuming knowledge of the variances of the strata, and it can most directly be interpreted as a process of minimising Chebyschev's inequality.

\subsection{Neyman allocation}\label{sec:neyman_sampling}

Neyman allocation identifies that in order to minimise the variance in the final estimate of the population, that sampling should be directly in proportion to the stratum variances multiplied by their sizes.
We present Neyman allocation rule with its proof to illustrate the connection with allocation rule with a concentration inequality:

\begin{theorem}[Neyman allocation]\label{thm:neyman_selection}
For $m$ strata, of sizes $N_i$, with variance $\sigma_i^2$. For a sample budget $n$, and there is a choice how much to sample from each strata $n_i$, in which sampling is done with replacement (ie. all samples are independent and identically distributed)
Then the selection of $n_i$ which minimises the variance of our population estimate $\mu$ is:
$$n_i = \frac{nN_i\sigma_i}{\sum_jN_j\sigma_j}$$
\end{theorem}
\begin{proof}
For any independant random variables $X$ and $Y$ (and for any $a,b\in \mathbb{R}$), $\text{Var}(aX+bY) = a^2\text{Var}(X)+b^2\text{Var}(Y)$.\\
So, if $X_{i,j}$ are the random variable of the $j$th sample from the $i$th stratum, then:
\begin{equation}\label{eq:variance_decomposition_for_strata_mean} \hat{\mu} = \sum_{i=1}^m\frac{N_i}{\sum_kN_k}\frac{1}{n_i}\sum_{j=1}^{n_i}X_{i,j} ~~~~~~~\text{and hence}~~~~~~~\text{Var}(\hat{\mu}) = \frac{1}{\left(\sum_kN_k\right)^2} \sum_{i=1}^m\frac{N_i^2}{n_i}\sigma_i^2 \end{equation}
To minimise the variance of our estimate $\text{Var}(\mu)$ by selecting $n_i$ subject to the constraint that $\sum_{i=1}^m n_i = n$, we form the Lagrangian (with Lagrange multiplier $\lambda$):
$$L = \text{Var}(\hat{\mu}) + \lambda\left(\sum_{i=1}^mn_i-n\right) = \sum_{i=1}^m\left(\frac{N_i^2\sigma_i^2}{n_i\left(\sum_kN_k\right)^2} + \lambda \left(n_i-\frac{n}{m}\right)\right)$$
Hence for any $i$ solving for $\frac{\partial L}{\partial n_i}=0$ leads to:
$n_i = \frac{N_i\sigma_i}{\lambda\sum_kN_k}$ and using $\sum_{i=1}^m n_i = n$, to eliminate $\lambda$ gives the result
\end{proof}

This allocation rule begs the question of why we would specifically want to minimise the variance of our population mean estimate.
And the primary reason is that that the variance of an estimate bounds the probability of error in the estimate, and this can be seen most directly by Chebyshev's inequality.

\begin{theorem}[Chebyshev's inequality]\label{thm:chebyshevs}
for any random variable --- in this case $\hat{\mu}$ is a random variable --- with variance $\text{Var}(\hat{\mu})$ then the error of $\hat{\mu}$ from its mean is probability bounded:
$$ \p\left(|\hat{\mu}-\mu|\ge k\sqrt{\text{Var}(\hat{\mu})}\right)\le\frac{1}{k^2} $$
\end{theorem}

Thus if $\mu$ is the true population mean, then the error in our stratified estimate of the population mean $\hat{\mu}$ is probability bounded by the variance of it.
So, for instance, Chebyshev's inequality guarantees that there will always be less than a 25\% chance that the error in our estimate of the population mean will be more than twice the square root of its variance.
In this context, Chebyshev's is an example of a \textit{concentration inequality}, as it provides probability bounds on the concentration of the estimate around its mean value.

An additional and perhaps more intuitive angle by which Neyman allocation may be seen to be appropriate, is that if sufficiently large numbers of samples have been taken then the sample means of the strata will tend to be Gaussian distributed by the Central Limit Theorem.
In this context the strata means have a distribution that is entirely characterised by their mean and variance, and hence so too therefore is the population mean.
In this context the variance of the sampled population mean is the only parameter controlling the error, and minimising it directly translates into improved accuracy.

One primary limitation of using Neyman allocation, is that is presupposes knowledge of the variances of the strata, which usually aren't available beforehand or in practice.
One relatively easy way of going around this problem is to go through a process to estimate the variances of the strata, perhaps as a prior step, and then using this knowledge with Neyman allocation to choose further samples.
%And indeed some proceses do this, particularly in Section \ref{sec:shapley} we will consider a method of sampling the Shapley Value using exactly this idea.
And while this idea certainly works, it leaves open the question about how much sampling should be done to estimate the strata variances against how much sampling should be left to sample by those estimated variances.

Neyman allocation is an orthodox sampling rule which directly extends from minimising a specific concentration inequality that unfortunately depends on known variances.
It is therefore suitable to ask if there are alternative concentration inequalities which do not depend on known variances, which can be minimised to form novel sampling rules for stratified sampling.

\subsection{Concentration inequalities and Chernoff bounds}

In order to consider what alternative concentration inequalities exist, we need to turn to the space of concentration inequalities generally.

Neyman allocation implicitly minimises Chebyshev's inequality, but Chebyshev's inequality is one example of many concentration inequalities.
Concentration inequalities are applied in a range of data science contexts for a variety of prediction, machine learning and hypothesis testing tasks, including:
change detection \citep{KiferShaiGehrke2004,8000571}
and classification \citep{Zia-UrRehman2012} in data streams;
outlier analysis in large databases \citep{Aggarwal2015};
online optimisation \citep{FlaxmanKalaiMcMahan2005,AgarwalDekelXiao2010};
online prediction and learning problems \citep{%Cesa-BianchiLugosi2006,%Maron1997,
Mnih:2008:EBS:1390156.1390241,DBLP:conf/aaai/ThomasTG15,Maurer50empiricalbernstein},
and in settings with bandit feedback \citep{AuerCesa-BianchiEtal_SIAM2003,AudibertBubeck_COLT2009,Tran-ThanhChapmanRJ_AAAI2009}.

There are many famous concentration inequalities such as Chebyshev's inequality~\citep{Chebyshev1}, Bernstein's inequalities~\citep{Burnstein1},
Hoeffding's inequalities~\citep{hoeffding1} and Bennett's inequalities~\citep{10.2307/2282438}.
New analysis has yielded a wider range of concentration inequalities and methods of generating them.
In particular, various innovations concern the concentration of more-general functions of random variables, such as
the Efron-Stein~\citep{efron1981} and
entropy methods~\citep{Boucheron_concentrationinequalities}.%, and Talagrand's concentration inequality~\cite{Talagrand1995}.

Recently, concentration inequalities have been developed which do not rely on variance information, but incorporate uncertainty about variance information via the sample variance, these bounds are sometimes called \textit{Empirical Bernstein Bounds} (EBB).
These concentration inequalities describe the likely difference of a sample mean from the population mean in terms of the \textit{sample} variance, some of the first EBBs given in literature are:

\begin{theorem}[\cite{Maurer50empiricalbernstein}]\label{MandPsEBB}
Let $X$ be a real-valued random variable that is bounded $a\le X\le b$, with $D=b-a$.  Then for $x_1,x_2,\dots,x_n$ independent samples of $X$ the mean $\hat{\mu}=\frac{1}{n}\sum_{i=1}^nx_i$ and sample variance $\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n(x_i-\hat{\mu})^2 $ are probability bounded by $t$ for any $t>0$:
\begin{equation}\label{maurersbound}
    \p\left(\mu-\hat{\mu}\ge\sqrt{\frac{2\hat{\sigma}^2\log(2/t)}{n}}+\frac{7D\log(2/t)}{3(n-1)}\right)\le t
\end{equation}
\end{theorem}

\begin{theorem}[\cite{10.1007/978-3-540-75225-7_15}]\label{AudibertsEBB}
In exactly the same context as in Theorem \ref{MandPsEBB}
\begin{equation}
    \p\left(\mu-\hat{\mu}\ge\sqrt{\frac{2\hat{\sigma}^2\log(3/t)}{2n}} + \frac{3D\log(3/t)}{2n}\right) \le t.
    \end{equation}
\end{theorem}

The derivation of these EBBs are of interest, because they illustrate how it is possible to derive concentration inequalities which involve variance information but do not depend directly on the variance itself, and which might be extended to develop alternative methods of stratified sampling.
Unfortunately these EBBs cannot directly be used to minimise the error in the aggregated population estimate, as the aggregated population estimate does not itself have a sample variance.
But in later section \ref{section:unionising_ebbs} we will show how these types of inequalities can be modified and subsequently minimised to create new methods of sample selection in stratified random sampling.

In section \ref{derivation} we will also show how it is possible to derive stronger EBBs, thus we introduce some of the components used in the process of deriving EBBs.

\subsection{Components of concentration inequality derivations}

One of the more famous concentration inequalities is \textit{Hoeffding's Inequality} which is one of a class of concentration inequalities called \textit{Chernoff bounds}.

\begin{lemma}[Chernoff Bound]\label{chernoff1}
If $\hat{\mu}$ is sample mean of $n$ independent and identically distributed samples of random variable $X$ then for any $s>0$ and $t$:
\[ \p(\hat{\mu}\ge t)\le\E\left[\exp(sX)\right]^n\exp(-snt) \]
\end{lemma}
\begin{proof}[Proof of Chernoff Bound - Lemma \ref{chernoff1}]
$$\p(\hat{\mu}\ge t) =  \p\left(\exp\left(s\sum_{i=1}^nx_i\right)\ge \exp(snt)\right)$$
$$\le \E\left[\exp\left(s\sum_{i=1}^nx_i\right)\right]\exp(-snt) \le \E\left[\exp\left(sX\right)\right]^n\exp(-snt)
$$
using Markov's inequality and the i.i.d of the samples, respectively.
\end{proof}

Here we have given the general form of Chernoff bounds for the sample mean of random variable, parameterised by a choice of $s>0$ and $t$.
Using this kind of lemma, many well-known examples of Chernoff bounds follow from the derivation of upper bounds for $\E\left[\exp(sX)\right]$, also known as the \textit{moment generating function}.

For any upper bound $\E[\exp(sX)]\le g(s)$ then $\p(\hat{\mu}>t)\le g^n(s)\exp(-snt)$ is an upper bound for the deviation of the sample mean, which can then be subsequently minimised with $s$, forming a concentration inequality; in this way a Chernoff concentration inequality can be deduced from a bound on the moment-generating-function.
Hoeffding's inequality \citep{hoeffding1} is an illustrative example of this process of deriving a Chernoff bound (which we will do in many times in this chapter), and we give a short refactored proof of this process:

\begin{theorem}[Hoeffding's inequality for mean zero]\label{hoeffdings_inequality}
Let $X$ be a real-valued random variable that is bounded $a\le X\le b$, with a mean $\mu$ of zero.  Then for $D=b-a$ and any $t>0$, the mean $\hat{\mu}$ of $n$ independent samples of $X$ is probability bounded by:
\begin{equation}\label{eq_no2}\p(\hat{\mu}\ge t)\le \exp\left(\frac{-2nt^2}{D^2}\right)
\end{equation}
\end{theorem}
\begin{proof}
If $X$ has a probability density function $f(x)$, then we can linearise $\exp(sx)$ as:
\begin{equation}\label{Hoeffdings_line_fitting}\E[\exp(sX)] = \int_a^bf(x)\exp(sx)dx \le \int_a^bf(x)\left(\frac{x-a}{b-a}\exp(sb) + \frac{b-x}{b-a}\exp(sa)\right)dx\end{equation}
Using the fact that the mean $\mu = \int_a^bf(x)xdx = 0$ thus:
\begin{equation}\E[\exp(sX)] \le \frac{1}{sb-sa}\left(sb\exp(sa) - sa\exp(sb) \right)\end{equation}
Given the fact that for any $\kappa>0,\gamma<0$:
\begin{equation}\label{Hoeffdings_lemma} \frac{\kappa\exp(\gamma)-\gamma\exp(\kappa)}{\kappa-\gamma}\le \exp\left(\frac{1}{8}(\kappa-\gamma)^2\right) \end{equation}
thus:
\begin{equation}\label{hoeffdings_lemma_eq}\E[\exp(sX)] \le \exp\left(\frac{1}{8}s^2(b-a)^2 \right)\end{equation}
Applying our Chernoff bound lemma \ref{chernoff1} we get:
$$ \p(\hat{\mu}\ge t) \le \exp\left(\frac{1}{8}s^2(b-a)^2 n-snt\right) $$
And minimising with respect to $s$ yields the required result.
\end{proof}

The most limiting feature of the derivation is the requirement that the mean is zero however this is immaterial and is used to simplify the derivation, as any statistic can be shifted such that its expectation value becomes zero, hence:

\begin{theorem}[Hoeffding's inequality]\label{Hoeffdings_inequality_proper}
Let $X$ be a real-valued random variable that is bounded $a\le X\le b$.  Then for $D=b-a$ and any $t>0$, the mean $\hat{\mu}$ of $n$ independent samples of $X$ is probability bounded by:
\begin{equation}\label{eq_no2}\p(\hat{\mu}-\mu\ge t)\le \exp\left(\frac{-2nt^2}{D^2}\right)
\end{equation}
Alternatively by rearranging:
\begin{equation}\label{eq_no2}\p\left(\hat{\mu}-\mu\ge \sqrt{\frac{D^2\log(1/t)}{2n}}\right)\le t
\end{equation}
\end{theorem}

%A limiting feature of Hoeffding's inequality is that it assumes that the data are bounded within a range $a\le X\le b$, which limits the application of Hoeffding's inequality.
%Similarly Chebyschev's inequality only sensibly holds in contexts where the data's distribution has a defined variance (consider that various heavy-tailed distributions may not have a finite variance, for instance).
%What is quite notable is that it is difficult to come to very meaningful conclusions about data processes without assuming atleast -something- about the distribution of data in question.
%And further that these inequalities can be seen as being largely a consequence of chosen assumptions.
%These assumptions may be based on prior information, expert opinion, or determined from the characteristics of the system under observation.

In this way Hoeffding's inequality is a rather friendly result that states that the concentration of sample mean is probability bounded by a Gaussian function, and this might be seen as an intuitive corollary of the Central Limit Theorem.
We provide the derivation of Hoeffding's inequality to illustrate the technique of deriving Chernoff bounds.
Additionally we will utilise Equation \ref{hoeffdings_lemma_eq} in further derivations - also called Hoeffding's Lemma:

\begin{lemma}[Hoeffding's Lemma]\label{Hoeffdings_lemma_lemma}
Let $X$ be a real-valued random variable that is bounded $a\le X\le b$, with a mean $\mu$ of zero, then for $D=b-a$ and any $s>0$:
$$\E[\exp(sX)] \le \exp\left(\frac{1}{8}s^2D^2 \right)$$
\end{lemma}

This process of deriving and minimising a bound for the moment-generating-function will be used repeatedly to create novel concentration inequalities in sections \ref{section:old_statistics} and \ref{section:SEBB}.

\subsubsection{A further note Chernoff bounds, sampling with or without replacement}

In many cases, the derivation of concentration inequalities assume that the values that the samples take are independently from each other.
This most naturally corresponds to the schema of sampling \textit{with} replacement, rather than sampling \textit{without} replacement.
However Chernoff probability bounds that assume independence of the samples are also suitable to the case of sampling without replacement, this is due to a result shown by \cite{hoeffding1}:

\begin{lemma}[Hoeffding's reduction]\label{hoeffdings_reduction}
let $X=(x_1,\dots,x_n)$ be a finite population of $n$ real points, let $X_1,\dots,X_n$ denote random samples without replacement from $X$ and $Y_1,\dots,Y_n$ denote random samples with replacement from $X$.
If $f:\mathbb{R}\rightarrow\mathbb{R}$ is continuous and convex, then:
$$\textstyle \E\left[f\left(\sum_{i=1}^nX_i\right)\right]\le\E\left[f\left(\sum_{i=1}^nY_i\right)\right] $$
\end{lemma}

Using the continuous and convex function $f(x)=\exp(sx)$ it can be seen via the construction of Chernoff bounds (Theorem \ref{chernoff1}) that this result implies that all Chernoff bounds developed for sampling with replacement also hold for sampling without replacement.

%Concentration inequalities that specifically describe sampling without replacement can give additional tightening over those that assume sampling with replacement. This refinement was first demonstrated in \cite{serfling1974} with a martingale argument.
%More recently, this result was improved with a reverse martingale argument and used to extend M\&P's EBB to the case of sampling without replacement \cite{bardenet2015}.
%These techniques are extended in section \ref{sec:without_replacement} we specifically give a tightening result specific to sampling without replacement, but it is sufficient to note that all bounds developed here for sampling with replacement are suitable for sampling without replacement by this Lemma.

\subsection{Other general probability lemmas}

There are further lemmas that are necessary for the further derivations for this chapter.
The first lemma is an often-used and rather weak result used to fuse simple statements of probability (provided with proof for completeness of information):
\begin{lemma}[Probability Union]\label{prob_union}
For any random variables $a,b$ and $c$:
\[\p(a>c) \; \le \; \p(a>b) + \p(b>c)\]
\end{lemma}
\begin{proof}[Proof of Probability Union - Lemma \ref{prob_union}]
For events $A$ and $B$
$$\p(A\cup B) \le \p(A)+\p(B)$$
hence for events $a>b$ and $b>c$: $$\p((a>b) \cup (b>c)) \le \p(a>b) + \p(b>c)$$
If $a>c$, then $(a>b) \cup (b>c)$ is true irrespective of $b$, so:
$$\p(a>c) \le \p((a>b) \cup (b>c))$$
\end{proof}
This relationship is a well known and useful tool for settings where the probability relationship between $a$ and $c$ is unknown but the relationship between $a$ and some $b$, and also between that $b$ and $c$ is known.
Although this relationship is quite useful, it is known to be a very weak relationship, as it holds with equality (ie. $\p(a>c)=\p(a>b)+\p(b>c)$) only if $\p((a>b)\cap(b>c))=0$.

Note also, that the same proof method for probability union also works straightforwardly for various substitutions of the $\ge$ symbol for the $>$ in the inner inequalities, with the exception that: $\p(a\ge c) \; \le \; \p(a>b) + \p(b>c)$ may be false.\footnote{ie. In the context of lemma \ref{prob_union} the following inequalities can be proven by much the same logic $\p(a>c)\le\p(a\ge c)\le\p(a\ge b)+\p(b>c)$ and $\p(a>c)\le\p(a\ge c)\le\p(a\ge b)+\p(b\ge c)$, however care must be taken because $\p(a\ge c)\le\p(a>b)+\p(b>c)$ may not be true}

A second Lemma is a straightforward and commonly known result of algebra that relates the sample squares about the mean and the mean squared, to the sample variance (provided with proof for completeness of information).
\begin{lemma}[Variance Decomposition]\label{variance1}
For $n$ samples $x_i$, sample mean $\hat{\mu} = \frac{1}{n}\sum_ix_i$, sample variance $\hat{\sigma}^2=\frac{1}{n-1}\sum_i(x_i-\hat{\mu})^2$, and average of sample squares $\hat{\sigma}_0^2 = \frac{1}{n}\sum_ix_i^2$, the following relationship holds:
% \begin{align*}
% \quad\hat{\sigma}^2
% &=\frac{1}{n-1}\sum_i\left(x_i-\frac{1}{n}\sum_jx_j \right)^2
%  =\frac{1}{n-1}\left(\sum_ix_i^2-\frac{1}{n}\sum_{i,j}x_ix_j \right) % \\
% &= \frac{n}{n-1}\left(\hat{\sigma}_0^2-\hat{\mu}^2\right)
% \end{align*}
% which rearranging gives:
\[
\hat{\sigma}_0^2=\hat{\mu}^2+\frac{n-1}{n}\hat{\sigma}^2
\]
\end{lemma}
\begin{proof}[Proof of Variance Decomposition - Lemma \ref{variance1}]
By expanding $\hat{\sigma}^2$ into parts:\\
$$\quad\hat{\sigma}^2=\frac{1}{n-1}\sum_i\left(x_i-\frac{1}{n}\sum_jx_j \right)^2
=\frac{1}{n-1}\left(\sum_ix_i^2-\frac{1}{n}\sum_{i,j}x_ix_j \right) = \frac{n}{n-1}\left(\hat{\sigma}_0^2-\hat{\mu}^2\right)$$
\end{proof}

This lemma will prove to be important to us, as we will use it as a means of constructing probability bounds on the error of the sample variance $\hat{\sigma}^2$ in terms of the error of the mean squared $\hat{\mu}^2$ and error of the average sample squares $\hat{\sigma}^2_0$.
Once constructed, integrating these probability bounds then allows us to eliminate the variance $\sigma^2$ from our equations, which is an essential part of deriving our new EBBs (such as per equations \ref{eq_no44} and \ref{eq2}).

%And we will eventually use this result to create bounds for the sample variance from bounds on the sample squares about the mean and the mean squared.


\documentclass[]{article}
\usepackage[margin=1.0in]{geometry}
\usepackage{amssymb}
\tiny

%title material
\title{Astronomy 400B Homework 1 Solutions}
\author{\vspace{-10ex}}

\date{\vspace{-10ex}}

%include latex definitions
\input{astro400B_definitions.tex}

%begin the document
\begin{document}

%make the title, goes after document begins
\maketitle

%first section
\section{Problem 1: Sparke \& Gallagher 1.3}

For Betelguese, $T\approx3500\K$, $\theta=0.045~\arcsec$ in diameter, and $d=140\pc$.

\begin{eqnarray}
R &=& \theta \times d\\
R &=& (0.045/2)~\arcsec \times (4.848137 \times 10^{-6} \rad / \arcsec) \times 140 \pc \nonumber \\ &\times& (3.0856\times10^{18}\cm/\pc) / (6.96\times10^{10} \cm/\Rsun) \\
R &=& 677\Rsun \approx 700\Rsun
\end{eqnarray}
\noindent
The luminosity is then
\begin{eqnarray}
L &=& 4\pi R^2 \sigma T^{4}\\
L &=& 4\pi (677\Rsun\times 6.96\times 10^{10}\cm/\Rsun)^{2} \times (5.67 \times10^{-5}~\erg~\s^{-1}~\cm^{-2}\K^{-4})\times(3500\K)^{4}\\
L &=& 2.37\times10^{38}~\erg~\s^{-1} / (3.839\times10^{33}~\erg~\s^{-1} / \Lsun) = 0.62\times10^{5}\Lsun\approx10^{5}\Lsun
\end{eqnarray}

\section{Problem 2: Sparke \& Gallagher 1.10}

Equation 1.10 from the book is
\begin{equation}
m_1 - m_2 = -2.5 \log_{10} (F_1/F_2)
\end{equation}
\noindent
If a source is dimmed by $\exp(-\tau_{\lambda})$, then
the attenuated magnitude $m_1$ relative to the unattenuated
magnitude $m_2$ can be found as
\begin{eqnarray}
m_1 - m_2 &=& -2.5 \log_{10} \exp(-\tau_{\lambda}) = 2.5 \tau_{\lambda} \log_{10} e = 1.086\tau_{\lambda}\\
m_1 &=& m_2 + 1.086\tau_{\lambda}\\
\therefore m_1 &=& m_2 + A_{\lambda};~~A_{\lambda} = 1.086\tau_{\lambda},
\end{eqnarray}
\noindent
where $A_{\lambda}$ is called the {\it extinction}.

\section{Problem 3: Sparke \& Gallagher 1.12}

The Milky Way luminosity is $L\approx2\times10^{10}\Lsun$,
and pretend it is a sphere with radius $R\approx5\kpc$.

Equation 1.3 is
\begin{equation}
L=4\pi R^2 \sigma T^4.
\end{equation}
\noindent
We need to find $T$, so
\begin{eqnarray}
T &=& \left( \frac{L}{4\pi R^2 \sigma}\right)^{1/4}\\
T &=& \left( \frac{2\times10^{10}\Lsun \times 3.839\times10^{33}~\erg~\s^{-1}/\Lsun}{4\pi\times(5\kpc \times 3.0857\times10^{21}\cm/\kpc)^2 \times (5.67\times10^{-5}\erg~\s^{-1}~\cm^{-2}~\K^{-4})}\right)^{1/4}\\
T &=& (452.7\K^{4})^{1/4} = 4.6\K \approx 5\K
\end{eqnarray}

\section{Problem 4: Sparke \& Gallagher 1.16}

Equation 1.25 from the book is
\begin{equation}
\rho_{L}(B_J) = \int_{0}^{\infty}\Phi(L)dL = n_{\star}L_{\star}\Gamma(\alpha+2) \approx 2 \times 10^{8}h\Lsun~\Mpc^{-3}.
\end{equation}
\noindent
Let's be more precise, and use $L_\star = 9\times10^{9}h^{-2}\Lsun$, $n_{\star} = 0.02 h^{3}\Mpc^{-3}$, and $\alpha=-0.46$ such that $\Gamma(2-0.46) = 0.888178$.  We then have
\begin{equation}
\rho_L (B_J) = n_{\star}L_\star\Gamma(2+\alpha) = (9\times10^{9}h^{-2}\Lsun) \times (0.02 h^{3}\Mpc^{-3}) \times (0.888178) = 1.599\times10^{8}h\Lsun~\Mpc^{-3}
\end{equation}
\noindent
For the universe to be at the critical density, we would need
the average mass to light ratio to be
\begin{equation}
M/L = \rhoc/\rho_L = \frac{2.8\times10^{11}h^{2}\Msun\Mpc^{-3}}{1.599\times10^{8}h\Lsun~\Mpc^{-3}} = 1751 h\Msun/\Lsun \approx 1700 h\Msun/\Lsun.
\end{equation}

\end{document}


\documentclass{exam}
%\printanswers
\begin{document}

\section{Decisions and MDPs}

\begin{questions}

\question Suppose you are a turtle guru, and you have found your way into a turtle-themed spinoff of "Who Wants To Be A Millionaire". You have gotten to the 50,000 turtle question, and would win 15,500 turtles if you decide to quit before seeing the 50,000 turtle question. You have a 14 percent chance of getting the next question right. What is the expected payoff of going for the next question? What is the optimal decision?
\begin{solution}
\end{solution}

\question Is a sequence of actions sufficient to get an agent in a nondeterministic environment to reach some goal?
\begin{solution}
\end{solution}

\question What are the components of an MDP?
\begin{solution}
\end{solution}

\question In class, we talked about an example environment where taking the "North" action was nondeterministic. Is it possible for all actions in an environment to be nondeterministic?
\begin{solution}
\end{solution}

\question \textbf{(No Answer Necessary)} Review any gopollock questions you got wrong at ter.ps/cmsc389fgp3
\begin{solution}
\end{solution}

\end{questions}

\section{Exploring OpenAI and GridWorld}

Note: This week, we will begin a very cursory exploration of OpenAI Gym and FrozenLake (a version of the GridWorld environment that we discussed in class). We will solve FrozenLake in later lectures and assignments using a variety of RL algorithms that we will learn. \\

\begin{questions}

\question
\textbf{OpenAI's FrozenLake}
\begin{itemize}
\item Open the view-only Python 3 Notebook at ter.ps/cmsc389fhw3
\item Click on "File", "Save a Copy in Drive"
\item Go to your copy of the notebook and complete the exercises
\item Share your finished notebook with cmsc389f@gmail.com, "HW2 - First Name, Last Name"
\end{itemize}

\end{questions}

% \ifprintanswers
% You're currently in LaTeX solution mode!
% \else
% NOTE: If you'd like to answer in LaTeX, uncomment \textbackslash printanswers at the top!
% \fi

\end{document}


\documentclass[11pt]{article}

\usepackage[margin=1in]{geometry}

\usepackage{enumitem}
\usepackage{amsmath}

\newcommand{\Delay}{\texttt{DELAY}}

\newcommand{\Force}{\texttt{FORCE}}

\usepackage{listings}

\lstloadaspects{formats}

\begin{document}

\begin{center}

  \textbf{\Large Compilation and Language Design in Julia}\\\vspace{3pt}
  \textbf{Multiple Dispatch JIT/AOT and More}

  {\large CIS561 Graduate Report}\\\vspace{3pt}

Alexandre Chen

\end{center}

\section{Introduction}\label{sec:Introduction}
Julia is a newly developed programming language that specializes in Computational Science and Numerical Algorithms. It was designed to solve the famous two-language problem: Developers or scientists would use high-level easy-to-use languages such as MATLAB/Python to test the prototype of an algorithm and then re-implement it via languages that have better performance such as C/C++/FORTRAN for actual deployments. The two-language problem significantly lengthens the development cycle. Also due to the inconsistency usage of prototyping language and implementing language, other issues would caused which further increases the maintenance costs.

Ideally, researchers who develop numerical algorithms to solve problems arising from science and engineering would prefer a language that is easy to use as MATLAB, open as Python, and fast as C/C++/FORTRAN. And Julia is designed to satisfy these requirements. Julia adopts Just-in-time (JIT) compilation, although not exactly the same with JIT compiler in Java, to make it faster than interpreted languages such as Python. Julia is also compatible with LLVM for code generation and optimization. Aside from being fast and easy to use, Julia also supports functional programming paradigm,the heritage of LISP which turns out to be very handy in computational science where mathematical function evaluations are heavily used.

\section{Multiple Dispatch}
As a dynamic programming language, Julia uses multiple dispatch, allowing extensions of functionality by overloading. Each function can consist of a large number of methods. (E.g. "+" method in Julia has 180 methods) The types of variables that each method can handle are declared. Julia will dispatch to the most specific method by for a given function call. Julia doesn't use class and class functions as other programming language such as Java or C++ where dynamic dispatch is adopted. The Struct in Julia is similar to the struct used in C.

\begin{lstlisting}
Struct Dual{T} // T is type of the variable
    s1::T
    s2::T
end

function Base.:(+)(a::Dual{T},b::Dual{T}) where T
    Dual{T}(a.s1 + b.s1, a.s2 + b.s2)
end

function Base.:(*)(a::Dual{T},b::Dual{T}) where T
    Dual{T}(a.s1 * b.s1, a.s2 * b.s2)
end
\end{lstlisting}{}

Suppose we then have function f(a,b) = a*b + a. For given variable a = 1, b = 2, when we call f(a,b). It will load operators "+" and "*" for Int type, with will return output Int(3). However, we can also have input variables given in the form of our self-defined Dual Struct. c = Dual(1.0, 1.0), d = Dual(2.0,2.0). If we call function f(a,b), by overloading arithmetic operators (+) and (*) that we defined for Dual Struct, it will return output Dual(3.0, 3.0).

Dispatching on a function for a call with argument type T in our example consists in picking a method \textit{m} from all the available methods of function f. The selection filters out methods whose type are not a super type of T, and it takes the method whose Type T' is the most specific of the left ones. Every position in the tuples T and T' have the same role, there is no single receiver position that takes precedence over another. This is different from single dispatch that is adopted in C++ and Java.

\section{JIT in Julia}\label{sec:JIT}
The internal levels of code generation follows the processes below. From high-level structure going all the way down.
\begin{itemize}
    \item Method Definition $\rightarrow$ methods, metaprogramming
    \item {code\_lowered $\rightarrow$ generated functions, simplified code structure}
    \item code\_typed \begin{itemize}
        \item {code\_typed} $\rightarrow$ precompiled modules (.jl) global inference
        \item local optimization
        \item code\_warntype $\rightarrow$ dynamic behavior annotations
    \end{itemize}
    \item code\_llvm $\rightarrow$ external codegen, llvmcall-2.0 julep
    \begin{itemize}
        \item Intermediate representation for low-level optimization
    \end{itemize}
    \item {code\_native} $\rightarrow$ static system image (.so/.dll/.dylib) Machine Code Representation
\end{itemize}

Just-in-time compilation (or dynamic translation or run-time compilation) is a way of executing computer code that involves compilation during the execution of a program at run-time rather than prior to execution, which is in contrast of ahead-of-time (AOT) compilation. \cite{Bezanson:2018:JDP:3288538.3276490} It is a combination of AOT and interpretation. Roughly the JIT compilation combines the speed of compiled code while being flexible like interpretation. In theory, since JIT is a dynamic compilation and allows adaptive optimizations including dynamic recompilation and micro-architecture-specific speedups, it can be faster than static compilation. The earliest version of JIT can be attributed to LISP by John McCarthy in 1960. JIT compilers are mostly known for Java and Lua.

Julia's LLVM-based just-in-time (JIT) compiler, combined with the its own language design allows it to have similar level of performance of C. However, the JIT used in Julia is not exactly the same as the JIT in Lua or Java. For example, LuaJIT is a Just-In-Time compiler for Lua programming language, but it offers APIs so i can be used as a general-purpose standalone language for different platforms. There is existing approach by applying LuaJIT to interpreted languages such as Python for better performance.

However, Julia used a different approach towards JIT implementation. Other JIT setups for scripting languages use something known as tracing JIT. It adopts probabilistic method to determine the frequencies of the parts of the code that are repetitively run and compiled these code. This tracing JIT approach is different from traditional JIT that works on a per-method basis. Julia, by comparison, is almost static. When running any Julia function, it will first take function calls and then auto-specialize it down to concrete data types. So f(x::Number) = x \^{} 3 will be specialized to Float64 when it is called as f(1.0). This type information propagates along type-inference as far as it can go. If you do a x\^{}3 internally, it will replace that expression with \^{}(::Float64,3) (Here 3 is the constant that we used), and then since it can determine all the variable types it will use, it will push this all the way down to the generate LLVM IR code. And from this level, machine code can be generated. Notice that we only generated the code for function f(x::Number) with Floating point input variables. If we are going to recall the function f(x::Number) with input value 2.0, We can use the same compiled code. But If we want to use input value 1 which is an Integer, Julia will compile a separate version for Int.

At first sight, this might seem highly inefficient because essentially you are going to compile different versions of the same function for different input variable types separately. But the advantage of this process is that it's clear and deterministic. Also since Julia is widely used in Computational Science, in many cases, functions are handling numerical data as input. Usually based on different purposes, from accuracy requirements to stability requirements the precision levels for certain types of variables are fixed before the program is written and compiled. Specifying the types of compiled functions doesn't suffer from efficiency too much as it might seem.

Adopting deterministic rules would also help developers know what's going behind the code generation and how to optimize for efficiency, this transparency is lacked in MATLAB, which is a traditional commercial software widely used in numerical algorithms and engineering.

However, there is certain drawback in this approach. In order this process can work well,  types of variables inside a function should be "type-stable". Consider the following example:

\begin{lstlisting}
function g(v,n)
    x = 0
    for i in 1:n
        x += v
    end
    return x
end
\end{lstlisting}

If we call this function with input variable v=1.0, n=5. It is weird that this function is not as fast as we would expect. The reason is because the type of x starts as Int. But in step x += v, Int + Float64 produces a Float64 type so the type of x gets changed here. In static programming language like C, you can't change the type of variables once declared. Julia is a dynamic typed language that can compile x in a way that is dynamically typed. The accompanying cost of the increase in agility is the decrease in efficiency, and this is why this code is slower than expected.

Even though Julia is a dynamic language, programmers are often advised to avoid data type conversions for the sake of efficiency. One way to achieve this is that Julia supports method overloading. We can define g(v,n) function for different input variables. In different versions of function g(v,n), we use instance variables matching the types of input variables. So Julia would know exactly which g(v,n) we would need to compile when given input v=1.0, n=5. However, doing this will certainly make duplication repeatedly and is not the most efficiently way. this is a task This can be avoided by using the same type with input variable v for instance variable x. Julia provides built-in functions such as zero(), ones(), similar() that take input variables and return variables of the same type used as instance variables very often in initialization processes. For example, we can replace our previous function with a generic type-stable one

\begin{lstlisting}
function g(v,n)
    x = zero(v)
    for i in 1:n
        x += v
    end
    return x
end
\end{lstlisting}

Compared with tracing JIT compilation, this semi-static approach for JIT implementation doesn't not require statistical data from run-time information, and is easier to introspect and diagnose the code generation process, especially when things are going wrong. This feature is extremely important for many computational scientists who don't have compiler knowledge but need to develop a package consists of tens of separate code files and make it compatible with other implemented packages. There are other packages in Python adopts similar approach with Julia, such as Numba. The difference is that this feature is a language-level feature of Julia, users have better direct handling of data types than Python + Numba, which allows Julia to scale well.

\section{LLVM in Julia}
LLVM is a compiler framework created by Chris Lattner and Vikram Adve at University of Illinois at Urbana-Champaign for the support of \textit{transparent, lifelong program analysis and transformation} for arbitrary programs. \cite{Lattner:2004:LCF:977395.977673}
LLVM defines a common, low-level code representation in Static Single Assignment (SSA) form. It has several novel features which includes a simple, language-independent type system, an instruction for typed address arithmetic, and a simple exception handling mechanism for high-level languages. LLVM currently supports many languages. The Clang is a part of LLVM framework that uses LLVM as its back end. It has become a popular alternative to GCC for better error message handling and is much faster and uses far less memory. Julia dynamically links to LLVM by default. The LLVM C Backend also allows translating Julia Code to C code so the algorithm developed in Julia can also be easily redeployed on servers that don't support C. Also there is high-level LLVM package in Julia that allows users to directly access LLVM IR code for profiling and diagnosis.

\section{Metaprogramming in Julia}
Julia provides various features for defining functions at compile-time and run-time. Most significantly, it provides Macros to generate code and reduce the need to call built-in evaluation function eval(). Macro maps a tuple of arguments to an expression that has been already compiled. By using macros, many duplicate or similar functions can be avoided. Many built-in Julia functions for performance analysis, code profiling have been implemented in macros, which simplifies the process for code bench marking and optimization. Metaprogramming also enables efficient use of already implemented functions without writing APIs to work with other functions. This has improved the potential collaborations among Julia users significantly, especially those from other engineering and science fields who lack background knowledge of programming languages and software methods.

\section{Conclusions}\label{sec:Conclusions}

Julia is a dynamic programming language specifically designed for numerical algorithms that has some similar properties with static ones to avoid run-time uncertainty for efficiency. It was built on top of modern compiler framework and absorbs many language designs from other programming language such as JIT compilation, Metaprogramming etc for the purpose of being fast and easy-to-use in computational science. It is an evolving languages with many new features and has a booming Julia ecosystem. Many new features including garbage collection are gradually added to Julia and getting improved. The metaprogramming property allows the low-level Julia source code to be written in native Julia, which is easy for Julia users to read and understand low-level functions that are used in their code compilation. It's also to easy to make different Julia packages compatible with others. Actually many Julia native functions are coming from merging user-generated packages. Although with certain limitations coming from the language design, Julia is becoming a popular and promising programming language in computational science.

\bibliographystyle{abbrv}

\bibliography{references.bib}

\end{document}


\documentclass[main.tex]{subfiles}
\begin{document}

\section*{Mon Jan 13 2020}

The distribution for a thermal source is more irregular than the Poissonian we get for the coherent light.

Recall: our coherent states are given by
%
\begin{align}
\ket{\alpha } = \exp(- \frac{1}{2} \abs{\alpha}^2) \sum _{n} \frac{\alpha^{n}}{\sqrt{n!}} \ket{n}
\,
\end{align}
%
where \(\ket{n}\) are the number eigenstates.

We can define the displacement operator:
%
\begin{align}
\hat{D} (\alpha ) = \exp( \alpha \hat{a}^\dag - \alpha^{*} \hat{a})
\,,
\end{align}
%
which, it can be proven, can give us the state \(\ket{\alpha }\) starting from the vacuum \(\ket{0}\): \(\hat{D} (\alpha ) \ket{0} = \ket{\alpha }\).

We discuss interactions with an electromagnetic field: we have the interaction term in the Hamiltonian:
%
\begin{align}
\hat{V}(t) = \int \dd[3]{r} \vec{j} (\vec{r},t) \cdot \hat{A} (\vec{r}, t)
\,,
\end{align}
%
where \(\hat{A}\) is the operator corresponding to the vector potential, and is given by
%
\begin{align}
\hat{A} =
\,.
\end{align}

In certain cases, we can only consider the dipole contribution since on the length scales of the problem the field is approximately constant.

\dots

We can find an expression for \(\alpha \) by integrating a coupling term.
The coherent states are not orthonormal: we have
%
\begin{align}
\braket{\beta }{\alpha } = \exp( \frac{1}{2} \qty(\beta ^{*} \alpha - \beta \alpha^{*} -\abs{\beta -\alpha }^2 ))
\,,
\end{align}
%
which can never be zero: its square norm is \(\exp(-\abs{\beta- \alpha }^2)\) there are no orthogonal vectors here. Nevertheless, we are in a Hilbert space so we can write a completeness relation, even though we do \emph{not} have a basis.

We have a distribution in the space of \(\alpha \): it is constrained by the uncertainty principle. We can use \emph{homodyne} detectors to select a quadrature to ``squeeze'' and get more resolution on. \emph{Heterodyne} means we split the signal and use both.

We can use our QFT of light to solve the problem of the interaction of an EM field with an atom: the transformed Hamiltonian is
%
\begin{align}
H' = \frac{1}{2m} \qty(\vec{P} + e \vec{A})^2 + V - e \Phi
\,,
\end{align}
%
which is an operator equation, even though I omit the hats.

If we consider both the Hilbert spaces, we will have a transition from an initial state \(\ket{a} \ket{n}\)  to either \(\ket{b} \ket{n+1}\) if a photon is emitted or to \(\ket{b} \ket{n-1}\) if a photon is absorbed.

We compute the probability amplitudes of these by sandwitching the interaction Hamiltonian.

The zero-point energy cannot be harvested for a transition: however, spontaneous emission can happen by interaction with the vacuum of the EM field.

The interaction Hamiltonian is separable: its EM part is either \(\hat{a} \) or \(\hat{a}^\dag\), so we immediately get the result that
%
\begin{align}
\frac{\abs{\expval{\text{emission}}}^2}{\abs{\expval{\text{absorption}}}^2} = \frac{n+1}{n}
\,,
\end{align}
%
so emission is slightly more probable.

In the interaction Hamiltonian we need to consider the actual shapes of the orbitals of the atoms: it is a difficult search.

\subsection{A crash course in LASER}

For this part, see the Saleh-Teich.

Let us say we have two states with energies \(E_{1, 2}\) with \(E_2 - E_1 = \hbar \omega \).
We can either have emission, absorption or stimulated emission, as we were discussing.

The dipole term is approximately constant.
The probability density of thhe deexcitation is given by
%
\begin{align}
w_{i} = \frac{\overline{n}  }{t _{\text{sp}}}
\,,
\end{align}
%
where \(t _{\text{sp}}\) is such that \(\mathbb{P} = 1 / t _{\text{sp}}\).
\todo[inline]{What? what are the units here?}

We'd like to have amplification of the emission between the states 2 and 1: however the emission is always more likely than the absorption, asymptotically they have the same probability.

The way to solve this is introducing more states: the easy way to do it is to introduce two of them, call them 0 and 3, one below and one above our 1 and 2. These are still excitation states of a certain atom.

The \emph{pumping} is the temporal and spatial density of the \(0 \to 3\) transitions, then the state descends through 2, 1, and finally 0.

We need to choose the atom appropriately, with a good probability of the atom absorbing the pumping, and a high probability of doing \(3 \to 2\). The \(2 \to 1\) transition must have a reasonably low probability.
Neodymium is a good candidate. For it, the characteristic time of state \(2\) is of the order of the hundreds of \SI{}{\micro \second}.

Recall the law of Boltzmann statistics:
%
\begin{align}
\frac{N_2 }{N_1 } = \exp(-\frac{E_2 - E_1 }{k_B T})
\,.
\end{align}

The construction we made creates an \emph{inversion} of this population: the population of state \(2\) becomes larger than that of state \(1\).

We are interested in \(N = N_2 - N_1 \): in the thermodynamical equilibrium case this is almost \(-N_1 \) since \(N_2 \) is negligible.
In our case, instead, it becomes \(N = W t _{\text{sp}}\).

It is useful to introduce the concept of \emph{optical gain}:
let's say we have a cavity of length \(d\), then the frequency corresponding to the lowest mode is \(\nu_{f} = c / 2d \) and we can consider modes like \(\nu = q \nu_{f}\).

The number flux is
%
\begin{align}
\Phi = \frac{I}{h \nu }
\,,
\end{align}
%
where \(I\) is the light intensity;
 we are interested in \(\dv*{\Phi }{z}\), where \(z \) is the coordinate along our cavity.
 It will look like:
 %
 \begin{align}
 \dv{\Phi }{z} = \gamma(\nu ) \Phi(z)
 \,,
 \end{align}
 %
where \(\gamma = N \sigma (\nu )\), \(N\) being the one from above, \(N_2 - N_1 \). If this is positive, then we have optical gain.

We want our laser to create a \emph{ray} of light: this is not trivial, the simplest thing is to create a cavity where one of the mirrors actually has a non-1 reflectivity, so that a certain portion of light escapes.

This can be summarized with the average permanence time of a photon in the cavity:
if the intensity in terms of the position looks like \(I(z)= I_0 \exp(-\alpha_{r} z)\) for some coefficient \(\alpha_{r}\), then we can also write \(\alpha_{r} = 1 / c \tau \), where we introduced the characteristic time \(\tau \).

The rule will then be \(\gamma > \alpha_{r}\): if this is the case, then the radiation is amplified more than it gets out.
If this is not the case, we basically have a thermal source.

\end{document}


\subsection{Comparison of results}

	Obviously the time of Local search is not comparable to the Exact Method and since I set a fixed time for Tabu Seach is useless to compare the CPU time between the algorithms used.

	Check the differences between the results is much more interesting, Figures \ref{fig:comparison} and \ref{fig:fb80-comparison}. We can say that:
	\begin{itemize}
		\item Local Search is the fastest methods.
		\item Tabu Search in almost all cases found the optimal solution or a solution near to optimum;
		\item In general the Tabu Search is better than the Local Search but we need to consider that it requires much more time, 30 seconds versus some milliseconds;
		\item Local Search performs worse for instances with a lot of nodes.
	\end{itemize}

	\vspace{2cm}

	\begin{figure}[bh]
		\centering
		\includegraphics[width=\linewidth]{img/fb80-comparison}
		\caption{Comparison in the fb80 instance. The worse instance for the meta heuristic methods.}
		\label{fig:fb80-comparison}
	\end{figure}

	\begin{figure}[h]
		\centering
		\includegraphics[width=\linewidth]{img/rnd-comparison}

		\vspace{1cm}

		\includegraphics[width=\linewidth]{img/fb-comparison}
		\caption{Comparison of the different methods for rnd and fb instances. \textbf{Note:} in order to mark the difference y-axes doesn't start from 0.}
		\label{fig:comparison}
	\end{figure}


\documentclass[a4paper]{article}

\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage{float}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage[colorinlistoftodos]{todonotes}

\title{\bfseries{COMP90016 -- Assignment 1 }}
\author{Haonan Li}

\date{\today}

\begin{document}
\maketitle

\section{Introduction}
\label{sec:introduction}

This assignment consists of three tasks.

For the first task, we first build a k-mer index for a reference file, then aligned all reads from a FASTQ reads file using the index.

The second task is to investigate the runtime of the aligner build in task 1. In this task, we compare the speed of the new aligner with the naive aligner proposed in group assignment. In addition, we also evaluate the relation between size of k-mer and the alignment speed.

For the last task, we extend our algorithm to further investigate mismatches to the reference.

\section{Results \& Discussion}
\label{sec:experiment}
\subsection{Task 2}

\begin{table}[H]
	\centering
	\begin{tabular}{c|c|c|c|c|c|c|c}
		\hline
		k & 0 & 4 & 8 & 13 & 16 & 32 & 50 \\
		\hline
		ref1 & 24.43 & 18.98 & 1.76 & 1.48 & 1.40 & 1.04 & 0.51 \\
		\hline
		ref2 & 50.87 & 35.34 & 1.83 & 1.49 & 1.49 & 1.14 & 0.66 \\
		\hline
		ref3 & 80.54 & 52.65 & 1.91 & 1.56 & 1.47 & 1.22 & 0.78\\
		\hline
	\end{tabular}
	\caption{\label{tab:1}Time consuming for k-mer indexed alignment. (k=0, refers to  the naive aligner.)}
\end{table}

Table \ref{tab:1} shows the time consuming for k-mer aligner work on the given datasets. $K=0$ refers to the naive aligner proposed in group assignment. From these results, we find that new aligner is much more efficiently than the naive alignments.

The result is not surprised. Because the  complexity of naive Hamming distance aligner is $O(ngl)$, where $n$ is the number of reads, $g$ the length of the genome, and $l$ the length of the reads. But for indexed aligner, the complexity is O($nl^2+g$), here are theoretically analysis.

First assuming the length of k-mer is k, where $ k\ll l$, then for each read, we have $l-k$ k-mers and suppose for each k-mer, the number of indexed positions is p. So we need to compre $O((l-k)*l*p)$ times for each read. As $k\ll l$, $O(l-k)=O(l)$. Here, we have a complexity of $O(nl^{2}p)$. We normally assume that we can build a linear spaced index with constant number of positions for each k-mer,\footnote{Actually, if k is large enough, p will equals 1} and length of genome maybe very large, so we need to traverse it to build the index. Finally, the complexity is $O(nl^{2} + g)$.

Figure \ref{aa} shows new aligner's alignment time with the reference length. We change reference length from 0 to 5000 using ref3.fa, with every 15 one point. This figure indicates that time consuming changes linearly with reference length from 0 to 5000,

Figure \ref{bb} shows new aligner's alignment time with the read length. We can identify it is a part of quadratic function.

Figure \ref{00} shows aligner's alignment time with k (the k-mer length), $k=0$ indicates the naive aligner's time. From which we find that when k is big enough. TIme consuming almost does not change with k.

Figure \ref{cc} is the visualization of two aligner's time comparison. We find that new aligner is faster then the naive aligner.
\begin{figure}[h]
	\centering
	\subfigure[Time consuming change with reference length]{
		\label{aa}
		\includegraphics[width=0.45\textwidth]{time_ref_len.png}}
	\subfigure[Time comsuming change with read length]{
		\label{bb}
		\includegraphics[width=0.45\textwidth]{time_read_len.png}}
	\subfigure[Time consuming change with k]{
		\label{00}
		\includegraphics[width=0.45\textwidth]{align_time.png}}
	\subfigure[TIme consuming visulation of fixed k]{
		\label{cc}
		\includegraphics[width=0.45\textwidth]{align_vis.pdf}}
	\caption{Time sonsuming analysis.}
	\label{fig:time}
\end{figure}

\subsection{Task 3}

Figure \ref{a}  shows the distribution of base quality scores of all the reads for each position, and Figure \ref{b} the distributions of quality scores of all mismatches for best alignment to the given reference ref1.fa  (also by position).

Compare these two images, we find that the range of each position's quality score of two pictures are similar, which indicates even a position has a high average quality score or most quality scores of this position is high, mismatches occurred.  In addition, both of them shows that the middle position's quality score commonly higher than the start and end of the reads, this might because of the sequencing method.

However, no matter the upper quartile, lower quartile and median, two pictures show some differences. For most positions, the box of upper quartile to lower quartile of mismatched picture always lower than that for all reads. This might due to more low quality scores appear in the mismatched set.  Algorithm there are exceptions that median for mismatches upper than it for all reads, which might because the dataset is small. Besides, as the position varies form 1 to 100, the average quality score becomes lower, mismatches increase. This was reflected by the median lines that get closer to the bottom and even overlapped with it as the positions near to 100.

\begin{figure}[!htb]
	\centering
	\subfigure[All the reads for each position]{
		\label{a}
		\includegraphics[width=0.47\textwidth]{base_qs.png}}
	\subfigure[All mismatches]{
		\label{b}
		\includegraphics[width=0.47\textwidth]{mis_qs.png}}
	\caption{Distribution of base quality scores.}
	\label{fig:2}
\end{figure}

%\begin{thebibliography}{9}
%\end{thebibliography}
\end{document}


\documentclass[twoside,11pt]{article}

\usepackage{style/jmlr2e2}
\usepackage{subcaption}
\usepackage{graphicx}
\graphicspath{ {./images/} }

\jmlrheading{Modified Template}{2021}{}{N/A}{N/A}{}{}

\begin{document}

\title{Machine Learning Scientific Report}

% \author{\name Anonymous author}

% \editor{}

\maketitle

\section{Problem framing}
The Covid‑19 pandemic is still developing today. Between December 2019 and May 2021, there have been over 160 million cases and over 3 million deaths globally. To better identify trends and assist in medical care, we can turn to machine learning (ML) for support.

In this paper, we build and compare predictive models using ML algorithms and epidemiological data from the Covid-19 outbreak. The dataset we use is by \cite{xu2020Epidemiological} and is publicly available on GitHub.\footnote{github.com/beoutbreakprepared/nCoV2019/} The problem we want to solve is of predicting confirmed cases' outcomes as either \emph{died} or \emph{discharged} (from hospital). The motivation for solving this problem is that the prediction model built may be helpful for quickly allocating medical care to patients at a time when there are limited resources.

\section{Experimental procedure}
\subsection{Data preparation}
First, to better understand the data, we produced a number of visualisations: we created a bar plot of up to the 10 most frequent values of each categorical attribute (e.g., \emph{symptoms}); we created a scatter plot of the \emph{latitude} and \emph{longitude} of each case; and, we plotted the daily and monthly number of cases globally and by country. From these visualisations, we got a strong overview of the spread of Covid-19 by location and by time.

Second, we began preparing the dataset for ML algorithms by removing irrelevant instances. These were instances that were not useful for the task. Since we were training models to predict outcome, irrelevant instances were those whose value of the target, \emph{outcome}, was not one of either \emph{died} or \emph{discharged}. To remove such instances, we had to correct structural errors, such as inconsistent capitalisation and wording among target values: we replaced \emph{Died}, \emph{dead}, or similar values with \emph{died}; and, we replaced \emph{Discharged}, \emph{recovered}, or similar values with \emph{discharged}. Instances with target values whose meaning was dissimilar to either of these two values, including \emph{null} values, were dropped.

Third, we kept attributes which we thought would be useful. The attributes kept, which became features, were \emph{age}, \emph{sex}, \emph{latitude}, \emph{longitude}, \emph{date\_onset\_symptoms}, \emph{symptoms}, \emph{chronic\_disease\_binary}, and \emph{travel\_history\_binary}. After this, we replaced the \emph{symptoms} and \emph{date\_onset\_symptoms} features with a new \emph{symptoms\_binary} feature, and cleaned up the dataset by dropping any remaining instances with missing values.

Fourth, we transformed numerical data. We performed data binning on the \emph{age} feature, and created categorical values for the age ranges \emph{0-14}, \emph{15-34}, \emph{35-59}, \emph{60-79}, and \emph{80+}. These were chosen because the \emph{age} feature was in fact already categorical, and the latter four categories were the four most frequent. We also performed feature scaling on the \emph{latitude} and \emph{longitude} features by standardisation, also known as Z-score normalisation, which rescales the feature values to have zero-mean and unit-variance.

Fifth, we transformed categorical data. We performed one-hot encoding on the \emph{age} and \emph{sex} features since they had nominal data. Also, we encode the target values so that \emph{discharged} corresponds to 1 and \emph{died} corresponds to 0. The reason for performing these transformations is that ML algorithms require features to be numbers.

Finally, we randomise the resulting dataset, which has 5,206 instances, as we do not want the order of the instances, which is irrelevant, to affect the model training process; and, we split the data into a training set, validation set, and test set to prevent overfitting. We create a test set by selecting a 20\% subset of the dataset so there are at least 1,000 test instances, which is large enough to yield statistically meaningful results. Since we suspect the \emph{chronic\_disease\_binary} attribute to be an important feature to predict \emph{outcome}, we chose to ensure that the the test set was representative of the overall \emph{chronic\_disease\_binary} distribution by doing stratified sampling. The overall distribution was that 98.2\% of values were \emph{False} while the remaining 1.8\% were \emph{True}. We also observe that we maintain an exact balance of train and test target values (75.3\% being \emph{discharged}), which makes it easier to train models.

\subsection{Model selection, training, and testing}
The three ML algorithms we chose to build predictive models were logistic regression classification (LRC), gradient boosting classification (GBC), and support vector classification (SVC). For each of these three models, we iteratively train the model on the training set, evaluate the model on the validation set, and tweak the model according to the results on the validation set. In particular, we employ 4-fold cross-validation so that the size of the validation set is equal to the size of the test set. This further reduces the chance of overfitting. Then, we pick the model with the set of hyperparameters that does best on the validation set and confirm the results on the test set. This approach allows us to compare different trained models in an unbiased way, by comparing model performance using the test set, which is kept apart from the training process and is therefore unseen data.

\subsection{Hyperparameter tuning}
We tried different hyperparameter combinations for each model. For LRC, we tried values 0.001, 0.01, 0.1, 1, 10, 100, and 1000 for $C$, the regularisation hyperparameter; and, we tried $L_1$ and $L_2$ regularisation for the norm used in the penalisation. For GBC, we tried values 2, 4, 8, and 16 for the minimum number of samples required to be at a leaf node (\emph{min\_samples\_leaf}); and, values 0.001, 0.01, and 0.1 for the learning rate. And, for SVC, we tried values 0.75, 0.85, 0.95, and 1 for $C$; and, linear, polynomial, radial-basis, and sigmoid kernel types.

For LRC, the best model achieved a 0.80 accuracy score with hyperparameter combination $C=1000$ and $L_2$ regularisation. For GBC, the best model achieved a 0.85 accuracy score with hyperparameter combination \emph{min\_samples\_leaf}$=2$ and learning rate 0.01. And, for SVC, the best model achieved a 0.84 accuracy score with hyperparameter combination $C=1$ and radial-basis kernel type.

\section{Results}
Having found the best hyperparameter combinations to train the best models for all three ML algorithms, we then evaluate the trained models on the test set. To enable evaluation, we produce the following: the confusion matrix, positive predictive value (PPV), negative predictive value (NPV), sensitivity, specificity, $F_1$ score, accuracy, and receiver operating characteristic (ROC) curve, for each model.

\begin{table}[!htb]
    \caption{Confusion matrices produced by each trained model on the test set.}
    \begin{subtable}{.33\linewidth}
      \centering
        \caption{LRC.}
        \begin{tabular}{|c|cc|c|}
            \hline
            Pred. & 0 & 1 & all \\
            Real. & & & \\
            \hline
            0 & 185 & 55 & 240\\
            1 & 148 & 654 & 802\\
            \hline
            all & 333 & 709 & 1042\\
            \hline
        \end{tabular}
    \end{subtable}%
    \begin{subtable}{.33\linewidth}
      \centering
        \caption{GBC.}
        \begin{tabular}{|c|cc|c|}
            \hline
            Pred. & 0 & 1 & all \\
            Real. & & & \\
            \hline
            0 & 112 & 128 & 240\\
            1 & 19 & 783 & 802\\
            \hline
            all & 131 & 911 & 1042\\
            \hline
        \end{tabular}
    \end{subtable}%
    \begin{subtable}{.33\linewidth}
        \centering
          \caption{SVC.}
          \begin{tabular}{|c|cc|c|}
            \hline
            Pred. & 0 & 1 & all \\
            Real. & & & \\
            \hline
            0 & 109 & 131 & 240\\
            1 & 24 & 778 & 802\\
            \hline
            all & 133 & 909 & 1042\\
            \hline
        \end{tabular}
      \end{subtable}
\end{table}

\begin{table}[!htb]
    \caption{Statistics produced by each trained model on the test set.}
    \centering
    \begin{tabular}{|c|c|c|c|c|c|c|}
        \hline
        \textbf{Model} & \textbf{NPV} & \textbf{PPV} & \textbf{Specificity} & \textbf{Sensitivity} & \textbf{$F_1$ Score} & \textbf{Accuracy} \\
        \hline
        LRC & 0.56 & $\mathbf{0.92}$ & $\mathbf{0.77}$ & 0.82 & 0.87 & 0.81 \\
        GBC & $\mathbf{0.85}$ & 0.86 & 0.47 & $\mathbf{0.98}$ & $\mathbf{0.91}$ & $\mathbf{0.86}$ \\
        SVC & 0.82 & 0.86 & 0.45 & 0.97 & $\mathbf{0.91}$ & 0.85 \\
        \hline
    \end{tabular}
    \label{tab:stats}
\end{table}

\begin{figure}[h]
    \centering
    \begin{subfigure}[b]{0.3\textwidth}
        \centering
        \includegraphics[width=\textwidth]{lr-roc}
        \caption{LRC.}
        \label{fig:lr-roc}
    \end{subfigure}
    \hfill
    \begin{subfigure}[b]{0.3\textwidth}
        \centering
        \includegraphics[width=\textwidth]{gb-roc}
        \caption{GBC.}
        \label{fig:gb-roc}
    \end{subfigure}
    \hfill
    \begin{subfigure}[b]{0.3\textwidth}
        \centering
        \includegraphics[width=\textwidth]{sv-roc}
        \caption{SVC.}
        \label{fig:sv-roc}
    \end{subfigure}
       \caption{ROC curves produced by each trained model on the test set.}
       \label{fig:roc}
\end{figure}

% \begin{figure}[h]
%     \centering
%     \begin{subfigure}[b]{0.3\textwidth}
%         \centering
%         \includegraphics[width=\textwidth]{lr-precrec}
%         \caption{LRC.}
%         \label{fig:lr-precrec}
%     \end{subfigure}
%     \hfill
%     \begin{subfigure}[b]{0.3\textwidth}
%         \centering
%         \includegraphics[width=\textwidth]{gb-precrec}
%         \caption{GBC.}
%         \label{fig:gb-precrec}
%     \end{subfigure}
%     \hfill
%     \begin{subfigure}[b]{0.3\textwidth}
%         \centering
%         \includegraphics[width=\textwidth]{sv-precrec}
%         \caption{SVC.}
%         \label{fig:sv-precrec}
%     \end{subfigure}
%        \caption{Three simple graphs}
%        \label{fig:precrec}
% \end{figure}

By looking at the area under the curve (AUC), which is a measure of separability, of the ROC curves in Figure \ref{fig:roc}, we see that GBC produces the highest value followed by LRC then SVC. Since a higher AUC value corresponds to more `idealistic' behaviour (i.e., more correct classifications), we determine that GBC has the best performance for solving our classification problem, followed by LRC then SVC. This ordering is also reflected by the statistics in Table \ref{tab:stats}, which show that GBC performs best in 4 metrics (highlighted in bold), followed by LRC in 2 metrics, and SVC in only 1.

\section{Discussions}
\subsection{Chosen models}
The reason we chose to implement LRC and GBC is that they were demonstrated in the practical \emph{Exercise for Logistic Regression}\footnote{tinyurl.com/COMP2261prac6} with breast cancer prediction. As such, we base our code on these demonstrations. As for SVC, we chose this algorithm because it was recommended for our problem after consulting the `ML Cheat Sheet' (for scikit-learn)\footnote{scikit-learn.org/stable/tutorial/machine\_learning\_map/index.html}, since we had less than 100k samples and no text data.

\subsection{Experimental procedure}
Although standardisation was useful for LRC and SVC, it does not affect tree based models such as GBC since they are not affected by linear transformations. Nonetheless, standardisation would have helped the model better deal with outliers. However, because we only performed standardisation, we do not know whether min-max normalisation would have produced better results. In future, we can try fitting the model to raw data, min-max normalised data, and standardised data, and compare the results to find the best.

Another procedure we could have done differently is to split the dataset into train and test sets before performing any data cleaning or transformation. Had we done so, we may have gotten different results.

\subsection{Limitations}
One limitation of the implementation is that the predictive models use only a handful of attributes that were present in the raw dataset to perform classification. We were going to include the \emph{country} feature in model training; however, we found that the values were skewed towards one country (\emph{Phillipines}), which might affect training performance. Similarly, we wanted to include the \emph{date\_confirmation} attribute; however, our initial exploration of the dataset showed that there were many outliers (spikes in the number of cases around the start or end of each month), and so the attribute might not contain much useful information.

Another limitation of the implementation is that we do not explicitly perform down sampling or up-weighted during model training to account for the imbalanced data set, since \emph{discharged} accounts for about 75\% of \emph{outcome} values. Had we done this, we may have observed fewer false positives or false negatives in the confusion matrix.

Finally, we could have included more hyperparameters to try in our combinations, such as the deviance and exponential loss functions to be optimised for GBC and degree of the polynomial kernel function for SVC.

\section{Conclusions and lessons learnt}
Overall, we conclude that GBC is the best ML algorithm at solving our problem of predicting confirmed cases' outcomes as either \emph{died} or \emph{discharged}. Moreover, we learned the lesson that we can never conduct enough experiments in ML, since there are always more algorithms to try, attributes to select, and hyperparameters to tune.

% \vskip 0.2in
\bibliography{references}

\end{document}