\documentclass[
    12pt, % Default font size, values between 10pt-12pt are allowed
    %spanish, % Uncomment for Spanish
]{fphw}

% Template-specific packages
\usepackage[utf8]{inputenc} % Required for inputting international characters
\usepackage[T1]{fontenc} % Output font encoding for international characters
\usepackage{fontspec,unicode-math} % Required for using utf8 characters in math mode
\usepackage{parskip}  % To add extra space between paragraphs
% \usepackage{mathpazo} % Use the Palatino font
\usepackage{graphicx} % Required for including images
\usepackage{booktabs} % Better horizontal rules in tables
\usepackage{hyperref} % For links (both internal and external)
\usepackage{listings} % Required for insertion of code
\usepackage{enumerate}% To modify the enumerate environment
\usepackage{cleveref} % Better \ref command -> \cref
\usepackage{import}   % This 4 packages and the command allow importing pdf
\usepackage{xifthen}  % figures generated with inkscape
\usepackage{pdfpages} % Source: https://castel.dev/post/lecture-notes-2/
\usepackage{mathtools}% Mathematical environments
\usepackage{relsize}  % \smaller and \mathsmaller commands
\usepackage{physics}  % A lot of good commands for mathematics
\usepackage[binary-units]{siunitx}  % To write magnitudes with units
\usepackage{multirow}
% \usepackage{multicolumn}
\usepackage{transparent}
\newcommand{\incfig}[1]{%
    \def\svgwidth{0.95\columnwidth}
    \small
        \import{./images/}{#1.pdf_tex}
}

\setlength{\parindent}{15pt}

%----------------------------------------------------------------------------------------
%    ASSIGNMENT INFORMATION
%----------------------------------------------------------------------------------------

\title{Assignment 1 \\ Optimization of a sequential application} % Assignment title

\author{Emilio Domínguez Sánchez} % Student name

\date{October 18th, 2020} % Due date

\institute{University of Murcia \\ Faculty of Informatics} % Institute or school name

\class{Arquitectura y Organización de Computadores} % Course or class name

\professor{Dr. José Manuel García Carrasco} % Professor or teacher in charge of the assignment

%----------------------------------------------------------------------------------------
%    Definitions
%----------------------------------------------------------------------------------------

\lstset{
    basicstyle=\footnotesize,        % the size of the fonts that are used for the code
    frame=L,
    captionpos=t
}
\newcommand{\tech}{\texttt}
\newcommand{\gcc}{\textit{gcc}}
\newcommand{\clang}{\textit{clang}}

\DeclareMathOperator{\performance}{Performance}
\DeclareMathOperator{\bandwidth}{Bandwidth}
\DeclareSIUnit\flop{FLOP}
\DeclareSIUnit[per-mode=fraction]\fps{\flop\per\s}
\DeclareSIUnit[per-mode=fraction]\gfps{\giga\flop\per\s}
\DeclareSIUnit[per-mode=fraction]\gBps{\giga\byte\per\s}
\DeclareSIUnit\op{operation}
\DeclareSIUnit\ins{instruction}
\DeclareSIUnit\cycle{cycle}
\DeclareSIUnit\core{core}
\DeclareSIUnit\socket{socket}
\DeclareSIUnit\node{node}
\DeclareSIUnit\query{query}
\DeclareSIUnit\channel{channel}
\DeclareSIUnit\controller{controller}

\begin{document}

\maketitle % Output the assignment title, created automatically using the information in the custom commands above

%----------------------------------------------------------------------------------------
%    ASSIGNMENT CONTENT
%----------------------------------------------------------------------------------------

\section*{Problem}

\begin{problem}

    Suppose you had a data center that was equiped with
a \textit{Haswell E5-2699 v3} general purpose processor,
an \textit{NVIDIA K80} GPU and
an TPU from \textit{Google} specifically designed for machine learning loads.
The power performance is described in \cref{tab:power}.
The performance characteristics vary depending on the type of load.
In \cref{tab:performance} you can see the details
for three different workloads $A$, $B$ and $C$.
Where $A$ and $B$ represent workloads typical of the programs that train neural networks
and $C$ represents a traditional workload.

\begin{enumerate}
    \item If the data center spends $70\%$ of its time on the workload $A$
and the rest of its time in the workload B,
what is the \textit{speedup} of using the TPUs over the GPUs? %TODO

    \item Assume that the power consumption for each processor is lineal between
its idle value and its max load value.
Specifically, we have measured that for the loads of the previous question
the general purpose processor is working at $59.4\%$ of its capacity,
the GPUs are doing it at $55.9\%$ and
the TPUs at $88.6\%$.
What is the power consumption for each of the three platforms?

    \item After some time, the data center has the following distribution of load.
$40\%$ of the load is of type $A$,
$10\%$ is of type $B$ and
$50\%$ of type $C$.
What is the \textit{speedup} of using GPUs or TPUs compared to using the platform
with general purpose processors.
\end{enumerate}

\end{problem}

\begin{table}[h]
    \footnotesize
    \centering
    \begin{tabular}{c c c c c}
        \multirow{2}{*}{System} & \multirow{2}{*}{Chip} & \multirow{2}{*}{TDP}
            & Power Consumption & Power Consumption \\
        & & & (stand-by) & (max. load) \\
        \hline \hline \\

        General purpose & \textit{Haswell E5-2699 v3} &
            $\SI{504}{\watt}$ & $\SI{159}{\watt}$ & $\SI{455}{\watt}$ \\
        GP-GPU & \textit{NVIDIA K80} &
            $\SI{1838}{\watt}$ & $\SI{357}{\watt}$ & $\SI{991}{\watt}$ \\
        ASIC & \textit{TPU(v1)} &
            $\SI{861}{\watt}$ & $\SI{290}{\watt}$ & $\SI{384}{\watt}$ \\
    \end{tabular}
    \caption{Power comsumption}
    \label{tab:power}
\end{table}

\begin{table}[h]
    \footnotesize
    \centering
    \begin{tabular}{c c c c c}
        & & \multicolumn{3}{c}{Productivity (Throughput)} \\
        System & Chip & A & B & C \\
        \hline \hline \\

        General purpose & \textit{Haswell E5-2699 v3} &
            $5482$ & $13194$ & $12000$ \\
        GP-GPU & \textit{NVIDIA K80} &
            $13461$ & $36465$ & $15000$ \\
        ASIC & \textit{TPU(v1)} &
            $225000$ & $280000$ & $2000$ \\
    \end{tabular}
    \caption{Performance parameters}
    \label{tab:performance}
\end{table}

%----------------------------------------------------------------------------------------

\newpage

\section*{Answer}

\begin{enumerate}
    \item The statement talks about the time spent in each workload,
    instead of the percentage of the workload that corresponds to a specific type.
    But then it asks about the speedup instead of the throughput increase.
    I will find the answer for the throughput thinking that the time spent is fixed
    and the speedup thinking that the percentages refer to the workload and not the time.

    \begin{enumerate}
        \item Percentages with respect to time.

        We will compute the throughput for each case:

        \begin{align*}
            \operatorname{Throughput}_{\text{GPU}} &=
            0.7 \times 13461 + 0.3 \times 36465 =
            20362.2, \\
            \operatorname{Throughput}_{\text{TPU}} &=
            0.7 \times 225000 + 0.3 \times 280000 =
            241500.
        \end{align*}

        \noindent
        The speedup of using the TPUs would be

        \begin{align*}
            \operatorname{speedup} =
            \frac{241500}{20362.2} \approx
            11.86.
        \end{align*}

        \item Percentages with respect to workload.

        The speedups for the tasks would be $\frac{225000}{13461} \approx 16.71$
        and $\frac{280000}{36465} \approx 7.68$.
        Giving a task speedup of (using \textit{Amdahl's argument})

        \begin{equation*}
            \operatorname{speedup} \approx
            \frac{1}{\frac{0.7}{16.71} + \frac{0.3}{7.68}} \approx
            12.35.
        \end{equation*}

    \end{enumerate}

    \item The expression for the power consumption has to be

    \begin{equation*}
        \operatorname{Power}(\text{load ratio}) =
        \operatorname{Power}_{\text{idle}} + \text{load ratio} \times
            \qty(\operatorname{Power}_{\text{max.}}-\operatorname{Power}_{\text{idle}}). \\
    \end{equation*}

    \noindent
    Hence, the power consumption for each platform can be calulated as

    \begin{align*}
        \operatorname{Power}_{\text{CPU}}(0.594) =
        159 + 0.594 \times (455 - 159) \approx
        334.82, \\
        \operatorname{Power}_{\text{CPU}}(0.559) =
        357 + 0.559 \times (991 - 357) \approx
        711.41, \\
        \operatorname{Power}_{\text{TPU}}(0.886) =
        290 + 0.886 \times (384 - 290) \approx
        373.28.
    \end{align*}

    \item The speedup with respect to using CPUs for each workload would be

    \begin{center}
        \begin{tabular}{cccc}
            System & A & B & C \\
            \hline \hline \\
            GP-GPU &
            $\frac{13461}{5482} \approx 2.46$ &
            $\frac{36465}{13194} \approx 2.76$ &
            $\frac{15000}{12000} = 1.25$ \\
            \\
            ASIC & $\frac{225000}{5482} \approx 41.0$ &
            $\frac{280000}{13194} \approx 21.2$ &
            $\frac{2000}{12000} \approx 0.1.7$
        \end{tabular}
    \end{center}

    Hence, the total speedup would be (using \textit{Amdahl's argument})

    \begin{align*}
        \operatorname{speedup}_{\text{GPU}} &\approx
        \frac{1}{\frac{0.3}{2.46} + \frac{0.1}{2.76} + \frac{0.5}{1.25}} \approx
        1.67, \\
        \operatorname{speedup}_{\text{GPU}} &\approx
        \frac{1}{\frac{0.3}{41.0} + \frac{0.1}{21.2} + \frac{0.5}{1.67}} \approx
        0.34.
    \end{align*}

\end{enumerate}

%----------------------------------------------------------------------------------------

\end{document}


\documentclass{article}

\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amsfonts}
\usepackage{tikz}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}
\usepackage{amssymb}

\usetikzlibrary{automata,positioning}

%
% Basic Document Settings
%

\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\pagestyle{fancy}
\lhead{\hmwkAuthorName}
\chead{\hmwkClass\ (\hmwkClassInstructor\ \hmwkClassTime): \hmwkTitle}
\rhead{\firstxmark}
\lfoot{\lastxmark}
\cfoot{\thepage}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

\setlength\parindent{0pt}

%
% Create Problem Sections
%

\newcommand{\enterProblemHeader}[1]{
    \nobreak\extramarks{}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
}

\newcommand{\exitProblemHeader}[1]{
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \stepcounter{#1}
    \nobreak\extramarks{Problem \arabic{#1}}{}\nobreak{}
}

\setcounter{secnumdepth}{0}
\newcounter{partCounter}
\newcounter{homeworkProblemCounter}
\setcounter{homeworkProblemCounter}{1}
\nobreak\extramarks{Problem \arabic{homeworkProblemCounter}}{}\nobreak{}

%
% Homework Problem Environment
%
% This environment takes an optional argument. When given, it will adjust the
% problem counter. This is useful for when the problems given for your
% assignment aren't sequential. See the last 3 problems of this template for an
% example.
%
\newenvironment{homeworkProblem}[1][-1]{
    \ifnum#1>0
        \setcounter{homeworkProblemCounter}{#1}
    \fi
    \section{Problem \arabic{homeworkProblemCounter}}
    \setcounter{partCounter}{1}
    \enterProblemHeader{homeworkProblemCounter}
}{
    \exitProblemHeader{homeworkProblemCounter}
}

%
% Homework Details
%   - Title
%   - Due date
%   - Class
%   - Section/Time
%   - Instructor
%   - Author
%

\newcommand{\hmwkTitle}{Homework\ \#3}
\newcommand{\hmwkDueDate}{January 31st, 2017}
\newcommand{\hmwkClass}{Elements of Algebra I}
\newcommand{\hmwkClassTime}{Section 161}
\newcommand{\hmwkClassInstructor}{Professor Deepam Patel}
\newcommand{\hmwkAuthorName}{Yao Xiao}

%
% Title Page
%

\title{
    \vspace{2in}
    \textmd{\textbf{\hmwkClass:\ \hmwkTitle}}\\
    \normalsize\vspace{0.1in}\small{Due\ on\ \hmwkDueDate\  }\\
    \vspace{0.1in}\large{\textit{\hmwkClassInstructor\ \hmwkClassTime}}
    \vspace{3in}
}

\author{\textbf{\hmwkAuthorName}}
\date{}

\renewcommand{\part}[1]{\textbf{\large Part \Alph{partCounter}}\stepcounter{partCounter}\\}

%
% Various Helper Commands
%

% Useful for algorithms
\newcommand{\alg}[1]{\textsc{\bfseries \footnotesize #1}}

% For derivatives
\newcommand{\deriv}[1]{\frac{\mathrm{d}}{\mathrm{d}x} (#1)}

% For partial derivatives
\newcommand{\pderiv}[2]{\frac{\partial}{\partial #1} (#2)}

% Integral dx
\newcommand{\dx}{\mathrm{d}x}

% Alias for the Solution section header
\newcommand{\solution}{\textbf{\large Solution}}

% Probability commands: Expectation, Variance, Covariance, Bias
\newcommand{\E}{\mathrm{E}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\Cov}{\mathrm{Cov}}
\newcommand{\Bias}{\mathrm{Bias}}

\begin{document}

\maketitle

\pagebreak

\begin{homeworkProblem}
\subsection{1)}
1. Not commutative, when \(x=1 \ y=0\), \(x*y=1 \ y*x=0\) \\
2. Not associative, when \[
(x+2y+xy)+2z+(x+2y+xy)z=x + 2 y + x y + 2 z + x z + 2 y z + x y z
\]
\[
x+2(y+2z+yz)+x(y+2z+yz)=x+2y+z4+2yz+xy+2xz+xyz
\]
3. There's no identity due to 0*y \(\neq\) x*0 and 2y \(\neq\) y\\
4. There's no inverse due to 3
\subsection{2)}
1. It's commutative, since when \(x>y \ |x-y|=x-y\ |y-x|=-(y-x)=x-y\)
when \(y>x \ |y-x|=y-x \ |x-y|=-(x-y)=y-x\) \\
2. It's not associative, when x=2,y=1,z=1
\[
||2-1|-1|=0
\]
while
\[
|2-|1-1||=2
\]
3. There's no identity since when \(x<0\), the result must be positive so there would be no e satisfied \\
4. There's no inverse due to 3
\subsection{3)}
1. It's commutative since \(max{x,y}=max{y,x}\)\\
2. It's associative since \(max(x, max(y, z))
= max(x, y, z) = max(max(x, y), z)\)\\
3. There's no identity, If \(e \in \mathbb{R}\) were an identity, then we can choose an integer \(x < e\) and for this integer and we could get \(max(x,e)=e\neq\ x\). So there's no identity\\
4. There's no inverse due to 3

\end{homeworkProblem}

\begin{homeworkProblem}
\[
\begin{split}
g \in G\\
e &= g^{-1} * g\\
(g^{-1})^{-1}*e &=(g^{-1})^{-1} * (g^{-1} * g)\\
 &=((g^{-1})^{-1} * g^{-1}) * g)\\
\end{split}
\]
Since \(g^{-1}*g=e\) Let \(g'=g^{-1}\) \(g'^{-1}*g'=e\)
So the formula above becomes
\[
\begin{split}
(g^{-1})^{-1}*e&=(e) * g)\\
(g^{-1})^{-1}&=g
\end{split}
\]
In order to show \((gh)^{-1}=h^{-1}g^{-1}\)
We just need to show the right hand side is the inverse of \(gh\).
\[\begin{split}
	gh(h^{-1}g^{-1})=g(h^{-1}h)g^{-1}\\
	&=geg^{-1}\\
	&=gg^{-1}\\
	&=e
\end{split}\]
So \((h^{-1}g^{-1})\) is the inverse of gh, as a result, \(gh^{-1}=h^{-1}g^{-1}\)
\end{homeworkProblem}

\begin{homeworkProblem}
Using the theory of Problem 2, we could get
\[\begin{split}
g^{-1}h^{-1}&=(hg)^{-1}\\
&=(gh)^{-1}\\
&=h^{-1}g^{-1}
\end{split}\]
So its inverse commutes as well.
\end{homeworkProblem}

\begin{homeworkProblem}
By Problem 2, we have
\[\begin{split}
(ab)^{-1}=b^{-1}a^{-1}=a^{-1}b^{-1}
\end{split}\]
If it commutes\\
So we could get
\[\begin{split}
(ab)^n=((ab)^{-1})^{-n}=(a^{-1}b^{-1})^{-n}=a^nb^n
\end{split}\]

\end{homeworkProblem}

\begin{homeworkProblem}
\begin{enumerate}
\item Closure. By the definition, If \( x,y\in  G\)
\[
\exists x^{-1}, y^{-1} \in S
\]
From problem 2, we know \(y^{-1}*x^{-1}=(xy)^{-1}\). If xy has an inverse, so it must be in G. We proved that \( x*y \in G\)
\item Associative, since G is a subset of S and S , it must hold
\item Identity, since \(e \in S\) , e has an inverse e, \(e \in G\) must hold
\item Inverse, let \( x \in G\), there must be an inverse \( x^{-1} \in S\) since \(x \in S\). We know that \( (x^{-1})^{-1}=x \) and \( x \in G\). There must be an element \( x^{-1} \in G\)

\end{enumerate}
\end{homeworkProblem}

\begin{homeworkProblem}
\includegraphics[scale=1]{table.png}
\end{homeworkProblem}

\begin{homeworkProblem}
We assume set \(\mathbb{Z}_n\), which 2 has an inverse. So \( \exists k,l \in \mathbb{Z}_n \) such that \(2k+1=nl \) so \(2k=nl-1\) , nl-1 is even, nl must be odd, which means n and l must be odd. n is odd.
\end{homeworkProblem}

\begin{homeworkProblem}
(12)(13) doesn't commute. (12)(13)=(132). (13)(12)=(123)
\end{homeworkProblem}

\end{document}


\documentclass[a4paper]{report}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{textcomp}
\usepackage{amsmath, amssymb}
\usepackage{xcolor}
\usepackage{tcolorbox}
\tcbuselibrary{theorems}
% Definitions & Theorems

\newtcbtheorem
  []% init options
  {definition}% name
  {Definition}% title
  {%
    colback=green!5,
    colframe=green!35!black,
    fonttitle=\bfseries,
  }% options
  {def}% prefix

\newtcbtheorem
  []% init options
  {theorem}% name
  {Theorem}% title
  {%
    colback=red!5,
    colframe=red!35!black,
    fonttitle=\bfseries,
  }% options
  {thrm}% prefix

\newtcbtheorem
  []% init options
  {example}% name
  {Example}% title
  {%
    colback=blue!5,
    colframe=blue!35!black,
    fonttitle=\bfseries,
  }% options
  {expl}% prefix
% figure support
\usepackage{import}
\usepackage{xifthen}
\pdfminorversion=7
\usepackage{pdfpages}
\usepackage{transparent}
\newcommand{\incfig}[1]{%
    \def\svgwidth{\columnwidth}
    \import{./figures/}{#1.pdf_tex}
}
\DeclareMathSymbol{\lsim}{\mathord}{symbols}{"18}
\usepackage{hyperref}
\hypersetup{
    colorlinks,
    citecolor=black,
    filecolor=black,
    linkcolor=black,
    urlcolor=black
}

\begin{document}

\chapter{Elementary Number Theory and Methods of Proof}

\section{Direct Proof and Counterexample I: Introduction}

\begin{definition}{Even and Odd}{label}
    An integer $n$ in \textbf{even} if, and only if, $n$ equals twice some integer. An integer $n$ is \textbf{odd} if, and only if,
    $n$ equals twice some integer plus $1$.
\end{definition}

\begin{definition}{Prime and Composite}{label}
    An integer $n$ is \textbf{prime} if, and only if, $n > 1$ and for all positive integers $r$ and $s$, if $n=rs$ then either
    $r$ or $s$ equals $n$. An integer $n$ is \textbf{composite} if, and only if, $n > 1$ and $n = rs$ for some integers
    $r$ and $s$ with $1 < r < n$ and $1 < s < n$.
\end{definition}

\subsection{Proving Existential Statements}

There are two ways to prove an existential statement: find one condition that satisfies the predicate,
or give a set of directions for finding that condition. These methods are called
\textbf{constructive proofs of existence}. A \textbf{nonconstructive proof of existence} shows that
the condition satisfying the predicate is guaranteed from some axiom/theorem, or showing that the lack
of such a condition would lead to a contradiction.

\subsection{Disproving Universal Statements}

\begin{definition}{Disproof by Counterexample}{label}
    To disprove a universal statement of the form $\forall x \in D, P(x) \to Q(x)$, simply find an
    $x$ for which $P(x)$ is true and $Q(x)$ is false.
\end{definition}

\subsection{Proving Universal Statements}

The \textbf{Method of Exhaustion}, although impractical, can work for small domains. For more general
cases, we use

\begin{definition}{Method of Generalizing from the Generic Particular}{label}
    To show that every element of a set satisfies a certain property, show that a particular but
    arbitrary chosen $x$ satisfies the property. When using this method on a universal conditional,
    this is known as the \textbf{method of direct proof}.
\end{definition}

\begin{definition}{Existential Instantiation}{label}
    If the existence of a certain kind of object is assumed or has been deduced then it can be
    given a name, as long as that name is not currently being used to denote something else.
\end{definition}

\subsection{Proof Guidelines}

\begin{enumerate}
    \item Copy the statement of the theorem to be proved on your paper.
    \item Clearly mark the beginning of your proof with the word \textbf{Proof}.
    \item Make your proof self-contained.
    \item Write your proof in complete, grammatically correct sentences.
    \item Keep your reader informed about the status of each statement in your proof.
    \item Give a reason for each assertion in your proof.
    \item Include the "little words and phrases" that make the logic of your arguments clear.
    \item Display equations and inequalities.
    \item Note: be careful with using the word if. Use because instead if the premise is not in doubt.
\end{enumerate}

\subsection{Disproving Existential Statements}

In order to prove that an existential statement is false, you simply have to prove that its negation
is true.

\section{Direct Proof and Counterexample II: Rational Numbers}

\begin{definition}{Rational Number}{label}
    A real number is \textbf{rational} if, and only if, it can be expressed as a quotient of
    two integers with a nonzero denominator. A real number that is not rational is \textbf{irrational}.
\end{definition}

\begin{theorem}{Rational Number Properties}{label}
    \begin{itemize}
        \item Every integer is a rational number.
        \item The sum of any two rational numbers is rational.
    \end{itemize}
\end{theorem}

\begin{definition}{Corollary}{label}
    A statement whose truth can be immediately deduced from a theorem that has already been proven.
\end{definition}

\section{Direct Proof and Counterexample III: Divisibility}

\begin{definition}{Divisibility}{label}
    If $n$ and $d$ are integers and $d \neq 0$ then $n$ is \textbf{divisible by} $d$ if, and only
    if, $n$ equals $d$ times some integer. The notation $d  \mid n$ is read "$d$ divides $n$".
    Symbolically, \[
    d  \mid n \leftrightarrow \exists k \in \mathbb{Z}  \mid n = dk
    .\]
    It then follows that \[
    d \nmid n \leftrightarrow \forall k \in Z | n \neq dk
    .\]
\end{definition}

\subsection{The Unique Factorization of Integers Theorem}

Because of its importance, this theorem is also called the \emph{fundamental theorem of arithmetic}.
It states that any integer greater than 1 either is prime or can be written as a product of
prime numbers in a way that is unique. Formally,

\begin{theorem}{Unique Factorization of Integers}{label}
    Given any integer $n > 1$ there exists a positive integer $k$, distinct prime numbers
    $p_1, p_2, \ldots p_k$, and positive integers $e_1,e_2, \ldots e_k$ such that \[
        n = p_1^{e_1} p_2^{e_2} \ldots p_k^{e_k}
    .\]
    When the values of $p$ are ordered in non decreasing order, the above is known as the
    \textbf{standard factored form} of $n$.
\end{theorem}

\section{Direct Proof and Counterexample IV: Division into Cases and the Quotient-Remainder Theorem}

\begin{theorem}{The Quotient-Remainder Theorem}{label}
    Given any integer $n$ and positive integer $d$, there exist unique integers $q$ and $r$ such that
    \[
    n = dq + r, 0 \le r < d
    .\]
    Note that if $n$ is negative, the remainder is still positive.
\end{theorem}

\subsection{div and mod}

From the quotient remainder theorem, div is the value $q$, and mod is the value $r$. Note that \[
    n \: mod \: d = n - d \cdot (n \: div \: d)
.\]

\subsection{Method of Proof by Division into Cases}

To prove a statement of the form "If $A_1$ or $A_2$ or $A_3$ or $\ldots$ or $A_n$, then $C$ prove that
$A_i$ for all $1 \le i \le n$ implies $C$. This is useful when a statement can be easily split
into multiple statements that fully encompass the original statement.

\begin{example}{}{label}
    Prove that the square of any odd integer has the form $8m + 1$ for some integer $m$.
\end{example}

\emph{Proof (Brief).} Suppose $n$ is an odd integer. By the quotient remainder theorem and using the fact
that the integer is odd, we can split the possible forms of $n$ into two cases: $4q+1$ or $4q+3$ for
some integer $q$. It can be proven through substitution that these two cases simplify to the form
$n^2=8m+1$.

\subsection{Absolute Value and the Triangle Inequality}

\begin{definition}{Absolute Value}{label}
    For any real number $x$, the \textbf{absolute value of x} is defined as follows:
    \begin{equation}
        |x| =
        \begin{cases}
            x & \text{if} \: x \ge 0 \\
            -x & \text{if} \: x < 0
        \end{cases}
    \end{equation}
\end{definition}

\begin{theorem}{Triangle Inequality}{label}
    For all real numbers $x$ and  $y$, $|x + y|  \le  |x| + |y|$.
\end{theorem}

\section{Indirect Argument: Contradiction and Contraposition}

Proof by contradiction is extremely intuitive and exactly what it sounds like. Assume that the negation
is true, and show that this assumption leads to a contradiction. Argument by contrapositive is equally
intuitive given the fact that a statement is logically equivalent to its contrapositive. Note that
proof by contraposition can only be used on universal conditionals.

\chapter{Sequences, Mathematical Induction, and Recursion}

The proof chapter is finally over! I did not like that chapter D:

\section{Sequences}

\begin{definition}{Sequence}{label}
    A \textbf{sequence} is a function whose domain is either all the integers between two given
    integers or all the integers greater than or equal to a given integer.
\end{definition}

The first term of a sequence is known as the \textbf{initial term}, and the last term is known as
the \textbf{final term}. An \textbf{explicit formula} or \textbf{general formula} is a rule
that shows how the values of $a_k$ depend on $k$.

\subsection{Summation Notation}

\begin{definition}{Summation Notation}{label}
    For integers $m$ and $n$ where $m \le n$, \[
        \sum_{k=m}^{n} a_k = a_m + a_{m+1} + \ldots + a_n
    .\]
    We call $k$ the \textbf{index} of the summation, $m$ the \textbf{lower limit} of the summation,
    and $n$ the \textbf{upper limit} of the summation.

    A recursive definition of summation notation: \[
        \sum_{k=m}^{m} a_k = a_m \; \text{and} \sum_{k=m}^{n} a_k = \sum_{k=m}^{n-1} a_k + a_n
    .\]
\end{definition}

\subsection{Product Notation}

\begin{definition}{Product Notation}{label}
    For integers $m$ and $n$ where $m \le n$, \[
        \prod_{k=m}^{n} a_k = a_m \cdot a_{m+1} \cdot \ldots \cdot a_n
    .\]

    The recursive definition of product notation is in essence the same idea as the one in summation
    notation.
\end{definition}

\subsection{Properties of Summations and Products}

\begin{theorem}{Properties}{label}
    \begin{enumerate}
        \item $\sum_{k=m}^{n} a_k + \sum_{k=m}^{n} b_k = \sum_{k=m}^{n} (a_k+b_k)$
        \item $c \cdot \sum_{k=m}^{n} a_k = \sum_{k=m}^{n} c \cdot a_k$
        \item $(\prod_{k=m}^{n} a_k) * (\prod_{k=m}^{n} b_k) = (\prod_{k=m}^{n} (a_k \cdot b_k))$
    \end{enumerate}
\end{theorem}

Substituting a new variable for $k$ is simple. Change the limits by setting $k$ equal to each of the
limits and noting the new value, then change the expression itself by substituting $k$.

\section{Mathematical Induction}

The general structure of mathematical induction mirrors a line of thinking somewhat like a domino effect:
If we can show that some property $P(n)$ is true for some integer $a$, and for all integers $k \ge a$,
if $P(k)$ is true then $P(k+1)$ is true, then the statement "for all integers $n \ge a, P(n)$ is true.
This is known as the \textbf{Principle of Mathematical Induction}. Showing that $P(n)$ is true for some
integer $a$ is known as the \textbf{basis step}, and proving that the truth of $P(k+1)$ follows from the
truth of $P(k)$ is known as the \textbf{inductive step}.

\begin{example}{Sum of the First $n$ Integers}{label}
    For all integers $n \ge 1$, \[
        1+2+\ldots+n=\frac{n*(n+1)}{2}
    .\]
\end{example}

\emph{Proof (Brief).} Let the property $P(n)$ be the given equation. It can be shown that $P(1)$ is true.
We assume that $P(k)$ is true ($P(k)$ is known as the \textbf{inductive hypothesis}) and use the fact
that $P(k)+k=P(k+1)$. Using algebraic manipulation, we find that $P(k+1)$ is true.

In general, use the inductive hypothesis to prove the inductive step.

\section{Strong Mathematical Induction}

\begin{definition}{Principle of Strong Mathematical Induction}{label}
    Let $P(n)$ be a property that is defined for integers $n$, and let $a$ and $b$ be fixed integers
    with $a \le b$. Suppose the following two statements are true:
    \begin{enumerate}
        \item $P(a), P(a+1), \ldots, P(b)$ are all true. (\textbf{basis step})
        \item For any integer $k \ge b$, if $P(i)$ is true for all integers $i$ from $a$ through $k$,
            then $P(k+1)$ is true. (\textbf{inductive step})
    \end{enumerate}
    Then the statement \[
        \text{for all integers } n \ge a, P(n)
    .\]
    is true.
\end{definition}

\begin{example}{Number of Multiplications Needed to Multiply $n$ Numbers}{label}
    Prove that for any integer $n \ge 1$, if $x_1, x_2, \ldots x_n$ are $n$ numbers, then no matter
    how the parentheses are inserted into their product, the number of multiplications used
    to compute the product is $n - 1$.
\end{example}

\emph{Proof (Brief).} $P(1)$ is evidently true, as it takes $0$ multiplications to multiply one number.
Using the inductive hypothesis that $P(i)$ is true for all integers $i$ from $1$ to $k$, we now
attempt to prove that $P(k+1)$ is also true. We do this by splitting the $k + 1$ integers into two
sides with $l$ ($1 \le l \le k$) factors on the left and $r$ $(1 \le r \le k)$ factors on the right.

By inductive hypothesis, we note that we end up with $(l-1)+(r-1)+1=l+r-1$ multiplications, which, when
considering that $l + r = k + 1$, proves the statement.

\begin{definition}{Well-Ordering Principle for the Integers}{label}
    Let $S$ be a set of integers containing one or more integers all of which are greater than some fixed integer. Then $S$ has a least element.
\end{definition}

A consequence of the well-ordering principle is the fact that any strictly decreasing sequence of nonnegative integers is finite. (This is because
from the well-ordering principle, there must be some least element $r_k$, which is the final element, because if there were some term $r_{k+1}$, then
$r_{k+1}<r_k$, which contradicts the well-ordering principle.

\section{Recursion}

Solving a problem recursively means to find a way to break it down into smaller subproblems each having the same form as the original problem.

\begin{example}{Tower of Hanoi}{label}
    What is the least number of steps to move 64 golden disks from pole A to pole C (given the information that
    there are three poles, all the disks are different sizes, and at any point, discs on every pole must be in
    increasing order of size)?
\end{example}

The key observation here is to note that if we know the solution for $k-1$ discs, we can use this information to
get the solution for $k$ discs:

\begin{enumerate}
    \item Move $k-1$ discs to pole B.
    \item Move the bottom disc of pole A to pole C.
    \item Move all the discs from pole B to pole C.
\end{enumerate}

It then follows that we get the recurrence $m_k=2m_{k-1}+1$ where $m_k$ is the number of moves to move $k$ discs from
one pole to another.

An explicit formula for this recurrence can be found through iteration and using the formula for the sum of a geometric
sequence. From this, we get the formula $m_n = 2^n-1$.

\subsection{Checking the Correctness of Explicit Formulas}

We can use mathematical induction to check the correctness of explicit formulas. For example, we can verify the formula for
Tower of Hanoi by showing that it is true for 1 ring, and then showing that it is true for all $k+1$ assuming that $k$ is true.
Check your induction proof carefully to make sure that no mistakes were made, and that the recursive form as well as the
explicit form were both used.

\end{document}


\documentclass[tikz,border=0cm]{standalone}
\usepackage{type1cm}
\usepackage{fp}
\usetikzlibrary{calc}
\usetikzlibrary{shadows}

\usepackage{fetamont}
\usepackage{cmbright}

\usepackage{bm,amsmath,amssymb}
\usepackage{soul}
\usepackage{multirow}

\newcommand{\clm}{Cl\'{e}ment }
\newcommand{\norm}[1]{\lVert#1\rVert}
\newcommand{\semi}[1]{\lvert#1\rvert}

\renewcommand\refname{}   % Don't want references title

\definecolor{edblue}{RGB}{4, 80, 124}
\definecolor{eyellow}{RGB}{248, 176, 23}
\definecolor{elblue}{RGB}{55, 172, 195}

\usepackage[a4paper, margin=1in]{geometry}
\usepackage{url}
\usepackage{setspace}
\renewcommand{\baselinestretch}{1.0}
\let\OLDthebibliography\thebibliography
\renewcommand\thebibliography[1]{
  \OLDthebibliography{#1}
  \setlength{\parskip}{0pt}
  \setlength{\itemsep}{0pt plus 0.3ex}
}

\begin{document}

\begin{tikzpicture}[x=21cm, y=29.7cm]
\fill[white, use as bounding box] (0, 0) rectangle (1, 1);

%%%%%%%%%%%%%%%%%%%%%%%%%%
% Title
%%%%%%%%%%%%%%%%%%%%%%%%%%
\begingroup
\draw[fill=eyellow, draw=none] {
  ($(current bounding box.north west)+(0cm, -20.0mm)$) --
  ($(current bounding box.north east)+(0cm, -20.0mm)$) --
  (current bounding box.north east) --
  (current bounding box.north west) --
  cycle
};
   %
\draw[elblue, line width=1.5mm]{
  ($(current bounding box.north east)-(0cm, 0.75mm)$) --
  ($(current bounding box.north west)-(0cm, 0.75mm)$)
};
  %
\draw[elblue, line width=1.5mm]{
  ($(current bounding box.north east)+(0cm, -19.25mm)$) --
  ($(current bounding box.north west)+(0cm, -19.25mm)$)
};
\node[anchor=center]
      (title)
      at ($(current bounding box.north)+(0cm, -1.0cm)$)
  {\begin{minipage}{\textwidth}
   \begin{center}
     {\ffmfamily\Large\bfseries\color{black}{fenicstools: supplementary materials}}\\
     {\ffmfamily\large\color{edblue}\textbf{M.~Mortensen, M.~Kuchta}}
   \end{center}
  \end{minipage}};
\endgroup

\node[anchor=north] at ($(current bounding box.north)+(0cm, -2.5cm)$)
{
  \begin{minipage}{1.2\textwidth}
    These are the supplementary materials for the poster \cite{poster} on
    fenicstools \cite{fenicstools} presented by the authors on FEniCS'16
    workshop.

%%%%%%%%%%%%%%%
% CLM
%%%%%%%%%%%%%%%
    \section*{\ffmfamily\color{edblue}{\clm interpolation}}
    The implementation of \clm interpolation in fenictools follows the
    exposition of the topic presented in \cite[ch. 2.6]{braess}. Therein the
    interpolant is constructed by finding the best patch-wise constant
    approximation. However, this is only a special case in the theory of
    \clm interpolants \cite{clement}. We note that the treatment of $H^1_0$
    interpolantion is discussed in \cite{scott}. Finally, we remark that the notion
    of finite element spaces on patches (more general polygonal domains) would allow
    for Virtual Finite Element Method \cite{vfem} to be used within FEniCS.

    Foundation for the averaging matrix $A$ is the summation matrix, which is a
    $(0, 1)$-matrix such that in $j$-th row the unit values are in columns that
    represent the cells that make up a patch surrounding the $j$-th degree of
    freedom. The matrix is constructed by using properties of the vertex quadrature.
    Let $K$ a cell and $f\in P_1(K)$ then
    \[
    \displaystyle \int_{K} f\,\mathrm{d}x = \frac{\semi{K}}{n_v} \sum_{j=1}^{n_v} f(x_j),
    \]
    where $n_v$ is the number of vertices of the simplex. Further let $\phi^K_i$ the
    $i$-th basis function of linear Lagrange element over $K$. Then integrating
    $\tfrac{n_v}{\semi{K}}\phi^K_i$ by the vertex quadrature gives the value one
    as desired. The averaging matrix is obtained by diagonally scaling the
    summation matrix by a vector of cell volumes. The total cost of constructing
    $A$ is close to assembling the mass matrix for scalar CG$_1$ space.

    In fenictools, the averaging matrix is constructed only for scalars. In
    order to interpolate tensor fields, each component is averaged and then assigned
    appropriately. The mapping from components to tensor is provided by DOLFIN's
    \emph{FunctionAssigner} class and is built only once. Its construction is
    the final step of setting up the \clm interpolator.

    Setting up \emph{FunctionAssigner} makes up for a significant part of the
    total construction time of \clm infrastructure, cf. Table \ref{tab:assign}.
    Overall, the new method is about four times, see Table \ref{tab:cstr}, more
    expensive to setup than $L^2$ interpolation (assembly of mass matrix + setting
    up conjugate gradient solver with multigrid preconditioning). However, with
    the action of \clm interpolator being more than eight times faster, the initial
    investment pays off during repetitive evaluation.

\begin{table}[ht]
\begin{center}
\caption{MPI-averaged setup cost of \clm interpolant infrastructure. The assembly of the
  averaging operator (in seconds) as a unit time for computing the mappings
for \emph{FunctionAssigner}.}
\label{tab:assign}
\footnotesize{
\begin{tabular}{l|ccccc}
\hline
\multirow{2}{*}{CPUs} & \multicolumn{5}{c}{Degrees of freedom}\\
\cline{2-6}
   &     132098 & 526338     &    2101250 &      8396802 & 14612418\\
\hline
  1 & (0.42, 0.84) & (1.46, 0.65) & (3.65, 1.17) & (16.16, 1.93) & (31.05, 1.75)\\
  2 & (0.11, 0.79) & (0.26, 1.06) & (1.04, 1.40) & (4.37, 1.63) & (8.02, 1.30)\\
  4 & (0.04, 0.70) & (0.12, 0.84) & (0.28, 0.98) & (1.20, 1.06) & (2.26, 0.96)\\
  8 & (0.01, 0.61) & (0.04, 0.60) & (0.12, 0.76) & (0.53, 0.98) & (0.95, 0.94)\\
  16 & (0.00, 0.57) & (0.01, 0.70) & (0.04, 0.70) & (0.13, 0.95) & (0.23, 0.82)\\
\hline
\end{tabular}
}
\end{center}
\end{table}
%
\begin{table}
\begin{center}
\caption{MPI-averaged setup cost (in seconds) of infrastructure for \clm interpolation and
$L^2$ projection.}
\label{tab:cstr}
\footnotesize{
\begin{tabular}{l|ccccc}
\hline
\multirow{2}{*}{CPUs} & \multicolumn{5}{c}{Degrees of freedom}\\
\cline{2-6}
   &     132098 & 526338     &    2101250 &      8396802 & 14612418\\
\hline
  1 &  (0.87, 0.31) & (2.81, 0.76) & (8.78, 3.04) & (51.01, 14.98) & (91.94, 26.94)\\
  2 &  (0.47, 0.17) & (1.23, 0.46) & (5.30, 1.75) & (24.03, 8.24) & (43.68, 16.20)\\
  4 &  (0.32, 0.09) & (1.06, 0.36) & (2.73, 0.99) & (12.18, 4.43) & (21.94, 11.69)\\
  8 &  (0.24, 0.06) & (0.67, 0.22) & (2.19, 0.77) & (10.09, 3.34) & (18.27, 5.74)\\
  16 & (0.16, 0.03) & (0.36, 0.10) & (1.32, 0.39) & (5.26, 1.65) & (9.29, 2.99)\\
\hline
\end{tabular}
}
\end{center}
\end{table}

%%%%%%%%%%%%%%%
% LP
%%%%%%%%%%%%%%%
    \section*{\ffmfamily\color{edblue}{Lagrangian tracking}}
    \emph{LagrangianParticles} class is organized as a dictionary, in which a
    cell containing some of the tracked particles is a key and a list of the
    particles contained in the said cell is a value. Based on the motion of
    particles, new cells are added to or removed from the dictionary.

    The data structure is very efficient in time integration where it eliminates
    the expensive search problem of locating for each particle a parent cell in
    which the velocity field is to be evaluated. In fenicstools the required
    cell is always available. Further advantage of the dictionary is a fact that
    for cell $K$ with multiple particles a restriction of the velocity field,
    $\boldsymbol{u}|_K$, is performed only once.

    The figures in the poster are part of the master thesis of Per Thomas Haga
    \cite{ptthesis} who used \emph{LagrangianParticles} to study drug delivery
    in spinal chord. The thesis developed into a recently submitted paper
    \cite{ptpaper}.

  \end{minipage}
};
\end{tikzpicture}

%%%%%%%%%%%%%%%%
% Refences
%%%%%%%%%%%%%%%%
\begin{tikzpicture}[x=21cm, y=29.7cm]
\fill[white, use as bounding box] (0, 0) rectangle (1, 1);

%%%%%%%%%%%%%%%%%%%%%%%%%%
% Title
%%%%%%%%%%%%%%%%%%%%%%%%%%
\begingroup
\draw[fill=eyellow, draw=none] {
  ($(current bounding box.north west)+(0cm, -20.0mm)$) --
  ($(current bounding box.north east)+(0cm, -20.0mm)$) --
  (current bounding box.north east) --
  (current bounding box.north west) --
  cycle
};
   %
\draw[elblue, line width=1.5mm]{
  ($(current bounding box.north east)-(0cm, 0.75mm)$) --
  ($(current bounding box.north west)-(0cm, 0.75mm)$)
};
  %
\draw[elblue, line width=1.5mm]{
  ($(current bounding box.north east)+(0cm, -19.25mm)$) --
  ($(current bounding box.north west)+(0cm, -19.25mm)$)
};
\node[anchor=center]
      (title)
      at ($(current bounding box.north)+(0cm, -1.0cm)$)
  {\begin{minipage}{\textwidth}
   \begin{center}
     {\ffmfamily\Large\bfseries\color{black}{fenicstools: supplementary materials}}\\
     {\ffmfamily\large\color{edblue}\textbf{M.~Mortensen, M.~Kuchta}}
   \end{center}
  \end{minipage}};
\endgroup

\node[anchor=north] at ($(current bounding box.north)+(0cm, -2.5cm)$)
{
  \begin{minipage}{1.2\textwidth}
    \section*{\ffmfamily\color{edblue}{References}}
    \bibliographystyle{plain}
    \vspace{-1cm}
    \bibliography{extras}
  \end{minipage}
};
\end{tikzpicture}

\end{document}


\documentclass{article}

\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{amsfonts}
\usepackage{tikz}
\usepackage{physics}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}
\usepackage{hyperref}

\usetikzlibrary{automata,positioning}

%
% Basic Document Settings
%

\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\pagestyle{fancy}
\lhead{\hmwkAuthorName}
\chead{\hmwkClass\ : \hmwkTitle}
\rhead{\firstxmark}
\lfoot{\lastxmark}
\cfoot{\thepage}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

\setlength\parindent{0pt}

%
% Create Problem Sections
%
\newcommand{\be}{\begin{equation}}
\newcommand{\ee}{\end{equation}}
\newcommand{\bes}{\begin{equation*}}
\newcommand{\ees}{\end{equation*}}
\newcommand{\bea}{\begin{flalign*}}
\newcommand{\eea}{\end{flalign*}}

\newcommand{\enterProblemHeader}[1]{
    \nobreak\extramarks{}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
}

\newcommand{\exitProblemHeader}[1]{
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \stepcounter{#1}
    \nobreak\extramarks{Problem \arabic{#1}}{}\nobreak{}
}

\setcounter{secnumdepth}{0}
\newcounter{partCounter}
\newcounter{homeworkProblemCounter}
\setcounter{homeworkProblemCounter}{1}
\nobreak\extramarks{Problem \arabic{homeworkProblemCounter}}{}\nobreak{}

%
% Homework Problem Environment
%
% This environment takes an optional argument. When given, it will adjust the
% problem counter. This is useful for when the problems given for your
% assignment aren't sequential. See the last 3 problems of this template for an
% example.
%
\newenvironment{homeworkProblem}[1][-1]{
    \ifnum#1>0
        \setcounter{homeworkProblemCounter}{#1}
    \fi
    \section{Problem \arabic{homeworkProblemCounter}}
    \setcounter{partCounter}{1}
    \enterProblemHeader{homeworkProblemCounter}
}{
    \exitProblemHeader{homeworkProblemCounter}
}

%
% Homework Details
%   - Title
%   - Due date
%   - Class
%   - Section/Time
%   - Instructor
%   - Author
%

\newcommand{\hmwkTitle}{Homework\ \#1}
\newcommand{\hmwkDueDate}{Due on 14th January, 2019}
\newcommand{\hmwkClass}{Dynamical Systems}
\newcommand{\hmwkClassTime}{}
\newcommand{\hmwkClassInstructor}{}
\newcommand{\hmwkAuthorName}{\textbf{Aditya Vijaykumar}}

%
% Title Page
%

\title{
    %\vspace{2in}
    \textmd{\textbf{\hmwkClass:\ \hmwkTitle}}\\
    \normalsize\vspace{0.1in}\small{\hmwkDueDate\ }\\
%    \vspace{3in}
}

\author{\hmwkAuthorName}
\date{}

\renewcommand{\part}[1]{\textbf{\large Part \Alph{partCounter}}\stepcounter{partCounter}\\}

%
% Various Helper Commands
%

% Useful for algorithms
\newcommand{\alg}[1]{\textsc{\bfseries \footnotesize #1}}

% For derivatives
\newcommand{\deriv}[1]{\frac{\mathrm{d}}{\mathrm{d}x} (#1)}

% For partial derivatives
\newcommand{\pderiv}[2]{\frac{\partial}{\partial #1} (#2)}

% Integral dx
\newcommand{\dx}{\mathrm{d}x}

% Alias for the Solution section header
\newcommand{\solution}{\textbf{\large Solution}}

% Probability commands: Expectation, Variance, Covariance, Bias
\newcommand{\E}{\mathrm{E}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\Cov}{\mathrm{Cov}}
\newcommand{\Bias}{\mathrm{Bias}}

\begin{document}

\maketitle
(\textbf{Acknowledgements} - I would like to thank Divya Jagannathan for discussions.)
\\

\begin{homeworkProblem}
	\textbf{Part (a)}\\
	Consider first the set $ E_u $. Any vector in this said would be given by $ V = \sum_i C_j w_j $, where $ w_j $ is a generalized eigenvector of this set. We know,
	\begin{equation*}
	(A - \lambda I)^m w_j = 0
	\end{equation*}
	for some $ m $. This also means that,
	\begin{equation*}
	(A - \lambda I) w_j = W_j \implies A w_j = \lambda w_j + W_j
	\end{equation*}
	where $ W_j \in \ker((A - \lambda I)^{m-1})$. Hence $ A w_j  \in E_u $. This also means that $ A^k w_j \in E_u $ for any whole number $ k $, which in turn means that, in general, $\sum_{k} c_k A^k w_j \in E_u$.

	Consider,
	\begin{align*}
	e^{At} V = \sum_{i,k} C_j \dfrac{t^k A^k}{k!} w_j \in E_u
	\end{align*}
	Hence Proved that $ E_u $ is an invariant subspace. The argument follows similarly for $ E_s, E_c $.

	If $ \alpha_j, \beta_j, \gamma_j $ are all eigenvectors as defined below, the most general solution of the system in given by,
	\begin{align*}
	\sum_{j} C_j w_j  &= \sum_{j, \alpha_j \in E_u} M_j \alpha_j + \sum_{j, \beta_j \in E_s} N_j \beta_j + \sum_{j, \gamma_j \in E_c} K_j \gamma_j\\
	\implies R^d &= E_u \oplus E_s \oplus E_c
	\end{align*}

	\textbf{Part (b)}\\
	For $x_0= \sum_{j} C_j w_j, w_j \in E_s $, $ e^{At}x_0 = \sum_{j}  C_j w_j e^{a_j t} e^{ i b_j t}, a_j < 0$. Hence, we can say,
	\begin{align*}
	\lim\limits_{t \rightarrow \infty } e^{At}x_0 &= \sum_{j}   C_j w_j \lim\limits_{t \rightarrow \infty } e^{a_j t} e^{ i b_j t} = 0 \\
	\lim\limits_{t \rightarrow - \infty } \abs{ e^{At}x_0 } &= \lim\limits_{t \rightarrow - \infty }  \abs{\sum_{j}   C_j w_j e^{a_j t} e^{ i b_j t}} = \infty
	\end{align*}

	\textbf{Part (c)}\\
	For $x_0= \sum_{j} C_j w_j, w_j \in E_u $, $ e^{At}x_0 = \sum_{j}  C_j w_j e^{a_j t} e^{ i b_j t}, a_j > 0$. Hence, we can say,
	\begin{align*}
	\lim\limits_{t \rightarrow -\infty } e^{At}x_0 &= \sum_{j}   C_j w_j \lim\limits_{t \rightarrow - \infty } e^{a_j t} e^{ i b_j t} = 0 \\
	\lim\limits_{t \rightarrow  \infty } \abs{ e^{At}x_0 } &= \lim\limits_{t \rightarrow  \infty }  \abs{\sum_{j}   C_j w_j e^{a_j t} e^{ i b_j t}} = \infty
	\end{align*}

	\textbf{Part (d)}\\
	For $x_0= \sum_{j} C_j w_j, w_j \in E_s $, $ e^{At}x_0 = \sum_{j}  C_j w_j e^{a_j t} e^{ i b_j t}, a_j < 0$. Hence, we can say,

\end{homeworkProblem}

\begin{homeworkProblem}
	\textbf{Part (a)}\\

	\textbf{Part (b)}\\
	As we are working with linear systems, any linear combination of the solutions will be linear too. We know that the columns of the fundamental matrix $ X $ are solutions to the ODE, and that they span the solution space. Right multiplying $ X $ with a constant, non-singular matrix $ C $ will give us a matrix which will have linear combination of the columns of $ X $ (ie the solutions to the ODE) according to the entries in $ C $. Due to the condition of non-singularity, this new matrix will also have linearly independent columns. Hence, the new matrix $ Y(t) = X(t) C $ will have columns which are solutions to the ODE and also span the solution space. Hence Proved.

	 Left multiplying $ X $ with a constant, non-singular matrix $ B $ will give us a matrix which will have linear combination of the \textit{rows} of $ X $ (which are not the solutions to the ODE) according to the entries in $ B $. This means that $ B X $ will not, in general, be a fundamental matrix for the system. Of course, $ BX $ can be a fundamental matrix if the rows of $ X $ are indeed solutions to the system, ie. if $ X^T = X $.
	\\

	\textbf{Part (c)}\\
	From the fact that $ X_1, X_2 $ are fundamental matrices and from the previous part, we can make the following statement,
	\begin{equation*}
	X_2(t) = X_1(t) C
	\end{equation*}
	where $ C $ is a non-singular matrix.

	Consider,
	\begin{align*}
	X_2(t + \omega) &= X_2(t) B_2\\
	X_1(t + \omega) C &= X_2(t) B_2\\
	X_1(t) B_1 C &= X_1(t)  C B_2\\
	\implies B_1 &= C B_2 C^{-1}
	\end{align*}
	where inverses could be taken in the last step only because the matrices are known to be nonsingular. Hence proved that $ B_1, B_2 $ are similar.
\end{homeworkProblem}
\end{document}


\documentclass[12pt]{article}
\newcommand{\kg}{\mathrm{kg}}
\newcommand{\m}{\mathrm{m}}
\newcommand{\cm}{\mathrm{cm}}
\newcommand{\s}{\mathrm{s}}
\newcommand{\ms}{\mathrm{ms}}
\begin{document}
\newcounter{problem}
\thispagestyle{empty}

\section*{NYU General Physics 1---Problem set 5}

\paragraph{Problem~\theproblem:}\refstepcounter{problem}%
A solid rubber ball of radius $1.5\,\cm$ is dropped from a height of
$1\,\m$ onto a hard surface.  It bounces.  The objective of this
problem is to figure out the magnitude of the contact force on the
ball \emph{during} the bounce.

\textsl{(a)} The contact force pushing the ball upwards will be far
larger in magnitude than the gravitational force pushing the ball
downwards.  Explain why in words.

\textsl{(b)} Estimate the mass $m$ of the ball and also the speed $v$
at which the ball will be traveling just before it hits the floor.
Use the internet or the library for the density of rubber.

\textsl{(c)} If the ball is in contact with the floor for about
one~millisecond, and if it bounces back upwards with about the ``equal
and opposite'' velocity to that it had before the contact, what is the
mean acceleration $\vec{a}$ of the ball during the bounce?

\textsl{(d)} What is the implied mean contact force $\vec{N}$ during
the bounce?

\textsl{(e)} I said the ball will be in contact with the floor for
$\sim 1\,\ms$.  How could you estimate this?  One option is to
consider the time it takes a sound wave to traverse the diameter of
the ball.  Look up the speed of sound in rubber and estimate this
time; is $1\,\ms$ reasonable?  Also, can you think of other ways to
make an estimate?

\paragraph{Problem~\theproblem:}\refstepcounter{problem}%
You are in the passenger seat of a car traveling fast in a straight
line.  You have your seatbelt on.  The driver slams on the brakes, so
you are accelerating with a magnitude of $12\,\m\,\s^{-2}$.

\textsl{(a)} If your mass is $50\,\kg$, calculate and also draw all of
the forces acting on your body during the acceleration (which most
normal people would call ``deceleration'').

\textsl{(b)} If the car plus contents has a mass of $1300\,\kg$, what
is the total force of the car on the road, from all four tires?  Give
direction and magnitude.

\textsl{(c)} If the road is slippery, the car will go into a skid.
What is the critical coefficient of friction $\mu$ below which the car
will slide?

\paragraph{Problem~\theproblem:}\refstepcounter{problem}%
When you are on a roller coaster, you feel heavier when the roller
coaster goes through the bottom of a dip, and you feel lighter when
the roller coaster goes over the crest of a hill.

\textsl{(a)} What force on your body in the roller coaster is larger
at the bottom of a dip and what force is smaller at the top of the
hill?  Hint: It isn't the gravitational force!  Be sure to be able to
explain \emph{why} the force is different in the different cases.

\textsl{(b)} The astronauts in the Space Station feel weightless; why?
Hint: It isn't because the gravitational force on them is small!

\end{document}


\documentclass[11pt, letterpaper]{article}
\setlength{\parindent}{0in}
\setlength{\textheight}{8.7in}
\setlength{\textwidth}{6.8in}
\setlength{\oddsidemargin}{-0.3in}
\setlength{\evensidemargin}{0.0in}
\addtolength{\topmargin}{-1in}
\setlength{\parskip}{0.1in}

\usepackage{amsmath, amsfonts, color}
\usepackage{amssymb}
\usepackage{bm}
\usepackage{booktabs}
\usepackage{enumerate}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{pdfpages}
\newcommand*{\justifyheading}{\raggedleft}

\renewcommand{\baselinestretch}{1.0}

\newcommand{\bx}{{\bm x}}
\newcommand{\bX}{{\bm X}}
\newcommand{\by}{{\bm y}}
\newcommand{\bY}{{\bm Y}}
\newcommand{\bW}{{\bm W}}
\newcommand{\bG}{{\bm G}}
\newcommand{\bR}{{\bm R}}
\newcommand{\bZ}{{\bm Z}}
\newcommand{\bV}{{\bm V}}
\newcommand{\bL}{{\bm L}}
\newcommand{\bz}{{\bm z}}
\newcommand{\be}{{\bm e}}
\newcommand{\bgamma}{{\bm \gamma}}
\newcommand{\bbeta}{{\bm \beta}}
\newcommand{\balpha}{{\bm \alpha}}
\newcommand{\bSigma}{{\bm \Sigma}}
\newcommand{\bmu}{{\bm \mu}}
\newcommand{\btheta}{{\bm \theta}}
\newcommand{\bepsilon}{{\bm \epsilon}}
\newcommand{\bone}{{\bm 1}}
\newcommand{\bzero}{{\bm 0}}
\newcommand{\bC}{{\bm C}}
\newcommand{\bI}{{\bm I}}
\newcommand{\bA}{{\bm A}}
\newcommand{\bB}{{\bm B}}
\newcommand{\bQ}{{\bm Q}}
\newcommand{\bS}{{\bm S}}
\newcommand{\bD}{{\bm D}}
\newcommand{\cQ}{\mathcal{Q}}
\newcommand{\cU}{\mathcal{U}}
\newcommand{\cI}{\mathcal{I}}
\newcommand{\cL}{\mathcal{L}}

\newcommand{\beas}{\begin{eqnarray*}}
\newcommand{\eeas}{\end{eqnarray*}}

\newenvironment{equationarrayright}{
                          \begin{eqnarray*}
                          \begin{array}{rcll}
                         }{
                          \end{array}
                          \end{eqnarray*}
                         }
\newcommand{\bear}{\begin{equationarrayright}}
\newcommand{\eear}{\end{equationarrayright}}

\renewcommand\arraystretch{1.3}

\DeclareMathOperator*{\argmin}{arg\,min}

\title{STAT/BIOST 571: Homework 1}
\date{\today}
\author{Philip Pham}

\begin{document}

\maketitle

\section*{Problem 1: Non-parametric linear regression (5 points)}
{In the context of random $\bX$ linear regression, consider $n$ independent observations $(x_i, Y_i)$ generated as follows.
First, the indepenent variable $x_i$ is selected according to
\[
x_i = \left\{ \begin{array}{cl} 0 & \textrm{w.p. 1/3} \\
                                          1 & \textrm{w.p. 1/3} \\
                                          3 & \textrm{w.p. 1/3} \end{array} \right. ,
\]
then the systematic component of $Y_i$ is set to
\[
\mu_i = \left\{ \begin{array}{cl} 0.6 & \textrm{if } x_i = 0 \\
                                           3.6& \textrm{if } x_i=1 \\
                                          6.8 &\textrm{if } x_i=3 \end{array} \right. ,
\]
and finally $Y_i \sim N(\mu_i , \sigma^2)$ with $\sigma^2=1$.
\begin{enumerate}[(a)]
{\item Simulate a single realization from the data-generating mechanism described above (with $n=20$)
and present the data in a scatterplot, with the $Y_i$ on the vertical axis and the $x_i$ on the horizontal axis.  Fit a simple linear regression model of the form
\[
Y_i = \beta_0 + x_i \beta_1 + \epsilon_i
\]
by ordinary least squares and overlay the fitted regression line on your scatterplot.  Comment on the approporiateness of fitting a linear model to these data.}

\begin{description}
\item[Solution:] See Figure \ref{fig:p1_ols}.

  \begin{figure}
    \centering
    \includegraphics{p1_ols.pdf}
    \caption{Ordinary least squares for data according to the model in Problem 1.}
    \label{fig:p1_ols}
  \end{figure}

  The line isn't a very good fit. It overestimates when $x = 0$ and
  underestimates when $x = 1$. This isn't surprising since the linear model is
  incorrect.

\end{description}
{\item Let $\hat{\beta}_1$ be the estimated slope parameter from fitting the model above, and define $\beta_1$ to be the quantity for which $\hat{\beta}_1$ is consistent, in the limit as $n \rightarrow \infty$.  Using the result on Slide 1.18 of the course lecture notes, calculate $\beta_1$ exactly.}

\begin{description}
\item[Solution:] $\beta_0 = 1$ and $\boxed{\beta_1 = 2.}$

  Let $\mathbb{P}$ be the probability measure over $x$ and $y$. Let
  $\phi\left(x \mid \mu, \sigma^2\right)$ be the density of the normal
  distribution with mean $\mu$ and variance $\sigma^2$. Consider the integral,
  \begin{align*}
    L\left(\gamma\right)
    &= \int \left(y - \left(\gamma_0 + \gamma_1x\right)\right)^2\,\operatorname{d}\mathbb{P}\left(x, y\right) \\
    &= \frac{1}{3}\int_{-\infty}^\infty\left(y - \gamma_0\right)^2\phi\left(y\mid 0.6, 1\right)\,\operatorname{d}y + \\
    &~~~~\frac{1}{3}\int_{-\infty}^\infty\left(y - \gamma_0 - \gamma_1\right)^2\phi\left(y\mid 3.6, 1\right)\,\operatorname{d}y + \\
    &~~~~\frac{1}{3}\int_{-\infty}^\infty\left(y - \gamma_0 - 3\gamma_1\right)^2\phi\left(y\mid 6.8, 1\right)\,\operatorname{d}y \\
    &= \frac{1}{3}\left(
      62.56 - 22\gamma_0 - 48\gamma_1 + 3\gamma_0^2 + 10\gamma_1^2 + 8\gamma_0\gamma_1
      \right).
  \end{align*}
  If we let $\beta = \argmin_\gamma L\left(\gamma\right)$. Taking the
  derviative, we setting
  $\frac{\partial L }{\partial \gamma}\left(\beta\right) = 0$ to obtain
  $\beta_0 = 1$ and $\beta_1 = 2$.
\end{description}

{\item Download White's seminal paper from 1980 on sandwich estimation from the course website .  Read (at least) the first few pages and explain how the value of $\beta_1$ you derived in part (b) also follows
  from Lemma 1 of the paper.}

\begin{description}
\item[Solution:] In fitting the model, we are assuming that the linear model is
  correct and Assumption 1 of the paper holds.

  Assumption 2 is satisfied so since $\mathbb{E}\left[\epsilon_i^2\right] = 1$
  for all $i$ and the expected covariance matrix =
  $\bar{M}_n = n^{-1}\sum_{i=1}^n\mathbb{E}\left[X_i^2\right]$ is just a matrix
  $1 \times 1$ with a single positive entry, so it is certainly nonsingular is
  with a positive determinant.

  Thus, Lemma 1 tells us that $\hat{\beta} \xrightarrow {\mathrm{a.s.}} \beta$,
  where $\hat{\beta} = \left(X^\intercal X\right)^{-1} X^\intercal Y$.

  Since $\hat{\beta}$ is also the maximum likelihood estimate which minimizes
  mean squared error, it converges almost surely to the value of $\beta$ that
  minimizes expected mean squared error.
\end{description}

{\item For finite $n$, we cannot discuss properties of the expectation of $\hat{\beta}_1$, at least not in the conventional sense.  Explain why not.}

\begin{description}
\item[Solution:] Note that
  $\hat{\beta} = \left(X^\intercal X\right)^{-1} X^\intercal Y$.

  If the linear model were correct, that is, $Y = X\beta + \epsilon$, taking the
  expection would give
  \begin{equation}
    \mathbb{E}\left[\hat{\beta}\right]
    = \beta + \mathbb{E}\left[
      \left( X^\intercal X \right)^{-1} X^\intercal \epsilon
    \right] = \beta.
  \end{equation}

  However, This calculation is incorrect when $Y$ depends on $X$ in a nonlinear
  way as with these data.
\end{description}

\end{enumerate}
For the remainder of this problem, condition on realizations of the data for which there exist $i$ and $i^\prime$ such that $x_i \neq x_{i^\prime}$.
\begin{enumerate}[(a)] \addtocounter{enumi}{4}
{\item Conduct a simulation study to assess the bias of $\hat{\beta}_1$ for estimating $\beta_1$.  Plot the estimated bias as a function of $n$ (for $n=2,4,6,\ldots,28,30,35,\ldots,95,100$), and overlay the standard error of $\hat{\beta}_1$ as a function of $n$.  Note that for this problem you should estimate the standard error of $\hat{\beta}_1$ based on the
  standard deviation of $\hat{\beta}_1$ across simulated realizations of the data, not using any further exact or asymptotic calculations.}

\begin{description}
\item[Solution:] See Table \ref{tab:p1_simulation_results} for the slope
  estimates $\hat{\beta}_1$ and standard error estimates $\hat{\sigma}$ for
  different values of $n$.

  \begin{table}
    \centering
    \input{p1_simulation_results.tex}
    \caption{The estimate for the slope ($\hat{\beta}_1$) and standard error for
      the estimate ($\hat{\sigma}$) were each calculated with $10^6$ trials. The
      last column $\mathbb{E}\left[\hat{\beta}_1\right]$ was calculated
      numerically by enumerating over the possible draws of $X$, which is detailed in Part (g).}
    \label{tab:p1_simulation_results}
  \end{table}

  \begin{figure}
    \centering
    \includegraphics{p1_simulation_results.pdf}
    \caption{Data from Table \ref{tab:p1_simulation_results} along with
      asymptotic slope ($\beta_1$), the bias ($\hat{\beta}_1 - \beta_1$), and
      variance ($\hat{\sigma}^2$).}
    \label{fig:p1_simulation_results}
  \end{figure}

  These data are plotted in Figure \ref{fig:p1_simulation_results}.
\end{description}

{\item Comment on your findings in part (e).  Is the bias in estimating $\beta_1$ surprising?  Why or why not?  How concerned should one be about this bias? }

\begin{description}
\item[Solution:] The bias is not suprising. As discussed in Part (d), the usual
  method of proving that the estimate is unbiased does not work since the true
  model is not linear.

  The bias decreases quickly with increasing $n$, so it is not too concerning
  with reasonable sample sizes. Moreover, the standard error is larger than the
  bias, so in the bias-variance decomposition,
  \begin{align*}
    \mathbb{E}\left[
      \left(\hat{\beta}_1 - \beta_1\right)^2
    \right]
    &=
      \mathbb{E}\left[
      \left(\hat{\beta}_1 - \beta_1\right)^2
      \right] =
    \mathbb{E}\left[\hat{\beta}_1^2\right] - 2\mathbb{E}\left[\hat{\beta}_1\right]\beta_1 + \beta_1^2  \\
    &= \operatorname{var}\left(\hat{\beta}_1\right) + \mathbb{E}\left[
      \hat{\beta}_1
      \right]^2  - 2\mathbb{E}\left[\hat{\beta}_1\right]\beta_1 + \beta_1^2  \\
    &= \left(\mathbb{E}\left[\hat{\beta}_1\right] - \beta_1\right)^2 + \operatorname{var}\left(\hat{\beta}_1\right) \\
    &= \left[\operatorname{Bias}\left(\hat{\beta}_1\right)\right]^2 + \operatorname{var}\left(\hat{\beta}_1\right),
  \end{align*}
  the variance term which is the standard error squared will dominate. Indeed,
  according to Figure \ref{fig:p1_simulation_results}, the variance is much
  larger than the bias when $n$ is small. When $n$ is large, both the bias and
  variance are small, so neither is too concerning.
\end{description}

{\item It is possible to derive exact values for the biases estimated in part (d) by enumerating all possible draws of $x_1, \ldots, x_n$.  Do this calculation for $n=2,4,6,8$.  Report your numerical findings and overlay them on your plot from part (e).   Show how you can do the calculation entirely by hand for $n=2$ (you may use a computer for matrix algebra).  Although it is highly advisable to use a computer for this calculation for $n>2$, simulation results are not sufficient.  Your calculation of the bias must be exactly correct up to the computer's floating point precision.}

\begin{description}
\item[Solution:] $\mathbb{E}\left[\hat{\beta}\right]$ can be efficiently
  calculated by noticing that
  \begin{equation}
    \mathbb{E}\left[\hat{\beta} \mid X \right] =
    \mathbb{E}\left[\left(X^\intercal X\right)^{-1}X^\intercal Y
      \mid X
    \right]
    = \left(X^\intercal X\right)^{-1}X^\intercal \mu.
  \end{equation}
  is the same for different permuations of $X$.

  Assume that each $X_{(j_0,j_1,j_3)} \in \mathbb{R}^n$ is a vector of $j_0$
  $0$s, $j_1$ $1$s, and $j_3$ $3$s, where $j_0 + j_1 + j_3 = n$ and
  $j_0,j_1,j_3 \in \left\{0,1,\ldots, n - 1\right\}$. The triplets $\left(j_0, j_1, j_3\right)$
  can be seen as coming from the multinomial distribution, so
  \begin{equation}
    \mathbb{P}\left(\left(j_0, j_1, j_3\right)\right) = \frac{n!}{j_0!j_1!j_3!}\left(\frac{1}{3}\right)^n.
  \end{equation}

  Therefore, we have that
  \begin{align*}
    \mathbb{E}\left[\hat{\beta}\right]
    &= \mathbb{E}\left[\mathbb{E}\left[
      \hat{\beta}
      \mid X
      \right]\right] \\
    &= \sum_{\left\{
    \left(j_0, j_1, j_3\right) \in \left\{0,1,\ldots, n - 1\right\}^3
      \mid j_0 + j_1 + j_3 = n\right\}}
      \mathbb{P}\left(\left(j_0, j_1, j_3 = n\right)\right)
      \left(X^\intercal X\right)^{-1}X^\intercal \mu \\
    &= \sum_{\left\{
    \left(j_0, j_1, j_3\right) \in \left\{0,1,\ldots, n - 1\right\}^3
      \mid j_0 + j_1 + j_3 = n\right\}}
      \frac{n!}{j_0!j_1!j_3!}\left(\frac{1}{3}\right)^n
      \left(X^\intercal X\right)^{-1}X^\intercal \mu.
  \end{align*}

  \begin{figure}
    \centering
    \includegraphics{p1_bias_comparison}
    \caption{Comparison of the simulated bias from Part (e) against the exact
      calculation in Part (g).}
    \label{fig:p1_bias_comparison}
  \end{figure}

  The results of this calculation can be seen in the last column of Table
  \ref{tab:p1_simulation_results}. The biases from the simulation are plotted
  against the exact calculation in Figure \ref{fig:p1_bias_comparison}.

  Both the table and figure show that the simulation and calulations
  agree. There appears to be more noise with larger $n$, however.
\end{description}
\end{enumerate}

\section*{Problem 2: Average slope interpretation of generalized least squares (5 points)}
{\em Consider simple linear regression with a
single covariate and an intercept, such that we are fitting
\[
E(Y_i)=\beta_0 + \beta_1 x_i
\]
with scalar observations $Y_1,\ldots,Y_n$. As we saw on slide 1.18, the OLS estimator of $\beta_1$ can
be written as a weighted average of the pairwise slopes $(Y_i-Y_j)/(x_i-x_j)$.
This is called the ``average slope'' representation.
}
\begin{enumerate}[(a)]
{\em \item Suppose that $\bSigma$, the covariance of $\bY$, is a diagonal matrix with known entries $\sigma_1^2,\ldots,\sigma_n^2$.
  Derive the weights for an ``average slope'' representation for the optimal generalized least squares (GLS) estimate of $\beta_1$ (see exercise 8.1 in the Wakefield text if you are not sure of the form of the optimal GLS estimate).  A formal proof is optional, but you should provide a convincing justification for the specific form of your answer.  HINT: One way to approach this problem is to think about what happens when the $\sigma^2_i$ are all rational numbers, i.e., they can be expressed as fractions.}

\begin{description}
\item[Solution:] The likelihood function is
  \begin{equation}
    L_{X,y,\Sigma}\left(\beta\right) = f_\beta\left(y\right)
    = \frac{1}{\sqrt{\det\left(2\pi\Sigma\right)}}\exp\left(
      -\frac{1}{2}\left(y - X\beta\right)^{\intercal}\Sigma^{-1}\left(y - X\beta\right)
    \right).
  \end{equation}
  Maximizing this likelihood by taking the $\log$ and derivative, we estimate
  $\beta$ to obtain the maximum likelihood estimate
  $\hat{\beta} =
  \left(X^\intercal\Sigma^{-1}X\right)^{-1}X^\intercal\Sigma^{-1}y$.

  Expanding this out, we find that
  \begin{equation*}
    \hat{\beta} =
    \left.
      \begin{pmatrix}
        \displaystyle
      \left(\sum_{i=1}^n\frac{x_i^2}{\sigma_i^2}\right)
      \left(\sum_{i=1}^n\frac{y_i}{\sigma_i^2}\right) -
      \left(\sum_{i=1}^n\frac{x_i}{\sigma_i^2}\right)
      \left(\sum_{i=1}^n\frac{x_iy_i}{\sigma_i^2}\right) \\
      \displaystyle
      \left(\sum_{i=1}^n\frac{1}{\sigma_i^2}\right)\left(\sum_{i=1}^n\frac{x_iy_i}{\sigma_i^2}\right) -
      \left(\sum_{i=1}^n\frac{x_i}{\sigma_i^2}\right)
      \left(\sum_{i=1}^n\frac{y_i}{\sigma_i^2}\right)
    \end{pmatrix}
    \middle/\left(
    \left(\sum_{i=1}^n\frac{1}{\sigma_i^2}\right)
      \left(\sum_{i=1}^n\frac{x_i^2}{\sigma_i^2}\right) -
      \left(\sum_{i=1}^n \frac{x_i}{\sigma_i^2}\right)^2
    \right)
  \right..
\end{equation*}

Let
$\bar{x} = \left.\sum_{i=1}^n \frac{1}{\sigma_i^2}x_i\middle/\sum_{i=1}^n
  \frac{1}{\sigma_i^2}\right.$ denote the weighted mean. We can rewrite our
estimate for the slope
\begin{equation*}
  \hat{\beta}_1 = \frac{\sum_{i=1}^n\frac{1}{\sigma_i^2}\left(x_i - \bar{x}\right)\left(y_i - \bar{y}\right)}
  {\sum_{i=1}^n\frac{1}{\sigma_i^2}\left(x_i - \bar{x}\right)^2}
  =
  \frac
  {
    \sum_{i=1}^n\frac{1}{\sigma_i^2}\sum_{j=1}^n\frac{1}{\sigma_j^2}\sum_{k=1}^n\frac{1}{\sigma_k^2}
    \left(x_i - x_j\right)\left(y_i - y_k\right)
  }
  {
    \sum_{i=1}^n\frac{1}{\sigma_i^2}\sum_{j=1}^n\frac{1}{\sigma_j^2}\sum_{k=1}^n\frac{1}{\sigma_k^2}
    \left(x_i - x_j\right)\left(x_i - x_k\right)
  }.
\end{equation*}

Now with some algebra,
\begin{align*}
  \sum_{i=1}^n\frac{1}{\sigma_i^2}\sum_{j=1}^n\frac{1}{\sigma_j^2}\sum_{k=1}^n\frac{1}{\sigma_k^2}
  \left(x_i - x_j\right)\left(y_i - y_k\right)
  &= \sum_{j=1}^n\frac{1}{\sigma_j^2}\sum_{i=1}^n\frac{1}{\sigma_i^2}\sum_{k=1}^n\frac{1}{\sigma_k^2}
    \left(x_i - x_j\right)\left(y_i - y_k\right) \\
  &= \left(\sum_{k=1}^n\frac{1}{\sigma_k^2}\right)
    \sum_{i=1}^n\frac{1}{\sigma_i^2}\sum_{j=1}^n\frac{1}{\sigma_j^2}
    \left(x_i - x_j\right)y_i \\
  &= \frac{1}{2}\left(\sum_{k=1}^n\frac{1}{\sigma_k^2}\right)
    \sum_{i=1}^n\frac{1}{\sigma_i^2}\sum_{j=1}^n\frac{1}{\sigma_j^2}
    \left(x_i - x_j\right)\left(y_i - y_j\right),
\end{align*}
where the last two steps take advantage of the symmetry of the $i$ and $j$
indices. Using this result, we have can write our slope estimate
\begin{align*}
  \hat{\beta}_1
  &=
    \frac{\sum_{i=1}^n\frac{1}{\sigma_i^2}\sum_{j=1}^n\frac{1}{\sigma_j^2}
    \left(x_i - x_j\right)\left(y_i - y_j\right)}
  {\sum_{i=1}^n\frac{1}{\sigma_i^2}\sum_{j=1}^n\frac{1}{\sigma_j^2}
    \left(x_i - x_j\right)^2} \\
  &= \boxed{\left.
    \sum_{i=1}^n\sum_{\substack{j=1 \\ x_i \neq x_j}}^n \left(\frac{\left(x_i - x_j\right)^2}{\sigma_i^2\sigma_j^2}\right)
    \left(\frac{y_i - y_j}{x_i - x_j}\right)
    \middle/
    \sum_{i=1}^n\sum_{j=1}^n \frac{\left(x_i - x_j\right)^2}{\sigma_i^2\sigma_j^2}
    \right..}
\end{align*}

Thus, the unnormalized weight for the slope between $i$ and $j$ is
$\boxed{\displaystyle \frac{\left(x_i - x_j\right)^2}{\sigma_i^2\sigma_j^2}.}$
The normalized weights are
$\boxed{\displaystyle w_{ij} = \left.\frac{\left(x_i - x_j\right)^2}{\sigma_i^2\sigma_j^2}
    \middle/\sum_{i=1}^n\sum_{j=1}^n \frac{\left(x_i - x_j\right)^2}{\sigma_i^2\sigma_j^2}\right..}$

Intuitively, this make sense. As $\sigma_i^2$ and $\sigma_j^2$ increase, the
weight decreases since our estimated slope is noisier. On the other hand, as
$(x_i - x_j)^2$ increases the weight increases because the slope estimate
becomes more robust to noise since the effect size is larger. To see this
mathematically, note that
$\frac{y_i - y_j}{x_i - x_j} = \beta_1 + \frac{\sigma_i\epsilon_i -
  \sigma_j\epsilon_j}{x_i - x_j}$, where
$\epsilon_i \sim \mathcal{N}\left(0,1\right)$. The error term increases in
magnitude as a function of $\sigma_i^2$ and $\sigma_j^2$ and decreases in
magnitude as a function of $(x_i - x_j)^2$.
\end{description}
{\item Now consider what the weights would be in an ``average slope''
representation for the optimal GLS estimate of $\beta_1$, for a general covariance matrix $\bSigma$.  Give some properties of the weights that you would expect to hold.  In answering this question, it might be helpful to think about how the weights would change as you vary individual entries in $\bSigma$.
Also discuss any situations where you would expect the weight for a particular pair of observations to
be infinite or zero.  Note that you are {\bf not} asked to derive an average slope representation for GLS.}

\begin{description}
\item[Solution:] Let $\mathbf{1}$ be a vector a in $\mathbb{R}^n$ of all
  $1$s. Denote the weighted mean,
  $\bar{x} =
  \left.\mathbf{1}^\intercal\Sigma^{-1}x\middle/\mathbf{1}^\intercal\Sigma^{-1}\mathbf{1}\right.$
  Denote the centered vector, $\tilde{x} = x - \bar{x}\mathbf{1}$.

  Then, we have that
  \begin{equation}
    \hat{\beta}_1 = \frac{\tilde{x}^\intercal\Sigma^{-1}\tilde{y}}{\tilde{x}^\intercal\Sigma^{-1}\tilde{x}}
    = \left.
      \sum_{i=1}^n\sum_{j=1}^n \Sigma^{-1}_{ij}\tilde{x}_i\tilde{y}_j
      \middle/
      \sum_{i=1}^n\sum_{j=1}^n \Sigma^{-1}_{ij}\tilde{x}_i\tilde{x}_j.
      \right.
    \end{equation}

    Let $\Delta \in \mathbb{R}^{n \times n}$ be the skew-symmetric matrix where
    $\Delta_{ij} = x_i - x_j$. Let $\Sigma_i$ denote the $i$th row or column of
    the covariance matrix (the matrix is symmetric). We can rewrite our slope
    estimate as
    \begin{equation*}
      \hat{\beta}_1 = \left.\sum_{i=1}^n\sum_{\substack{j=1 \\ x_i \neq x_j}}^n
      \frac{y_i - y_j}{x_i - x_j}\left(x_i - x_j\right)\operatorname{tr}\left(
      \Delta\Sigma_i^{-1}\left(\Sigma_j^{-1}\right)^\intercal
    \right)
    \middle/
    \sum_{i=1}^n\sum_{j=1}^n\left(x_i - x_j\right)\operatorname{tr}\left(
      \Delta\Sigma_i^{-1}\left(\Sigma_j^{-1}\right)^\intercal
    \right),\right.
    \end{equation*}
    where $\operatorname{tr}$ is the trace operator. Since
    \begin{align}
      -\left(x_i - x_j\right)\operatorname{tr}\left(
        \Delta\Sigma_i^{-1}\left(\Sigma_j^{-1}\right)^\intercal
      \right)
      &= \left(x_i - x_j\right)\sum_{k=1}^n\sum_{l=1}^n
        \left(x_k - x_l\right)\Sigma_{ik}^{-1}\Sigma_{jl}^{-1} \nonumber \\
      &= \left(x_i - x_j\right)\sum_{k=1}^n\sum_{l=k + 1}^n
        \left(x_k - x_l\right)\det\begin{pmatrix}
          \Sigma_{ik}^{-1} & \Sigma_{il}^{-1} \\
          \Sigma_{jk}^{-1} & \Sigma_{jl}^{-1}
        \end{pmatrix},
    \end{align}
    we can see the pairwise slopes are weighted according to the entries from
    the precision matrix. Moreover, the situation differs from the independent
    case in that the weight of each slope depends on all possible pairwise
    differences $x_j - x_l$, which intuitively makes sense since there may be
    some correlation with the other observations.

    Let $\tilde{\rho}_{ij}$ be the partial correlation coefficient between $Y_i$
    and $Y_j$ given the remaining $Y_k$. The partial correlation measures the
    correlation between observations controlling for cofounders. Entries of the
    precision matrix are the scaled partial correlations,
    $\Sigma_{ij}^{-1} = \tilde{\rho}_{ij}\sqrt{\Sigma_{ii}^{-1}\Sigma_{jj}^{-1}}$.

    With this identity, we can rewrite unnormalized weights for the slope
    estimate
    \begin{equation}
      -\left(x_i - x_j\right)\operatorname{tr}\left(
        \Delta\Sigma_i^{-1}\left(\Sigma_j^{-1}\right)^\intercal
      \right) = \left(x_i - x_j\right)
      \Sigma_{ii}^{-1}\Sigma_{jj}^{-1}
      \sum_{k=1}^n\sum_{l=k + 1}^n
      \left(x_k - x_l\right)\det\begin{pmatrix}
        \tilde{\rho}_{ik} & \tilde{\rho}_{il} \\
        \tilde{\rho}_{jk} & \tilde{\rho}_{jl}
        \end{pmatrix},
        \label{eqn:p2_slope_correlation}
    \end{equation}

    Equation \ref{eqn:p2_slope_correlation} gives us insight into how the
    covariance matrix $\Sigma$ affects the weights for the pairwise slope
    between observations $i$ and $j$.

    As before, large differences $x_i - x_j$ increase the magnitude of the
    weight, and large variances, manifested as small entries of
    $\Sigma_{ii}^{-1}$ and $\Sigma_{jj}^{-1}$, decrease the magnitude of the
    weight. Now, the differences $x_j - x_l$ also affect the weight in a similar
    manner to $x_i - x_j$. The partial correlation coefficients govern the
    degree of this effect. If there is no partial correlation, that is,
    $\tilde{\rho}_{ik} = 0$ and $\tilde{\rho}_{il} = 0$ or
    $\tilde{\rho}_{jk} = 0$ and $\tilde{\rho}_{jl} = 0$, then the term involving
    $x_{k} - x_{l}$ is zeroed out. Of course, when the observations are
    independent only the terms with $\left(k,l\right) = \left(i,j\right)$ or
    $\left(k,l\right) = \left(j,i\right)$ are nonzero, and we recover the same
    weights as the previous part.
  \end{description}
\end{enumerate}

\section*{Problem 3: Cross-sectional/longitudinal effects, partitioning the exposure, mean model misspecification (10 points)}
{\em Consider a made-up computer literacy trial similar to the one discussed in lecture (slides 1.35--1.48).  There are $n$ subjects, indexed by $i=1,\ldots,n$, each one of which is observed at
$m$ followup times, indexed by $j=1,\ldots,m$.  Let $Y_{ij}$ be the literacy score of subject $i$ at follow-up time $j$, at which time the subject's age is $x_{ij}$.  Suppose the design is fixed, meaning that the $x_{ij}$ are all
deterministic and known in advance, and assume the true data-generating
mechanism can be written
\begin{equation}
\label{eqn:p3_eq1}
Y_{ij}=f(x_{i1})+\beta_L(x_{ij}-x_{i1})+\epsilon_{ij}
\end{equation}
with i.i.d. $\epsilon_{ij}\sim N(0,\sigma^2)$.  The unknown function $f(\cdot)$
represents a potentially nonlinear cohort effect and $\beta_L$ is the coefficient for the
linear longitudinal effect.
Suppose we try to estimate $\beta_L$ using
the exposure partioning method proposed on slide 1.44
(see, also, Diggle et al. page 16), based on the linear regression estimating equations
with the mean model
\begin{equation}
\label{eqn:p3_eq2}
E(Y_{ij})=\beta_0 + \beta_C x_{i1} + \beta_L (x_{ij}-x_{i1}).
\end{equation}}
\begin{enumerate}[(a)]
{\em \item Give a sufficient condition on the data-generating mechanism for $Y_{ij}$ in equation~(\ref{eqn:p3_eq1}) to guarantee
  unbiased estimation of $\beta_L$ and valid model-based standard errors.}

\begin{description}
\item[Solution:] Let $\tilde{X}$ be the matrix with $1$s in the first columns,
  $x_{i1}$ (repeated as necessary) in the second column and $x_{ij} - x_{i1}$ in
  the third column. Let $Y$ be a vector of the corresponding literacy scores. We
  can estimate $\beta_L$ with the least squares estimator
  \begin{equation}
    \hat{\beta} =
    \begin{pmatrix}
      \hat{\beta}_0 \\
      \hat{\beta}_C \\
      \hat{\beta}_L
    \end{pmatrix}
    =
    \left(\tilde{X}^\intercal\tilde{X}\right)^{-1}\tilde{X}^\intercal Y.
  \end{equation}
  If we substitute Equation \ref{eqn:p3_eq1} for $Y$,
  $\mathbb{E}\left[\hat{\beta}\right] = \beta$ if $f$ is a linear function, that
  is, $f\left(x_{i1}\right) = \beta_0x_{i1}$ for some constant $\beta_0$.

  Given $f$ is linear, the covariance matrix for our estimator is
  \begin{equation}
    \operatorname{var}\left(\hat{\beta}\right)
    =
    \sigma^2\left(\tilde{X}^\intercal \tilde{X}\right)^{-1}.
  \end{equation}
\end{description}

{\em \item Give a sufficient condition on the choice of follow-up times in the study design to guarantee
unbiased estimation of $\beta_L$, even if the condition in part (a) is violated.  Thinking about
when confounding bias is a problem will be helpful here.}

\begin{description}
\item[Solution:] To control for the cofounding factors, we can use
  randomization. In particular, if follow-up times are selected independently of
  the subjects' ages, any systematic bias from incorrectly predicting base
  literacy rates will be eliminated.

  To see this, note that if the covariates are independent
  $\mathbb{E}\left[\tilde{X}^\intercal \tilde{X}\right]$ is a diagonal matrix
  (without loss of generality assume $\tilde{X}$ is centered), so the
  coefficients are estimated independently of each other.
\end{description}

{\em \item Download the modified made-up literacy data from the course web site (you can find these data with the homework files, not in the general dataset section), and generate
one or more exploratory plots.  Describe how you can graphically see evidence that neither of
the above conditions is satisfied.}

\begin{description}
\item[Solution:] See the plotted data in Figure \ref{fig:p3_data}.

  \begin{figure}
    \centering
    \includegraphics{p3_data.pdf}
    \caption{In the computer literacy data, each subject has 3 observations
      taken at similar follow-up periods.}
    \label{fig:p3_data}
  \end{figure}
  Looking at the red points, one sees that $f$ is clearly not linear, so the
  assumption in Part (a) is violated. Each subject was followed up with
  twice. $x_{i3} - x_{i2}$ appears to be similar to $x_{i2} - x_{i1}$. For older
  subjects, the follow up periods appear to be shorter. Clearly, the follow up
  times were not selected independently of the subjects' ages, so the assumption
  Part (b) is violated.
\end{description}

{\em \item Propose an alternative regression model that can still be used to unbiasedly estimate $\beta_L$ and give valid model-based standard errors.  Fit this model to the downloaded data using the $\texttt{lm()}$ function in $\texttt{R}$ and report your findings (point estimate and standard error).
  Compare to what you obtain by fitting model~(\ref{eqn:p3_eq2}) to the data.}

\begin{description}
\item[Solution:] We can fit a model directly to the difference
  $Y_{ij} - Y_{i1} = \beta_L\left(x_{ij}- x_{i1}\right) + \left(\epsilon_{ij} -
    \epsilon_{i1}\right)$, $j \neq 1$. In this way, we discard one-third of our
  observations. Let $\tilde{X}_3$ be the third column of $\tilde{X}$ and let
  $\tilde{Y}$ be the vector of $Y_{ij}- Y_{i1}$. Assume that we have discarded
  the $j = 1$ entries. Then, we can our new model can estimated
  $\hat{\beta}_L =
  \left(\tilde{X}_3^\intercal\tilde{X}_3\right)^{-1}\tilde{X}_3^\intercal\tilde{Y}$.
  Since the expected residual is $0$ this estimator is always unbiased.

  Fitting this model to the data, we find that
  $\boxed{\hat{\beta}_L = 1.02185154}$ with corresponding standard error
  $\boxed{\sqrt{\hat{\operatorname{var}}\left(\hat{\beta}_L\right)} =
    0.032418976.}$

  If we fit the model in Equation \ref{eqn:p3_eq2}, we have
  $\boxed{\hat{\beta}_L = 1.24721578}$ with corresponding standard error
  $\boxed{\sqrt{\hat{\operatorname{var}}\left(\hat{\beta}_L\right)} = 1.06297.}$
  Thus, the misspecified model leads to a much higher standard error.
\end{description}

{\em \item Design and conduct a simulation study to illustrate the following
\begin{enumerate}
\item Model~(\ref{eqn:p3_eq2}) can fail to give unbiased estimates of $\beta_L$ without at least one of the additional conditions from parts (a) and (b).
\item Model~(\ref{eqn:p3_eq2}) does give unbiased estimates if either of the additional conditions from parts (a) and (b) is satisfied.
\item The model you propose in part (d) works, regardless.
\end{enumerate}
You will need to consider a few choice of $f(\cdot)$ and study designs $\{x_{ij}\}$
to illustrate points (i) through (iii).  Note that in earlier parts of the problem, you should have
argued theoretically that each of these points is true; the purpose of the simulation study is to verify
and illustrate your conclusions. }

\begin{description}
\item[Solution:] I fixed $\beta_L = 2$, $\sigma^2 = 1$, and use 64
  subjects. If I specified 2 follow-up times for each subject and let
  $f\left(x\right) = 200*\exp\left(-x\right)$, both assumptions are
  violated. Indeed, fitting the model in Equation \ref{eqn:p3_eq2} led to a
  biased estimate of $2.16$. The alternative model from the previous section
  handle this case fine and produced an unbiased estimate.

  Next, I enforced the condition in Part (a) and made $f$ linear,
  $f\left(x\right) = 200 - 12x$. Both the model in Equation \ref{eqn:p3_eq2} and
  the alternative model result in unbiased estimates.

  When it's enforced that the follow up times are independent of the base age,
  again both models produce unbiased estimates even if the $f$ is a nonlinear
  function like $f\left(x\right) = 200*\exp\left(-x\right)$.

  Code for all these simulations is found in the Appendix.
\end{description}

{\em \item In your simulation study, what do you notice about the standard error estimates from the model in equation~(\ref{eqn:p3_eq2}) when the condition from part (b) is satisfied but
  the condition from part (a) is not satisfied?  Do sandwich standard errors fix the problem?  }

\begin{description}
\item[Solution:] With the usual linear regression estimate for the variance of
  $\hat{\beta}_L$, the standard errors are overestimated. My simulations show
  that the 95\% confidence interval actually covered the true mean 98.1\% of the
  time.

  Sandwich estimation seems to help a little, but the situation is the same with
  the same: the 95\% confidence interval with the sandwich standard errors
  covers the true mean 97.75\% of the time, so it also overestimates the
  standard error.
\end{description}

{\em \item Explain, theoretically, what is going wrong in part (f) of this problem.
This involves some fairly careful thinking about misspecified mean models and what is required
for sandwich estimation to be valid.}

\begin{description}
\item[Solution:] Sandwich estimation protects against misspecification of the
  variance by using empirical estimates. The \emph{bread} part of the sandwich
  estimator in the case of ordinary least squares is $X^\intercal\hat{\Sigma}X$,
  where $\hat{\Sigma}_{ii}$ is diagonal, and
  $\hat{\Sigma}_{ii} = \left(Y_i - X\hat{\beta}\right)^2$. Thus, the diagonal
  contains empirical residual estimates. Those estimates depend on our mean
  model, which in this case is incorrect, biased, and inconsistent since the $f$
  is nonlinear. Thus, the results of the sandwich estimator will also be biased
  and inconsistent.
\end{description}

\end{enumerate}

\section*{Appendix}

Code for fitting models and generating plots is attached.

\includepdf[pages=-]{p1_simulation.pdf}
\includepdf[pages=-]{p3_regression.pdf}

\end{document}


\documentclass[journal, a4paper]{IEEEtran}

\usepackage{graphicx}
\begin{document}

% Define document title and author

\font\myfont=cmr12 at 16pt

	\title{\myfont  Speed of Light Measurements with a Helium-Neon Laser }
	\author{Litawn Gan
	\thanks{}}
	\markboth{Physics 107. Lab 2}{}
	\maketitle

\begin{abstract}
We measure the speed of light $c$ using laser interferometry techniques. By varying the length of a Helium-Neon laser cavity, we obtain the frequency spacing between longitudinal modes. Based on the relationship between the speed of light and the free spectral range frequency, we deduce a value of $2.98 \pm .02 m/s$ for $c$. We analyze the limitations of this particular experiment, specifically the imprecision of the initial length measurements as well as the systematic effects of frequency pulling and pushing.
\end{abstract}

% Each section begins with a \section{title} command
\section{Introduction}

The speed of light is a universal physical constant that bridges the fundamental concepts of space and time. Its value specifies the maximum speed at which matter and information can be propagated in the universe. The precise measurement of the quantity has a storied history, as generations of theoretical and experimental physicists have devoted energy to the problem \cite{measure}  The accepted value today is 299,792,458 m/s, with the modern meter defined to satisfy this relation \cite{speed}.

Measuring the speed of light can be accomplished with a Helium-Neon laser. While lasers are designed to emit coherent light, a given laser cavity has several distinct longitudinal modes associated with it. Light at certain  modal frequencies constructively interfere when reflecting off the cavity's reflecting surfaces, while all other frequencies are suppressed by destructive interference.  When light from multiple modes is projected onto a photodiode, the photocurrent oscillates at the beat frequency, the frequency difference between adjacent modes.

\begin{figure}[!hbt]
		% Center the figure.
		\begin{center}
		% Include the eps file, scale it such that it's width equals the column width. You can also put width=8cm for example...
		\includegraphics[width=\columnwidth]{longitudinal_modes.png}
		% Create a subtitle for the figure.
		\caption{The evenly spaced longitudinal modes of a typical laser. \cite{beat}  As can be seen, laser light must be pass a certain gain threshold in order for multiple modes to be present. Two longitudinal modes are outputted in the laser above.}

		\label{fig:tf_plot}
		\end{center}
	\end{figure}

This beat frequency is the free spectral range of a laser and is functionally related with the speed of light $c$. It is the round-trip time necessary for a beam of light to travel through a resonator of length $L$ and group index of refraction $n$.
$$ \nu_{FSR} =  \frac{c}{2nL} $$
Using this  relation, we can estimate the speed of light by measuring the free spectral range at different cavity lengths.

There are two primary drawbacks to this approach: the imprecision of the initial cavity length and index of refraction measurements. We elaborate on the expected magnitude of these effects and how to circumvent these problems in future experiments.

\section{Experimental Methods}

The central apparatus in the experiment is a 10 mW Helium-Neon laser designed to project photons with wavelengths of 632.8 nm. The laser cavity consists of a mirror on one end and a Brewster window on the opposite end. The Brewster window filters out polarizations orthogonal to the Brewster plane, ensuring that the measured beat frequency will in fact be the free spectral range frequency.

Laser alignment is accomplished on an optical table with the HeNe discharge tube, irises, high-reflector mirrors, and output-coupler mirrors. Our experimental data was produced with a laser power output of .7 mW, measured at the input of the photodiode. The irises were placed in the laser cavity to restrict gain from the center of the optical axis, isolating the $TEM_{00}$ mode. Isolating one transverse mode is necessary because higher-order modes have different beat frequencies that can complicate the measurement.\cite{beat}

The output of the laser is directed into a fast photodiode which communicates with an RF frequency spectrum analyzer.  For every iteration of length, the location of the beat frequency is noted. We fit the experimental data with the functional relationship between the speed of light, resonator length, and free spectral range.

\section{Uncertainty Analysis and Results}

Figure 2 represents measurement data for 27 cavity lengths.  The uncertainty in our initial cavity length $ L $  was $\pm .5$ cm, determined by uncertain locations of the ends of the laser cavity. The uncertainty in our length increment $\Delta L$ was minuscule in comparison, $\pm .01 $ cm, thanks to the high precision of the micrometer.

Unlike $L$ and $\Delta L$, uncertainty in the beat frequency $\nu$ was not dominated by measurement errors, but rather frequency pushing and pulling. Frequency pulling occurs when the index of refraction in the laser cavity effects each transverse mode differently (it is lower for frequencies beneath the resonance transition and higher for frequencies above it)\cite{beat}. It effectively decreases the beat frequency between modes and the free spectral range as modes are pulled towards the center of the gain curve. Fluctuations from this effect are about .03 to .04 MHz\cite{pulling}. A way to address frequency pulling in future experiments is to use a Fabry-Perot interferometer to equalize the intensities of the laser's longitudinal modes. This minimizes the refractive index difference between longitudinal modes, and reduces the pulling effect.

Frequency pushing, on the other hand, is related with the total gain in the laser cavity. In general, as gain in the laser cavity is increased, the beat frequency measured on the photodiode increases \cite{beat}. This systematic error manifests when variations in the voltage from the power supply effect the behavior of the oscillator. Fluctuations due to this effect are about .01 MHz for small $\pm 10 \%$ variations in total laser amplitude.  Like frequency pulling, frequency pushing can be minimized with the use of a  Fabry-Perot interferometer to ensure that the two longitudinal modes measured are at the same total amplitude.

Our best estimate of the magnitude of frequency pulling and pushing effects, $\pm .05$ MHz, masks out the measurement uncertainty of $\pm .001$ MHz from the spectrum analyzer.
\begin{figure}[!hbt]
		% Center the figure.
		\begin{center}
		% Include the eps file, scale it such that it's width equals the column width. You can also put width=8cm for example...
		\includegraphics[width=\columnwidth]{figure_2.png}
		% Create a subtitle for the figure.
		\caption{The frequency locations for 26 different cavity lengths. Error bars are too small to display on this scale; the measurement uncertainty in frequency is about $.001 $ MHz, dictated by the scale of the spectrum analyzer.}

		\label{fig:tf_plot}
		\end{center}
	\end{figure}

Another source of systematic error originates from the environment of the laser. Minor issues with signal consistency were observed due to the dust and air particles. While the effect was small in still air, small breezes would perturb our data significantly. This uncertainty affected the amplitude of our free spectral range measurements, but should have a much smaller effect on the actual frequencies measured as compared to the $\pm .05$ MHz uncertainty from frequency pushing and pulling. With no vacuum chamber, we resorted to shielding the laser beam path with makeshift covers.

\begin{figure}[!hbt]
		% Center the figure.
		\begin{center}
		% Include the eps file, scale it such that it's width equals the column width. You can also put width=8cm for example...
		\includegraphics[width=\columnwidth]{deltaLbeat.png}
		% Create a subtitle for the figure.
		\caption{Length increments $\Delta L$ in relation to inverse beat frequency $\frac{1}{\nu_{FSR}}$ with a linear regression plotted over the data points. Error bars are too small to see at this scale. The slope of this line is $ 1.490 \pm .009 \times 10^8 m/s$  with uncertainties derived from the linear regression.}

		\label{fig:tf_plot}
		\end{center}
	\end{figure}
Figure 3 shows the graphed relationship between length increments $\Delta L $ and inverse beat frequency $\frac{1}{\nu_{FSR}}$.  According to the free spectral range equation, the slope of this line is $c/2n$. By modeling the index of refraction contribution as mostly from air ($n_{air} = 1.00027$), we obtain the speed of light:

$$c = 2.98 \pm .02 m/s $$
\section{Conclusion}

Two limits on precision remain after the analysis outlined in this experiment. The length of the entire laser cavity $L_{HeNe}$ and the index of refraction of the material constituting the cavity $n_{HeNe}$ are both relevant when considering higher order $TEM_{ij}$ modes and the pattern of beat frequencies produced. The allowed frequencies for higher order modes can be summarized in the following equation \cite{beat}:

$$f_{Nij} = \frac{c}{2L} [N + \frac{1}{\pi}(i + j + 1 )cos^{-1}(\sqrt{g_{1}g_{2}})]$$
where $g_{1}g_{2}$ represents the resonator's stability. While this particular experiment utilized the $TEM_{00}$ modes, and could therefore relegate the $L_{HeNe}$ and $n_{HeNe}$ terms to the y-intercept of the $\Delta L$ versus $\frac{1}{\nu_{FSR}}$ graph,  precise measurements of these unknown parameters would be a natural evolution of a further exploration into the speed of light.

\begin{thebibliography}{5}

	\bibitem{measure} % Conference paper
	K. M. Evenson, J.S. Wells, F.R. Peterson, B.L. Danielson \& G.W. Day, {\em Speed of Light from Direct Frequency and Wavelength Measurements of the Methane-Stabilized Laser.} Phys. Rev. Lett. 29, 1346 – Published 6 November 1972.

    \bibitem{modes}
    Dean L. Farnham, Robert S. Van Dyck, Jr., and Paul B. {\em Determination of the Electron's Atomic Mass and the Proton/Electron Mass Ratio via Penning Trap Mass Spectroscopy.} Schwinberg
	Phys. Rev. Lett. 75, 3598

    \bibitem{beat} % Transaction paper
	Daniel J. D'Orazio et. al, “Measuring the speed of light using beating longitudinal modes in an open-cavity HeNe laser” Appl. Opt. 3(2), 267–275 (1964).

	\bibitem{speed} %
    NIST Reference on Constants, Units, and Uncertainties, 		      																	 http://physics.nist.gov/cuu/Units/meter.html.

    \bibitem{pulling} %
    A. M. Lindberg, Mode frequency pulling in He-Ne
    Lasers," Am. J. Phys. 67, 350-353 (1999).

\end{thebibliography}

\end{document}