%!TEX root = ../thesis.tex
% ******************************* Thesis Appendix B ********************************

\ifpdf
    \graphicspath{{Appendix2/Figs/Raster/}{Appendix2/Figs/PDF/}{Appendix2/Figs/}}
\else
    \graphicspath{{Appendix2/Figs/Vector/}{Appendix2/Figs/}}
\fi

\chapter{Math proofs and other ideas}

\section{Math} % (fold)
\label{sec:math}

This section develops what I understand so far about the theory.
It is also an opportunity to gather all the quantities at the same place to give them unique names and notations.

For now I skip the latent variables that describes what happen between the collision and the measurement.
For now I also skip the trigger because it only add more complexity to the problem without fundamental changes.

\subsection{Notation and context} % (fold)
\label{sub:notation_and_context}

Notation :
\begin{itemize}
	\item $\alpha$ = nuisance parameter
	\item $\beta$ = poisson law parameter for the number of backgrounds
	\item $\gamma$ = Poisson law parameter for the number of signals according to current theory
	\item $\mu$ = deviation from the expected number of signals from theory
	\item $z$ or $z_i$ = latent variables
	\item $n$ = number of event
	\item $s$ = number of signals
	\item $b$ = number of backgrounds
	\item p(x) = probability density of an event $x$
	\item p(x|S) = conditional probability density of an event $x$ knowing it is a signal ie signal likelihood
	\item p(x|B) = conditional probability density of an event $x$ knowing it is a background ie background likelihood
\end{itemize}

Let's consider a counting exepriment in a collider.
Inside the collider two bunch of particles are meeting at some point.
From this meeting two types of collisions are possible :
\begin{itemize}
	\item Soft collision which does not produce interesting physics
	\item Hard collision which produces interesting physics called \textbf{event}
\end{itemize}

The hard collisions (events) are way rarer than the soft one.
The process creating the high energy particles is fundamentally stochastic.
Meaning that the nature and properties (eg. kinematics) of the process producing particles are not fully predictable but follow probability distributions whose shapes and properties are deterministics.

The data can be modeled by counting iid Bernouilli rare process which is well approximated by a Poisson distribution \needcite (cf wikipedia) when the number of samples is large.
\begin{equation}
	n \sim Poisson(\text{some parameter})
\end{equation}

The events are splitted into 2 categories : the signal events and the background events.
The signal events are events generated with a peculiar process (e.g. $H\to \tau^+ \tau^-$) that we want to study.
The backgroud events gather all the other processes.
The expected amount of background events $\beta$ is known (with high precision) thanks to previous studies/experiments and to measurements made in the data space where few or no signals can be found.

\subsection{Estimate mu in a simple case} % (fold)
\label{sub:estimate_mu_in_a_simple_case}

Let's first write everything without nuisance parameters.

The total number of events is the sum of signal events and background events.
\begin{equation}
	n = s + b
\end{equation}
The number of signal events is following a poisson distribution of parameter $\mu \gamma$.
\begin{equation}
	s \sim Poisson(\mu \gamma)
\end{equation}
The number of background events is following a poisson distribution of parameter $\beta$.
\begin{equation}
	b \sim Poisson(\beta)
\end{equation}
Since the sum of Poisson random variables is also Poisson distributed \needcite (cf wikipedia) :
\begin{equation}
	n \sim Poisson(\mu \gamma + \beta)
\end{equation}

The objective of the study is to infer $\mu$ from data.
We are using the maximum likelihood estimator.
\begin{equation}
    \hat \mu = \argmax_\mu p(n|\mu) =  \argmax_\mu Poisson(n|\mu)
\end{equation}
This can be done by hand. Assuming that we know $\gamma$ and  $\beta$.
\begin{equation}
    \frac{\partial \log p(n|\mu)}{\partial \mu} =  \gamma \frac{n}{\mu\gamma + \beta} - \gamma
\end{equation}
\begin{equation}
    \frac{\partial \log p(n|\mu)}{\partial \mu} = 0 \iff \mu = \frac{n-\beta}{\gamma}
\end{equation}
Note $\EE[n] = \mu\gamma + \beta$ and $\max Poisson = \EE[Poisson]$ so maybe there is a simpler way to find this ?

\subsection{Binning} % (fold)
\label{sub:binning}

The objective of this document is to clarify a few questions :
\begin{itemize}
	\item What is the link between the probability density of an event $p(x)$ and the number of events $n$ ?
	\item Why is it better to divide the data space into bins ?
	\begin{itemize}
		\item What is the link between signal purity (ie maximum $s/b$) and the variance of the estimator of $\mu$ ?
		\item Is it better to keep only the bin with maximum signal purity (ie maximum $s/b$) ?
		\item Is signal purity $s/b$ or $\gamma / \beta$ ? Are those definition equivalent ?
	\end{itemize}
\end{itemize}

Let's start with the link between the number of events ($s$, $b$, $n$) and the number of events in a bin ($s_i$, $b_i$, $n_i$).
A bin is a subspace of the data space where the event can appear.
We define $\Omega_i$ the subspace of the i-th bin and $\Omega_{tot} = \bigcup_i^K \Omega_i $ the full data space as the union of my $K$ bins.
Begining with the obvious quantity :
\begin{equation}
	n = |D| = \sum_{x\in D} 1
\end{equation}
The number of events in a bin is simply the fraction of events that falls inside a bin.
\begin{equation}
	n_i = \sum_{x\in D} \mathbbm{1} [x\in \Omega_i]
\end{equation}

On average (or in the infinite sample case) an event falls inside the i-th bin with probability $p(x \in \Omega_i)$ defined as:
\begin{equation}
	p(x \in \Omega_i) = \int_{x \in \Omega_i} p(x) dx
\end{equation}
Leading to :
\begin{equation}
	\label{eq:nb_events_in_bin}
	\EE [n_i] = \EE [n] \times p(x \in \Omega_i)  = \EE[n] \times \int_{x \in \Omega_i} p(x) dx
\end{equation}
Let's keep pushing open doors for clarity.
An event is either a signal event $S$ or a background event $B$.
Therefore we can decompose the density into two parts :
\begin{equation}
	p(x) = p(x|S) p(S) + p(x|B) p(B)
\end{equation}
Injecting this into \autoref{eq:nb_events_in_bin}
\begin{align}
	\EE [n_i] &= \EE[n] \times \int_{x \in \Omega_i} p(x) dx \\
	\EE [n_i] &= \EE[n] \times \int_{x \in \Omega_i} p(x|S) p(S) + p(x|B) p(B) dx \\
	\EE [n_i] &= \EE[n] \times \left ( \int_{x \in \Omega_i} p(x|S) p(S) dx + \int_{x \in \Omega_i}  p(x|B) p(B) dx \right )\\
	\EE [n_i] &= \EE[n] \times \int_{x \in \Omega_i} p(x|S) p(S) dx \quad+\quad \EE[n] \times \int_{x \in \Omega_i}  p(x|B) p(B) dx \\
	\EE [n_i] &= \EE[n] \times p(S) \int_{x \in \Omega_i} p(x|S) dx \quad+\quad \EE[n] \times p(B) \int_{x \in \Omega_i}  p(x|B) dx \\
	\EE [n_i] &= \EE[s] \int_{x \in \Omega_i} p(x|S) dx \quad+\quad \EE[b] \int_{x \in \Omega_i}  p(x|B) p(B) dx \\
\end{align}
We use $\EE[s] = \EE[n] \times p(S)$ and $\EE[b] = \EE[n] \times p(B)$ which is again obvious statement since the expected number of signal events is the expected total number of event times the probability of an event to be a signal.
Replacing expenctancies of the number of events with their values according to our Poisson parameter
\begin{equation}
	\EE[n_i] = (\mu\gamma + \beta) \times \int_{x \in \Omega_i} p(x) dx
\end{equation}
\begin{equation}
	\EE[s_i] = \mu\gamma \times \int_{x \in \Omega_i} p(x|S) dx
\end{equation}
\begin{equation}
	\EE[b_i] = \beta \times \int_{x \in \Omega_i} p(x|B) dx
\end{equation}

The answer to the question : what is the link between $p(x)$ and $n$ is \autoref{eq:nb_events_in_bin}.

\subsection{Link between bins and classifier and variance of MLE} % (fold)
\label{sub:link_between_bins_and_classifier_and_variance_of_mle}

Here we explore the reasons for dividing the data space into bins and also for the use of classifer to produce space region very rich in signal events.

\subsubsection{Why making the estimation in a signal richer region is better ?}

There is 2 arguments.
The first one is that we want the likelihood to be narrow near the maximum.
The likelihood function $L(\mu) : \mu \to p(n|\mu) = Poisson(n|\mu\gamma + \beta)$ is narrow near the maximum if the second derivative is large in absolute value near the maximum (or the second derivative of the log).
\begin{equation}
    \frac{\partial^2 \log L(\mu)}{\partial \mu^2} = -\frac{n \gamma^2}{(\mu\gamma + \beta)^2}
\end{equation}
\begin{equation}
    \left . \frac{\partial^2 \log L(\mu)}{\partial \mu^2}  \right |_{\mu=\hat \mu = \frac{n-\beta}{\gamma}} = -\frac{\gamma^2}{n}
\end{equation}

So maximizing the second derivative requires to maximize $\gamma^2 / n$ ie maximize the ratio between expected number of signals and the total number of event.

The second argument is from the Cramer-Rao bound\needcite.
\begin{equation}
    \VV[\hat \mu] \geq \frac{1}{I(\mu)}
\end{equation}
With $N$ the number of independent observation (is $N$ the number of bins ?) and $I(\mu)$ the Fisher information.

\begin{equation}
    I(\mu) = - N \times \EE\left [\frac{\partial^2 \log L(\mu)}{\partial \mu^2} \right ]
\end{equation}
giving
\begin{equation}
    I(\mu) = - N \times \EE\left [-\frac{n \gamma^2}{(\mu\gamma + \beta)^2} \right ] = N \times \frac{\gamma^2}{\mu\gamma + \beta}
\end{equation}
\begin{equation}
    \VV[\hat \mu] \geq  \frac{\mu\gamma + \beta}{N\gamma^2} = \frac{\EE[n]}{N\gamma^2}
\end{equation}
Again minimizing the variance requires to either increase the number of independent observation or maximizing the extected number of signals.

This observation motivates the use of a signal vs background classifier to select a richer region of signals.

\subsubsection{Why dividing the data space into smaller regions ?}

The intuition is that there is more knowledge in knowing that $n_1 = 10$, $n_2 = 7$ and $n_3 = 3$ than knowing that $\sum_i n_i = 20$.
As long as the bins contains enough samples to keep the Poisson approximation then it is better to have more (ie smaller) bins.

The objective is to get a narrow likelihood.
\begin{equation}
    p(n | \mu) = Poisson(n | \mu (\sum_i \gamma_i) + (\sum_i \beta_i))
\end{equation}
\begin{equation}
    p({n_i} | \mu) = \prod_i Poisson(n_i | \mu \gamma_i + \beta_i)
\end{equation}

The second one is getting narrower as we increase the number of bins (multiply numbers $ 1 \leq $).

Also from discussion :
\begin{equation}
    p({n_i} | \mu) = \prod_i \int_{\gamma_i, \beta_i} Poisson(n_i | \mu \gamma_i + \beta_i) p(\gamma_i, \beta_i)
\end{equation}
But this is assuming $\gamma$ and $\beta$ are nuisance parameter.
We can be more precise with $\gamma(\alpha)$ and $\beta(\alpha)$.

\subsection{Other questions and remarks} % (fold)
\label{sub:other_questions_and_remarks}

\subsubsection{Is Poisson approximation still accurate in small bins ?} % (fold)

We assume that the number of events is following a Poisson distribution.
This is very accurate in the large sample limit which is the case at the LHC (huge amount of events).

The same goes for the number of signal events.
But I recall that when taking into account the nuisance parameters and only selecting one very pure bin there is only a few dozen events.
We are far from the large sample limit !

The same argument applies for selecting many bins and using the Poisson likelihood in each one.

First answer : the Poisson counting is not on the signal but on the signal + background.

So the relevant question is : is there enough events (signal + background) in each bin to consider the Poisson approximation accurate ?

\section{Proof}
\label{sec:proof}

\subsection{Binned likelihood narrower}

This section gives the details of the proof that the binned likelihood is narrower than the non binned one.
\begin{equation}
    L_0 = \frac{(\mu \gamma + \beta)^n }{n!} e^{-(\mu \gamma + \beta)}
\end{equation}
with $n = \sum_{i=1}^K n_i $, $\gamma = \sum_{i=1}^K \gamma_i $ and $\beta = \sum_{i=1}^K \beta_i $.
\begin{equation}
    L_K = \prod_{i=1}^M \frac{(\mu \gamma_i + \beta_i)^{n_i} }{n_i!} e^{-(\mu \gamma_i + \beta_i)}
\end{equation}
Now there is 2 option that I can see :  first prove that $L_0 > L_K$ which seems possible with concavity of the log function and $\log L_0 > \log L_k$;
second is to use the Cramer-Rao bound of the variance and prove that
$\left (\frac{\partial^2 \log L_0}{\partial \mu^2}\right )^{-1} > \left (\frac{\partial^2 \log L_K}{\partial \mu^2}\right )^{-1}$
which also seems possible with concave or convex argument somewhere.

Proof that $L_0 > L_K$ does not seems to be what we should be looking for.

\subsection{Pure signals bin is better}

The question is : how to optimize the choice of space regions $\Omega_i$ in order to improve inference.
The expected answer is : using a histogram (or something like it) on the score of a classifier trained to separate signals and backgrounds.

\victor{TODO NEED HELP : How do we prove it ? Where are the papers about it ?}

Let $\hmu$ be a unbiased estimator.
The Cramer-Rao bound for a scalar parameter states that :
\begin{equation}
    \VV[\hmu] \geq \left ( \EE \left [ \frac{\partial^2 log \mathcal L}{\partial \mu^2}  \right ] \right ) ^{-1}
\end{equation}
where $\mathcal L$ is the likelihood function.

In our problem with a binned poisson likelihood (see \autoref{eq:binned_poisson_likelihood})
\begin{equation}
    \EE \left [ \frac{\partial^2 log \mathcal L}{\partial \mu^2}  \right ] = \EE \left[ \sum_{i=1}^K n_i \frac{\gamma_i^2}{(\mu \gamma_i + \beta_i)^2}  \right ]
\end{equation}
\begin{equation}
    \EE \left [ \frac{\partial^2 log \mathcal L}{\partial \mu^2}  \right ] = \sum_{i=1}^K \EE[n_i] \frac{\gamma_i^2}{(\mu \gamma_i + \beta_i)^2}
\end{equation}
recall that $\EE[n_i] = \mu \gamma_i + \beta_i$ because $n_i \sim Poisson(\mu \gamma_i + \beta_i)$
\begin{equation}
    \EE \left [ \frac{\partial^2 log \mathcal L}{\partial \mu^2}  \right ] = \sum_{i=1}^K  \gamma_i^2 \frac{\EE[n_i]}{\EE[n_i]^2}
\end{equation}
\begin{equation}
    \EE \left [ \frac{\partial^2 log \mathcal L}{\partial \mu^2}  \right ] = \sum_{i=1}^K  \frac{\gamma_i^2}{\EE[n_i]} = \sum_{i=1}^K  \frac{\EE[s_i]^2}{\EE[n_i]}
\end{equation}

\textbf{Small note} : In the one bin case with the old notation using $s$ and $b$ as the expected number of signals ad background we find back :
\begin{equation}
    \VV[\hmu] \geq \frac{s+b}{s^2}
\end{equation}
Hence the estimator of the standard deviation :
\begin{equation}
    \frac{1}{\hshmu} = \sqrt{\frac{s^2}{s+b}} = \frac{s}{\sqrt{s+b}}
\end{equation}
So I feel that I am going in the right direction. \textbf{End of small note}.

In order to minimize the variance (or its lower bound at least) we need to maximise $\EE \left [ \frac{\partial^2 log \mathcal L}{\partial \mu}  \right ]$ \ie
\begin{equation}
    \min \left [ \sum_{i=1}^K  \frac{\gamma_i^2}{\EE[n_i]} \right ] ^{-1} = \max \sum_{i=1}^K  \frac{\gamma_i^2}{\EE[n_i]} = \max \sum_{i=1}^K  \frac{\gamma_i^2}{(\mu \gamma_i + \beta_i)}
\end{equation}

With $\sum_{i=1}^K \gamma_i = \gamma$ and $\sum_{i=1}^K \beta_i = \beta$ fixed.
This can be seen as a budget $\gamma$ and $\beta$ and we are looking for a good or the best repartition of this budget in our bins.

\textbf{Informally} : it is better to put all of our (expected signals) $\gamma$ in one bin \ie $\gamma_1 = \gamma$ and $\gamma_{>1} = 0$ and all the (expected backgrounds) $\beta$ in another bin $\beta_2 = \beta$ and $\beta_{\neq 2} = 0$.
Giving :
\begin{equation}
	\sum_{i=1}^K  \frac{\gamma_i^2}{(\mu \gamma_i + \beta_i)} = \frac{\gamma^2}{\mu \gamma + 0} + \frac{0}{0 +\beta} = \frac{\gamma}{\mu}
\end{equation}
Any other repartition of the budget is smaller ?
\begin{equation}
	\sum_{i=1}^K  \frac{\gamma_i^2}{(\mu \gamma_i + \beta_i)} = \frac{\gamma_0^2}{\mu \gamma_0 + \beta_0} + \frac{\gamma_1^2}{\mu \gamma_1 +\beta_1} \leq \frac{\gamma}{\mu}
\end{equation}
I believe it's true because $f : x \to x^2$ is a convex function so $\gamma_0^2 + \gamma_1^2 \leq (\gamma_0 + \gamma_1)^2 $.

And here I'm stuck.
Assuming that my intuition is right : How do I provide a clean proof that this is the best choice ?

The usefull property of a classifier is to separate signals and backgrounds.
Meaning we can have one one very pure bin of signals using a classifier or at least it is good way of providing pure bins.

\section{More statistics} % (fold)
\label{sec:more_statistics}

\victor{Il faudra virer cette section après. Mais elle va me servir à explorer les maths}

\subsection{Why a classifier} % (fold)
\label{sub:why_a_classifier}

Aishik asks : Why do we use a classifer to make estimation or hypothesis test ?

The main connection is that for a Bayes optimal classifier \needcite :
\begin{equation}
	c(x) = \frac{f_s p(x|S)}{(1-f_s) p(x|B) + f_s p(x|S)}
\end{equation}
The classifier score $c(x)$ is linked to the likelihood of a event $x$ assuming it is signal $ p(x|S)$ or assuming it is a background $p(x|B)$.
$f_s$ is the fraction of signal events in the training set.
Usually $f_s = 0.5$ in which can is simplifies itself.

Note that the ratio between signal and event likelihood can easily extracted :
\begin{equation}
	\frac{p(x|S)}{p(x|B)} = \frac{c(x)}{(1-c(x))} \frac{(1-n_s)}{n_s}
\end{equation}

So if you can connect what you are doing to $\frac{p(x|S)}{p(x|B)}$ then you can use a classifier to do the job.

Neal's paper \cite{Neal:2007zz} is connecting this ratio to parameter estimation.

The connection with hypothesis testing is also possible.
\textbf{I'll try to give it a shot here but I may do mistakes or forgot to mention required hypothesis or approximation that I am unaware of.}

The probability density of an event is :
\begin{equation}
 	p(x) = p(x|S) p(S) + p(x|B) p(B)
\end{equation}
Where $p(S)$ and $p(B)$ are the probability to be a signal or a background event and can be viewed as mixture coefficients.

Data are \emph{independent and identically distributed}.
Meaning we can write the density of the full data $D = \{x_i\}_{i=1}^N$
\begin{equation}
	p(D) = \prod_{x\in D} p(x) = \prod_{x\in D} p(x|S) p(S) + p(x|B) p(B)
\end{equation}

We now consider the 2 hypothesis.
The null hypothesis assumes that there is no signal meaning that $p(S) = 0$ and $p(B) = 1$
\begin{equation}
	p(D |H_0) = \prod_{x\in D} p(x|B)
\end{equation}
The alternative hypothesis is tha there is some signal $p(S) > 0$ and $p(B) < 1$
\begin{equation}
	p(D | H_1) = \prod_{x\in D} p(x|S) p(S) + p(x|B) p(B)
\end{equation}

According to Neyman-Pearson lemma \cite{neyman_pearson_1933} the likelihood ratio between the 2 hypothesis is a sufficient summary statistic for the hypothesis test (or the most powerfull quantity one can compute).
\begin{align}
	\lambda & = \frac{p(D | H_1)}{p(D |H_0)} = \frac{\prod_{x\in D} p(x|S) p(S) + p(x|B) p(B)}{\prod_{x\in D} p(x|B)} \\
	\lambda & = \prod_{x\in D} \frac{ p(x|S) p(S) + p(x|B) p(B)}{p(x|B)} \\
	\lambda & = \prod_{x\in D}  \frac{p(x|S)}{p(x|B)}  p(S) + p(B)
\end{align}
We find back our ratio $\frac{p(x|S)}{p(x|B)}$ that can be computed with a classifier.
$p(S)$ and $p(B)$ are constant and does not depend of the data...

This demo skecth is very similar to what is done in Neal's paper.
But I think that the reason for using a classifier score to do the dimension reduction is conected to this demo sketch if not exactly it.

Papers I still need to master :
\begin{itemize}
 	\item \url{https://arxiv.org/pdf/physics/9711021.pdf} The Unified approach to build confidence interval. It explains why the profiled likelihood ratio method is the prefered way to build a confience interval. Note to self : do not get angry at the useless comments about why one should be frequentists or bayesians...
 	\item \url{https://arxiv.org/pdf/1007.1727.pdf} Describes in details hypothesis test. It does not tackle the problem of Monte Carlo methods taking ages to converge when the dimensionality of the data is large. SO nothing on classification but still I believe I should know these things.
 	\item \url{https://arxiv.org/pdf/1506.02169.pdf} Describes the link between classification and likelihood ratio tests. But it is a recent one (2015) and Higgs discovery (2012) was already using classifier so I guess this is not the original paper on why classification is nice.
 \end{itemize}

About Bayesian vs Frequentist stupid war.
The section 2 of Neal's paper is criticising Feldman and Cousins' paper on the Unified approach.
So according to those folks if you want a Bayesian approach to confidence interval you do marginalization like Neal.
But if you want a Frequentist approach you use the profiled likelihood ratio method.

A scientist may consider that we need experimental evidence to choose between one or another.
Then you can read : \url{https://arxiv.org/pdf/physics/0403059.pdf} for coverage experiment of the Unified method.
And read \url{https://arxiv.org/pdf/physics/0408039.pdf} for coverage of the Bayesiand approach.
I read only introduction and conclusion of these papers and they state that Frequentist method leads to a small under-coverage and the Bayesian approach leads to a small over-coverage.

My conclusion is that the only way of doing good statistics is to understand and master both methods.
Test them on a problem and finally be at peace because both are giving bassically the same numbers at the end...


\documentclass[11pt,]{article}
\usepackage{lmodern}
\usepackage{amssymb,amsmath}
\usepackage{ifxetex,ifluatex}
\usepackage{fixltx2e} % provides \textsubscript
\ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex
  \usepackage[T1]{fontenc}
  \usepackage[utf8]{inputenc}
\else % if luatex or xelatex
  \ifxetex
    \usepackage{mathspec}
  \else
    \usepackage{fontspec}
  \fi
  \defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase}
\fi
% use upquote if available, for straight quotes in verbatim environments
\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
% use microtype if available
\IfFileExists{microtype.sty}{%
\usepackage{microtype}
\UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
}{}
\usepackage[margin=1in]{geometry}
\usepackage{hyperref}
\hypersetup{unicode=true,
            pdfborder={0 0 0},
            breaklinks=true}
\urlstyle{same}  % don't use monospace font for urls
\usepackage{graphicx}
% grffile has become a legacy package: https://ctan.org/pkg/grffile
\IfFileExists{grffile.sty}{%
\usepackage{grffile}
}{}
\makeatletter
\def\maxwidth{\ifdim\Gin@nat@width>\linewidth\linewidth\else\Gin@nat@width\fi}
\def\maxheight{\ifdim\Gin@nat@height>\textheight\textheight\else\Gin@nat@height\fi}
\makeatother
% Scale images if necessary, so that they will not overflow the page
% margins by default, and it is still possible to overwrite the defaults
% using explicit options in \includegraphics[width, height, ...]{}
\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio}
\IfFileExists{parskip.sty}{%
\usepackage{parskip}
}{% else
\setlength{\parindent}{0pt}
\setlength{\parskip}{6pt plus 2pt minus 1pt}
}
\setlength{\emergencystretch}{3em}  % prevent overfull lines
\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
\setcounter{secnumdepth}{0}
% Redefines (sub)paragraphs to behave more like sections
\ifx\paragraph\undefined\else
\let\oldparagraph\paragraph
\renewcommand{\paragraph}[1]{\oldparagraph{#1}\mbox{}}
\fi
\ifx\subparagraph\undefined\else
\let\oldsubparagraph\subparagraph
\renewcommand{\subparagraph}[1]{\oldsubparagraph{#1}\mbox{}}
\fi

%%% Use protect on footnotes to avoid problems with footnotes in titles
\let\rmarkdownfootnote\footnote%
\def\footnote{\protect\rmarkdownfootnote}

%%% Change title format to be more compact
\usepackage{titling}

% Create subtitle command for use in maketitle
\providecommand{\subtitle}[1]{
  \posttitle{
    \begin{center}\large#1\end{center}
    }
}

\setlength{\droptitle}{-2em}

  \title{}
    \pretitle{\vspace{\droptitle}}
  \posttitle{}
    \author{}
    \preauthor{}\postauthor{}
    \date{}
    \predate{}\postdate{}

\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhead[L]{MATH 1\\ Section \# 101}
\fancyhead[R]{Name: \underline{\hspace{1.5 in}}\\Jan 1 2001}
\usepackage{mathpazo}
\usepackage{mdframed}
\newmdenv[linecolor=black]{sol}
\usepackage{comment}
\specialcomment{answer}{\begingroup\hspace{1pt}\begin{sol}\sffamily\small\textbf{Answer}}{\end{sol}\endgroup}

\begin{document}

\hypertarget{quiz-1}{%
\section{Quiz \# 1}\label{quiz-1}}

\hypertarget{truefalse}{%
\subsection{True/False}\label{truefalse}}

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  \(5-3=1\)
\end{enumerate}

\begin{answer}

False.

\end{answer}

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\setcounter{enumi}{1}
\tightlist
\item
  \(2+4=6\)
\end{enumerate}

\begin{answer}

True.

\end{answer}

\end{document}


% Thank you Josh Davis for this template!
% https://github.com/jdavis/latex-homework-template/blob/master/homework.tex

\documentclass{article}

\newcommand{\hmwkTitle}{Final Report}

% \input{ShortcutsStatistics}

% ----------

% Packages

\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{amsfonts}
\usepackage{tikz}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}
\usepackage{enumitem}
\usepackage{chngcntr}
\usepackage{blkarray}

% Libraries

\usetikzlibrary{automata, positioning, arrows}

%
% Basic Document Settings
%

\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\pagestyle{fancy}
\lhead{\hmwkAuthorName}
\chead{}
\rhead{\hmwkClass\ (\hmwkClassInstructor): \hmwkTitle}
\lfoot{\lastxmark}
\cfoot{\thepage}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

\setlength\parindent{0pt}
\setcounter{secnumdepth}{0}

\newcommand{\hmwkClass}{MATH 5345 / Regression Analysis}        % Class
\newcommand{\hmwkClassInstructor}{Dr. Sun}           % Instructor
\newcommand{\hmwkAuthorName}{\textbf{Gabriela Lara and Joshua Mitchell}} % Author

%
% Title Page
%

\title{
    \vspace{2in}
    \textmd{\textbf{\hmwkClass:\ \hmwkTitle}}\\
    \normalsize\vspace{0.1in}\small\vspace{0.1in}\large{\textit{\hmwkClassInstructor}}
    \vspace{3in}
}

\author{\hmwkAuthorName}
\date{}

\renewcommand{\part}[1]{\textbf{\large Part \Alph{partCounter}}\stepcounter{partCounter}\\}

% Integral dx
\newcommand{\dx}{\mathrm{d}x}

%
% Various Helper Commands
%

% For derivatives
\newcommand{\deriv}[1]{\frac{\mathrm{d}}{\mathrm{d}x} (#1)}

% For partial derivatives
\newcommand{\pderiv}[2]{\frac{\partial}{\partial #1} (#2)}

% Alias for the Solution section header
\newcommand{\solution}{\textbf{\large Solution}}

% Formatting commands:

\newcommand{\mt}[1]{\ensuremath{#1}}
\newcommand{\nm}[1]{\textrm{#1}}

\newcommand\bsc[2][\DefaultOpt]{%
  \def\DefaultOpt{#2}%
  \section[#1]{#2}%
}
\newcommand\ssc[2][\DefaultOpt]{%
  \def\DefaultOpt{#2}%
  \subsection[#1]{#2}%
}
\newcommand{\bgpf}{\begin{proof} $ $\newline}

\newcommand{\bgeq}{\begin{equation*}}
\newcommand{\eeq}{\end{equation*}}

\newcommand{\balist}{\begin{enumerate}[label=\alph*.]}
\newcommand{\elist}{\end{enumerate}}

\newcommand{\bilist}{\begin{enumerate}[label=\roman*)]}

\newcommand{\bgsp}{\begin{split}}
% \newcommand{\esp}{\end{split}} % doesn't work for some reason.

\newcommand\prs[1]{~~~\textbf{(#1)}}

\newcommand{\lt}[1]{\textbf{Let: } #1}
     							   %  if you're setting it to be true
\newcommand{\supp}[1]{\textbf{Suppose: } #1}
     							   %  Suppose (if it'll end up false)
\newcommand{\wts}[1]{\textbf{Want to show: } #1}
     							   %  Want to show
\newcommand{\as}[1]{\textbf{Assume: } #1}
     							   %  if you think it follows from truth
\newcommand{\bpth}[1]{\textbf{(#1)}}

\newcommand{\step}[2]{\begin{equation}\tag{#2}#1\end{equation}}
\newcommand{\epf}{\end{proof}}

\newcommand{\dbs}[3]{\mt{#1_{#2_#3}}}

\newcommand{\sidenote}[1]{-----------------------------------------------------------------Side Note----------------------------------------------------------------
#1 \

---------------------------------------------------------------------------------------------------------------------------------------------}

% Analysis / Logical commands:

\newcommand{\br}{\mt{\mathbb{R}} }       % |R
\newcommand{\bq}{\mt{\mathbb{Q}} }       % |Q
\newcommand{\bn}{\mt{\mathbb{N}} }       % |N
\newcommand{\bc}{\mt{\mathbb{C}} }       % |C
\newcommand{\bz}{\mt{\mathbb{Z}} }       % |Z

\newcommand{\ep}{\mt{\epsilon} }         % epsilon
\newcommand{\fa}{\mt{\forall} }          % for all
\newcommand{\afa}{\mt{\alpha} }
\newcommand{\bta}{\mt{\beta} }
\newcommand{\mem}{\mt{\in} }
\newcommand{\exs}{\mt{\exists} }

\newcommand{\es}{\mt{\emptyset} }        % empty set
\newcommand{\sbs}{\mt{\subset} }         % subset of
\newcommand{\fs}[2]{\{\uw{#1}{1}, \uw{#1}{2}, ... \uw{#1}{#2}\}}

\newcommand{\lra}{ \mt{\longrightarrow} } % implies ----->
\newcommand{\rar}{ \mt{\Rightarrow} }     % implies -->

\newcommand{\lla}{ \mt{\longleftarrow} }  % implies <-----
\newcommand{\lar}{ \mt{\Leftarrow} }      % implies <--

\newcommand{\av}[1]{\mt{|}#1\mt{|}}  % absolute value

\newcommand{\prn}[1]{(#1)}
\newcommand{\bk}[1]{\{#1\}}

\newcommand{\ps}{\mt{\operatorname{+}} }
\newcommand{\ms}{\mt{\operatorname{-}} }

\newcommand{\ls}{\mt{\operatorname{<}} }
\newcommand{\gr}{\mt{\operatorname{>}} }

\newcommand{\lse}{\mt{\operatorname{\leq}} }
\newcommand{\gre}{\mt{\operatorname{\geq}} }

\newcommand{\eql}{ \mt{\operatorname{=}} }

\newcommand{\pr}{\mt{^\prime}} 		   % prime (i.e. R')
\newcommand{\uw}[2]{#1\mt{_{#2}}}
\newcommand{\uf}[2]{#1\mt{^{#2}}}
\newcommand{\frc}[2]{\mt{\frac{#1}{#2}}}
\newcommand{\lmti}[1]{\mt{\displaystyle{\lim_{#1 \to \infty}}}}
\newcommand{\limt}[2]{\mt{\displaystyle{\lim_{#1 \to #2}}}}

\newcommand{\bnm}[2]{\mt{#1\setminus{#2}}}
\newcommand{\bnt}[2]{\mt{\textrm{#1}\setminus{\textrm{#2}}}}
\newcommand{\bi}{\bnm{\mathbb{R}}{\mathbb{Q}}}

\newcommand{\urng}[2]{\mt{\bigcup_{#1}^{#2}}}
\newcommand{\nrng}[2]{\mt{\bigcap_{#1}^{#2}}}
\newcommand{\nck}[2]{\mt{{#1 \choose #2}}}

\newcommand{\nbho}[3]{\textrm{N(}#1, #2\textrm{) }\cap \textrm{ #3} \neq \emptyset}
     							   %  N(x, eps) intersect S \= emptyset
\newcommand{\nbhe}[3]{\textrm{N(}#1, #2\textrm{) }\cap \textrm{ #3} = \emptyset}
     							   %  N(x, eps) intersect S  = emptyset
\newcommand{\dnbho}[3]{\textrm{N*(}#1, #2\textrm{) }\cap \textrm{ #3} \neq \emptyset}
     							   %  N*(x, eps) intersect S \= emptyset
\newcommand{\dnbhe}[3]{\textrm{N*(}#1, #2\textrm{) }\cap \textrm{ #3} = \emptyset}
     							   %  N*(x, eps) intersect S = emptyset

\newcommand{\eqn}[1]{\[#1\]}
\newcommand{\splt}[1]{\begin{split}#1\end{split}}

\newcommand{\txt}[1]{\text{#1}} % Not new command, but remember \text for text in eqns
\newcommand{\tl}{\mt{\thicksim} }
\newcommand{\mn}[1]{\mt{\overline{#1}}}
\newcommand{\sg}{\mt{\sigma}}
\newcommand{\ssq}{\mt{\sigma^2}}

\newcommand{\bh}[1]{\mathbf{\hat{\text{$#1$}}}}
\newcommand{\bth}{\mt{\bh{\beta}}}
\newcommand{\yh}{\mt{\bh{Y}}}

\newcommand{\exv}[1]{\txt{E[}#1\txt{]}}
\newcommand{\vrn}[1]{V[#1]}

\newcommand{\gv}{ \mt{|} }

\newcommand{\cov}[2]{\txt{Cov(#1, #2)}}

\newcommand{\img}[1]{
\begin{figure}[h]
  \includegraphics[width=0.5\linewidth]{#1}
\end{figure}
}
\newcommand{\simg}[1]{
  \includegraphics[width=0.35\linewidth]{#1}
}
\newcommand{\wimg}[1]{
\begin{figure}[h]
  \includegraphics[width=1\linewidth]{#1}
\end{figure}
}
\newcommand\tab[1][1cm]{\hspace*{#1}}
\newcommand{\sumin}[1]{\mt{\sum_{i = 1}^n #1}}

\newcommand{\brm}[1]{\begin{pmatrix} #1 \end{pmatrix}}

\newcommand{\inm}[1]{\mt{\left\[ \begin{smallmatrix} #1 \end{smallmatrix} \right\]}}

\newcommand{\lbm}[4]{
	  \begin{blockarray}{#1} % a c for every row, plus the c-labels
        #2 \\ % & c-label1 & c-label2 & c-label3...
      \begin{block}{c(#3)} % a c for every column only
        #4 % r-label1 & data & data & data ... \\
           % r-label2 & data & data & data ... \\
           % ...
           % r-labeln & data & data & data ... \\
      \end{block}
    \end{blockarray}
}

% Example:
% \[\lbm{ccccc}{& H & y & d}{cccc}{H & 4 & 4 & 4 \\
% Y & 3 & 3 & 3 \\
% D & 2 & 2 & 2 \\
% D & 2 & 2 & 2 \\}\]

\newcommand{\unds}[2]{\mt{\underset{#1}{#2}}} % stuff underneath!
% ----------

\begin{document}

\maketitle

\newpage

\tableofcontents

\listoftables

\listoffigures

\newpage

\section{Abstract}

  A multiple linear regression was calculated to predict the gas mileage of a car (its MPG) based on the number of cylinders, displacement, horsepower, weight, acceleration, model year, and origin of the car. The initial model had problems with non-constant variance and multicollinearity. A transformation on the response variable and an additional interaction regressor was added to improve the homoscedasticity and the explanatory power of the model, resulting in an \uf{R}{2} value of 0.89. Only a subset of the seven regressors and an interaction term turned out to be significant.

\newpage

\section{Introduction}

  The Auto MPG data set, from the UC Irvine Machine Learning Repository, contains 397 samples of various models of vehicles. The goal of the analysis is to learn to predict a vehicle's gas mileage (mpg) given other attributes of the vehicle (e.g. its weight, model year, origin, etc).

  The data contains a total of nine attributes. Three are discrete (origin, model year, and number of cylinders), one is alphanumeric (model name), and the rest are continuous. The model name column was thrown out, and model year and number of cylinders were interpreted as continuous variables.

  The data also had some missing values (6), but they constituted a negligible amount of the data, so they were removed.

\newpage

\section{Discussion of Models and Analysis Results}

\subsection{Original Full Model Analysis}

\textbf{Original Full Model:}

\textbf{mpg (c) \tl wgt (c) + modelyr (mvd) + origin (mvd) + hp (c) + displ (c) + cylnum (mvd) + acc (c)}

\begin{table}[ht]
\centering
\begin{tabular}{rrrrrr}
  \hline
 & Estimate & Std. Error & t value & Pr($>$$|$t$|$) & Significance \\
  \hline
(Intercept) & -18.3106 & 4.6933 & -3.90 & 0.0001 & *** \\
  wgt\_c & -0.0067 & 0.0007 & -10.23 & 0.0000 & ***  \\
  modelyr\_mvd & 0.7805 & 0.0519 & 15.03 & 0.0000 & ***  \\
  origin\_mvd2 & 2.6340 & 0.5665 & 4.65 & 0.0000 & ***  \\
  origin\_mvd3 & 2.8557 & 0.5528 & 5.17 & 0.0000 & ***  \\
  hp\_c & -0.0174 & 0.0137 & -1.27 & 0.2056 & \\
  displ\_c & 0.0241 & 0.0077 & 3.14 & 0.0018 & ** \\
  cylnum\_mvd & -0.5123 & 0.3222 & -1.59 & 0.1126 & \\
  acc\_c & 0.0845 & 0.0984 & 0.86 & 0.3913 & \\
   \hline
\end{tabular}
\caption{R Summary of original full model (relating mpg to the rest)}
\label{tab:myfirsttable}
\end{table}

\begin{table}[ht]
\centering
\begin{tabular}{lrrrrrr}
  \hline
 & Df & Sum Sq & Mean Sq & F value & Pr($>$F) & Significance\\
  \hline
wgt\_c & 1 & 16470.05 & 16470.05 & 1505.92 & 0.0000 & ***  \\
  modelyr\_mvd & 1 & 2756.94 & 2756.94 & 252.08 & 0.0000 & ***  \\
  origin\_mvd & 2 & 261.23 & 130.61 & 11.94 & 0.0000 & ***  \\
  hp\_c & 1 & 8.96 & 8.96 & 0.82 & 0.3659 & \\
  displ\_c & 1 & 77.03 & 77.03 & 7.04 & 0.0083 & ** \\
  cylnum\_mvd & 1 & 29.10 & 29.10 & 2.66 & 0.1037 & \\
  acc\_c & 1 & 8.06 & 8.06 & 0.74 & 0.3913 &  \\
  Residuals & 382 & 4177.89 & 10.94 &  & & \\
   \hline
\end{tabular}
\caption{R ANOVA of original full model (relating mpg to the rest)}
\label{tab:myfirsttable}
\end{table}

\begin{table}[ht]
\centering
\begin{tabular}{rrrr}
  \hline
 & GVIF & Df & GVIF\verb|^|(1/(2*Df)) \\
  \hline
wgt\_c & 11.07 & 1.00 & 3.33 \\
  modelyr\_mvd & 1.30 & 1.00 & 1.14 \\
  origin\_mvd & 2.09 & 2.00 & 1.20 \\
  hp\_c & 9.98 & 1.00 & 3.16 \\
  displ\_c & 22.87 & 1.00 & 4.78 \\
  cylnum\_mvd & 10.74 & 1.00 & 3.28 \\
  acc\_c & 2.62 & 1.00 & 1.62 \\
   \hline
\end{tabular}
\caption{VIF of each regressor in the Full Original Model}
\label{tab:fulloriginalvif}
\end{table}

\clearpage
\newpage

\begin{figure}
	\centering
	\includegraphics[width=1\linewidth]{1p_ScPlMtr}
	\caption[Scatterplot Matrix of the Original Full Model]
	{A scatterplot matrix between all the variables in the automobile data set}
\end{figure}

\clearpage
\newpage

\begin{figure}
	\centering
	\includegraphics[width=1\linewidth]{2-10p_resall}
	\caption[Residual Plots of the Original Full Model]
	{A Residual vs Fitted, a Residual vs R-Norm, and Residual vs Regressors plots of the Original Full Model}
\end{figure}

\clearpage
\newpage

\begin{table}[ht]
\centering
\begin{tabular}{rlrrl}
  \hline
 & Regressor & F\_Statistic & P\_Value & Significance \\
  \hline
1 & Displacement & 9.89 & 0.00 & ** \\
  2 & Weight & 104.63 & 0.00 & *** \\
  3 & HP & 1.61 & 0.21 & none \\
  4 & Cylinder Num & 2.53 & 0.11 & none \\
   \hline
\end{tabular}
\caption{A partial F test on each of the regressors with high VIF scores}
\label{tab:partialfhighvif}
\end{table}

\begin{figure}
	\centering
	\includegraphics[width=1\linewidth]{11-14p_PrRgall4}
	\caption[Partial regression plots on high VIF regressors]
	{Partial regression plots on each of the regressors with high VIF scores}
\end{figure}

\clearpage
\newpage

A full multiple linear regression was fitted on the data set with mileage (mpg) as the response variable and number of cylinders (cylnum), displacement (displ), horsepower (hp), weight (wgt), acceleration (acc), model year (modelyr), and origin (origin) as the predictor variables. The following five key assumptions were examined:

\balist
\item Normality of errors: Based on figure 2, it appears that the residuals are mostly normally distributed with a light tail.
\item Independence of errors: Based on figure 2, our errors might be nontrivially dependent due to the quadratic shape of the Residuals vs Fitted plot.
\item Constant variance of errors: Based on figure 2's Residuals vs Fitted plot, it appears that, due to the inconsistency of the spread of the residuals, constant variance is not upheld.
\item An expected error value of 0: Based on figure 2's Residuals vs Fitted plot, it is likely that this is upheld (due to half the data being roughly above 0 and the other half being below).
\item Linearity: Based on the significance of multiple coefficient estimates for each regressor in Table 1, it appears that our response variable is linearly related to our predictor variables. However, in the scatterplot (Figure 1), some regressors have a linear relationship with the response variable and some don't. In the Residuals vs Fitted plot (Figure 2), it takes a somewhat quadratic shape, but both ways can be argued. There is a gray area, but overall, linearity can be argued for.
\elist

Correlation among variables was also examined. Figure 1 and Table 3 indicate a strong relationship between hp, wgt, displ, and cylnum. Further inspection was done with partial regression plots and a partial f test for each of said variables. At this point, due to the significance of some variables and lack of significance of others from the partial f tests, it was concluded that a log transformation should be done on the response variable (mpg), due to the quadratic pattern in the Residual vs Fitted plot, and possibly an addition of an interaction term.

\clearpage
\newpage

\subsection{Transformed Full Model Analysis}

\textbf{Transformed Full Model:}

\textbf{log(mpg) (c) \tl wgt (c) + modelyr (mvd) + origin (mvd) + hp (c) + displ (c) + cylnum (mvd) + acc (c)}

\begin{figure}[h]
	\centering
	\includegraphics[width=0.95\linewidth]{15-23t_resall}
	\caption[Residual Plots of the Transformed Full Model]
	{A Residual vs Fitted, a Residual vs R-Norm, and Residual vs Regressors plots of the Transformed Full Model}
\end{figure}

\clearpage
\newpage

\begin{table}[ht]
\centering
\begin{tabular}{rlrrl}
  \hline
 & Regressor & F\_Statistic & P\_Value & Significance \\
  \hline
1 & Displacement & 8.39 & 0.00 & ** \\
  2 & Weight & 126.68 & 0.00 & *** \\
  3 & HP & 9.10 & 0.00 & ** \\
  4 & Cylinder Num & 6.35 & 0.01 & * \\
   \hline
\end{tabular}
\caption{A partial F test on each of the regressors with high VIF scores on the Transformed Full Model}
\label{tab:partialfhighvif}
\end{table}

\begin{figure}
	\centering
	\includegraphics[width=1\linewidth]{24-27t_PrRgall4}
	\caption[Partial regression plots on high VIF regressors on the Transformed Full Model]
	{Partial regression plots on each of the regressors with high VIF scores on the Transformed Full Model}
\end{figure}

\clearpage
\newpage

A new model was created with a log transformation on the response variable and then examined. Fortunately, the transformation dampened the quadratic pattern (increasing the independence of the errors) and stabilized the variance of the residuals.

\

However, the partial regression plots and partial f tests were redone with the same 4 regressors with mixed results. The transformation made the insignificant regressors (hp and cylnum) more significant (**, and *, respectively), however, the partial regression plots showed that the explanatory power of these two regressors was not significant.

\clearpage
\newpage

\subsection{Interaction Terms Analysis}

\begin{table}[ht]
\centering
\begin{tabular}{rlrrr}
  \hline
 & Model & R\_Sq & AR\_Sq & MS\_res \\
  \hline
1 & Interaction & 0.88 & 0.87 & 7.90 \\
  2 & Transformed + Interaction & 0.90 & 0.90 & 0.01 \\
   \hline
\end{tabular}
\caption{A chart comparing the Untransformed Full Model with all combinations of interaction terms for the high VIF regressors against the same model with a log transformation on the response variable (mpg)}
\label{tab:myfirsttable}
\end{table}

It was suspected, due to the somewhat significant difference between the partial f tests and partial regression plots of the transformed full model, that there was possible multicollinearity, so it was decided that an interaction term should be added. Due to the number of possible interaction terms, it was decided that all regressors plus all possible combinations of the regressors with high VIFs should be modeled and then compared to the original full model. The addition of these interaction terms resulted in a significant increase in \uf{R}{2}. A plain full model with these interaction terms was also compared to a transformed full model with the same interaction terms, and it was concluded that, given that interaction terms were going to be included, the transformed model was better based on its \uf{R}{2}. In the next section, other model metrics will be considered.

\newpage

\subsection{Model Selection Analysis}

\begin{table}[ht]
\centering
\begin{tabular}{rlrrrr}
  \hline
 & Selection\_Method & Num\_Regressors & R\_Sq & Adj\_R\_Sq & MS\_res \\
  \hline
1 & Forward & 6.00 & 0.89 & 0.89 & 0.01 \\
  2 & Backward & 16.00 & 0.90 & 0.90 & 0.01 \\
  3 & Stepwise & 6.00 & 0.89 & 0.89 & 0.01 \\
   \hline
\end{tabular}
\caption{Statistics about the models outputted from Forward, Backward, and Stepwise Selection algorithms in R (note that the model selected by Forward and Stepwise selection is identical, so just the Forward model will be considered in further sections)}
\label{tab:selectionalgorithmmodelcomparison}
\end{table}

\begin{table}[ht]
\centering
\begin{tabular}{rrrr}
  \hline
 & GVIF & Df & GVIF\verb|^|(1/(2*Df)) \\
  \hline
wgt\_c & 13.83 & 1.00 & 3.72 \\
  modelyr\_mvd & 1.27 & 1.00 & 1.13 \\
  origin\_mvd & 1.74 & 2.00 & 1.15 \\
  hp\_c & 37.47 & 1.00 & 6.12 \\
  acc\_c & 2.61 & 1.00 & 1.62 \\
  wgt\_c:hp\_c & 58.06 & 1.00 & 7.62 \\
   \hline
\end{tabular}
\caption{VIF of each regressor in the Forward Model}
\label{tab:forwardmodelvif}
\end{table}

\begin{table}[ht]
\centering
\begin{tabular}{rrrr}
  \hline
 & GVIF & Df & GVIF\verb|^|(1/(2*Df)) \\
  \hline
wgt\_c & 3110.02 & 1.00 & 55.77 \\
  modelyr\_mvd & 1.44 & 1.00 & 1.20 \\
  origin\_mvd & 3.01 & 2.00 & 1.32 \\
  hp\_c & 2806.06 & 1.00 & 52.97 \\
  displ\_c & 10568.29 & 1.00 & 102.80 \\
  cylnum\_mvd & 975.92 & 1.00 & 31.24 \\
  acc\_c & 3.64 & 1.00 & 1.91 \\
  wgt\_c:hp\_c & 27058.36 & 1.00 & 164.49 \\
  hp\_c:displ\_c & 39680.75 & 1.00 & 199.20 \\
  wgt\_c:displ\_c & 11724.72 & 1.00 & 108.28 \\
  wgt\_c:cylnum\_mvd & 9069.37 & 1.00 & 95.23 \\
  hp\_c:cylnum\_mvd & 9125.25 & 1.00 & 95.53 \\
  displ\_c:cylnum\_mvd & 19861.83 & 1.00 & 140.93 \\
  wgt\_c:hp\_c:cylnum\_mvd & 41867.56 & 1.00 & 204.62 \\
  wgt\_c:displ\_c:cylnum\_mvd & 15077.88 & 1.00 & 122.79 \\
  hp\_c:displ\_c:cylnum\_mvd & 44842.29 & 1.00 & 211.76 \\
   \hline
\end{tabular}
\caption{VIF of each regressor in the Backward Model}
\label{tab:backwardmodelvif}
\end{table}

\clearpage
\newpage

\begin{figure}
	\centering
	\includegraphics[width=1\linewidth]{28ti_resfittednorm}
	\caption[Residuals vs Fitted and vs Random Normal plots for Forward, Backward, and Stepwise Models]
	{Residuals vs Fitted and vs Random Normal plots for Forward, Backward, and Stepwise Models}
\end{figure}

\clearpage
\newpage

In order to select the most significant variables, several tests were applied: Forward, Backwards and Stepwise (Table 7).

\

The Forward and Stepwise selection algorithms both selected a model with 6 variables (5 original variables and 1 interaction term). The Backwards selection algorithm selected a model with 16 variables (mostly interaction terms).

\

The Backward selection model has a slightly higher \uf{R}{2}, but with slightly less degrees of freedom (due to the increased parameters) and possibly more complexity.

\

The VIF values of each regressor of each model were also examined. As expected, the VIF for the known correlated variables was still high, but, more importantly, the other variables showed little correlation.

\

The 5 key assumptions for each model were also checked (see Figure 6) and no major problems were observed. Outliers and influential points are examined in the next section.

\clearpage
\newpage

\subsection{Influential Points Analysis}

\begin{table}[ht]
\centering
\begin{tabular}{rlrlr}
  \hline
 & Model & Num\_Infl\_Pnts & Percent\_Infl\_Pnts & Common\_Infl\_Pnts \\
  \hline
1 & Forward & 20.00 & 5.12\% & 14.00 \\
  2 & Backward & 36.00 & 9.21\% & 14.00 \\
   \hline
\end{tabular}
\caption{Influential point comparison of Forward Model vs Backward Model}
\label{tab:myfirsttable}
\end{table}

\begin{table}[ht]
\centering
\begin{tabular}{rlrrr}
  \hline
 & Model & R\_Sq & AR\_Sq & MS\_res \\
  \hline
1 & Forward w/o Infl & 0.91 & 0.91 & 0.01 \\
  2 & Backward w/o Infl & 0.90 & 0.90 & 0.01 \\
   \hline
\end{tabular}
\caption{Forward Model with no influential points vs Backward Model with no influential points}
\label{tab:forwardvsbackwardnoinfluential}
\end{table}

It was observed that the Backward model had almost twice as many points deemed influential as the Forward model, indicating that the Forward model might be either more resilient to influential points or a better model of the data in general, since both models have a roughly equal \uf{R}{2} and \uw{MS}{residuals}.

\clearpage
\newpage

\subsection{Final Model Choice}

\textbf{log(mpg) (c) \tl modelyr (mvd) + origin (mvd) + hp (c) + acc (c) + wgt (c) * hp (c)}

\begin{table}[ht]
\centering
\begin{tabular}{rrrrrr}
  \hline
 & Estimate & Std. Error & t value & Pr($>$$|$t$|$) & Significance\\
  \hline
(Intercept) & 2.1373 & 0.1735 & 12.32 & 0.00001 & *** \\
  wgt\_c & -0.0004 & 0.0000 & -14.76 & 0.00001 & *** \\
  modelyr\_mvd & 0.0309 & 0.0018 & 17.59 & 0.00001 & *** \\
  origin\_mvd2 & 0.0558 & 0.0177 & 3.14 & 0.00180 & ** \\
  origin\_mvd3 & 0.0455 & 0.0180 & 2.52 & 0.01210 & * \\
  hp\_c & -0.0064 & 0.0009 & -7.06 & 0.00001 & *** \\
  acc\_c & -0.0053 & 0.0034 & -1.59 & 0.11180 & \\
  wgt\_c:hp\_c & 0.0000013 & 0.0000002 & 6.71 & 0.00001 & *** \\
   \hline
\end{tabular}
\caption{R Summary of the final model}
\label{tab:summaryfinalmodel}
\end{table}

\begin{table}[ht]
\centering
\begin{tabular}{lrrrrrr}
  \hline
 & Df & Sum Sq & Mean Sq & F value & Pr($>$F) & Significance \\
  \hline
wgt\_c & 1 & 34.62 & 34.62 & 2714.64 & 0.0000 & *** \\
  modelyr\_mvd & 1 & 4.72 & 4.72 & 369.94 & 0.0000 & *** \\
  origin\_mvd & 2 & 0.25 & 0.12 & 9.78 & 0.0001 & *** \\
  hp\_c & 1 & 0.11 & 0.11 & 8.87 & 0.0031 & ** \\
  acc\_c & 1 & 0.001 & 0.00 & 0.26 & 0.6078 & \\
  wgt\_c:hp\_c & 1 & 0.57 & 0.57 & 45.04 & 0.0000 & *** \\
  Residuals & 383 & 4.88 & 0.01 &  &  & \\
   \hline
\end{tabular}
\caption{R ANOVA of the final model}
\label{tab:anovafinalmodel}
\end{table}

Many model combinations were considered, but the simplicity, high \uf{R}{2}, low \uw{MS}{residuals}, and resilience to influential points of the Forward model made it the number one choice. Tables 12 and 13 indicate the R Summary and R ANOVA of the chosen model.

\newpage

\section{Conclusion}

A model that mostly satisfies the 5 key assumptions of multiple linear regression and also explains 89\% of the variance in the response variable (mpg) was found, which, by most metrics is a good job. It does appear that there is a significant linear relationship between the gas mileage of a car and other attributes like its weight and model year.

\

The initial model appeared to violate 2 of the 5 key assumptions (independent errors and constant variance of residuals) and seemed to have some multicollinearity problems among regressors. A log transformation on the response variable eased the 2 key assumption violations (as well as increased the predictive ability of the model), and the addition of an interaction term compensated for the correlation of two of the variables (as well as increased the predictive ability of the model. Since the models generated by the selection algorithms had similar predictive power, the one that had the least influential points was chosen.

\newpage

\section{References}

The data was acquired from UC Irvine's Machine Learning Repository at the following web address:

https://archive.ics.uci.edu/ml/datasets/auto+mpg

\end{document}


\documentclass[]{article}

\usepackage[utf8]{inputenc}
\usepackage{fancyhdr,verbatim,parskip}
\usepackage[margin=1in]{geometry}
\usepackage{enumitem}
\usepackage{pdfpages}
\usepackage{graphicx}
\usepackage{mathtools,amsmath,amssymb,amsfonts} % for math stuff

% for code
\usepackage{minted}

\pagestyle{fancy}
\fancyhf{}
\rhead{YOUR NAME, STUDENT \#}
\lhead{COURSE NAME TODO}
\rfoot{Page \thepage}

% Optional: add word count info
% Input the word count
% \immediate\write18{detex \jobname | wc -w > \jobname_wordcount.txt}
% \lfoot{{\input{\jobname_wordcount.txt}} words (using \texttt{detex})}

% Potentially add a bibliography
%\usepackage{biblatex}
%\addbibresource{refs.bib}

\begin{document}

% Potentially include a cover sheet
% \includepdf[pages={1}]{coversheet.pdf}

\begin{comment}
I like to start with a block comment outline of what I will write about
\end{comment}

\section*{Title}
Some content, more content.

Some code:
\begin{minted}[tabsize=4,python3,breaklines]{python}
	import math
	print(math.sqrt(2.0))
\end{minted}

Refer to a figure (Figure~\ref{fig:1}).
\begin{figure}[h]
		\centering
		\includegraphics[width=0.6\linewidth]{example-image-a}
		\caption{a caption}
		\label{fig:1}
\end{figure}

% Answer questions with letter numbers
\begin{enumerate}[label=(\alph*)]
	% =================================================================
	% Part a
	% =================================================================
	\item\label{part-a}

	Part~\ref{part-a} refers to Figure~\ref{fig:1}.

	% =================================================================
	% Part b
	% =================================================================
	\newpage
	\item\label{part-b}
	Another answer.
\end{enumerate}

%\printbibliography

\end{document}


%
% This is a borrowed LaTeX template file for lecture notes for CS267,
% Applications of Parallel Computing, UCBerkeley EECS Department.
% Now being used for CMU's 10725 Fall 2012 Optimization course
% taught by Geoff Gordon and Ryan Tibshirani.  When preparing
% LaTeX notes for this class, please use this template.
%
% To familiarize yourself with this template, the body contains
% some examples of its use.  Look them over.  Then you can
% run LaTeX on this file.  After you have LaTeXed this file then
% you can look over the result either by printing it out with
% dvips or using xdvi. "pdflatex template.tex" should also work.
%

\documentclass[twoside]{article}
\setlength{\oddsidemargin}{0.25 in}
\setlength{\evensidemargin}{-0.25 in}
\setlength{\topmargin}{-0.6 in}
\setlength{\textwidth}{6.5 in}
\setlength{\textheight}{8.5 in}
\setlength{\headsep}{0.75 in}
\setlength{\parindent}{0 in}
\setlength{\parskip}{0.1 in}

%
% ADD PACKAGES here:
%

\usepackage{amsmath,amsfonts,graphicx}

%
% The following commands set up the lecnum (lecture number)
% counter and make various numbering schemes work relative
% to the lecture number.
%
\newcounter{lecnum}
\renewcommand{\thepage}{\thelecnum-\arabic{page}}
\renewcommand{\thesection}{\thelecnum.\arabic{section}}
\renewcommand{\theequation}{\thelecnum.\arabic{equation}}
\renewcommand{\thefigure}{\thelecnum.\arabic{figure}}
\renewcommand{\thetable}{\thelecnum.\arabic{table}}

%
% The following macro is used to generate the header.
%
\newcommand{\lecture}[4]{
   \pagestyle{myheadings}
   \thispagestyle{plain}
   \newpage
   \setcounter{lecnum}{#1}
   \setcounter{page}{1}
   \noindent
   \begin{center}
   \framebox{
      \vbox{\vspace{2mm}
    \hbox to 6.28in { {\bf EE302 - Feedback Systems
	\hfill Spring 2019} }
       \vspace{4mm}
       \hbox to 6.28in { {\Large \hfill Lecture #1 \hfill} }
       \vspace{2mm}
       \hbox to 6.28in { {\it Lecturer: #2 \hfill } }
      \vspace{2mm}}
   }
   \end{center}
   \markboth{Lecture #1}{Lecture #1}

   \vspace*{4mm}
}
%
% Convention for citations is authors' initials followed by the year.
% For example, to cite a paper by Leighton and Maggs you would type
% \cite{LM89}, and to cite a paper by Strassen you would type \cite{S69}.
% (To avoid bibliography problems, for now we redefine the \cite command.)
% Also commands that create a suitable format for the reference list.
\renewcommand{\cite}[1]{[#1]}
\def\beginrefs{\begin{list}%
        {[\arabic{equation}]}{\usecounter{equation}
         \setlength{\leftmargin}{2.0truecm}\setlength{\labelsep}{0.4truecm}%
         \setlength{\labelwidth}{1.6truecm}}}
\def\endrefs{\end{list}}
\def\bibentry#1{\item[\hbox{[#1]}]}

%Use this command for a figure; it puts a figure in wherever you want it.
%usage: \fig{NUMBER}{SPACE-IN-INCHES}{CAPTION}
\newcommand{\fig}[3]{
			\vspace{#2}
			\begin{center}
			Figure \thelecnum.#1:~#3
			\end{center}
	}
% Use these for theorems, lemmas, proofs, etc.
\newtheorem{theorem}{Theorem}[lecnum]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{definition}[theorem]{Definition}
\newenvironment{proof}{{\bf Proof:}}{\hfill\rule{2mm}{2mm}}

% **** IF YOU WANT TO DEFINE ADDITIONAL MACROS FOR YOURSELF, PUT THEM HERE:

\begin{document}

% Lecture Details
\lecture{7}{Asst. Prof. M. Mert Ankarali}

\par

\section{Time Domain Analysis}

    \begin{minipage}[h]{0.5\linewidth}
    \begin{center}
      \includegraphics[width=0.9\textwidth]{sys}
    \end{center}
  \end{minipage}
    \begin{minipage}[h]{0.5\linewidth}
    \begin{center}
      \begin{align*}
      	Y(s) &= G(s) U(s)
	\\
	y(t) &= \mathcal{L} \lbrace G(s) U(s) \rbrace^{-1}
      \end{align*}
    \end{center}
  \end{minipage}

  Objective:

\begin{itemize}
	\item Calculate $y(t)$ for different $u(t)$,
	\item Understand the relation between the parameters and output behavior.
\end{itemize}

    \subsection{First Order Systems}

    Simplest first order system is an integrator, which is also the fundamental block for higher order systems.

    \begin{center}
        \begin{minipage}[h]{0.5\linewidth}
    \begin{center}
      \includegraphics[width=0.9\textwidth]{int}
    \end{center}
  \end{minipage}
      \end{center}

      \textbf{Ex 1:} Compute the step-response of the integrator system, $G(s) = \frac{1}{s}$.

      \textbf{Solution:} We assume that initial conditions are zero

          \begin{minipage}[h]{0.7\linewidth}
    \begin{center}
      \includegraphics[width=1\textwidth]{signals}
    \end{center}
  \end{minipage}
    \begin{minipage}[h]{0.3\linewidth}
    \begin{center}
      \begin{align*}
      	U(s) &= \frac{1}{s}
	\\
	Y(s) &= G(s) U(s) = \frac{1}{s^2}
	\\
	y(t) &= \mathcal{L}^{-1} \left\lbrace \frac{1}{s^2}  \right\rbrace
	\\
	&= t \ , \ \mathrm{for} \ t \geq 0
      \end{align*}
    \end{center}
  \end{minipage}

     \vspace{12pt}

     \textbf{Ex 2:} Compute the step-response of the following first order system

         \begin{minipage}[h]{0.7\linewidth}
    \begin{center}
      \includegraphics[width=1\textwidth]{firstex}
    \end{center}
  \end{minipage}
    \begin{minipage}[h]{0.3\linewidth}
    \begin{center}
      \begin{align*}
	G(s) &= \frac{K}{s+K} \
	\\
	Y(s) &= \frac{K}{s (s+K)} = \frac{K}{s (s+K)}
	\\
	&= \frac{A}{s} + \frac{B}{s+K}
      \end{align*}
    \end{center}
  \end{minipage}

  We can compute $A$ and $B$ as
  %
  \begin{align*}
  	A &= \lim_{s \to 0} \left[  s Y(s) \right] = \frac{K}{K} = 1
	\\
	B &= \lim_{s \to -K} \left[  (s+K) Y(s) \right] = \frac{K}{-K} = -1
   \end{align*}
%
Then, we can compute $y(t)$ as
%
  \begin{align*}
y(t) &= \mathcal{L}^{-1} \left\lbrace \frac{1}{s} - \frac{1}{s+K} \right\rbrace
\\
&= \left[ 1 - e^{-K t} \right] \ , \ \mathrm{for} \ t \geq 0
\end{align*}

         \begin{minipage}[h]{0.5\linewidth}
    \begin{center}
      \includegraphics[width=1\textwidth]{step}
    \end{center}
  \end{minipage}
    \begin{minipage}[h]{0.5\linewidth}
        \begin{center}
          Proportional Controller
          %
      \begin{align*}
	\lim_{t \to \infty} y(t) &= 1
	\\
	\lim_{t \to \infty} e(t) &= \lim_{t \to \infty} (y(t) - u(t)) = 0
      \end{align*}
      Zero steady-state error $\forall K > 0$

      ``Convergence speed'' $ \nearrow$ as $K \nearrow$
          \end{center}
  \end{minipage}

\textbf{Ex 2:} Find the unit-ramp response for the same system
%
\begin{align*}
	Y(s) &= \frac{K}{s^2 (s+K)}
	\\
	&= \frac{A}{s} + \frac{B}{s^2} + \frac{C}{s+K}
\end{align*}
%
$A$, $B$, and $C$ can be computed as
%
\begin{align*}
	C &= \lim_{s \to -K} \left[  (s+K) Y(s) \right] = \frac{1}{K}
	\\
	B &= \lim_{s \to 0} \left[  s^2 Y(s) \right] = 1
	\\
	A &= \lim_{s \to 0} \frac{d}{d s} \left[  s^2 Y(s) \right] =
	 \lim_{s \to 0} \frac{d}{d s} \left[  \frac{K}{(s+K)}  \right]
	=  \lim_{s \to 0} \left[  \frac{-K}{(s+K)^2}  \right]
	= \frac{-1}{K}
\end{align*}
%
Then, we can compute $y(t)$ as
%
  \begin{align*}
y(t) &= \frac{-1}{K} + t + \frac{1}{K} e^{-K t} \ ,  \ \mathrm{for} \ t \geq 0
\\
y(t) &= t + \frac{-1}{K} \left[ 1 -  \frac{1}{K} e^{-K t} \right] \ ,  \ \mathrm{for} \ t \geq 0
\end{align*}
%
Note that $r(t) = t \ , \ \mathrm{for} \ t \geq 0$.

\vspace{12pt}

         \begin{minipage}[h]{0.5\linewidth}
    \begin{center}
      \includegraphics[width=1\textwidth]{ramp}
    \end{center}
  \end{minipage}
    \begin{minipage}[h]{0.5\linewidth}
        \begin{center}
          Proportional Controller
          %
      \begin{align*}
	&e(t) = r(t) - y(t) = \frac{1}{K} \left[ 1 -  \frac{1}{K} e^{-K t} \right]
	\\
	&\lim_{t \to \infty} e(t) = \frac{1}{K}
      \end{align*}
      Non-zero steady-state error

      Steady-state error $ \searrow$ as $K \nearrow$
          \end{center}
  \end{minipage}

% **** This ENDS THE EXAMPLES. DON'T DELETE THE FOLLOWING LINE:
\end{document}


\documentclass[a4paper,14pt]{extreport}

%PACKAGES
%Graphics
\usepackage{graphicx}
\graphicspath{ {./images/} }
%Math
\usepackage{mathtools}
\usepackage{amsmath}
\usepackage{bm}

%DOCUMENT START
\title{Attitude Control? \\
\large An Informal Intro to Spacecraft Attitude Control}
\author{Brian Leung\\University of Michigan, Class of 2021}

\begin{document}

\maketitle
\setcounter{tocdepth}{0}
\tableofcontents{}
\chapter{Preface}
\section{Prerequisites}
Knowledge of Diff Eq is fairly necessary for this.
Some basic controls knowledge is also helpful.

\chapter{What is State Space?}

\chapter{Nonlinear Stability Definitions}

\chapter{Lyapunov Stability}

\chapter{Lyapunov Attitude Control}

\chapter{Attitude Control Implementations}

\end{document}


\documentclass[a4paper,10pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{float}
\usepackage{xcolor}

\usepackage[tmargin=1.0in,bmargin=1.0in, lmargin=1.25in, rmargin=1.25in]{geometry}

%opening
\title{Readme}
\author{Thomas Bellotti}

\begin{document}

\maketitle

\tableofcontents

\section{Brief description of the most important classes}
Since the whole project uses templates, almost all the classes are implemented in .h files stored in the folder \texttt{include}.
We now provide a brief description of what the most important classes do.
\subsection{Cell}
This class implements the notion of cell in a quadtree. Normally, the user does not have to directly use or interact with this class, which is the base brick of the class QuadTree and Pixel which will be introduced further.
\subsection{QuadTree}
This class implements the concepts of quadtree mesh, containing a smart pointer to a Cell, which is top of the tree.
The foundamental calls which a user should be aware of are:
\begin{itemize}
 \item The constructor, of course.
 \item \texttt{void buildUniform(const unsigned char)}, which constructs a uniform mesh with level specified by its argument.
 \item \texttt{void updateWithLevelSet(const LipschitzFunction<T> \&)}, which updates the mesh according to a criterion based on a level-set function (see presentation).
 \item \texttt{void updateQuadTree(const RefinementCriterion<T> \&)}, which does the same according to an arbitrary criterion given as argument.
 \item \texttt{void exportMeshTikz(const std::string \&, bool) const}, which lets us to export a \LaTeX file with a TiKz figure inside which allows us to plot the mesh we have constructed.
 \item \texttt{T simpleIntegration(const std::function<T(Point<T>)> \&) const}, which performs a ``naive'' (see presentation) integration of a function (\texttt{T} is the template parameter).
 \item \texttt{T thirdOrderGaussianIntegration(const std::function<T(Point<T>)> \&) const}, does the same using a third order Gaussian quadrature rule.
\end{itemize}

\subsection{Pixel}
This class inherits from Cell and adds just an RGB value on each cell. It implements the notion of ``generalized'' pixel of an image.

\subsection{Image}
Though our concept of image is represented by a quadtree, this class does not inherit from QuadTree since most of the features we introduce are not exactly the same and we prefered to keep them separated.
The most important functions of this class from the user's point of view are:
\begin{itemize}
 \item \texttt{void createFromFile(const std::string \&)}, which given a file in PPM format (we wanted to avoid to use external libraries for the sake of simplicity), parses and stores the corresponding image in our data structure.
 \item \texttt{void simplifyImage(double)}, given a threshold as argument, is compresses (simplifies) the image based on the color variance (see presentation). Typical values of the threshold are between $0.01$ (high quality, low compression rate) and $0.05$ (very low quality, high compression rate).
 \item \texttt{void saveImage(const std::string \&) const}, saving the image, as for the quadtree, in \LaTeX~format.
\end{itemize}

\subsection{RefinementCriterion}
This is a small yet important class of the project. It is an abstract class containing the following \texttt{virtual bool operator()(std::shared\textunderscore ptr<Cell<T>> arg) const} which should tell the system if the cell pointed by the argument should be refined (split) or not.

Custom criteria (some of which are already implemented) must inherit from this class and override the method we have just presented.

\section{Compile the project}
In order to \textbf{compile} the whole project, simply use:
\begin{center}
 \begin{verbatim}
  make
 \end{verbatim}
\end{center}
in the project folder. Notice that we have slightly changed the flags, using \texttt{C++14} instead of \texttt{C++11} in order to slightly use its features and no flag is applied to the compilation of the \texttt{MPI} sources since it would stop compilation due to errors, which are not generated by our source code, but rather by the way the \texttt{MPI} distribution has been compiled.
As usual, one can use
\begin{center}
 \begin{verbatim}
  make distclean
 \end{verbatim}
\end{center}
to clean the whole project.

\section{Examples and binaries}
Rather than a ready-to-use application, we provide a lot of examples and test which use all (or almost) the features of our classes.
We describe them in this section.

\subsection{Ready-to-use binaries}
\begin{itemize}
 \item \textbf{bin/imagecompression}. This is implemented in \texttt{src/main/imagecompression.cpp} and should be called with three arguments: the name of the input file (in .ppm format), the name of the output file (which will be a \LaTeX file) and the threshold for the image compression.

 It performs the compression of the image according to our method. Notice that the image must be square, with number of pixels per side which is a power of $2$, for the sake of simplicity.
\end{itemize}

\subsection{Ready-to-use functions}\label{foo1}
\begin{itemize}
 \item \textbf{src/parallel\textunderscore integration.cpp}, which is compiled to give \texttt{build/parallel\textunderscore integration.o}, which contains the parallel function:
 \begin{center}
  \begin{verbatim}
double parallel_integration(const std::function<double(Point<double>)> & func,
    unsigned order, const Point<double> & l_left, const Point<double> & u_right,
    unsigned char min_lev, unsigned char max_lev,
    const RefinementCriterion<double> & criterion)
  \end{verbatim}
 \end{center}
This functions integrates the function \texttt{func} with the naive formula (\texttt{order} = 0) or the 3rd order Gaussian formula (\texttt{order} = 1) on a square domain of lower left corner \texttt{l\textunderscore left} and upper right corner  \texttt{u\textunderscore right} on a mesh of minimum level and maximum level given by \texttt{min\textunderscore lev} and \texttt{max\textunderscore lev}, which is refined according to \texttt{criterion}.

Notice that there are many requirements on the number of cores used and the minimum size of the mesh which are explained inside the code, so have a look at it for further information.
\end{itemize}

\subsection{Example binaries}
\textcolor{red}{\textbf{Caveat}: In many examples, since inside the .cpp file we use absolute paths to provide ouput files, one may need to change them according to its distribution of the project in order to test the code. We do apologize.}
\subsubsection{General}
\begin{itemize}
 \item \textbf{bin/basic\textunderscore bricks}, compiled starting from \texttt{src/test/basic\textunderscore bricks.cpp}, which tests some of the basic calls which make the code working. It is not really interesting, it has been used mainly for debugging purpose.
 \end{itemize}
 \subsubsection{Quadtree meshes}
 \begin{itemize}
 \item \textbf{bin/test/quadtree\textunderscore highly\textunderscore adaptive\textunderscore mesh}, which has been compiled starting from the source code \texttt{src/test/quadtree\textunderscore highly\textunderscore adaptive\textunderscore mesh.cpp}. It tests the mesh adaptation with different criteria, outputing the figures in folders which can be specified in the source code (it needs to be adapted if the user changes path, which is likely to happen).
 \begin{figure}[H]
\begin{center}
\includegraphics[width=0.4\textwidth]{./figures/readme/level_set.pdf}
\end{center}
\end{figure}
This figure shows an example of output (once compiled with \LaTeX), but many more are possible.
\item \textbf{bin/test/quadtree\textunderscore time\textunderscore evolving\textunderscore mesh}, which tests the possibility of making a mesh evolving with time in order to implement selective refinement for time-dependent problems. It is compiled starting from the source code \texttt{src/test/quadtree\textunderscore time\textunderscore evolving\textunderscore mesh.cpp}
The logic is almost the same than in the previous test.
 \begin{figure}[H]
\begin{center}
\includegraphics[width=0.3\textwidth]{./figures/readme/frame0}
\includegraphics[width=0.3\textwidth]{./figures/readme/frame9}
\includegraphics[width=0.3\textwidth]{./figures/readme/frame15}
\end{center}
\end{figure}
\item \textbf{bin/test/quadtree\textunderscore other\textunderscore meshes}, which tests the mesh refinement using fancy criteria, also in order to build a fractal structure such as the Mandelbrot set.
 \begin{figure}[H]
\begin{center}
\includegraphics[width=0.95\textwidth]{./figures/readme/fractal}
\end{center}
\end{figure}
The source file is located at \texttt{src/test/quadtree\textunderscore other\textunderscore meshes.cpp}.
\item \textbf{bin/test/quadtree\textunderscore integration}, which tests the integration functions in QuadTree (without parallelism). It is implemented in \texttt{src/test/quadtree\textunderscore integration.cpp}
\end{itemize}
\subsubsection{Parallel integration on quadtree meshes}
These are nothing but different tests for the parallel integration function presented in Section \ref{foo1}. They are also shown in the presentation, so we present them rapidly:
\begin{itemize}
 \item \textbf{bin/test/mpi\textunderscore pi}, implemented in \texttt{src/test/mpi\textunderscore pi.cpp}
  \item \textbf{bin/test/mpi\textunderscore gauss}, implemented in \texttt{src/test/mpi\textunderscore gauss.cpp}
   \item \textbf{bin/test/mpi\textunderscore square}, implemented in \texttt{src/test/mpi\textunderscore square.cpp}
    \item \textbf{bin/test/mpi\textunderscore unif}, implemented in \texttt{src/test/mpi\textunderscore unif.cpp}
\end{itemize}
They all take the following arguments: minimum level and maximum level, type of quadrature rule (0 for naive and 1 for Gaussian) and the type of mesh we use for the test (1,2 or 3), see inside the source or in the presentation for more on it.
Notice that the bash script \texttt{batch\textunderscore mpi\textunderscore test.sh} in the root of the project executes them in batch varying all the parameters and the number of cores.
\subsubsection{Image compression}
\begin{itemize}
 \item \textbf{bin/test/image\textunderscore compression}, which is compiled from \texttt{src/test/image\textunderscore compression.cpp}.
 It tests the compression of some sample images (contained in \texttt{media/images} for different threshold values. Notice that the output folder may need to be changed.
  \begin{figure}[H]
\begin{center}
\includegraphics[width=0.95\textwidth]{./figures/readme/s4_comp_small}
\includegraphics[width=0.95\textwidth]{./figures/readme/s2_comp_small}
\end{center}
\end{figure}
These are some of the examples of image compression.
\end{itemize}

\section{How to compile output images}
In order to compile the \LaTeX ~ files which are the output used to visualize the quadtree structures, both as meshed and images, one must have a complete \LaTeX ~ distribution of this system with the TiKz package installed.

For small figures, writing
\begin{center}
 \begin{verbatim}
  pdflatex file_name.tex
 \end{verbatim}
\end{center}
is sufficient. Nevertheless, if the figure is really complex, this could saturate the buffer of \texttt{pdflatex}. The easiest way of fixing this issue is to use \texttt{lualatex} instead:
\begin{center}
 \begin{verbatim}
  lualatex file_name.tex
 \end{verbatim}
\end{center}
which is slightly slower but allows to represent large images or meshes.

\end{document}


%	This is written by Zhiyang Ong as a template for writing algorithms in LaTeX.

%	The MIT License (MIT)

%	Copyright (c) <2014> <Zhiyang Ong>

%	Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

%	The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

%	THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

%	Email address: echo "cukj -wb- 23wU4X5M589 TROJANS cqkH wiuz2y 0f Mw Stanford" | awk '{ sub("23wU4X5M589","F.d_c_b. ") sub("Stanford","d0mA1n"); print $5, $2, $8; for (i=1; i<=1; i++) print "6\b"; print $9, $7, $6 }' | sed y/kqcbuHwM62z/gnotrzadqmC/ | tr 'q' ' ' | tr -d [:cntrl:] | tr -d 'ir' | tr y "\n"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Algorithms}
\label{chp:Algorithms}

A template for typesetting algorithms is shown in {\sc Procedure} \ref{lst:MyAlgorithm}.

\begin{codebox}
\Procname{$\proc{NAME OF THE ALGORITHM}({\it ARGUMENTS})$}
\label{lst:MyAlgorithm}
\zi \Comment {\it Input ARGUMENT \#1: Definition1}
\zi \Comment {\it Input ARGUMENT \#2: Definition2}
\li BODY OF THE PROCEDURE
\zi \Comment {\it A while loop.}
\li \While [condition]
	\Do
\li	[Something]
	\End
\zi \Comment {\it A for loop.}
\li \For \id{Var} $\gets$ [initial value] \To [final value]
	\Do
\li	[Something]
	\End
\zi \Comment {\it An if-elseif-else block.}
\li	\If $[$Condition1$]$
	\Then
\li		Blah\dots
\li	\ElseIf $[$Condition2$]$
	\Then
\li		Blah\dots
\li	\ElseIf $[$Condition3$]$
	\Then
\li		Blah\dots
%	\li	\ElseNoIf $[$Condition$]$
%		\Then
%	\li		Blah\dots
\li	\Else
\li		Blah\dots
	\End
\zi \Comment {\it A variable assignment.}
\li $\id{blah} \gets A[j]$
\zi	\>	\Comment {\it This is indented with a tab.}
\zi	\Comment {\it What is the output of this procedure?}
\li	\Return
\end{codebox}


\documentclass[12pt]{article}
\usepackage{fullpage}
\usepackage{amsthm}
\usepackage{amsfonts,amsmath, amssymb,latexsym,mathrsfs}
\usepackage[margin=1.15in]{geometry}
\usepackage{enumitem}
\setlength{\parindent}{0pt}
\usepackage{tikz-cd}
\usepackage{fancyhdr}
\usepackage{multicol}

\theoremstyle{definition}
\newtheorem{thm}{Theorem}[section]

\newtheorem{prop}{Proposition}[section]

\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]

\theoremstyle{remark}
\newtheorem*{remark}{Remark}

\theoremstyle{definition}
\newtheorem{example}{Example}[section]

\theoremstyle{definition}
\newtheorem{lem}{Lemma}[section]

\theoremstyle{definition}
\newtheorem{cor}{Corollary}[section]

\date{}
\title{Final Exam Summary (Everything, formula)}

\begin{document}
\maketitle
\section{Integral}
\section{Riemann Sums}
\begin{enumerate}
	\item$\int_a^bf(x)dx= \lim_{n\rightarrow \infty} \sum_{i=1}^n f(x_i)\Delta x\text{\bf (Limit of Right-hand sum RIGHT(n))}$

	\item$\int_a^bf(x)dx= \lim_{n\rightarrow \infty} \sum_{i=0}^{n-1} f(x_i)\Delta x\text{\bf (Limit of Left-hand sum LEFT(n))}$

	\item$\int_a^bf(x)dx\approx  \sum_{i=0}^{n-1} f(\frac{x_i+x_{i+1}}{2})\Delta x\text{\bf (Limit of Mid sum MID(n))}$

	\item$\int_a^bf(x)dx\approx \sum_{i=0}^{n-1} \frac{f(x_i)+f(x_{i+1})}{2}\Delta x\text{\bf (Limit of Trapezoid sum TRAP(n))}$

	\item$\Delta(x)=\dfrac{b-a}{n}$

	\item$\frac{LEFT(n)+RIGHT(n)}{2}=TRAP(n)$
	\item$MID(n)\neq TRAP(n)$

	\item Error estimation: $|LEFT(n) - f(x)|<|LEFT(n)-RIGHT(n)|=(f(b)-f(a))\Delta x$. This usually gives a bound for $n$.
\end{enumerate}

\subsection{Properties of Riemann sums:}
\begin{enumerate}
\item If the graph of $f$ is increasing on $[a,b]$, then $LEFT(n)\leq \int^b_a f(x) dx \leq RIGHT(n)$
\item If the graph of $f$ is decreasing on $[a,b]$, then $RIGHT(n)\leq \int^b_a f(x) dx \leq LEFT(n)$
\item If the graph of $f$ is concave up on $[a,b]$, then $MID(n)\leq \int^b_a f(x) dx \leq TRAP(n)$
\item If the graph of $f$ is concave down on $[a,b]$, then $TRAP(n)\leq \int^b_a f(x) dx \leq MID(n)$
\end{enumerate}
\subsection{Properties of Definite Integrals}
	\begin{enumerate}
		\item $int^a_b f(x) dx = -\int^b_a f(x) dx$
		\item $\int^a_b f(x) dx+\int^b_c f(x) dx=\int^a_c f(x) dx$
		\item $\int^a_b (f(x)\pm g(x)) dx=\int^a_b f(x) dx \pm \int^a_b g(x) dx$
		\item $\int^a_b cf(x) dx = c \int^a_b f(x) dx$
		\item Symmetry due to the oddity of the function.
		\item Average value of function $f(x)$ in $[a,b]$ is $\frac{1}{b-a} \int_{a}^{b}f(x)dx$.
	\end{enumerate}

\begin{thm}
\textbf{The Fundamental Theorem of Calculus}:

If $f$ is continuous on interval $[a,b]$ and $f(t)=F'(t)$, then $\int^b_a f(t) dt = F(b)-F(a).$
\textbf{Second FTC (Construction theorem for Antiderivatives)}
If $f$ is a continuous function on an interval, and if $a$ is any number in that interval then the function $F$ defined on the interval as follows is an antiderivative of $f$:

\[F(x)=\int^x_a f(t) dt\]
\end{thm}

\begin{enumerate}
\item	$\int C dx = 0$
\item	$\int kdx=kx+C$
\item	$\int x^ndx=\frac{x^{n+1}}{n+1}+C, (n \neq -1)$
\item	$\int \frac{1}{x}dx = \ln|x|+C$
\item	$\int e^xdx=e^x+C $
\item	$\int \cos xdx=\sin x + C $
\item	$\int \sin xdx=-\cos x + C $

\end{enumerate}

Properties of antiderivatives:

\begin{enumerate}
\item $\int (f(x) \pm g(x))dx=\int f(x) dx \pm \int g(x) dx$
\item $\int cf(x) dx = c \int f(x) dx$
\end{enumerate}

\subsection{Integration Techniques}
\begin{enumerate}
	\item Guess and Check
	\item Substitution $du=f(x)'dx$ if $u=f(x)$
	\item By parts $\int u dv=uv-\int v du$
	\item Partial fractions $\frac{p(x)}{(x+c_1)^2(x+c_2)(x^2+c_3)}=\frac{A}{x+c_1}+\frac{B}{(x+c_1)^2}+\frac{C}{x+c_2}+ \frac{Dx+E}{x^2+c_3}$

\end{enumerate}

\section{Find Area/Volumes by slicing}

\begin{enumerate}
	\item Compute the area: Think about slicing the area into parallel line segments.
	\item Disk Method:\\
	Horizontal axis of revolution ($x$-axis): $V = \int_a^b \pi(f(x)^2 - g(x)^2)dx$\\
	Vertical axis of revolution ($y$-axis): $V = \int_a^b \pi(f(y)^2 - g(y)^2)dy$
	\item Shell Method:\\
	Horizontal axis of revolution ($x$-axis): $V = \int_a^b 2\pi y(f(y) - g(y))dy$\\
		Vertical axis of revolution ($y$-axis): $V = \int_a^b 2\pi x(f(x) - g(x))dx$

\end{enumerate}

\subsection{Mass}
The basic formula we are doing is:\begin{enumerate}
\item One dimensional: $M=\delta l$ where $M$ is the total mass, $\delta$ is the density, $l$ is line.
\item
Two dimensional: $M=\delta A$ where $M$ is the total mass, $\delta$ is the density, $A$ is Area.
\item Three dimensional (real world): $M=\delta V$ where $M$ is the total mass, $\delta$ is the density, $V$ is Volume.
\end{enumerate}

\subsection{Work}

Key formula we are using:\\
$\text{Work done = Force} \cdot \text{Distance}$ or
$W=F\cdot s$

Integration version:
$W = \int^b_a F(x) dx$

\subsection{L'Hopital's rule}

L’Hopital’s rule: If $f$ and $g$ are differentiable and (below $a$ can be $\pm \infty$)\\
i)$f(a) = g(a) = 0$ for finite $a$, \\
Or ii)$\lim_{x\to a} f(x)=\lim_{x\to a} g(x)= \pm \infty$,\\
Or iii)$\lim_{x\to \infty} f(x)= \lim_{x\to \infty} g(x) = 0$
then
\[\lim_{x\to a}\frac{f(x)}{g(x)} = \lim_{x\to a} \frac{f'(x)}{g'(x)} \]

\subsection{Dominance}
We say that $g$ dominates $f$ as $x \to \infty$ if $\lim_{x\to \infty}f(x)/g(x) = 0$.

\section{Improper integral}
There are two types of improper integral.

\begin{itemize}
	\item The first case is where we have the limit of the integration goes to infinity, i.e. $\lim_{b \to \infty} \int^b_a f(x) dx$.
	\item The integrand goes to infinity as $x \to a$.
\end{itemize}

\subsection{Converges or diverges?}

\begin{enumerate}
	\item Check by definition, this means check the limit directly.
	\item $p$-test.\\
	\includegraphics*[width=0.9\textwidth]{1.png}
	\item Exponential decay test.
	\[\int^\infty_0 e^{-ax} dx\] converges for $a>0$.
	\item Comparison test.\\
	If $f(x)\geq g(x) \geq 0$ on the interval $[a,\infty]$ then,\begin{itemize}
	\item If $\int^\infty_a f(x) dx$ converges then so does $\int^\infty_a g(x) dx$.
	\item If $\int^\infty_a g(x) dx$ diverges then so does $\int^\infty_a f(x) dx$.
	\end{itemize}
	\item Limit Comparison theorem.\\
	Limit Comparison Test. If $f(x)$ and $g(x)$ are both positive  on the interval $[a,b)$ where $b$ could be a real number or infinity.
	and
	\[\lim_{x\to b}\frac{f(x)}{g(x)} = C\] such that $0 < C < \infty$
	then the improper integrals $\int^b_a f(x) dx$ and $\int^b_a g(x) dx$ are either both convergent or both divergent.

\end{enumerate}

\section{Probability}
\subsection{PDF and CDF}
\begin{definition}
A function $p(x)$ is a \textbf{probability density function} or PDF if it satisfies the following conditions
\begin{itemize}
\item $p(x) \geq 0$ for all $x$.
\item $\int_{-\infty}^\infty p(x) = 1.$
\end{itemize}
\end{definition}

\begin{definition}
A function $P(t)$ is a \textbf{Cumulative Distribution Function} or cdf, of a density function $p(t)$, is defined by
$P(t) =\int_{-\infty}^t p(x) dx $,
which means that $P(t)$ is the antiderivative of $p(t)$ with the following properties:
\begin{itemize}
\item $P(t)$ is increasing and $0\leq P(t)\leq 1$ for all $t$.
\item $\lim_{t \to \infty}P(t)=1.$
\item $\lim_{t \to -\infty}P(t)=0.$
\end{itemize}
\end{definition}

Moreover, we have $\int_a^b p(x)dx=P(b)-P(a)$.

\subsection{Probability, mean and median}

\subsubsection*{Probability}
Let us denote $X$ to be the quantity of outcome that we care ($X$ is in fact, called the random variable).
$\mathbb{P}\{a\leq X\leq b\}=\int_a^b p(x)dx=P(b)-P(a)$\\
$\mathbb{P}\{X\leq t\}=\int_{-\infty}^t p(x)dx=P(t)$\\
$\mathbb{P}\{X\geq s\}=\int_{s}^\infty p(x)dx=1-P(s)$

\subsubsection*{The mean and median}
\begin{definition}
A \textbf{median} of a quantity $X$ is a value $T$ such that the probability of $X\leq T$ is $1/2$. Thus we have  $T$ is defined such that
$\int_{-\infty}^T p(x) dx=1/2$  or $P(T)=1/2$.
\end{definition}
\begin{definition} A \textbf{mean} of a quantity $X$ is the value given by
	\[ Mean= \frac{\text{Probability of all possible quantity}}{\text{Total probability}}= \frac{\int_{-\infty}^{\infty}xp(x)dx}{\int_{-\infty}^{\infty}p(x)dx}=\frac{\int_{-\infty}^{\infty}xp(x)dx}{1}=\int_{-\infty}^{\infty}xp(x)dx. \]
\end{definition}

\subsubsection*{Normal Distribution}
\begin{definition}
A normal distribution has a density function of the form
\[p(x)=\frac{1}{\sigma \sqrt{2 \pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}\] where $\mu$ is the mean of the distribution and $\sigma$ is the standard deviation, with $\sigma > 0$.
The case $\mu = 0$, $\sigma = 1$ is called the standard normal distribution.

\end{definition}

\section{Sequences and Series}
\subsection{Sequence}

If a sequence $s_n$ is bounded and monotone, it converges.

\subsection{Series}

Convergence Properties of Series:
\begin{enumerate}
\item If $\sum_{n=1}^{\infty} a_n$ and $\sum_{n=1}^{\infty} b_n$ converge and if $k$ is a constant, then

$\sum_{n=1}^{\infty} (a_n+b_n)$ converges to$\sum_{n=1}^{\infty} a_n + \sum_{n=1}^{\infty} b_n$.\\
$\sum_{n=1}^{\infty} ka_n$ converges to $k\sum_{n=1}^{\infty} a_n$

\item Changing a finite number of terms in a series does not change convergence,
\item  If $\lim_{n \to \infty}a_n\neq 0$ or $\lim_{n \to \infty}a_n$ does not exist, then
$\sum_{n=1}^{\infty} a_n$ diverges. (\textbf{!})
\item If $\sum_{n=1}^{\infty} a_n$ diverges, then $\sum_{n=1}^{\infty} a_n$ diverges if $k\neq 0$.
\end{enumerate}
Moreover, there are several test to determine if a series is convergent.
\begin{enumerate}
\item \textbf{The Integral Test}\\
Suppose $a_n = f(n)$, where $f(x)$ is decreasing and positive.
\\a. If $\int_1^\infty f(x) dx$ converges, then $\sum_{n=1}^{\infty} a_n$ an converges.
\\b. If $\int_1^\infty f(x) dx$ diverges, then $\sum_{n=1}^{\infty} a_n$ an diverges.

\item \textbf{p-test}\\
The $p$-series $\sum_{n=1}^{\infty} 1/n^p$ converges if $p > 1$ and diverges if $p \leq 1$.

\item \textbf{Comparison Test}\\
Suppose $0 \leq a_n \leq b_n$ for all $n$ beyond a certain value.
\\ a. If $\sum_{n=1}^{\infty} b_n$ converges, then $\sum_{n=1}^{\infty} a_n$ converges.
\\ b. If $\sum_{n=1}^{\infty} a_n$ diverges, then $\sum_{n=1}^{\infty} b_n$ diverges.

\item \textbf{Limit Comparison Test}\\
Suppose $a_n > 0$ and $b_n > 0$ for all $n$. If
$\lim_{n\to \infty}a_n/b_n= c$ where $c > 0$,
then the two series $\sum_{n=1}^{\infty} a_n$ and $\sum_{n=1}^{\infty} b_n$ either both converge or both diverge.

\item \textbf{Convergence of Absolute Values Implies Convergence}\\
If $\sum_{n=1}^{\infty}|a_n|$ converges, then so does $\sum_{n=1}^{\infty} a_n$.

\item \textbf{The Ratio Test}
For a series $\sum_{n=1}^{\infty} a_n$, suppose the sequence of ratios $|a_{n+1}|/|a_n|$ has a limit:
$\lim_{n\to \infty}|a_{n+1}|/|a_n| = L$, then
\begin{itemize}
\item If $L < 1$, then $\sum_{n=1}^{\infty} a_n$ converges.
\item If $L > 1$, or if $L$ is infinite, then $\sum_{n=1}^{\infty} a_n$ diverges.
\item If $L = 1$, the test does not tell anything about convergence of $\sum_{n=1}^{\infty} a_n$ (!).
\end{itemize}

\item \textbf{Alternating Series Test}
A series of the form $\sum_{n=1}^{\infty} (-1)^{n-1}a_n = a_1 - a_2 + a_3 - a_4 + \ldots + (-1)^{n-1}a_n + \ldots$
converges if
$0 < a_{n+1} < a_n$ for all $n$ and $lim_{n \to \infty}a_n = 0$.

Error of alternating test: let $S = \lim_{n\to \infty}S_n$, then have $|S - S_n| < a_{n+1}$.
\end{enumerate}

Notably, We say that the series $\sum_{n=1}^{\infty} a_n$ is
\begin{itemize}
\item absolutely convergent if $\sum_{n=1}^{\infty} a_n$ and $\sum_{n=1}^{\infty}|a_n|$ both converge.
\item conditionally convergent if $\sum_{n=1}^{\infty} a_n$ converges but $\sum_{n=1}^{\infty}|a_n|$ diverges.
\end{itemize}
\includegraphics[width=1\textwidth]{program2.png}

\begin{multicols}{2}
Test we consider for proving convergence:
\begin{enumerate}
	\item The integral test
	\item p-test
	\item Comparison test
	\item Limit comparison test
	\item Check the absolute convergence of the series
	\item Ratio Test
	\item Alternating Series Test
\end{enumerate}
\columnbreak

Test we consider for proving divergence:
\begin{enumerate}
	\item The integral test
	\item p-test
	\item Comparison test
	\item Limit comparison test
	\item Ratio Test
	\item Check $\lim_{n \to \infty} \neq 0$ or $\lim_{n \to \infty}$ does not exist.
\end{enumerate}
\end{multicols}

\includegraphics[width=1\textwidth]{program.png}

\subsection{Geometric Series}
There is a special series that we learn about, which is the Geometric Series, notice that the formula on the right hand side is what we called closed form.
A finite geometric series has the form
\[a + ax + ax^2 + \cdots + ax^{n−2} + ax^{n−1}=\frac{a(1-x^n)}{1-x}\text{ For } x \neq 1\]
An infinite geometric series has the form
\[a + ax + ax^2 + \cdots + ax^{n−2} + ax^{n−1}+ax^n +\cdots=\frac{a}{1-x}\text{ For } |x| < 1\]

\newpage

\subsection{Power Series}
\begin{definition}
	A power series about $x = a$ is a sum of constants times powers of $(x - a)$: \\
	$C_0 + C_1(x - a) + C_2(x - a)^2 + \ldots + C_n(x - a)^n + \ldots =	\sum_{n
	=0}^{\infty}	C_n(x - a)^n$.
\end{definition}
Moreover, each power series falls into one of the three following cases, characterized by its radius of convergence, $R$.
\begin{itemize}
\item The series converges only for $x = a$; the radius of convergence is defined to be $R = 0$.
\item The series converges for all values of $x$; the radius of convergence is defined to be
$R = \infty$.
\item There is a positive number $R$, called the radius of convergence, such that the series
converges for $|x - a| < R$ and diverges for $|x - a| > R$.
\end{itemize}
How to find radius of convergence: consider ratio test

The interval of convergence is the interval between $a - R$ and $a + R$, including any
endpoint where the series converges.

\subsection{Taylor Polynomial}
\textcolor{red}{Taylor Polynomial of Degree $n$ Approximating $f(x)$ for $x$ near $a$} is \[f(x) \approx P_n(x)
= f(a) + f'(a)(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \frac{f'''(a)}{3!}(x-a)^3  + \ldots + \frac{f^{(n)}(a)}{n!} (x-a)^n\]
We call $P_n(x)$ the Taylor polynomial of degree $n$ centered at $x = a$, or the Taylor poly
nomial about $x =a$.
\subsection{Taylor Series}
\textcolor{red}{Taylor Series for $f(x)$ about $x=a$} is \[f(x) = f(a) + f'(a)(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \frac{f'''(a)}{3!}(x-a)^3  + \ldots + \frac{f^{(n)}(a)}{n!} (x-a)^n+ \ldots \]
We call $P_n(x)$ the Taylor polynomial of degree $n$ centered at $x = a$, or the Taylor poly
nomial about $x =a$.

$f^{(n)}(a)= \{\text{coefficient of }x^n\}*n!$.

Moreover, there are \textcolor{red}{several important cases} that we consider, each of them is an Taylor expansion of a function about $x=0$:
\begin{itemize}
\item $e^{x}= 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \frac{x^4}{4!} + \frac{x^5}{5!} + \frac{x^6}{6!} + \frac{x^7}{7!} + \frac{x^8}{8!} + \cdots\text{ converges for all } x$
\item $\sin(x)=\sum\limits_{n=0}^\infty \dfrac{x^{2n+1}}{(2n+1)!}\cdot(-1)^n = x-\dfrac{x^3}{3!}+\dfrac{x^5}{5!}-\dfrac{x^7}{7!}+\dots\text{ converges for all } x$
\item $\cos(x)=\sum\limits_{n=0}^\infty \dfrac{x^{2n}}{(2n)!}\cdot(-1)^n = 1-\frac{x^2}{2!}+\frac{x^4}{4!}-\frac{x^6}{6!}+\dots \text{ converges for all } x$
\item $(1 + x)^p = \sum_{k=0}^{\infty} \binom{p}{k} x^k= \sum_{k=0}^{\infty} \frac{p!}{k!(p-k)!} x^k=1 + px + \frac{p(p - 1)}{2!}x^2 + \frac{p(p - 1)(p - 2)}{3!}x^3 + \cdots \text{ converges for } -1 < x < 1$.
\item $\ln(1+x) =\sum_{n = 0}^{\infty}\frac{(-1)^nx^{n+1}}{n+1}= x-\frac{x^2}{2}+\frac{x^3}{3}-\frac{x^4}{4}+\cdots,$

\end{itemize}
Moreover, we can definitely find Taylor Series based on the existing series using \textcolor{red}{four methods}:\\
Substitude/Differentiate/Integrate	/Multiply

\section{Parametric Equations and Polar Coordinate}
\subsection{Parametric Equations}

Summarize, we have the \textcolor{red}{slope}:$\frac{dy}{dx}=\frac{dy/dt}{dx/dt}$
and the \textcolor{red}{concavity} of the parametrized curve to be
$\frac{d^2y}{dx^2}=\frac{(dy/dx)/dt}{dx/dt}$

The quantity $v_x = dx/dt$ is the instantaneous velocity in the $x$-direction; $v_y = dy/dt$ is the
instantaneous velocity in the $y$-direction.
And we call that $(v_x,v_y)$ to be the velocity vector.

The \textcolor{red}{instantaneous speed} :$v = \sqrt{(dx/dt)^2 + (dy/dt)^2} =\sqrt{(v_x)^2 + (v_y)^2}$.\\

Moreover, the \textcolor{red}{distance} traveled from time $a$ to $b$ is $\int^b_a v(t) dt = \int_a^b \sqrt{(dx/dt)^2 + (dy/dt)^2} dt$

\subsection{Polar Coordinate}

\subsubsection{Relation between Cartesian and Polar}

\textcolor{red}{Cartesian to Polar}: $(x,y) \to (r= \sqrt{x^2 + y^2}, \theta) \text{ (Here we have that } \tan \theta = \frac{y}{x} \text{)}$
$\theta$ does not have to be $\arctan(\frac{y}{x})$!

Polar to Cartesian: $(r,\theta) \to (x=r \cos \theta, y=r \sin \theta)$

\subsubsection{Slope, Arc length and Area in Polar Coordinates}
\textcolor{red}{slope} of to be
$\frac{dy}{dx}=\frac{dy/d\theta}{dx/d\theta}$

The \textcolor{red}{arc length} from angle $a$ to $b$ is $\int_a^b \sqrt{(dx/d\theta)^2 + (dy/d\theta)^2} d\theta=\int_a^b \sqrt{r^2 + (dr/d\theta)^2} d\theta$

Fact: \textcolor{red}{area of the sector} is $1/2 r^2 \theta$, we have that for a curve $r = f(\theta)$, with \textcolor{red}{$f(\theta)$ continuously of the same sign}, the area of the region enclosed is $\frac{1}{2}\int^{b}_{a}f(\theta)^2 d\theta$

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage{wrapfig}
\usepackage[super,square]{natbib}
\usepackage{tabularx}

\title{
    % \vspace{-2.5cm}
    Stewardship of Dynamical Systems
}
\author{Luke J. Pereira}
\date{}

\begin{document}

\maketitle

\section{What is water? }
To ask the reader to introspect on dynamical systems is similar to asking a fish about the water they're swimming in.\footnote
{
    This is in reference to the commencement speech delivered by the late American novelist David Foster Wallace, titled This is Water\cite{dfw}. In the speech, Wallace uses the water metaphor to bring attention to the fact that "the most obvious, ubiquitous, important realities are often the ones that are the hardest to see and talk about."
    %, with the end goal of encouraging new graduates toward disciplining their consciousness' tendencies of falling into patterns of self-centeredness, boredom, and vice.
}
It can be difficult to detach ourselves from the processes that we're actively participating in so that we can attempt to analyse them, but in fact, a deep understanding of these processes may be the only way to improve them.

A \emph{dynamical system} can describe nearly every situation that changes or evolves over time. It is usually defined by a set of transition rules that take an initial state of a system and modifies it as time progresses. At the core of these systems, there is a sense of balance: between cause and effect, possible choices and what their branching future choices could be or cease to be, and ultimately, a balance between deterministic and stochastic behaviour. Many of us are familiar with what determinism is -- if we are 100\% sure something will happen, without any uncertainty, then it must be determined. A \emph{stochastic} process, on the other hand, produces unpredictable and random outcomes as a result of interactions between variables which all have their own likelihoods of happening. To illustrate how these seemingly very different concepts interact, we will examine a system which displays both stochastic and deterministic behaviour.

%Upon examination, we find that our mathematical models of dynamical systems have representations in nearly all natural occurring systems in reality and even in consciousness.

\section{A simple machine}
 Suppose we have a machine that follows a simple set of instructions to decide if a cell in a large paper grid will be punched in or left intact. The machine determines the state of a cell depending on the states of the nearby cells in the previous row. To make things easier to write, we can encode these elementary states with binary values $1$ and $0$, corresponding to a cell being punched in or left intact. Now we can say that if the three cells directly below a new cell are $111, 100$, or $000$, then the new cell will have state $0$, otherwise, it will be $1$.

\begin{wrapfigure}{r}{0.5\textwidth}
%   \vspace*{-0.4cm}
  \begin{center}
    \includegraphics[width=0.5\textwidth]{class4.png}
    \caption{Magnified Rule 110}
  \end{center}
\end{wrapfigure}

From this very simple and deterministic rule, we would expect to find well-formed and predictable patterns. Instead, the machine generates results that pass the strictest mathematical tests for randomness and yet, it is not simply generating noise. The deterministic rules produce locally recognizable patterns but the global patterns are highly stochastic. The interplay between formal rules and unpredictability generates outcomes that we didn't expect and the machine appears to display its own form of creativity. If we wanted to find the state of a particular cell in a distant iteration, there would be no way for us to know what it is without running through all the computations of the cells below it.

% The computations and the machine itself are deterministic, but the state is completely unknowable without running all the computations before it.

The machine we've described is a kind of dynamical system known as a \emph{cellular automaton}. It consists of a regular grid of cells, each of which is in one of a finite number of states. The particular system we examined is known as \emph{Rule 110} and is categorized as a class IV automaton. Members of this class of automaton all show the property we found in Rule 110, that is, their future states can not be predicted without simulating every step before it. This automaton was examined in depth by Stephen Wolfram in his 2002 book, \emph{A New Kind of Science} \cite{nks}. In it, Wolfram claims that the universe, including all the activity on our planet and in our minds, is part of a class IV automaton. For Wolfram, this claim along with the argument that no simulator could run faster than the Universe itself is sufficient grounds to argue for the existence of our free will. He suggests that even though our decisions might be predetermined because they exist in a  deterministic universe, they will still be unpredictable since the system we live in is a class IV automaton. Thus, there is no way to determine the future except to let it unfold.

\section{An excellent servant but a terrible master}

Although not all phenomena fit perfectly into this cellular-automata conception of the world, it's undeniable that dynamical systems can be found all around us and have serious impacts on our daily lives and our well-being. Despite all of this, we still have little insight into many of their inner workings, like their transition rules or possible states. It's likely that, because we are overwhelmed by the complexity and unpredictability of the systems, we assume the rules which define them are equally complex so we avoid trying to understand or safeguard them. Moreover, because of the growing complexity of the systems around and within us, it's more important now than ever before in the past for us to attempt to steward these systems so they do not run amok, which could result in depletion of our natural resources, geopolitical conflicts, and even mass mental health issues. It has only recently become crucial for future generations to master these systems which were automatic and unconscious to us in previous generations.

There is a Buddhist saying, ``The mind makes an excellent servant but a terrible master''. It expresses an understanding that the mind requires guidance developed from self-discipline and spiritual training, otherwise, it has a tendency of falling into patterns of self-centred thoughts, boredom, and can form habits of vice. The saying brings to light the balance between our conscious deterministic thoughts and our unconscious stochastic mental patterns. This sentiment regarding the mind can also be expressed about many other dynamical systems of our modern society that are also prone to fall into patterns with destructive consequences. Unrestrained capitalist and consumerist global markets tend to result in environmental and wealth distribution issues, a lack of family planning policies can result in overpopulation, not providing young people with mental health guidance and training can result in increased depression and suicide rates, etc. It is important for us to acknowledge that even though issues may exist on very complex global scales, they are likely restrained to a simple set of rules that dictate their interactions and future states. From this perspective, it is entirely possible and perhaps necessary for individuals to make an effort to introduce safeguards in the form of intelligent and creatively designed rules and regulations that can help push our global society towards a more favourable and sustainable future.

\begin{thebibliography}{2}
\bibitem{dfw}
Foster., Wallace, David (2009). This is water: some thoughts, delivered on a significant occasion about living a compassionate life. Kenyon College. (1st ed.). New York: Little, Brown. ISBN 0316068225.

\bibitem{nks}
Wolfram, Stephen (2002). A New Kind of Science. Champaign, IL. Wolfram Media. 1579550088.

\end{thebibliography}
\end{document}