\documentclass{article}
\usepackage{amsmath, amssymb, amsfonts, amsthm, mathtools}
\usepackage[utf8]{inputenc}
\usepackage[inline]{enumitem}
\usepackage{cancel}
\usepackage{soul}
\usepackage{hyperref}
\usepackage{datetime2}
\newtheorem{theorem}{Theorem}
\newtheorem{lem}{Lemma}
\newtheorem{defn}{Definition}
\usepackage{hyperref}

\setlength\parindent{0pt}

\usepackage{xcolor}
\definecolor{mybgcolor}{RGB}{50, 50, 50} %46, 51, 63

\usepackage{pagecolor}
\pagecolor{mybgcolor}
\color{white}

\usepackage{geometry}
\geometry{
    a4paper,
    total={170mm,257mm},
    left=20mm,
    top=20mm,
}

\usepackage{fancyhdr}
\usepackage[english]{babel}

\pagestyle{fancy}
\fancyhf{}
\rhead{- Aryaman Maithani}
\lhead{MA 214 Notes}

\title{MA 214: Numerical Analysis Notes}
\author{Aryaman Maithani}
\date{\DTMnow\\
Until: Lecture 12}

\begin{document}
\maketitle
\begin{center}
	\textsc{Disclaimer}
\end{center}
This is just a collection of formulae/algorithms compiled together.\\
In the case of algorithms, I explain the procedure concisely. However, do not take this as a substitute for lecture slides as I don't go into the theory at all.\\
Also, I've modified some things compared to the lecture slides wherever I felt it was an error. It also is possible that I've made a typo of my own. So, be warned.\\
Sometimes, I also change the order in the notes compared to how it was taught in slides if I feel like that's more efficient.

\hrulefill

\section{Interpolation}\label{sec:inter}
\begin{enumerate}
	\itemsep1em
	\item \textbf{Lagrange Polynomials}\\
	Let $x_0,\;x_1,\;\ldots,\;x_{n}$ be $n+1$ distinct points in $[a, b].$ Let $f:[a, b] \to \mathbb{R}$ be a function whose value is known at those aforementioned points. \\
	We want to construct a polynomial $p(x)$ of degree $\le n$ such that $p(x_i) = f(x_i)$ for all $i \in \{0, \ldots, n\}.$\\
	Towards this end, we define the polynomials $I_k(x)$ for $k \in \{0, \ldots, n\}$ in the following manner:
	\[I_k(x) := \prod_{i = 0, i \neq k}^{n}\frac{x - x_i}{x_k - x_i}.\]
	(Intuitive understanding: $I_k$ is a degree $n$ polynomial such that $I_k(x_j) = 0$ if $k \neq j$ and $I_k(x_k) = 1.$)\\
	Now, define $p(x)$ as follows:
	\[p(x) := \sum_{i=0}^{n}f(x_i)I_i(x)\]
	\item \textbf{Newton's form}\\
	Let $x_0, x_1, \ldots, x_n$ be $n+1$ distinct points in $[a, b].$ Let $f:[a, b] \to \mathbb{R}$ be a function whose value is known at those aforementioned points. \\
	We want to construct a polynomial $P_n(x)$ of degree $\le n$ such that $p(x_i) = f(x_i)$ for all $i \in \{0, \ldots, n\}.$\\~\\
	We define the divided differences (recursively) as follows:
	\begin{align*}
		f[x_0] &:= f(x_0)\\
		f[x_0, x_1, \ldots, x_k] &:= \dfrac{f[x_1, \ldots, x_k] - f[x_0, \ldots, x_{k-1}]}{x_k - x_0} & \text{ for all } 1 < k \le n
	\end{align*}
	With this in place, the desired polynomial $P_n(x)$ is (not so) simply:
	\begin{align*}
		P_n(x) := f[x_0]  +& f[x_0, x_1](x - x_0)\\
						 +& f[x_0, x_1, x_2](x - x_0)(x - x_1)\\
						 +& \ldots\\
						  &\vdots\\
						 +& f[x_0, x_1, \ldots, x_n](x - x_0)(x - x_1)\cdots(x - x_{n-1})
	\end{align*}
	\emph{Remarks.} Note that $x - x_n$ does not appear in the last term.\\
	Note that given $P_n(x),$ it is simple to construct $P_{n+1}(x).$
	\item \textbf{Osculatory Interpolation}\\
	This is essentially the same as the previous case. \\
	I'll state the problem in the form I think is the simplest. (Any other form can be reduced to this.)\\
	Suppose we are given $k+1$ distinct points $x_0, \ldots, x_k$ in $[a, b]$ and a function $f:[a, b] \to \mathbb{R}$ which is sufficiently differentiable.\\
	Suppose we are given the following values:\\
	\begin{align*}
		f^{(0)}(x_0), f^{(1)}(x_0)&, \ldots, f^{(m_1-1)}(x_0)\\
		f^{(0)}(x_1), f^{(1)}(x_1)&, \ldots, f^{(m_2-1)}(x_1)\\
		&\vdots\\
		f^{(0)}(x_k), f^{(1)}(x_k)&, \ldots, f^{(m_k-1)}(x_k)\\
	\end{align*}
	(Notation: $f^{(0)}(x) = f(x)$ and $f^{(n)}(x)$ is the $n^{\text{th}}$ derivative.)\\
	Thus, we are given $m_1 + m_2 + \cdots m_k =: n+1$ data. As usual, we now want to compute a polynomial $P_n(x)$ that agrees with $f$ at all the data. (That is, all the given derivatives must also be same.) As it goes without saying, $P_n(x)$ must have degree $\le n.$\\~\\
	To do this, we list the above points as follows:
	\[\underbrace{x_0, x_0, \ldots x_0}_{m_1}, \underbrace{x_1, x_1, \ldots, x_1}_{m_2}, \ldots, \underbrace{x_k, x_k, \ldots, x_k}_{m_k}.\]
	Now, we just apply the above (Newton's) formula with the following modification in the definition of the divided difference:
	\[f[\underbrace{x_i, x_i, \ldots, x_i}_{p+1 \text{ times}}] := \dfrac{f^{(p)}(x_i)}{p!}.\]
	\item \label{rich}\textbf{Richardson Extrapolation}\\
	Suppose that for sufficiently small $h \neq 0,$ we have the formula:
	\[M = N_1(h) + k_1h + k_2h^2 + k_3h^3 + \cdots,\]
	for some constants $k_1, k_2, k_3,\ldots.$\\
	Define the following:
	\[N_j(h) := N_{j-1}(h/2) + \frac{N_{j-1}(h/2) - N_{j-1}(h)}{2^{j-1} - 1} \quad \text{ for } j \ge 2.\]
	Choose some $h$ \emph{sufficiently small} (whatever that means). Then, $N_j(h)$ keeps becoming a better approximation of $M$ as $j$ increases.\\
	We create a table of $h$ and $N_j(h)$ as follows:\\
		\[
		\begin{array}{c|c|c|c|c}
			h & N_1(h) & N_2(h) & N_3(h) & N_4(h)\\
			\hline
			h & N_1(h) & & &\\
			h/2 & N_1(h/2) & N_2(h) & & \\
			h/4 & N_1(h/4) & N_2(h/2) & N_3(h) & \\
			h/8 & N_1(h/8) & N_2(h/4) & N_3(h/2) & N_4(h) \\
		\end{array}
		\]
	$N_4(h)$ will be a good approximation, then.\\
	(Look at slide 15 of Lecture 7 for an example.)\\~\\
	\textbf{Special case}\\
	Sometimes, we may have the following scenario:
	\[M = N_1(h) + k_2h^2 + k_4h^4 + \cdots.\]
	In this case, we define:
	\[N_j(h) := N_{j-1}(h/2) + \frac{N_{j-1}(h/2) - N_{j-1}(h)}{4^{j-1} - 1} \quad \text{ for } j \ge 2.\]
	Then, we do the remaining stuff as before.\\
	We define
	\[R_h^k := \frac{N_k(h) - N_k(2h)}{N_k(h/2) - N_k(h)}.\]
	The closer that $R_h^k$ is to $4^k,$ the better is the approximation.
\end{enumerate}

\section{Numerical Integration}
\[I = \int_{a}^{b} f(x) \text{d}x\]
\begin{enumerate}
	\itemsep1em
	\item \textbf{Rectangle Rule}
		\[I \approx (b - a)f(a)\]
		\[E^R = f'(\eta)\frac{(b-a)^2}{2},\;\text{for some }\eta\in[a, b]\]

	\item \textbf{Midpoint Rule}
		\[I \approx (b - a)f\left(\frac{a + b}{2}\right) \]
		\[E^M = \frac{f''(\eta)}{24}(b - a)^3,\;\text{for some }\eta\in[a, b]\]

	\item \textbf{Trapezoidal Rule}
	\[I \approx \frac{1}{2}(b - a)[f(a) + f(b)]\]
	\[E^T = -f''(\eta)\frac{(b - a)^3}{12},\;\text{for some }\eta\in[a, b]\]

	\item \textbf{Corrected Trapezoidal}
	\[I \approx \frac{1}{2}(b - a)[f(a) + f(b)] + \frac{(b - a)^2}{12}(f'(a) - f'(b))\]
	\[E^{CT} = f^{(4)}(\eta)\frac{(b - a)^5}{720},\;\text{for some }\eta\in[a, b]\]
	\item \textbf{Composite Trapezoidal}
	\[I \approx \frac{h}{2}\left[f(x_0) + 2\sum_{i=1}^{N-1}f(x_i) + f(x_N)\right]\]
	\[E_C^{T} = -f''(\xi)\frac{h^2(b-a)}{12},\;\text{for some }\xi\in[a, b]\]
	Here, $Nh = b-a$ and $x_i = a + ih.$\\
	\item \textbf{Simpson's Rule}
	\[I \approx \frac{b - a}{6}\left\{f(a) + 4f\left(\frac{a + b}{2}\right) + f(b)\right\}\]
	\[E^S = -\frac{1}{90}f^{(4)}(\eta)\left(\frac{b-a}{2}\right)^5,\;\text{for some }\eta\in[a, b]\]
	\item \textbf{Composite Simpson's}
	\[I \approx \frac{h}{6}\left[f(x_0) + 2\sum_{i=1}^{N-1}f(x_i) + 4\sum_{i=1}^{N}f\left(x_{i-1} + \frac{h}{2}\right) + f(x_N)\right]\]
	\[E_C^{S} = -f^{(4)}(\xi)\frac{(h/2)^4(b-a)}{180},\;\text{for some }\xi\in[a, b]\]
	Here, $Nh = b-a$ and $x_i = a + ih.$\\
	\item \textbf{Gaussian Quadrature}\\
	Let $Q_{n+1}(x)$ denote the $(n+1)^{\text{th}}$ Legendre polynomial.\\
	Let $r_0, \ldots, r_{n+1}$ be its roots. (These will be distinct, symmetric about the origin and will lie in the interval $[-1, 1].$)\\
	For each $i \in \{0, \ldots, n\},$ we define $c_i$ as follows:
	\[c_i := \int_{-1}^{1} \left(\prod_{k = 0, k \neq i}^{n}\frac{x - x_k}{x_i - x_k}\right) \text{d}x.\]
	Then, we have
	\[\int_{-1}^{1} f(x) \text{d}x \approx \displaystyle\sum_{i=0}^{n}f(r_i)c_i.\]
	Moreover, if $f$ is a polynomial of degree $\le 2n+1,$ then the above is ``approximation'' is exact.\\~\\
	Standard values: \\
	$n = 0:$ $Q_1(x) = x$ and $x_0 = 0.$ $c_0 = 2.$\\
	$n = 1:$ $Q_2(x) = x^2 - \frac{1}{3}$ and $x_0 = -\frac{1}{\sqrt{3}},\;x_1 = \frac{1}{\sqrt{3}}.$ $c_0 = c_1 = 1.$\\
	$n = 2:$ $Q_3(x) = x^3 - \frac{3}{5}x$ and $x_0 = -\sqrt{\frac{3}{5}},\;x_1 = 0,\; x_2 = \sqrt{\frac{3}{5}}.$ $c_0 = c_2 = 5/9,\;c_1 = 8/9.$
	\item \textbf{Improper integrals using Taylor series method}\\
	Suppose we have $f(x) = \frac{g(x)}{(x - a)^p}$ for some $0 < p < 1$ and are asked to calculate $I = \displaystyle\int_{a}^{b} f(x) \text{d}x.$\\
	For the sake of simplicity, I assume $a = 0$ and $b = 1.$\\
	Let $P_4(x)$ denote the fourth Taylor polynomial of $g$ around $a.$ (In this case $0$.)\\
	Now, compute $I_1 = \displaystyle\int_{0}^{1} \frac{P_4(x)}{x^p} \text{d}x.$ This can be integrated exactly. \hfill (Why?)\\
	Now, we approximate $I - I_1.$\\
	Define
	\[G(x) := \left\{\begin{array}{ll}
		\frac{g(x) - P_4(x)}{x^p} & \text{ if } 0 < x \le 1\\
		0 & \text{ if } x = 0
	\end{array}
	\right.\]
	Then, approximate $I_2 = \displaystyle\int_{0}^{1} G(x) \text{d}x$ using the composite Simpson's rule.\\
	Then, $I = I_1 + I_2.$\\~\\
	For the case of $a = 0,\; b = 1$ and $N = 2$ for the composite Simpson's part, we get that \\
	$I_2 \approx \frac{1}{12}[2G(0.5) + 4G(0.25) + 4G(0.75) + G(1)].$\\~\\
	That is, finally:
	\[I \approx \int_{0}^{1} \frac{P_4(x)}{x^p} \text{d}x + \frac{1}{12}[2G(0.5) + 4G(0.25) + 4G(0.75) + G(1)].\]
	%
	\item \textbf{Adaptive Quadrature}\\
	Let $I = \displaystyle\int_{a}^{b} f(x) \text{d}x$ be the integral that we want to approximate.\\
	Suppose that $\epsilon$ is the accuracy to which we want $I.$ That is, we want a number $P$ such that $|P - I| < \epsilon.$\\
	Here is what you do:\\
	Subdivide $[a, b]$ into $N$ intervals: $[x_0, x_1],\;[x_1, x_2],\;\ldots,\;[x_{n-1}, x_n].$ (Naturally, $a = x_0 \le x_1 \le \ldots \le x_n = b$.)\\
	Now, for each subinterval, compute the following values:\\~\\
	$S_i = \dfrac{h}{6}\left(f(x_i) + 4f\left(x_i + \dfrac{h}{2}\right) + f\left(x_{i+1}\right)\right),$ and\\~\\
	$\overline{S_i} = \dfrac{h}{12}\left(f(x_i) + 4f\left(x_i + \dfrac{h}{2}\right) + 2f\left(x_i + \dfrac{h}{2}\right) + 4f\left(x_i + \dfrac{3h}{4}\right) + f(x_{i+1})\right).$\\~\\
	Now, calculate $E_i = \frac{1}{15}|\overline{S_i} - S_i|.$\\~\\
	Now, if $E_i \le \frac{x_{i} - x_{i-1}}{b - a}\epsilon,$ then move on to the next interval.\\
	Otherwise, subdivide again to better approximate $\displaystyle\int_{x_{i-1}}^{x_i} f(x) \text{d}x.$\\~\\
	Finally, sum up all the $\overline{S_i}$s and that's the answer. That is,
	\[I \approx P = \sum_{i=1}^{n}\overline{S_i}.\]
	Check slide 25 of Lecture 6 for example.
	%
	\item \textbf{Romberg Integration}\\
	Essentially the baby of composite Trapezoidal rule and Richardson extrapolation.\\
	Suppose we want to calculate $\displaystyle\int_{a}^{b} f(x) \text{d}x.$\\
	Let $N$ be a power of $2$.\\
	$T_N := \dfrac{h}{2}\left[f(x_0) + 2\displaystyle\sum_{i=1}^{N-1}f(a + ih) + f(x_N)\right],$ where $Nh = b-a.$\\~\\
	Note that $T_N$ can be computed using $T_{N/2}$ (assuming $N \neq 2^0$) as:
	\[T_N = \frac{T_{N/2}}{2} + h\sum_{i=1}^{N/2}f\left(a + (2i - 1)h\right).\]
	In the above, we have $2Nh = b - a.$ (That is, it's not the same as the previous $h.$)\\
	Now for $m \ge 1,$ we define:
	\[T^{m}_N = T^{m-1}_N + \frac{T^{m-1}_N - T^{m-1}_{N/2}}{4^m - 1}.\]
	(Where $T^{0}_N$ is just $T_N.$)\\
	(Also, for some reason, $T'_N$ has been used instead of $T^1_N$.)\\
	Note that $\frac{N}{2^m}$ must be an integer for $T^m_N$ to be defined.
	We create a table as follows:\\
		\[
		\begin{array}{c|c|c|c|c}
			N & T_N & T'_N & T^2_N & T^3_N\\
			\hline
			1 & T_1 & & &\\
			2 & T_2 & T^1_2 & & \\
			4 & T_4 & T^1_4 & T^2_4 & \\
			8 & T_8 & T^1_8 & T^2_8 & T^3_8 \\
		\end{array}
		\]
	$T^3_8$ will be a good approximation, then.\\
	(Look at slide 25 of Lecture 7 for an example.)\\~\\
	\emph{Remark.} It can be shown that $I = T_N + c_2h^2 + c_4h^4 + \cdots.$ This is why we used the special case formula of \ref{sec:inter}. \ref{rich}.
\end{enumerate}

\section{Numerical Differentiation}
\begin{enumerate}
	\itemsep1em
	\item \[f'(a) \approx \frac{f(a + h) - f(a)}{h}\]
	\[E(f) = -\frac{1}{2}hf''(\eta) \quad \text{ for some }\eta\in[a, a+h].\]
	\item \textbf{Central Difference Formula}\\
	\[f'(a) \approx \frac{f(a + h) - f(a-h)}{2h}\]
	\[E(f) = -\frac{1}{6}h^2f^{(3)}(\eta) \quad \text{ for some }\eta\in[a-h, a+h].\]
	Note that this is an $O(h^2)$ approximation. Thus, we can use the special case of \S\ref{sec:inter}. \ref{rich}. for better accuracy.
	\item \[f'(a) \approx \frac{-3f(a) + 4f(a + h) - f(a + 2h)}{2h}\]
	\[E(f) = \frac{1}{3}h^2f^{(3)}(\eta) \quad \text{ for some }\eta\in[a, a+2h].\]\\
	\\
	Formula 2 is always the better one whenever applicable. At end points, formula 3 is better than formula 1.
	\item \textbf{Central difference for second derivative}\\
	\[f''(x_0) = \frac{f(x_0 + h) - 2f(x_0) + f(x_0 - h)}{h^2} - \frac{h^2}{12}f^{(4)}(\xi),\]
	for some $\xi \in (x_0 - h, x_0 + h).$
	%
	\item \textbf{Solving boundary-value problems in ODE}\\
	Suppose that we want to solve the following (linear) ODE:
	\[y''(x) + f(x)y'(x) + g(x)y = q(x)\]
	in the interval $[a, b]$ such that we know $y(a) = \alpha,$ and $y(b) = \beta.$ \\~\\
	Set $h := \frac{b - a}{N}$ for some $N \in \mathbb{N}$ and $x_i = a + ih$ for $h \in \{0, 1, \ldots, N\}.$\\
	Using central difference approximation, we set up $N-1$ linear equations as follows:
	\[\frac{y_{i-1} - 2y_i + y_{i+1}}{h^2} + f(x_i)\frac{y_{i+1} - y_{i-1}}{2h} + g(x_i)(y_i) = q(x_i)\]
	\[i = 1, 2, \ldots, N-1\]
	The above equations can be rearranged as:
	\[\left(1 - \frac{hf_i}{2}\right)y_{i-1} + (-2 + h^2g_i)y_i + \left(1 + \frac{hf_i}{2}\right)y_{i+1} = h^2q_i,\]
	for $i = 1, \ldots, N-1;$ where $f_i = f(x_i)$ and so on.\\
\end{enumerate}

\section{Solution of non-linear equations}
Let $f$ be a continuous function on $[a_0, b_0]$ such that $f(a_0)f(b_0) < 0$ in all these cases. We want to find a root of $f$ in $[a_0, b_0].$ (Existence in implied.)
\begin{enumerate}
	\itemsep1em
	\item \textbf{Bisection Method}\\
	Set $n = 0$ to start with.\\
	Loop over the following:\\
	Set $m = \frac{a_n + b_n}{2}.$\\
	If $f(a_n)f(m) < 0,$ then set $a_{n+1} = a_n$ and $b_{n+1} = m.$\\
	Else, set $a_{n+1} = m$ and $b_{n+1} = b_n.$\\
	Increase $n$ by one.\\
	We still have a root in $[a_n, b_n].$
	%
	\item \textbf{Regula-falsi or false-position method}\\
	Set $n = 0$ to start with.\\
	Loop over the following:\\
	Set $w = \frac{f(b_n)a_n - f(a_n)b_n}{f(b_n) - f(a_n)}.$\\
	If $f(a_n)f(w) < 0,$ then set $a_{n+1} = a_n$ and $b_{n+1} = w.$\\
	Else, set $a_{n+1} = w$ and $b_{n+1} = b_n.$\\
	Increase $n$ by one.\\
	We still have a root in $[a_n, b_n].$
	%
	\item \textbf{Modified regula-falsi}\\
	Set $n = 0$ and $w_0 = a_0$ to start with.\\
	Loop over the following:\\
	Set $F = f(a_n)$ and $G = f(b_n).$\\
	Set $w_{n+1} = \dfrac{Ga_n -Fb_n}{G - F}.$\\
	If $f(a_n)f(w_{n+1}) \le 0,$ then set $a_{n+1} = a_n$ and $b_{n+1} = w_{n+1}$ and $G =f(w_{n+1}).$\\
	Furthermore, if we also have $f(w_{n})f(w_{n+1}) > 0,$ set $F = \frac{F}{2}.$\\
	Else, set $a_{n+1} = w_{n+1}$ and $b_{n+1} = b_n$ and $F = f(w_{n+1}).$\\
	Furthermore, if we also have $f(w_{n})f(w_{n+1}) > 0,$ set $G = \frac{G}{2}.$\\
	Increase $n$ by one.\\
	We still have a root in $[a_n, b_n].$
	%
	\item \textbf{Secant method}\\
	Set $x_0 = a,$ $x_1 = b$ and until satisfied, keep computing $x_n$ given by
	\[x_{n+1} = \frac{f(x_n)x_{n-1} - f(x_{n-1})x_n}{f(x_n) - f(x_{n-1})} \qquad \text{ for } n \ge 1.\]
	\emph{Remark.} This process will be forced to stop if we arrive at $f(x_n) = f(x_{n-1})$ at some point.
\end{enumerate}

\section{Iterative methods}
\begin{enumerate}
	\itemsep1em
	\item \textbf{Newton's Method} \\
	You are given a function $f$ which is continuously differentiable and you want to find its root. You are also given some $x_0.$\\
	Compute the following sequence recursively until satisfied:
	\[x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)} \qquad \text{ for } n \ge 0.\]
	%
	\item \textbf{Fixed point iteration}\\
	Let $I$ be a closed interval in $\mathbb{R}.$ Let $f:I\to I$ be a differentiable function such that there exists some $K \in [0, 1)$ such that $|f'(x)| \le K$ for all $x \in I.$\\
	Then, there is a unique $\xi \in I$ such that $f(\xi) = \xi.$ To find this fixed point, choose any $x_0 \in I$ and define the sequence
	\[x_n := f(x_{n-1}) \quad n \ge 1.\]
	Then, $x_n \to \xi.$
	\item \textbf{Aitken's} $\mathbf{\Delta^2}$ \textbf{Process}\\
	\textbf{Definition.} Given a sequence $(x_n),$ let $\Delta x_n := x_{n+1} - x_n.$\\
	Then, $\Delta^2x_n = x_{n+2} - 2x_{n+1} + x_n.$\\~\\

	Given a sequence $x_0, x_1, \ldots$ converging to $\xi,$ calculate $\widehat{x_1}, \widehat{x_2}, \ldots$ by
	\[\widehat{x_n} := x_{n+1} - \frac{(\Delta x_n)^2}{\Delta^2x_{n-1}}.\]
	Then, $\widehat{x_n} \to \xi.$\\~\\
	%
	If the sequence $x_0, x_1, \ldots$ converges linearly to $\xi,$ that is, if
	\[\xi - x_{n+1} = K(\xi - x_n) + \theta(\xi - x_n), \qquad \text{ for some } K \neq 0\]
	then $\widehat{x_n} = \xi + O(\xi - x_n),$ that is, $\dfrac{\widehat{x_n} - \xi}{x_n - \xi} \to 0.$
	\item \textbf{Steffensen iteration}\\
	Let $g(x)$ be the function whose fixed point is desired. Let $y_0$ be some given point.\\
	Set $n = 0$ to start with.\\
	Loop over the following:\\
	Set $x_0 = y_n.$\\
	Set $x_1 = g(x_0),\;x_2 = g(x_1).$\\
	Calculate $\Delta x_1$ and $\Delta^2x_0.$\\~\\
	Set $y_{n+1} = x_2 - \dfrac{(\Delta x_1)^2}{\Delta^2x_0}.$\\~\\
	Increase $n$ by 1.\\~\\
	Note that we get a sequence $y_0, y_1, y_2, \ldots.$ However, we only ever have $x_0, x_1$ and $x_2.$
\end{enumerate}

\begin{defn}
	Let $x_0, x_1, x_2, \ldots$ be a sequence that converges to $\xi$ and set $e_n = \xi - x_n.$\\
	If there exists a number $P$ and a constant $C \neq 0$ such that
	\[\lim_{n\to \infty}\frac{|e_{n+1}|}{|e_n|^P} = C,\]
	then $P$ is called the \textbf{order of convergence} and $C$ is called \textbf{asymptotic error constant}.
\end{defn}

\textbf{Examples.}
\begin{enumerate}
	\item \textbf{Fixed point iteration}\\
	$\xi$ fixed point of $g:I\to I$ and $g'(\xi) \neq 0.$\\
	$P = 1$ and $C = |g'(\xi)|.$
	\item \textbf{Newton's method}\\
	$\displaystyle\lim_{n\to \infty}\frac{|e_{n+1}|}{|e_n|} = \frac{1}{2}\left|\frac{f''(\xi)}{f'(\xi)}\right|.$\\
	(If $\xi$ is a double root, then $P = 1.$)
	\item \textbf{Secant method}
	\[|e_{n+1}| = C|e_n||e_{n-1}|\]
	$P = \frac{1 + \sqrt{5}}{2} = 1.618\ldots.$\\
	\[\lim_{n\to \infty}\frac{|e_{n+1}|}{|e_n|^P} = \left|\frac{1}{2}\frac{f''(\xi)}{f'(\xi)}\right|^{1/P},\;\text{ provided }f'(\xi) \neq 0.\]
\end{enumerate}

\begin{theorem}
	Let $f:[a, b] \to \mathbb{R}$ be in $C^2[a, b]$ and let the following conditions be satisfied:
	\begin{enumerate}[nosep]
		\item $f(a)f(b) < 0,$
		\item $f'(x) \neq 0,$ for all $x \in [a, b],$
		\item $f''(x)$ doesn't change sign in $[a, b]$ (might be zero at some points),
		\item
		\[\frac{|f(a)|}{|f'(a)|} \le b - a \text{ and }\frac{|f(b)|}{|f'(b)|} \le b - a.\]
	\end{enumerate}
	Then, the Newton's method converges to the unique solution $\xi$ of $f(x) = 0$ in $[a, b]$ for any choice $x_0 \in [a, b].$
\end{theorem}

\section{Solving systems of linear equations}
\subsection{LU Factorisation}

	We want solve $Ax = b$ where $A$ is some known $n\times n$ matrix, $b$ a known $n\times1$ matrix and $x$ is unknown.\\
	\emph{Assumption}: $Ax = b$ can be solved without any row interchange.\\
	We define (finite) sequences of matrices $A^{(n)} = [a^{(n)}_{ij}]$ and $b^{(n)}.$ \\
	Define $A^{(1)} := A.$ Let $m_{ji} := \dfrac{a^{(i)}_{ji}}{a^{(i)}_{ii}}.$\\~\\
	Define $M^{(1)}$ as
	\[M^{(1)} :=
	\begin{bmatrix}
		1 & 0 & 0 & \cdots & 0\\
		-m_{21} & 1 & 0 & \cdots & 0\\
		-m_{31} & 0 & 1 & \cdots & 0\\
		\vdots & \vdots & \vdots & \ddots & \vdots\\
		-m_{n-1, 1} & 0 & 0 & \cdots & 0\\
		-m_{n1} & 0 & 0 & \cdots & 1\\
	\end{bmatrix}\]
	Thus, we can write $M^{(1)}A^{(1)}x = M^{(1)}b.$\\
	Let $A^{(2)} := M^{(1)}A^{(1)}$ and $b^{(2)} = M^{(1)}b^{(1)}.$\\
	Note that $A^{(2)}$'s first column will just have the top element non-zero and everything below will be zero.\\~\\
	We can similarly construct the later matrices that perform the row operations. In general, we have:
	\[M^{(k)} :=
	\begin{bmatrix}
		1 & 0 & \cdots & 0 & \cdots & 0\\
		0 & 1 & \cdots & 0 & \cdots & 0\\
		\vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\
		0 & 0 & \cdots & 1 & \cdots & 0\\
		0 & 0 & \cdots & -m_{k+1, k} & \cdots & 0\\
		\vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\
		0 & 0 & \cdots & -m_{n, k} & \cdots & 1\\
	\end{bmatrix},\]
along with
\[A^{(k+1)} = M^{(k)}A^{(k)} = M^{(k)}\cdots M^{(1)}A, \text{ and}\]
\[b^{(k+1)} = M^{(k)}b^{(k)} = M^{(k)}\cdots M^{(1)}b.\]
Finally, set $U = A^{(n)}$ and $L = [M^{(1)}]^{-1}\cdots \left[M^{(n-1)}\right]^{-1}.$\\
Then, we have
\[L =
\begin{bmatrix}
	1 & 0 & 0 & \cdots & 0\\
	m_{21} & 1 & 0 & \cdots & 0\\
	m_{31} & m_{32} & 1 & \cdots & 0\\
	\vdots & \vdots & \vdots & \ddots & \vdots\\
	m_{n-1, 1} & m_{n-1, 2} & m_{n-1, 3} & \cdots & 0\\
	m_{n1} & m_{n2} & m_{n3} & \cdots 1
\end{bmatrix}.\]
Thus, we have $A = LU.$ Now, set $y = Ux.$ We solve $Ly = b$ for $y.$ This is easy because $L$ is lower triangular.\\
Then, we solve $Ux = y$ for $x.$\\~\\
Check slide 27 of Lecture 11 for example.\\~\\
LU factorisation requires $\theta(n^3/3)$ multiplication and division and $\theta(n^3/3)$ addition and subtraction. However, once the factorisation is done, it can be used again and again to solve $Ax = b$ for different values of $b.$ The number of equations then taken to solve $Ax = b$ is $\theta(2n^2).$

\subsection{LPU Factorisation}
In the previous case, we assumed that we aren't doing any row operations on $A$ when doing the Gauss elimination. Now, we relax that condition.\\
Suppose, if possible, we have done some row exchanges while doing Gauss elimination on $A.$ Construct the matrix $P$ which is obtained upon doing those exact row exchanges on $I.$ (Only the row exchanges.)\\
Suppose that the the final product of doing the Gauss elimination following the previous procedure gave us the upper triangular matrix as $U$ and lower one as $L.$\\
Then, that means $PA = LU.$ Thus, solving $Ax = b$ can first be reduced to $PAx = Pb = b'.$ Now, converting $PA$ to $LU$ does not take any row exchange, so we're back in business. (Back to the previous case, that is.)\\
Check slide 15 of Lecture 12 for example.

\subsection{Cholesky's Algorithm}
\begin{defn}
	A matrix $A$ is positive definite if:
	\begin{enumerate}
		\item $A = A^t,$ and
		\item $x^tAx > 0$ for all $x \neq 0.$
	\end{enumerate}
\end{defn}
Given any positive definite matrix $A,$ we can write $A = LL^t,$ where $L$ is lower triangular. (And thus, $L^T$ would be upper triangular.)\\
The algorithm to do so is as follows:\\
Let $L = (l_{ij}).$ We now construct these entries.\\~\\
\texttt{
	set $l_{ii} := \sqrt{a_{11}}.$\\
	for $j = 2, 3, \ldots, n:$ \\
	\phantom{123456} set $l_{j1} := \frac{a_{j1}}{l_{11}}.$\\
	for $i = 2, 3, \ldots, n - 1:$\\~\\
	\phantom{123456} set $l_{ii} := \sqrt{\displaystyle a_{ii} - \sum_{k=1}^{i-1}l_{ik}^2}$\\~\\
	\phantom{123456} for $j = i+1, \ldots, n:$ \\
	\phantom{123456}\phantom{123456}set $l_{ji} := \displaystyle a_{ji} - \frac{\displaystyle\sum_{k=1}^{i-1}l_{jk}l_{ik}}{l_{ii}}.$\\~\\
	set $l_{nn} := \sqrt{a_{nn} - \displaystyle\sum_{k=1}^{n-1}l_{nk}^2}.$
}\\~\\
Note that in the above, we've only defined $l_{ij}$ for when $i \ge j.$ The others are obviously $0.$\\~\\
Cholevsky's algorithm requires $\theta(n^3/6)$ multiplication and division and $\theta(n^3/6)$ addition and subtraction.

\subsection{Scaled partial pivoting}
Suppose we want to solve $Ax = b.$ This could lead to round off errors (check slide 17 of Lecture 12). One possible way to combat this is to do the following:\\~\\
First, define the scale factor $s_i$ of row $i$ as the maximum modulus of any element in the row. More precisely, $s_i := \displaystyle\max_{1 \le j \le n} |a_{ij}|.$\\
(Note that we assume that $A$ is invertible and thus, every $s_i$ will be nonzero.)\\~\\
Now, along the first column, we find the row which has the greatest ratio $\dfrac{|a_{k1}|}{s_k}.$ Suppose this is achieved for row $p,$ that is,
\[\frac{|a_{p1}|}{s_p} = \max_{1 \le k \le n}\frac{|a_{k1}|}{s_k}.\]
If $p \neq 1,$ then we perform $R_1 \leftrightarrow R_p.$\\
(Now, when we refer to $s_p$, we will mean the scale factor of the new $p,$ that is, the original $s_1.$ Similarly, for $s_1.$)\\
Now, we make everything below $a_{11}$ zero using the standard row operation.\\~\\
Now, we repeat the same thing by going along column 2 and finding the row which has the greatest ratio $\dfrac{|a_{k2}|}{s_k}.$ If the row is not 2, then we perform the interchange and go on.\\~\\
Two things I would like to point out:\\
1. We never change the value of $s_i$ except the interchanging that we do. In particular, after doing the row operations like $R_2 - m_{21}R_1,$ I will \textbf{not} recalculate the value of $s_2.$\\
2. When considering the ratios $\dfrac{|a_{ki}|}{s_k}$ for column $i,$ we will consider the $a_{ki}$ that is currently present. That is, in this case, we will consider the values that we get after performing all the operations that have been performed until that point.\\~\\
Consider the example given in slide 22 of Lecture 12. After doing $R_1 \leftrightarrow R_3, R_2 - \frac{4.01}{1.09}R_1,$ and $R_3 - \frac{2.11}{1.09}R_1,$ we \emph{do not} recalculate $s_i.$ (We do exchange the values.)\\
However, when considering the ratios, we consider the new matrix for taking $a_{k2}$ obtained after the above subtractions.
\end{document}


\chapter{Methodology}
\label{ch:methodology}

\begin{itemize}
    \item Expound \#4 (\emph{What is your idea to address this gap?}), \#5 (\emph{Why do you think your idea will work?}), and \#10 (\emph{Introduce the idea on how to solve this gap}).
    \item Propose a solution. Use math models, diagrams, and algorithms to formalize your method.
    \item Cite related literature of the building blocks used in the proposed method.
    \item Justify every decision in your proposed method (\eg we used batch normalization to stabilize the training)
\end{itemize}

% Extracted from the CVPR template
\section{Style and Formatting}
\label{sec:style-formatting}

\subsection{Mathematics}

Please number all of your sections and displayed equations as in these examples:
\begin{equation}
  E = m\cdot c^2
  \label{eq:important}
\end{equation}
and
\begin{equation}
  v = a\cdot t.
  \label{eq:also-important}
\end{equation}
It is important for readers to be able to refer to any particular equation.
Just because you did not refer to it in the text does not mean some future reader might not need to refer to it.
It is cumbersome to have to use circumlocutions like ``the equation second from the top of page 3 column 1''.
(Note that the ruler will not be present in the final copy, so is not an alternative to equation numbers).
All authors will benefit from reading Mermin's description of how to write mathematics:
\url{http://www.pamitc.org/documents/mermin.pdf}.

\begin{figure}[t]
  \centering
  \fbox{\rule{0pt}{2in} \rule{0.9\linewidth}{0pt}}
   %\includegraphics[width=0.8\linewidth]{egfigure.eps}

   \caption[Example of a shortened caption.]{Example of a long caption title to demonstrate how to use the optional caption parameter.
   It is set in Roman so that mathematics (always set in Roman: $B \sin A = A \sin B$) may be included without an ugly clash.}
   \label{fig:onecol}
\end{figure}

\subsection{Footnotes}

Please use footnotes\footnote{This is what a footnote looks like.
It often distracts the reader from the main flow of the argument.} sparingly.
Indeed, try to avoid footnotes altogether and include necessary peripheral observations in the text (within parentheses, if you prefer, as in this sentence).
If you wish to use a footnote, place it at the bottom of the page on which it is referenced.
Use Times 10-point type, single-spaced.

\subsection{Cross-references}

For the benefit of author(s) and readers, please use the
{\small\begin{verbatim}
  \Cref{...}
\end{verbatim}}  command for cross-referencing to figures, tables, equations, or sections.
This will automatically insert the appropriate label alongside the cross-reference as in this example:
\begin{quotation}
  To see how our method outperforms previous work, please see \Cref{fig:onecol} and \Cref{tab:example}.
  It is also possible to refer to multiple targets as once, \eg~to \Cref{fig:onecol,fig:short-a}.
  You may also return to \Cref{sec:style-formatting} or look at \Cref{eq:also-important}.
\end{quotation}
If you wish to abbreviate the label, you can use the
{\small\begin{verbatim}
  \cref{...}
\end{verbatim}}
command. Here is an example:
\begin{quotation}
  \cref{fig:onecol} is also quite important.
\end{quotation}

%-------------------------------------------------------------------------
\subsection{References}

List and number all bibliographical references in 12-point Times, single-spaced, at the end of your manuscript.
To make a reference in the text, use the
{\small\begin{verbatim}
  \autocite{...}
\end{verbatim}} command instead of plain \verb`\cite{}`, for example~\autocite{Authors14}.
Where appropriate, include page numbers and the name(s) of editors of referenced books.
When you cite multiple papers at once, please make sure that you cite them in numerical order like this \autocite{Alpher02,Alpher03,Alpher05,Authors14b,Authors14}.
If you use the template as advised, this will be taken care of automatically.

\begin{table}
  \centering
  \caption[Results]{Results. Ours is better. Note that table captions should be placed above, in contrast to figure captions which are positioned below.}
  \begin{tabular}{@{}lc@{}}
    \toprule
    Method & Frobnability \\
    \midrule
    Theirs & Frumpy \\
    Yours & Frobbly \\
    Ours & Makes one's heart Frob\\
    \bottomrule
  \end{tabular}
  \label{tab:example}
\end{table}

%-------------------------------------------------------------------------
\subsection{Illustrations, graphs, and photographs}

All graphics should be centered.
In \LaTeX, avoid using the \texttt{center} environment for this purpose, as this adds potentially unwanted whitespace.
Instead use
{\small\begin{verbatim}
  \centering
\end{verbatim}}
at the beginning of your figure.
Please ensure that any point you wish to make is resolvable in a printed copy of the paper.
Resize fonts in figures to match the font in the body text, and choose line widths that render effectively in print.
Readers (and reviewers), even of an electronic copy, may choose to print your paper in order to read it.
You cannot insist that they do otherwise, and therefore must not assume that they can zoom in to see tiny details on a graphic.

When placing figures in \LaTeX, it's almost always best to use \verb+\includegraphics+, and to specify the figure width as a multiple of the line width as in the example below
{\small\begin{verbatim}
   \usepackage{graphicx} ...
   \includegraphics[width=0.8\linewidth]
                   {myfile.pdf}
\end{verbatim}
}

%-------------------------------------------------------------------------
\subsection{Color}

If you use color in your plots, please keep in mind that a significant subset of reviewers and readers may have a color vision deficiency; red-green blindness is the most frequent kind.
Hence avoid relying only on color as the discriminative feature in plots (such as red \vs green lines), but add a second discriminative feature to ease disambiguation.

\subsection{Miscellaneous}

\noindent
Compare the following:\\
\begin{tabular}{ll}
 \verb'$conf_a$' &  $conf_a$ \\
 \verb'$\mathit{conf}_a$' & $\mathit{conf}_a$
\end{tabular}\\
See The \TeX book, p165.

The space after \eg, meaning ``for example'', should not be a sentence-ending space.
So \eg is correct, {\em e.g.} is not.
The provided \verb'\eg' macro takes care of this.

When citing a multi-author paper, you may save space by using ``et alia'', shortened to ``\etal'' (not ``{\em et.\ al.}'' as ``{\em et}'' is a complete word).
If you use the \verb'\etal' macro provided, then you need not worry about double periods when used at the end of a sentence as in Alpher \etal.
However, use it only when there are three or more authors.
Thus, the following is correct:
   ``Frobnication has been trendy lately.
   It was introduced by Alpher~\autocite{Alpher02}, and subsequently developed by
   Alpher and Fotheringham-Smythe~\autocite{Alpher03}, and Alpher \etal~\autocite{Alpher04}.''

This is incorrect: ``... subsequently developed by Alpher \etal~\autocite{Alpher03} ...'' because reference~\autocite{Alpher03} has just two authors.

\begin{figure*}
  \centering
  \begin{subfigure}{0.68\linewidth}
    \fbox{\rule{0pt}{2in} \rule{.9\linewidth}{0pt}}
    \caption{An example of a subfigure.}
    \label{fig:short-a}
  \end{subfigure}
  \hfill
  \begin{subfigure}{0.28\linewidth}
    \fbox{\rule{0pt}{2in} \rule{.9\linewidth}{0pt}}
    \caption{Another subfigure.}
    \label{fig:short-b}
  \end{subfigure}
  \caption{Example of a short caption title. It should be centered.}
  \label{fig:short}
\end{figure*}

\section{Chapter Summary}

\textbf{Start Strong:} For every chapter (except possibly the Problem Statement), make an introduction (2 or 3 paragraphs) on what the chapter is all about.

\textbf{Stay Strong:} Explain ideas in the simplest and most direct way that many people in your field can understand. If a certain topic is a bit specialized or hard to remember, make a concise introduction. Point the reader to a reference for further understanding. Each chapter should be complete or stand-alone and concise.

\textbf{Finish Strong:} At the end, make a summary (2 or 3 paragraphs) to re-emphasize the points discussed in the chapter.


\chapter{Goal}

The goal is to find more efficient(in terms of solution cost) short-cutting heuristics which are used in the last step of Christofides' algorithm for finding the TSP tour.


\lab{Pandas 2: Plotting}{Pandas 2: Plotting}
\objective{Clear, insightful visualizations are a crucial part of data analysis.
To facilitate quick data visualization, pandas includes several tools that wrap around matplotlib.
These tools make it easy to compare different parts of a data set, explore the data as a whole, and spot patterns and correlations the data.}

\begin{info}
This lab will be done using Colab Notebooks.
These notebooks are similar to Jupyter Notebooks but run remotely on Google's servers.
Open a Google Colab notebook by going to your Google Drive account and creating a new Colaboratory file.
If making a Colaboratory file is not an option, download the application Colaboratory onto your Google Drive.
Once opening a new Colab Notebook, upload the file \li{pandas2.ipynb}.
To make the data files accessible, run the following at the top of the lab:
\begin{lstlisting}
>>> from google.colab import files

>>> uploaded = files.upload()
\end{lstlisting}
This will prompt you upload files for this notebook.
For this lab, upload \li{crime_data.csv} and \li{college.csv}.

Once the lab is complete, delete BOTH lines of code used for uploading files (the import statement and the upload statement) and download as a \li{.py} file to your git repository.
Push the newly made \li{pandas2.py} file.
\end{info}

\section*{Overview of Plotting Tools} % =======================================

The main tool for visualization in pandas is the \li{plot()} method for \li{Series} and \li{DataFrames}.
The method has a keyword argument \li{kind} that specifies the type of plot to draw.
The valid options for \li{kind} are detailed below.

\begin{table}[H]
\begin{tabular}{r|c|l}
Plot Type & \li{plot()} ID & Uses and Advantages \\ \hline
Line plot & \li{"line"} & Show trends ordered in data; easy to compare multiple data sets \\
Scatter plot & \li{"scatter"} & Compare exactly two data sets, independent of ordering \\
Bar plot & \li{"bar"}, \li{"barh"} & Compare categorical or sequential data \\
Histogram & \li{"hist"} & Show frequencies of one set of values, independent of ordering \\
% Kernel Density Estimation & \li{"kde"} & Estimate a smooth distribution \\
Box plot & \li{"box"} & Display min, median, max, and quartiles; compare data distributions \\
Hexbin plot & \li{"hexbin"} & 2D histogram; reveal density of cluttered scatter plots \\
\end{tabular}
\caption{Types of plots in pandas.
The plot ID is the value of the keyword argument \li{kind}.
That is, \li{df.plot(kind="scatter")} creates a scatter plot.
The default \li{kind} is \li{"line"}.}
\label{table:pandas-plot-options}
\end{table}

The \li{plot()} method calls \li{plt.plot()}, \li{plt.hist()}, \li{plt.scatter()}, and other matplotlib plotting function, but it also assigns axis labels, tick marks, legends, and a few other things based on the index and the data.
Most calls to \li{plot()} specify the \li{kind} of plot and which \li{Series} to use as the \li{x} and \li{y} axes.
By default, the \li{index} of the \li{Series} or \li{DataFrame} is used for the \li{x} axis.

% In Pandas, a \li{DataFrame} is an ordered collection of \li{Series}.
% A Series is similar to a dictionary, with values assigned to various labels, or indices.
% Each Series becomes a column in the data frame, with each row corresponding to an index.
% When several Series are combined into a single data frame, it becomes very easy to compare and visualize data.
% Each entry has an associated index and column.

\begin{lstlisting}
>>> import pandas as pd
>>> from matplotlib import pyplot as plt

>>> budget = pd.read_csv("new_budget.csv", index_col="Year")
>>> budget.plot(y="Rent")  # Plot rent against the index (date).
\end{lstlisting}

\begin{figure}[H]
\centering
\includegraphics[width=.7\textwidth]{figures/rent.pdf}
\end{figure}

In this case, the call to the \li{plot()} method is essentially equivalent to the following code.

\begin{lstlisting}
>>> plt.plot(budget.index, budget['Rent'], label='Rent')
>>> plt.xlabel(budget.index.name)
>>> plt.xlim(min(budget.index), max(budget.index))
>>> plt.legend(loc='best')
\end{lstlisting}

The \li{plot()} method also takes in many keyword arguments for matplotlib plotting and annotation functions.
For example, setting \li{legend=False} disables the legend, providing a value for \li{title} sets the figure title, \li{grid=True} turns a grid on, and so on.
For more customizations, see \url{https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.plot.html}.

\subsection*{Visualizing an Entire Data Set} % --------------------------------

A good way to start analyzing an unfamiliar data set is to visualize as much of the data as possible to determine which parts are most important or interesting.
For example, since the columns in a \li{DataFrame} share the same index, the columns can all be graphed together using the index as the $x$-axis.
By default, the \li{plot()} method attempts to plot \textbf{every} \li{Series} (column) in a \li{DataFrame}.
This is especially useful with sequential data, like the budget data set.

\begin{lstlisting}
>>> # Plot all columns together against the index.
>>> budget.plot(linewidth=1)

>>> # Plot all columns together except for 'Living Expenses' and 'Rent'.
>>> budget.drop(["Living Expenses", "Rent"], axis=1).plot(linewidth=1)
\end{lstlisting}

\begin{figure}[H] % Plot everything together.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/all_budget.pdf}
    \caption{All columns of the budget data set on the same figure, using the index as the $x$-axis.}
    \label{fig:budget-all}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/all_budget2.pdf}
    \caption{All columns of the budget data set except \li{"Living Expenses"} and \li{"Rent"}.}
    \label{fig:budget-all-v2}
\end{subfigure}
\caption{}
\end{figure}

While plotting every \li{Series} at once can give an overview of all the data, the resulting plot is often difficult for the reader to understand.
For example, the budget data set has 9 columns, so the resulting figure, Figure \ref{fig:budget-all}, is fairly cluttered.

One way to declutter a visualization is to examine less data.
Notice that \li{'Living Expenses'} has values much bigger than the other columns.
Dropping this column, as well as \li{'Rent'}, gives a better overview of the data, shown in Figure \ref{fig:budget-all-v2}.

\begin{warn}
Often plotting all data at once is unwise because columns have \textbf{different units of measure}.
Be careful not to plot parts of a data set together if those parts do not have the same units or are otherwise incomparable.
\end{warn}

Another way to declutter a plot is to use subplots.
To quickly plot several columns in separate subplots, use \li{subplots=True} and specify a shape tuple as the \li{layout} for the plots.
Subplots automatically share the same $x$-axis.
Set \li{sharey=True} to force them to share the same $y$-axis as well.

\begin{lstlisting}
>>> budget.plot(y=['Dining Out','Gas','Out With Friends', 'Netflix'],
...                 subplots=True, layout=(2,2), sharey=True,
...                 style=['-','--','-.',':'])
\end{lstlisting}

\begin{figure}[H]
    \includegraphics[width=.8\textwidth]{figures/line_subplots.pdf}
\end{figure}

As mentioned previously, the \li{plot()} method can be used to plot different kinds of plots.
One possible kind of plot is a histogram.
Since plots made by the \li{plot()} method share an $x$-axis by default, histograms turn out poorly whenever there are columns with very different data ranges or when more than one column is plotted at once.

\begin{lstlisting}
>>> # Plot three histograms together.
>>> budget.plot(kind='hist',y=['Gas','Dining Out','Out With Friends'],
...		   	     alpha=.7,bins=10)

>>> # Plot three histograms, stacking one on top of the other.
>>> budget.plot(kind='hist',y=['Gas','Dining Out','Out With Friends'],
...		    	    bins=10,stacked=True)
\end{lstlisting}

\begin{figure}[H] % Try to compare histograms.
\centering
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/bad_hist_unstacked.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/bad_hist_stacked.pdf}
\end{subfigure}
\caption{Two examples of histograms that are difficult to understand because multiple columns are plotted.}
\end{figure}

Thus, histograms are good for examining the distribution of a \textbf{single} column in a data set.
For histograms, use the \li{hist()} method of the \li{DataFrame} instead of the \li{plot()} method.
Specify the number of bins with the \li{bins} parameter.
Choose a number of bins that accurately represents the data; the wrong number of bins can create a misleading or uninformative visualization.

\begin{lstlisting}
>>> budget[["Dining Out","Gas","Other","Out With Friends"]].hist(grid=False, bins=10)
\end{lstlisting}

\begin{figure}[H]
    \includegraphics[width=.7\textwidth]{figures/hist_subplots.pdf}
   \caption{Histograms of \li{"Dining Out"}, \li{"Gas"}, \li{"Other"}, and \li"{Out With Friends"}.}
\end{figure}

% Problem 1
\begin{problem}
Create 2 visualizations for the data in \li{crime_data.csv}.
Make one of the visualizations a histogram.
The visualizations should be well labeled and easy to understand.
Include a short description of your plots as a caption.
\label{prob:hist}
\end{problem}

\section*{Patterns and Correlations} % ========================================

After visualizing the entire data set initially, a good next step is to closely compare related parts of the data.
This can be done with different types of visualizations.
For example, Figure \ref{fig:budget-all-v2} suggests that the \li{"Dining Out"} and \li{"Out With Friends"} columns are roughly on the same scale.
Since this data is sequential (indexed by time), start by plotting these two columns against the index.
Next, create a scatter plot of one of the columns versus the other to investigate correlations that are independent of the index.
Unlike other types of plots, using \li{kind="scatter"} requires both \li{x} and \li{y} columns as arguments.

\begin{lstlisting}
>>> # Plot 'Dining Out' and 'Out With Friends' as lines against the index.
>>> budget.plot(y=["Dining Out", "Out With Friends"])

>>> # Make a scatter plot of 'Dining Out' against 'Out With Friends'
>>> budget.plot(kind="scatter", x="Dining Out", y="Out With Friends",
...                 alpha=.8)
\end{lstlisting}

\begin{figure}[H] % Plot murder against rape.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/line_compare.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/scatter_compare.pdf}
\end{subfigure}
\caption{Correlations between \li{"Dining Out"} and \li{"Out With Friends"}.}
\end{figure}

The first plot shows us that more money is spent on dining out than being out with friends overall.
However, both categories stay in the same range for most of the data.
This is confirmed in the scatter plot by the block in the upper right corner, indicating the common range spent on dining out and being out with friends.

\begin{warn}
When analyzing data, especially while searching for patterns and correlations, \textbf{always} ask yourself if the data makes sense and is trustworthy.
What lurking variables could have influenced the data measurements as they were being gathered?

The crime data set from Problem \ref{prob:hist} is somewhat suspect in this regard.
The murder rate is likely accurate, since murder is conspicuous and highly reported, but what about the rape rate?
Are the number of rapes increasing, or is the percentage of rapes being reported increasing?
It's probably both!
Be careful about drawing conclusions for sensitive or questionable data.
\end{warn}

Another useful visualization used to understand correlations in a data set is a scatter matrix.
The function \li{pd.plotting.scatter_matrix()} produces a table of plots where each column is plotted against each other column in separate scatter plots.
The plots on the diagonal, instead of plotting a column against itself, displays a histogram of that column.
This provides a very quick method for an initial analysis of the correlation between different columns.

\begin{lstlisting}
>>> pd.plotting.scatter_matrix(budget[['Living Expenses','Other']])
\end{lstlisting}

\begin{figure}[H]
    \includegraphics[width=.7\textwidth]{figures/scatter_table.pdf}
    \caption{Scatter matrix comparing \li{"Living Expenses"} and \li{"Other"}.}
\end{figure}

\subsection*{Bar Graphs}

Different types of graphs help to identify different patterns.
Note that the data set \li{budget} gives monthly expenses.
It may be beneficial to look at one specific month.
Bar graphs are a good way to compare small portions of the data set.

As a general rule, horizontal bar charts (\li{kind="hbar"}) are better than the default vertical bar charts (\li{kind="bar"}) because most humans can detect horizontal differences more easily than vertical differences.
If the labels are too long to fit on a normal figure, use \li{plt.tight_layout()} to adjust the plot boundaries to fit the labels in.

\begin{lstlisting}
>>> # Plot all data for the last month in the budget
>>> budget.iloc[-1,:].plot(kind='barh')
>>> plt.tight_layout()

>>> # Plot all data for the last month without 'Rent' and 'Living Expenses'
>>> budget.drop(['Rent','Living Expenses'],axis=1).iloc[-1,:].plot(kind='barh')
>>> plt.tight_layout()
\end{lstlisting}

\begin{figure}[H] % Plot the early 1980s and the last year.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/all_expenses.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/some_expenses.pdf}
\end{subfigure}
\caption{Bar graphs showing expenses paid in the last month of \li{budget}.}
\end{figure}

% Problem 2
\begin{problem}
Using the crime data from the previous problem, identify if a trend exists between \li{Forcible Rape} and the following variables:
\begin{enumerate}
\item \li{Violent}
\item \li{Burglary}
\item \li{Aggravated Assault}
\end{enumerate}

Make sure each graph is clearly labelled and readable.
Include a caption explaining whether there is a visual trend between the variables.
\end{problem}

\section*{Distributional Visualizations} % ====================================

While histograms are good at displaying the distributions for one column, a different visualization is needed to show the distribution of an entire set.
A \emph{box plot}, sometimes called a ``cat-and-whisker'' plot, shows the five number summary: the minimum, first quartile, median, third quartile, and maximum of the data.
Box plots are useful for comparing the distributions of relatable data.
However, box plots are a basic summary, meaning that they are susceptible to  miss important information such as how many points were in each distribution.

\begin{lstlisting}
>>> # Compare the distributions of four columns.
>>> budget.plot(kind="box", y=["Gas","Dining Out","Out With Friends","Other"])

>>> # Compare the distributions of all columns but 'Rent' and 'Living Expenses'.
>>> budget.drop(["Rent", "Living Expenses"], axis=1).plot(kind="box",
>>>												        	vert=False)
\end{lstlisting}

\begin{figure}[H] % Use box plots to compare distributions.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/box1.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/box2.pdf}
\end{subfigure}
\caption{Vertical and horizontal box plots of \li{budget} dataset.}
\end{figure}

% TODO: KDE plot?

% TODO: hexbin plots

% Bar plots, on the other hand, are particularly useful for comparing several categories of data over time, or whenever there is a sense of progression in the index.
% Line plots are better suited to show a more continuous index (such as each year in a century), whereas bar plots are better for a discrete index (a few distinct years).
% Line and bar plots work well when there is a logical progression in the index, such as time.
% However, when frequency of occurrence is more important than the location of the data, histograms and box plots can be more informative.
% Use \li{plot(kind="hist")} to produce a histogram.
% Standard histogram options, such as the number of bins, are also accepted as keyword arguments.
% The \li{alpha} keyword argument makes each bin slightly transparent.

% Sometimes it is helpful to visualize a distribution of values using the box-and-whisker plot which displays the median, first and third quartiles, and outliers.

% Scatter plots are commonly used in a myriad of areas and have a simple implementation in pandas.
% Unlike other plotting commands, \li{scatter} needs both an \li{x} and a \li{y} column as arguments. % TODO: incorporate above!

% The scatter plot option includes many features which can be used to make the plots easier to understand.
% For example, we can change the size of the point based on another column.

\begin{comment} % This should be a pivot table example.
Consider the pydataset \li{HairEyeColor}, which contains the hair and eye color of various individuals.
A scatter plot of hair color vs eye color is relatively useless unless we can see the frequencies with which each combination occurs.
Including the keyword argument \li{s} allows us to control the size of each point.
This can be set to a fixed value or the value in another column.
In the example below, the size of each point is set to the frequency with which each observation occurs.

\begin{lstlisting}
>>> hec = data("HairEyeColor")
>>> X = np.unique(hec["Hair"], return_inverse=True)
>>> Y = np.unique(hec["Eye"], return_inverse=True)
>>> hec["Hair"] = X[1]
>>> hec["Eye"] = Y[1]
>>> hec.plot(kind="scatter", x="Hair", y="Eye", s=hec["Freq"]*10)
>>> plt.xticks([0,1,2,3], X[0])
>>> plt.yticks([0,1,2,3], Y[0])
\end{lstlisting}

\begin{figure}[H]
    \centering
    \includegraphics[width=.7\textwidth]{figures/hair_eye_scatter.pdf}
    \caption{Frequency of Hair-Eye Color Combinations}
\end{figure}
\end{comment}

\subsection*{Hexbin Plots} % --------------------------------------------------

A scatter plot is essentially a plot of samples from the joint distribution of two columns.
However, scatter plots can be uninformative for large data sets when the points in a scatter plot are closely clustered.
\emph{Hexbin plots} solve this problem by plotting point density in hexagonal bins---essentially creating a 2-dimensional histogram.

The file \texttt{sat\_act.csv} contains 700 self reported scores on the SAT Verbal, SAT Quantitative and ACT, collected as part of the Synthetic Aperture Personality Assessment (SAPA) web based personality assessment project.
The obvious question with this data set is ``how correlated are ACT and SAT scores?''
The scatter plot of ACT scores versus SAT Quantitative scores, Figure \ref{fig:pandas-act-scatter}, is highly cluttered, even though the points have some transparency.
A hexbin plot of the same data, Figure \ref{fig:pandas-act-hexbin}, reveals the \textbf{frequency} of points in binned regions.

\newpage

\begin{lstlisting}
>>> satact = pd.read_csv("sat_act.csv", index_col="ID")
>>> list(satact.columns)
<<['gender', 'education', 'age', 'ACT', 'SATV', 'SATQ']>>

# Plot the ACT scores against the SAT Quant scores in a regular scatter plot.
>>> satact.plot(kind="scatter", x="ACT", y="SATQ", alpha=.8)

# Plot the densities of the ACT vs. SATQ scores with a hexbin plot.
>>> satact.plot(kind="Hexbin", x="ACT", y="SATQ", gridsize=20)
\end{lstlisting}

\begin{figure}[H]
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/scores_scatter.pdf}
    \caption{ACT vs. SAT Quant scores.}
    \label{fig:pandas-act-scatter}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \includegraphics[width=\textwidth]{figures/scores_hexbin.pdf}
    \caption{Frequency of ACT vs. SAT Quant scores.}
    \label{fig:pandas-act-hexbin}
\end{subfigure}
\caption{Scatter plots and hexbin plot of SAT and ACT scores.}
\end{figure}

Just as choosing a good number of \li{bins} is important for a good histogram, choosing a good \li{gridsize} is crucial for an informative hexbin plot.
A large \li{gridsize} creates many small bins and a small \li{gridsize} creates fewer, larger bins.

\begin{info}
Since hexbins are based on frequencies, they are prone to being misleading if the dataset is not understood well. For example, when plotting information that deals with geographic position, increases in frequency may be results in higher populations rather than the actual information being plotted.

\end{info}

See \url{http://pandas.pydata.org/pandas-docs/stable/visualization.html} for more types of plots available in Pandas and further examples.

% Problem 3
\begin{problem}
Use \li{crime_data.csv} to display the following distributions.
\begin{enumerate}
        \item The distributions of \li{Burglary}, \li{Violent}, and \li{Vehicle Theft} across all crimes,
        \item The distributions of \li{Vehicle Theft}s over the values of \li{Robbery}.
\end{enumerate}

As usual, all plots should be labeled and easy to read.
\end{problem}

\section*{Principles of Good Data Visualization} % ============================

Data visualization is a powerful tool for analysis and communication.
When writing a paper or report, the author must make many decisions about how to use graphics effectively to convey useful information to the reader.
Here we will go over a simple process for making deliberate, effective, and efficient design decisions.

\subsection*{Attention to Detail} % -------------------------------------------

Consider the plot in Figure \ref{fig:nolabels}.
It is a scatter plot of positively correlated data of some kind, with \li{temp}--likely temperature--on the $x$ axis and \li{cons} on the $y$ axis.
However, the picture is not really communicating anything about the dataset.
It has not specified the units for the $x$ or the $y$ axis, nor does it tell what \li{cons} is.
There is no title, and the source of the data is unknown.

\begin{figure}[H]
    \centering
    \includegraphics[width=.7\textwidth]{figures/ice_cream_bad.pdf}
    \caption{Non-specific data.}
    \label{fig:nolabels}
\end{figure}

\subsection*{Labels and Citations} % ------------------------------------------

In a homework or lab setting, we sometimes (mistakenly) think that it is acceptable to leave off appropriate labels, legends, titles, and sourcing.
In a published report or presentation, this kind of carelessness is confusing at best and, when the source is not included, even plagiaristic.
Data needs to be explained in a useful manner that includes all of the vital information.

Consider again Figure \ref{fig:nolabels}.
This figure comes from the \li{Icecream} dataset within the \li{pydataset} package, which we store here in a dataframe and then plot:
\begin{lstlisting}
>>> from pydataset import data
>>> icecream = data("Icecream")
>>> icecream.plot(kind="scatter", x="temp", y="cons")
\end{lstlisting}

This code produces the rather substandard plot in Figure \ref{fig:nolabels}.
Examining the source of the dataset can give important details to create better plots.
When plotting data, make sure to understand what the variable names represent and where the data was taken from.
Use this information to create a more effective plot.

The ice cream data used in Figure \ref{fig:nolabels} is better understood with the following information:
\begin{enumerate}
    \item The dataset details ice cream consumption via four-weekly observations from March 1951 to July 1953 in the United States.
    \item \li{cons} corresponds to ``consumption of ice cream per head'' and is measured in pints.
    \item \li{temp} corresponds to temperature, degrees Fahrenheit.
    \item The listed source is: ``Hildreth, C. and J. Lu (1960) \emph{Demand relations with autocorrelated disturbances}, Technical Bulletin No 2765, Michigan State University.''
\end{enumerate}

This information gives important details that can be used in the following code.
As seen in previous examples, pandas automatically generates legends when appropriate.
Pandas also automatically labels the $x$ and $y$ axes, however our data frame column titles may be insufficient.
Appropriate titles for the $x$ and $y$ axes must also list appropriate units.
For example, the $y$ axis should specify that the consumption is in units of \emph{pints per head}, in place of the ambiguous label \li{cons}.

\begin{lstlisting}
>>> icecream = data("Icecream")
# Set title via the title keyword argument
>>> icecream.plot(kind="scatter", x="temp", y="cons", title="Ice Cream Consumption in the U.S., 1951-1953",)
# Override pandas automatic labelling using xlabel and ylabel
>>> plt.xlabel("Temp (Farenheit)")
>>> plt.ylabel("Consumption per head (pints)")
\end{lstlisting}

To add the necessary text to the figure, use either \li{plt.annotate()} or \li{plt.text()}.
Alternatively, add text immediately below wherever the figure is displayed.

\begin{lstlisting}
>>> plt.text(20, .1, r"Source: Hildreth, C. and J. Lu (1960) \emph{Demand"
...     "relations with autocorrelated disturbances}\nTechnical Bulletin No"
...     "2765, Michigan State University.", fontsize=7)
\end{lstlisting}

Both of these methods are imperfect but can normally be easily replaced by a caption attached to the figure.
Again, we reiterate how important it is that you source any data you use; failing to do so is plagiarism.

Finally, we have a clear and demonstrative graphic in Figure \ref{fig:labels}.

\begin{figure}[H]
    \centering
    \includegraphics[width=.7\textwidth]{figures/ice_cream_good.pdf}
    \caption{Source:  Hildreth, C. and J. Lu (1960) \emph{Demand relations with autocorrelated disturbances}, Technical Bulletin No 2765, Michigan State University.}
    \label{fig:labels}
\end{figure}

\begin{warn}
Visualizing data can inherit many biases of the visualizer and as a result can be intentionally misleading.
Examples of this include, but are not limited to, visualizing subsets of data that do not represent the whole of the data and having purposely misconstrued axes.
Every data visualizer has the responsibility to avoid including biases in their visualizations to ensure data is being represented informatively and accurately.
\end{warn}
% Problem 4
\begin{problem}
The dataset \li{college.csv} contains information from 1995 on universities in the United States.
To access information on variable names, go to \url{https://cran.r-project.org/web/packages/ISLR/ISLR.pdf}.
Create 3 plots that compare variables or universities.
These plots should answer questions about the data, e.g. what is the distribution of graduation rates or do schools with more PhD students also take more students from the Top 10 percent of their high school class.
These plots should be easy to understand, have clear variable names, and citations.
\end{problem}

\begin{comment}
\begin{problem}
The \li{pydataset} module contains numerous data sets, each stored as a pandas \li{DataFrame}.
Read data on road accident deaths in the United States using the data set \li{road} in the \li{pydataset} module.
Using this data, answer the following questions:

\begin{enumerate}
\item Does higher population density indicate higher fuel consumption?
\item Does a correlation exist between number of drivers and number of deaths in a state?
\item Does the temperature of a state in January affect its population density?
\end{enumerate}

Support each claim with a visualization and make sure the visualizations are clearly labeled and easy to understand.
Include any citations provided by the data set.

(Hint: Consider how outliers may be skewing the data.)
\end{problem}

% Problem 5
\begin{problem}
The file \li{new_york_crime_clean.csv} contains data on New York City crimes and felonies from 2000-2017 taken from \url{https://www1.nyc.gov/site/nypd/stats/crime-statistics/historical.page}.
Use this data to answer the following questions and give visualizations supporting each claim:
\begin{enumerate}
\item What are the most common and least common crimes?
\item Does a trend exist between robbery and drug felonies? If so, what is the trend?
\item Which crimes have had the largest distributions?
\end{enumerate}
Use a different style of visualization for each question.
Make sure that each plot is well labeled and contains citations.

(Hint: Use plt.axis('scaled') or plt.axis('square') to make sure x- and y-axis are the same.)
\end{problem}
\end{comment}


\documentclass[]{article}
\usepackage[utf8]{inputenc}
\usepackage{cite}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{algorithmic}
\usepackage{graphicx}
\usepackage{textcomp}
\usepackage{xcolor}
\usepackage{multicol}
\usepackage{cuted}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{hyperref}

\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    filecolor=magenta,
    urlcolor=cyan}

\title{ENPM667\textemdash Control of Robotic Systems \\ Project~1: Unscented Kalman Filter}
\author{{Trevian~Jenkins} \\
    {\textit{Maryland Applied Graduate Engineering}} \\
    {\textit{University of Maryland, College Park}} \\
    {College Park, Maryland, USA} \\
    {trj0011@umd.edu}
\\ \\
    {Markose~Jacob} \\
    {\textit{Maryland Applied Graduate Engineering}} \\
    {\textit{University of Maryland, College Park}} \\
    {College Park, Maryland, USA} \\
    {markj11@umd.edu}}
\date{27 November 2020}

%\def\BibTeX{{\rm B\kern-.05em{\sc i\kern-.025em b}\kern-.08em
%    T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}

\begin{document}

\maketitle
\thispagestyle{empty}

\begin{abstract}
The Kalman filter is a widely-used technique for extracting  meaningful information out of input data in the presence of disturbances. While the Kalman filter succeeds in situations where the system model is linear, it fails to approximate the true data correctly beyond the range of linearity. The extended Kalman filter provides a generalization of the Kalman filter for use in nonlinear systems, and it performs well when the disturbance is normally distributed, which is usually assumed.  However, the extended Kalman filter suffers from bias when the data falls under some other statistical distribution. Such a change in distribution may for example occur, as we will see, from switching from polar to Cartesian coordinates. The unscented Kalman filter circumvents this issue by transforming the distribution along with the data. This report summarizes our implementation of the unscented Kalman filter and describes its advantage over the linear and extended Kalman filters, as well as plotting some examples.
\end{abstract}

\newpage

\thispagestyle{empty}
\tableofcontents
\listoffigures

\newpage

\section{Introduction}
The Kalman filter was popularized by Rudolf Kalman in 1960, and it has become the \textit{de facto} standard of statistical estimation and prediction when dealing with data that contains disturbances, such as that of robotic sensors. In a linear system, or a subset of data that can be linearized within a certain domain, the basic linear Kalman filter can sufficiently estimate the true data. However, when the system is generally nonlinear, the standard Kalman filter quickly loses accuracy when estimating the values of state variables. Thus, we seek to use a generalization the Kalman filter for use in nonlinear systems.

We referred to "New extension of the Kalman filter to nonlinear systems," by Julier and Uhlmann, to implement the unscented Kalman filter. In this report, we will first describe the mathematical theory behind the linear, extended, and unscented Kalman filters. We will use a simple sinusoidal example to illustrate the unscented Kalman filter in use. We will also replicate Fig. 1 and the space vehicle trajectory example from the paper to show the practical use of the unscented Kalman filter.

\section{Mathematical Theory}

\subsection{Linear Kalman Filter}

Let us first look at the standard Kalman filter, or the \textit{linear} Kalman Filter. When using the Kalman filter, we desire to regularly update our estimation of the state variables based on the previous estimation of the state variables, the uncertainty of the accuracy of our current state vector, and the interval of time $\Delta t_{P}$ between predictions. Every time the value of the state vector is predicted, the uncertainty, which is stored in a covariance matrix $\boldsymbol{P}$, grows, due to the fact that we are effectively extrapolating the current position since any information was last received. We will use the example of a sine wave to illustrate. The equations of motion we will use for this example are as follows:

\begin{equation}
x(t) = \alpha\sin{\omega t} + \beta\cos{\omega t}
\label{equ:eom1}
\end{equation}
\begin{equation}
\cfrac{d}{dt} x(t) = \dot{x}(t) = \alpha\omega\cos{\omega t} - \beta\omega\sin{\omega t} \\
\label{equ:eom2}
\end{equation}
\begin{equation}
\cfrac{d^{2}}{dt^{2}} x(t) = \ddot{x}(t) = -\alpha\omega^{2}\sin{\omega t} - \beta\omega^{2}\cos{\omega t} \\
\label{equ:eom3}
\end{equation}

We may linearize equations (\ref{equ:eom1}) through (\ref{equ:eom3}) by using the small angle approximation as follows. If the value of $x$ is assumed to be small (\textless \ 0.2 rad), then $\sin{x} \approx x$. The small angle approximation is derived from the Maclaurin series approximation:
\[ f(x) = \sum_{k=0}^{\infty}{\frac{x^{k}f^{(k)}(x)}{k!}} \]
The Maclaurin series for the sine function is
\[\begin{split}
\sin{x} & = \sum_{k=0}^{\infty}{(-1)^{k} \cfrac{x^{2k + 1}}{(2k + 1)!}} \\
        & = x - \frac{x^{3}}{6} + \frac{x^{5}}{120} - \frac{x^{7}}{5040} + \dots \\
        & \approx x,
\end{split}\]
with the final approximation neglecting higher-order terms. A similar derivation is used to approximate the cosine function as
\[\begin{split}
\cos{x} & = \sum_{k=0}^{\infty}{(-1)^{k} \cfrac{x^{2k}}{(2k)!}} \\
        & = 1 - \frac{x^{2}}{2} + \frac{x^{4}}{24} - \frac{x^{6}}{720} + \dots \\
        & \approx 1 - \frac{x^{2}}{2}.
\end{split}\]
For the linear Kalman filter, it is necessary that we make the slightly less accurate, but still reasonable approximation,
\[\begin{split}
\cos{x} \approx 1.
\end{split}\]
This linear approximation is reasonable for many applications and is a precursor for the use of the linear Kalman filter. With the small-angle approximation, the equations of motion then become
\begin{equation}
\begin{split}
x(t) & \approx \alpha t + \beta \\
\dot{x}(t) & \approx \alpha\omega - \beta\omega t \\
\ddot{x}(t) & \approx -\alpha\omega^{2} t - \beta\omega^{2} \approx -\omega^{2}x
\end{split}.
\label{equ:eom_linear}
\end{equation}

Using our equations of motion and the small angle approximation, we may define the state vector $\boldsymbol{x}$ as
\begin{equation}
\boldsymbol{x} = \begin{bmatrix} x_{1} \\ x_{2} \end{bmatrix}
                    = \begin{bmatrix} x \\ \dot{x} \end{bmatrix}.
\label{equ:state_variables}
\end{equation}

The state transition matrix $A$ may then be defined as follows:
\begin{equation}
\boldsymbol{A} = \begin{bmatrix} 1 & \Delta t \\
                -\omega^{2}\Delta t & 1 \end{bmatrix}
\label{equ:a_lkf}
\end{equation}

The measurement matrix $\boldsymbol{H}$ describes the relation between a new measurement $z$ and the state vector $\boldsymbol{x}$. For example, if we have a system that is capable of measuring $x$ but not $\dot{x}$, then $h$ would be $\begin{bmatrix} 1 & 0 \end{bmatrix}$.

With equations (\ref{equ:state_variables}) and (\ref{equ:a_lkf}), we find that the next (\textit{a priori}) values of the state vector and covariance matrix are predicted to be
\begin{equation}
\begin{split}
\boldsymbol{x}_{n}^{-} & = \boldsymbol{A} x_{n-1} \\
\boldsymbol{P}_{n}^{-} & = \boldsymbol{A}\boldsymbol{P}_{n-1}\boldsymbol{A}^{T} + \boldsymbol{Q},
\end{split}
\label{equ:state_predict_lkf}
\end{equation}
where $\boldsymbol{Q}$ is the covariance matrix describing the process noise (from random disturbances, etc.). When new data is received, the measurement step updates $\boldsymbol{x}$ and $\boldsymbol{P}$ as follows:
\begin{equation}
\begin{split}
\boldsymbol{K}_{n} & = \boldsymbol{P}_{n}^{-} \boldsymbol{H}^{T}
(\boldsymbol{H} \boldsymbol{P}_{n}^{-} \boldsymbol{H}^{T}
+ \boldsymbol{R})^{-1} \\
\boldsymbol{x}_{n} & = \boldsymbol{x}_{n}^{-} + \boldsymbol{K}_{n} (z_{n} - H\boldsymbol{x}_{n}^{-}) \\
\boldsymbol{P}_{n} & = (\boldsymbol{I} - \boldsymbol{K}_{n}\boldsymbol{H}) \boldsymbol{P}_{n}^{-}
\end{split}.
\label{equ:state_measure_lkf}
\end{equation}

\subsection{Extended Kalman Filter}

In the case of the extended Kalman filter, we run the prediction and measurement steps as before. The major change, however, is that instead of using matrices to define the state transitions, we simply use a vector function. Equations (\ref{equ:eom1}) through (\ref{equ:eom3}) dictate that the next (\textit{a priori}) value of the state vector is predicted to be
\begin{equation}
\begin{split}
\boldsymbol{x}_{n}^{-} & = f(\boldsymbol{x}_{n-1}) \\
& = \begin{bmatrix} f_{1}(x_{1}, x_{2}) \\
                    f_{2}(x_{1}, x_{2}) \end{bmatrix} \\
& = \begin{bmatrix} x_{1} + x_{2} \Delta t \\
                    x_{2} -\omega^{2} x_{1} \Delta t \end{bmatrix},
\end{split}
\label{equ:state_predict_ekf}
\end{equation}
and when this prediction is made, the uncertainty matrix becomes
\begin{equation}
\boldsymbol{P}_{n}^{-} = \boldsymbol{A}_{n} \boldsymbol{P}_{n-1} \boldsymbol{A}_{n}^{T}
                       + \boldsymbol{W}_{n} \boldsymbol{Q}_{n-1} \boldsymbol{W}_{n}^{T},
\label{equ:uncertainty_predict}
\end{equation}
where $\boldsymbol{A}$ is the Jacobian matrix
\[ \boldsymbol{A}_{ij} = \cfrac{\partial f_{i}}{\partial x_{j}}(\boldsymbol{x}_{n-1}), \]
$\boldsymbol{Q}$ is the covariance matrix describing the process noise $w$, and $\boldsymbol{W}$ is the Jacobian matrix
\[ \boldsymbol{W}_{ij} = \cfrac{\partial f_{i}}{\partial w_{j}}(\boldsymbol{x}_{n-1}). \]

The function $h(\boldsymbol{x})$ describes the relation between a new measurement $z$ and the state vector $\boldsymbol{x}$. Whenever new measurements are retrieved, such as azimuth information in our case, the next (\textit{a posteriori}) value of the state matrix is updated to be
\begin{equation}
\label{equ:state_measure}
\boldsymbol{x}_{n} = \boldsymbol{x}_{n}^{-} + \boldsymbol{K}_{n}[z_{n}
- h(\boldsymbol{x}_{n}^{-})],
\end{equation}
where
\[ \boldsymbol{K}_{n} = \boldsymbol{P}_{n}^{-} \boldsymbol{H}_{n}^{T}
(\boldsymbol{H}_{n} \boldsymbol{P}_{n} \boldsymbol{H}_{n}^{T}
+\boldsymbol{V}_{n} \boldsymbol{R}_{n} \boldsymbol{V}_{n}^{T})^{-1}, \]
$\boldsymbol{H}$ is the Jacobian matrix
\[ \boldsymbol{H}_{ij} = \cfrac{\partial h_{i}}{\partial x_{j}}(\boldsymbol{x}_{n}), \]
$\boldsymbol{R}$ is the covariance matrix describing the measurement noise $v$, and $\boldsymbol{V}$ is the Jacobian matrix
\[ \boldsymbol{V}_{ij} = \cfrac{\partial h_{i}}{\partial v_{j}}(\boldsymbol{x}_{n}). \]
With the arrival of new measurements, uncertainty always decreases, and the new uncertainty matrix $\boldsymbol{P}$ becomes
\begin{equation}
\boldsymbol{P}_{n} = (\boldsymbol{I} - \boldsymbol{K}_{n} \boldsymbol{H}_{n}) \boldsymbol{P}_{n}^{-}.
\label{equ:uncertainty_measure}
\end{equation}

The ($-$) superscript in the $\boldsymbol{x}^{-}$ vector and $\boldsymbol{P}^{-}$ matrices indicate that those values are \textit{a priori} values that were calculated from the predict step, before the arrival of measurement data. We repeat the cycle of obtaining measurements of the current state vector and using those measurements to infer about the next values of the state vector. The output of the Kalman filter provides an improved estimate over the raw sensor data, which is to be used as e.g. the feedback within a control system, or for visualization.

\subsection{Complex-Step Derivative Approximation}

We utilized the method described in this section to compute the Jacobian matrices for the Kalman filter. Traditionally, the derivative of a function $f(x)$ is estimated using the finite difference approximation
\begin{equation}
\label{equ:derivative_approx}
f'(x) \approx \frac{f(x+h) - f(x)}{h}
\end{equation}
for, say, a small stepsize $h \approx 10^{-7}$. However, we wish to avoid cancellation error arising from taking differences of such small values. Thus, the partial derivatives of the aforementioned Jacobian functions are computed using the complex-step derivative approximation described as follows. The derivative of $f(x)$ with respect to $x$ is calculated as
\begin{equation}
\cfrac{\partial f}{\partial x} = \cfrac{\Im [f(x + ih)]}{h},
\label{equ:complex_step}
\end{equation}
where $\Im [z]$ returns the imaginary part of the complex number $z$. The complex step method results from the Taylor series approximation about a real point x:
\begin{equation}
f(x + ih) = f(x) + ihf'(x) - \frac{h^{2}f''(x)}{2!} - \frac{ih^{3}f'''(x)}{3!} + \cdots
\label{equ:complex_step_taylor}
\end{equation}
Taking the imaginary part of equation (\ref{equ:complex_step_taylor}) and dividing by $h$ results in
\begin{equation}
\begin{split}
f'(x) & = \frac{\Im [f(x + ih)]}{h} + \frac{h^{2}f'''(x)}{3!} + \frac{h^{4} f^{(5)}(x)}{5!} + \cdots \\
      & = \frac{\Im [f(x + ih)]}{h} + \mathcal{O}(h^{2}) \\
      & \approx \frac{\Im [f(x + ih)]}{h}.
\end{split}
\label{equ:complex_step_approx}
\end{equation}
The final result truncates the higher-order terms, giving a truncation error $\mathcal{O}(h^{2})$. Since this calculation does not involve a subtraction operation, it is not prone to cancellation errors resulting from subtractions. Therefore, we consider the complex step method superior to the more traditional derivative approximation in equation (\ref{equ:derivative_approx}) for small stepsize $h$, hence our use in the extended Kalman filter.

\subsection{Unscented Kalman Filter}

The basic setup of the unscented Kalman filter is similar to the extended kalman filter. We still run the prediction and measurement steps as we always have. However, we now implement what is known as an \textit{unscented transformation}. This operation transforms the mean and covariance information about a distribution along with the data itself.

The following steps are performed to make this transformation happen:
\begin{enumerate}
    \item Creating the sigma points through a sigma point selection algorithm.
    \item Calculating the transformed set:
        \[\hat{x}_{a,n} = f[x_{a,n}]\]
    \item Predicting the mean:
        \[\hat{\mu}_{a,n} = \sum_{i=0}^{p}{W^{(i)} \hat{x}_{a,n}^{(i)}}\]
    \item Predicting the covariance:
        \[\hat{K}_{a,n} = \sum_{i=0}^{p}{W^{(i)} (\hat{x}_{a,n}^{(i)} - \hat{\mu}_{a,n})(\hat{x}_{a,n}^{(i)} - \hat{\mu}_{a,n})^{T} }\]
    \item Instantiating prediction points:
        \[y_{n}^{(i)} = g[x_{a,n}^{(i)}]\]
    \item Predicting the observation:
        \[y_{n} = g[y_{n}^{(i)}]\]
    \item Calculating the innovation covariance:
        \[\hat{S}_n = \sum_{i=0}^{p}{W^{(i)} (\hat{y}_{n}^{(i)} - \hat{y}_n)(\hat{y}_{n}^{(i)} - \hat{y}_n)^{T} }\]
    \item Calculating the cross covariance matrix:
        \[\hat{K}^{xy}_{n} = \sum_{i=0}^{p}{W^{(i)} (\hat{x}_{n}^{(i)} - \hat{\mu}_n)(\hat{x}_{n}^{(i)} - \hat{\mu}_n)^{T} }\]
    \item The update is performed as per the typical Kalman filter operation:
        \[\begin{split}
        \mu_{n} & = \hat{\mu}_{n} + W_{n}\nu_{n} \\
        K_{n} & = \hat{K}_{n} - W_{n} \hat{S}_{n} W_{n}^{T} \\
        \nu_{n} & = y_{n} - \hat{y}_{n} \\
        W_{n} & = \hat{K}^{xy}_{n} \hat{S}_{n}^{-1}
        \end{split}\]
\end{enumerate}

From step (9) above, we obtain our new \textit{a posteriori} estimate of the mean value of the state vector $\mu_{n}$ and error covariance matrix $K_{n}$.

\section{Results}

\subsection{Example Sinusoidal Wave}
\label{sec:sine}

The results we achieved are as follows. From our sinusoidal example described before, we illustrate the true curve and the filtered data in Fig. \ref{fig:ukf_sine}. The true value of the wave, the measurements, and the filtered value of the wave are indicated in the plot, with the filtered value of the wave at any given time instant $t$ in reasonable correspondence with the true value at that time.

\begin{figure}[t]
\centerline{\includegraphics[height=6cm]{sine_example.png}}
\caption{Unscented Kalman filter for sinusoidal wave.}
\label{fig:ukf_sine}
\end{figure}

\subsection{Coordinate System Conversion}
\label{sec:cart_polar}

To illustrate the effects on nonlinear transformation, we will show the transformation of data points representing location data from a sensor tracking a target placed at $(x, y)$ coordinates (0, 1). The sensor used in the example from the original paper returned a mean radius of 1.0 m, with a standard deviation of 0.02 m. The mean angle returned by the sensor was $\frac{\pi}{2}$ rad, while the standard deviation of the angle was $\frac{\pi}{20}$ rad. This data was converted into Cartesian coordinates as
\begin{equation}
\begin{split}
x & = r\cos{\theta} \\
y & = r\sin{\theta}
\end{split}
\label{equ:polar_to_cartesian}
\end{equation}
The data points are plotted in the left plot, and the irregular distribution resulting from the coordinate transformation is apparent.

On the right are plots containing the statistical information contained within the data. The x symbol indicates the true mean of the data, and the dashed ellipse is the true covariance ellipse. The linearized version of the sensor data has a mean indicated by the o symbol, and the dotted ellipse is the corresponding covariance ellipse. As one can see, the data after a linearization suffers from bias created by the linearization of the data points without transforming the statistical data as well.

\begin{figure}[ht!]
\centering\noindent
\includegraphics[width=0.7\textwidth]{fig_1.png}
\captionof{figure}{Transformation of coordinates from polar to Cartesian, and the effect on the mean and covariance.}
\label{fig:cart_polar}
\end{figure}

\subsection{Vehicular Trajectory Example}
\label{sec:vehicle}

For the practical example we decided to illustrate, we attempted to replicate the results of Section VI\textemdash \textquotedblleft Applying the UT to Recursive Estimation\textquotedblright\textemdash from the original paper by Julier and Uhlmann\cite{b1}. This section details the estimation of the state of a vehicle as it is reentering Earth's atmosphere. The vehicle is being observed by a radar on Earth's surface, near the projected landing location.

The following state variables are tracked by the filter:
\begin{itemize}
    \item $x_{1}$: The $x$-component of the position of the vehicle
    \item $x_{2}$: The $y$-component of the position of the vehicle
    \item $x_{3}$: The $x$-component of the velocity of the vehicle
    \item $x_{4}$: The $y$-component of the velocity of the vehicle
    \item $x_{5}$: The aerodynamic coefficient, which is a property of the aerodynamics of the vehicle
\end{itemize}

The vehicle has the following state dynamics:
\begin{equation}
\begin{split}
    \dot{x}_{1}(k) & = x_{3}(k) \\
    \dot{x}_{2}(k) & = x_{4}(k) \\
    \dot{x}_{3}(k) & = D(k) x_{3}(k) + G(k) x_{1}(k) + v_{1}(k) \\
    \dot{x}_{4}(k) & = D(k) x_{4}(k) + G(k) x_{2}(k) + v_{2}(k) \\
    \dot{x}_{5}(k) & = v_{3}(k) \\
\end{split}
\label{equ:dynamics}
\end{equation}

The force $D(k)$ is the drag force at iteration $k$. The force $G(k)$ is the force due to gravity, and $v_{k}$ is the process noise. The distance to the vehicle from the center of Earth is the function $R(k) = \sqrt{x_{1}^{2}(k) + x_{2}^{2}(k)}$, while the speed of the vehicle is $V(k) = \sqrt{x_{3}^{2}(k) + x_{4}^{2}(k)}$. The drag and gravitational forces, as well as other important variables and functions, are as follows:
\begin{equation}
\begin{split}
    D(k) & = -\beta(k) \exp{\left( \cfrac{R_{0} - R(k)}{H_{0}} \right)} V(k) \\
    G(k) & = -\cfrac{Gm_{0}}{r^{3}(k)} \\
    \beta (k) & = \beta_{0}\exp{(x_{5}(k))} \\
    \boldsymbol{v}(k) & = \begin{bmatrix} v_{1}(k) & v_{2}(k) & v_{3}(k) \end{bmatrix}^{T} \\
        & = \begin{bmatrix} 2.4064 \times 10^{-5} & 2.4064 \times 10^{-5} & 0 \end{bmatrix}^{T}
\end{split}
\label{equ:drag_gravity}
\end{equation}
Some important constants are defined below:
\begin{equation}
\begin{split}
    \beta_{0} & = 0.59783 \\
    H_{0} & = 13.406 \\
    Gm_{0} & = 3.9860 \times 10^{5}\ \text{(G constant $\times$ mass of Earth)} \\
    R_{0} & = 6374\ \text{(Radius of Earth)} \\
\end{split}
\label{equ:constants}
\end{equation}
\textbf{One important note: The value of $\boldsymbol{\beta_{0}}$ was preceded by a \textquotedblleft$\boldsymbol{-}$\textquotedblright\ sign in the paper, which is incorrect since it results in a double negative, given that the drag force $\boldsymbol{D(k)}$ already contains the appropriate negative sign. This issue gave us some trouble in simulating an accurate model of the system dynamics until we figured out the error.}

Using the dynamics of equation (\ref{equ:dynamics}), we modeled the trajectory of the vehicle in Fig.~\ref{fig:fig8}, which matches closely with the original paper as expected. We repeated iterations of the dynamics until the vehicle landed on the surface.

\begin{figure}[ht!]
\centering\noindent
\includegraphics[width=0.95\textwidth]{Fig_8_Trajectory.png}
\captionof{figure}{Trajectory of reentry of a vehicle into the atmosphere of Earth, obtained from equations of motion indicated in \ref{sec:vehicle}. Simulated from Fig. 8 in Julier and Uhlmann\cite{b1}.}
\label{fig:fig8}
\end{figure}

Like in the original paper, we assigned to the system the initial conditions
\[x(0) = \begin{bmatrix} 6500.4 & 349.14 & -1.8093 & -6.7967 & 0.6932 \end{bmatrix}^{T}\]
and
\[P(0) = \begin{bmatrix}
10^{-6} & 0 & 0 & 0 & 0 \\
0 & 10^{-6} & 0 & 0 & 0 \\
0 & 0 & 10^{-6} & 0 & 0 \\
0 & 0 & 0 & 10^{-6} & 0 \\
0 & 0 & 0 & 0 & 0
\end{bmatrix},\]
while we let the unscented Kalman filter assume the initial conditions
\[\mu_{0} = \begin{bmatrix} 6500.4 & 349.14 & -1.8093 & -6.7967 & 0 \end{bmatrix}^{T}\]
and
\[K_{0} = \begin{bmatrix}
10^{-6} & 0 & 0 & 0 & 0 \\
0 & 10^{-6} & 0 & 0 & 0 \\
0 & 0 & 10^{-6} & 0 & 0 \\
0 & 0 & 0 & 10^{-6} & 0 \\
0 & 0 & 0 & 0 & 1
\end{bmatrix},\]
indicating that there is relative initial certainty of the value of every variable except the aerodynamic coefficient. The assumed process noise of the filter was adapted from the actual process noise $v(k)$,
\[Q(k) = \begin{bmatrix}
2.4064 \times 10^{-5} & 0 & 0 \\
0 & 2.4064 \times 10^{-5} & 0 \\
0 & 0 & 0 \end{bmatrix}.\]
The measurements and updates were performed at a frequency of 10 Hz, and the simulation was performed for a duration of 200 seconds, for a total of 1000 iterations.

 For visual reference, we provided a plot of the estimated trajectory of the vehicle, compared to the vehicle's true trajectory, in Fig.~\ref{fig:est_trajectory}. Our results shown in Fig.~\ref{fig:results} are of the mean squared error and variance for the unscented Kalman filter for $x_{1}$, $x_{3}$, and $x_{5}$, which represent the vehicle's $x$-position, $x$-velocity, and aerodynamic coefficient, respectively. We only plotted the error of the mean, as well as the covariance, from the unscented Kalman filter. As seen in Fig.~\ref{fig:est_trajectory}, our Kalman filter implementation tracks the trajectory of the vehicle reasonably well up to the termination of the estimation program, represented by the red \color{red}x \color{black} in the plot. We could not quite replicate the error estimation of the original simulation, likely due to differences in the way we calculated and/or reported our error calculations. This divergence could also be a result of the difference in randomization between the original simulation and our implementation. Nonetheless, our trajectory tracking via unscented Kalman filter has performed similarly well.

\begin{figure}[htb]
\centering\noindent
\includegraphics[width=0.95\textwidth]{Estimated_Trajectory.png}
\caption{Result of estimated trajectory versus true trajectory.}
\label{fig:est_trajectory}
\end{figure}

\begin{figure}[ht]
\centering\noindent
\begin{subfigure}{0.48\textwidth}
    \includegraphics[width=\linewidth, height=5cm]{x1.png}
    \captionof{figure}{Error of mean and covariance of $x_{1}$.}
    \label{fig:results_x1}
\end{subfigure}
\begin{subfigure}{0.48\textwidth}
    \includegraphics[width=\linewidth, height=5cm]{x3.png}
    \captionof{figure}{Error of mean and covariance of $x_{3}$.}
    \label{fig:results_x3}
\end{subfigure}
\begin{subfigure}{0.48\textwidth}
    \includegraphics[width=\linewidth, height=5cm]{x5.png}
    \captionof{figure}{Error of mean and covariance of $x_{5}$.}
    \label{fig:results_x5}
\end{subfigure}
\caption{Results of our simulation, as described in section \ref{sec:vehicle}.}
\label{fig:results}
\end{figure}

\section{Conclusion}

This project revolved around the implementation of the unscented Kalman filter for estimation of variables in a nonlinear system with non-Gaussian-distributed noise. Leading to this implementation, we also implemented the standard linear Kalman filter, as well as the extended Kalman filter for nonlinear systems. We also showed a comparison of the linear and unscented Kalman filters with a sinusoidal example in section \ref{sec:sine}. From the original paper by Julier and Uhlmann, we managed to replicate the results of the transformation of statistical data\textemdash mean and covariance\textemdash\ shown in this report in Fig.~\ref{fig:cart_polar} and described in section \ref{sec:cart_polar}. We also managed to replicate results of the vehicle trajectory estimation in section \ref{sec:vehicle}.

\subsection{Future work}

This Kalman filter implementation has room for improvement, as well as potential for further exploration. The primary aspect that can be improved is the separation of the prediction and measurement stages. As of right now, the prediction and measurement steps are alternated continuously. This process can be generalized for the case of multiple predictions between measurements, as well as, conversely, multiple measurements at once. This update would be practically useful for, say, a sensor that provides few measurements far in between, so that the filter would have to continuously estimate the change in the state vector and error covariance matrix before receiving another measurement. Likewise, in practice, a system may have multiple sensors, with inputs possibly arriving simultaneously, in which case one would wish to run the measurement step multiple times in succession before the need to predict future states.

The original paper by Julier and Uhlmann contained some examples of the use of the unscented transform. Replicating more of the results from their work, particularly these unscented transformation examples, would provide more understanding of the uses and limits of the unscented transformation.

Finally, readers who seek to learn more about the situations where various estimation strategies succeed over others would benefit from an extensive direct comparison of the Kalman filters. Theoretical and practical examples utilizing different Kalman filters could be the subject of a future project or report.

\subsection{Acknowledgements}

We would like to thank Dr. Waseem Malik at The University of Maryland, as well as our teaching assistant Srikumar Muralidharan, for giving us a theoretical foundation for completing this project.

\begin{thebibliography}{00}
\bibitem{b1} S. J. Julier, and J. K. Uhlmann, "New extension of the Kalman filter to nonlinear systems," \textit{Proc. SPIE}, vol. 3068, pp. 182-193, July 1997, doi: 10.1117/12.280797.
\bibitem{b2} J. R. R. A. Martins, P. Sturdza, and J. J. Alonso, "The complex-step derivative approximation," \textit{ACM Trans. Math. Softw.}, vol. 29, no. 3, pp. 245-262, Sep. 2003, doi:~10.1145/838250.838251
\bibitem{b3} G. Welch, and G. Bishop, "An introduction to the Kalman filter," Dept. Comp. Sci., Univ. of North Carolina at Chapel Hill, Chapel Hill, NC, USA, Rep. TR 95-041, 2006.

\end{thebibliography}

\end{document}