\documentclass{article}
%
%   get various packages
%
\usepackage[margin=1.0in]{geometry}                                     % adjust margins
\geometry{letterpaper}                                                  % ... or a4paper or a5paper or ...
\usepackage{url}                                                        % need this to use URLs in bibtex
\usepackage{setspace}													% need this for \setstrech{...}
\usepackage{scrextend}                                                  % need this for addmargin
\usepackage[export]{adjustbox}                                          % need this to get frame for includegraphics
\usepackage{bigints}                                                    % bigger integral symbol
%
%   tikz et al
%
\usepackage{tikz}
\usetikzlibrary{patterns,angles,quotes,shapes,intersections,decorations,backgrounds}
\usepackage{pgfplots}
\usepackage{relsize}
\usepackage{float}
%
%	amsmath stuff
%
\usepackage{amsthm}
\usepackage{amsmath}
%
%	watermarks
%
% \usepackage{draftwatermark}
% \SetWatermarkText{Draft}
% \SetWatermarkScale{5}
% \SetWatermarkLightness {0.9}
% \SetWatermarkColor[rgb]{0.7,0,0}
%
%
\newtheorem{thm}{Theorem}[section]
\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]
\newtheorem{proposition}{Proposition}[section]
\newtheorem{lemma}{Lemma}[section]
\newtheorem{example}{Example}[section]
%
\newcommand{\argmax}{\operatornamewithlimits{argmax}}
\newcommand{\argmin}{\operatornamewithlimits{argmin}}
%
%	title, etc
%
\title{A Few Notes on the Brachistochrone Problem}
\author{David Meyer \\ dmm@1-4-5.net}
\date{Last update: \today}                                                                           % Activate to display a given date or no date
%
%	be compatible
%
\pgfplotsset{compat=1.17}
%
%
\begin{document}
\maketitle
%
%
%
\section{Introduction}
Johann Bernoulli posed the "problem of the brachistochrone" to the
readers of \emph{Acta Eruditorum} in June, 1696 \cite{1696acta},
which asks the following question:

\bigskip
\begin{addmargin}[2em]{7em}                                                                             % 2em left, 7em right
Given two points $A$ and $B$ in a vertical plane, what is the
curve traced out by a point acted on only by gravity, which
starts at $A$ and reaches $B$ in the shortest time?
\end{addmargin}

\bigskip
\noindent
The scenario Bernoulli envisioned is depicted in Figure
\ref{fig:brachistochrone}.

\bigskip
\bigskip
%
%	brachistochrone problem setup
%
\begin{figure}[H]
  \centering
  \resizebox{0.80 \textwidth}{!} {                                                                      % resize the figure if you want
     \begin{tikzpicture} [framed]                                                                       % draw the picture inside a black frame (requires \usetikzlibrary{backgrounds})
%
%	set up some coordinates
%
        \coordinate (A)		at (0.0,  0.0);								% put the origin at A
        \coordinate (Ax)	at (1.2,  0.0);								% x axis here
        \coordinate (Ay)	at (0.0, -1.2);								% y axis positive in the down direction
        \coordinate (Ball)	at (0.5, -0.5);								% draw the ball here
        \coordinate (B)		at (4.0, -1.0);								% other end of the curve
        \coordinate (Gy0)	at (3.5,  0.25);							% draw the gravity vector starting here
        \coordinate (Gy1)	at (3.5, -0.5);								% gravity vector ends here
        \coordinate (D0)	at (0.65,-0.65);							% draw motion arrow starting here
        \coordinate (D1)	at (1.0, -0.945);							% motion arrow ends here
%
%	draw the picture
%
        \draw[blue, thick] (A) to [bend right=50] (B);                 	% draw the curve in blue
        \node[circle, draw=black, fill=red, scale=0.70] at (Ball) {};	% draw a red ball with a black outline on the curve
        \draw[-stealth, thin] (D0) -- (D1);								% draw an arrow in the direction the ball is moving
        \fill (A) circle (0.035);										% put a dot on the curve at A
        \fill (B) circle (0.035);										% put a dot on the curve at B
        \draw[draw=none] (A) node[xshift=-0.10cm, font=\footnotesize, left] {$A \; (x_1,y_1)$};		% label A
        \draw[draw=none] (B) node[xshift=0.10cm, font=\footnotesize, right] {$B \; (x_2,y_2)$};		% label B
        \draw[-stealth, thin] (A) -- (Ax) coordinate [label={[font=\scriptsize, above] $x$}];		% draw x axis
        \draw[-stealth, thin] (A) -- (Ay) coordinate [label={[font=\scriptsize, left] $y$}];		% draw y axis
        \draw[-stealth, thick] (Gy0) -- (Gy1) coordinate [label={[font=\footnotesize, midway, right] $\vec{g}$}];	% draw the gravity vector (g)
      \end{tikzpicture}													% end \tikzipcture
    }																	% end \resizebox
  \caption{The Brachistochrone Problem Setup}
  \label{fig:brachistochrone}
\end{figure}

\medskip
\section{A Bit of History}
\label{sec:history}
The problem posed by Johann Bernoulli was the called the
Brachistochrone\footnote{The word brachistochrone comes from the
Ancient Greek: \emph{brachis} (shortest) + \emph{chronos} (time)
\cite{enwikisource:6202943}.} problem, or the path of fastest
descent. Bernoulli issued his challenge in \emph{Acta Eruditorum}
\cite{1696acta}, writing

\bigskip
\begin{addmargin}[2em]{7em}
I, Johann Bernoulli, address the most brilliant mathematicians in
the world. Nothing is more attractive to intelligent people than
an honest, challenging problem, whose possible solution will
bestow fame and remain as a lasting monument. Following the
example set by Pascal, Fermat, etc., I hope to gain the gratitude
of the whole scientific community by placing before the finest
mathematicians of our time a problem which will test their
methods and the strength of their intellect. If someone
communicates to me the solution of the proposed problem, I shall
publicly declare him worthy of praise.
\end{addmargin}

\noindent
Bernoulli wrote the problem statement as:

\bigskip
\begin{addmargin}[2em]{7em}
Given two points A and B in a vertical plane, what is the curve traced
out by a point acted on only by gravity, which starts at A and reaches
B in the shortest time.
\end{addmargin}

\bigskip
\noindent
Figure \ref{fig:brachistochrone} attempts to capture Bernoulli's
problem statement.

\bigskip
\noindent
Galileo had attempted to solve this problem in his \emph{Two New
Sciences} \cite{wiki:two_new_sciences} and had concluded, based
on geometric arguments, that the solution was a circular
path. Galileo's approach is shown in Figure
\ref{fig:galileo}. Apparently Galileo hedged a bit on his
solution. In fact, it is said that he had reservations about his
conclusion and suggested that a “higher mathematics” would
possibly find a better solution \cite{brachistochrone_problem}.

\bigskip
\begin{figure}[H]
\center{\includegraphics[frame,scale=0.27] {images/galileo.png}}
\caption{Galileo considered a mass falling along different chords
of a circle starting at A.  He proved that the path along ABG was
quicker than along AG, and ABCG was quicker than ABG, and ABCDG
was quicker than ABCG, etc.  In this way he showed that the path
along the circular arc was quicker than any set of chords. From
this he (incorrectly) inferred that the circle was the path of
quickest descent. Galileo held out reservations about his
solution, and rightfully so.}
\label{fig:galileo}
\end{figure}

\noindent
Then in 1659, while studying the physics of pendulums and time
keeping, Christiaan Huygens \cite{wiki:huygens} recognized that a
perfect harmonic oscillator, one whose restoring force was linear
in the displacement of the oscillator, would produce the perfect
time piece. Unfortunately, while the pendulum (as proposed by
Galileo) was the simplest oscillator to construct, Huygens knew
that it was not a perfect harmonic oscillator. In particular, he
knew that the period of oscillation became smaller when the
amplitude of the oscillation became larger. In order to “fix” the
pendulum, he searched for a curve of equal time, called the
\emph{tautochrone} \cite{wiki:tautochrone}, that would allow all
amplitudes of the pendulum to have the same period. He found the
solution and recognized it to be a cycloid \cite{wiki:cycloid}.
The cycloid is shown in Figure \ref{fig:cycloid}.

\bigskip
\begin{figure}[H]
  \centering
  \resizebox{0.63 \textwidth}{!} {										% resize the figure if you want
    \begin{tikzpicture} [framed]										% draw the picture inside a black frame (requires \usetikzlibrary{backgrounds})
     \coordinate (O) at (0,0);
     \coordinate (A) at (0,3);
     \def\r{1}															% r is the radius
     \def\c{1.4}														% c is the center
     \coordinate (C) at (\c, \r);

     \draw[-latex] (O) -- (A) node[anchor=south] {$y$};
     \draw[-latex] (O) -- (2.6*pi,0) node[anchor=west] {$x$};
     \draw[red, domain=-0.5*pi:2.5*pi, samples=50, line width=1] plot ({\x - sin(\x r)},{1 - cos(\x r)});
     \draw[blue, line width=1] (C) circle (\r);
     \draw[] (C) circle (\r);
%
%       set up some variables and coordinates
%
     \def\x{0.4}														% x
     \def\y{0.83}														% y
     \def\xa{0.3} 														% coordinate x for arc left
     \def\ya{1.2}														% coordinate y for arc left
     \coordinate (X)     at (\x, 0 );
     \coordinate (Y)     at (0, \y );
     \coordinate (XY)    at (\x, \y );
     \node[anchor=north] at (X) {$x$};
     \draw[fill=black] (O) circle (1pt);								% draw a dot at the origin
     \draw[fill=blue]  (C) circle (1pt);   								% draw center of circle
     \draw[] (C) -- node[anchor=south] {\; $a$} (XY);					% draw radius of the circle
     \coordinate (B) at (\c, 0);    									% bottom of circle, radius to the bottom
     \draw[] (C) -- (B) node[anchor=north] {$a \, \theta$};
%
%       projections of point XY
%
     \draw[dotted] (XY) -- (X);
     \draw[dotted] (XY) -- (Y) node[anchor=east, xshift=1mm] {$\quad y$};
%
%       arc theta
%       start arc
%
     \coordinate (S) at (\c, 0.4);
     \draw[->] (S) arc (-90:-165:0.6);
     \node[xshift=-2mm, yshift=-2mm] at (C) {\scriptsize $\theta$};
%
%       arc above
%
     \coordinate (AA) at (\xa, \ya);
     \draw[-latex, rotate=25] (AA) arc (-220:-260:1.3);
%
%       arc below
%
     \def\xb{2.5}														% coordinate x for arc bottom
     \def\yb{0.8}														% coordinate y for arc bottom
     \coordinate (AB) at (\xb, \yb);
     \draw[-latex, rotate=-10] (AB) arc (-5:-45:1.3);
     \draw[fill=black] (XY) circle (1pt);								% draw XY dot
%
%       top label
%
     \coordinate (T) at (pi, 2);
     \node[anchor=south] at (T)  {$(\pi a, 2 a )$} ;
     \draw[fill=black] (T) circle (1pt);
%
%       equations
%
     \coordinate (E) at ( 4,1.2);
     \coordinate (F) at ( 4,0.9);
     \node[] at (E) {\scriptsize $x=a(\theta - \sin \theta)$};
     \node[] at (F) {\scriptsize $y=a(1 - \cos \theta)$};
%
%       label 2pi a
%
     \coordinate (TPA) at (2*pi, 0);
     \node[anchor=north] at (TPA) {$2 \pi a$};
     \draw[fill=black] (TPA) circle (1pt);								% draw a dot on the x axis at 2 \pi a

    \end{tikzpicture}
  }
 \caption{The Cycloid Curve}
 \label{fig:cycloid}
\end{figure}

\bigskip
\begin{figure}[H]
\center{\includegraphics[scale=0.60] {images/cycloid_coordinates.png}}
\caption{Finding the Coordinates of the Point D on a Cycloid}
\label{fig:cycloid_coordinates}
\end{figure}

\bigskip
\noindent
BTW, how are the cycloid's $(x,y)$ coordinates computed? Consider
the scenario depicted in Figure \ref{fig:cycloid_coordinates}
\cite{cycloid_coordinates}. To find the $(x,y)$ coordinates of
the point D we look at the distances involved. We can see from
the figure that

\medskip
\begin{itemize}
 \item The x-coordinate is the length of OT minus the length of DB
 \item The y-coordinate is the length of CT minus the length of CB
 (note that CT = $a$ is the radius of C)
 \item The length of an arc that is subtended by a central angle is
       equal to the radius times the central angle
\end{itemize}

\bigskip
\noindent
Now, since the distance that circle C has rolled must be equal to
the length of arc DT and the arc DT is equal to $a \theta$ we know
that $a\theta = \text{arc DT} = \text{OT}$. We also know from
trigonometry that $\text{DB} = a \sin \theta \text{ and } \text{CB}
= a \cos \theta$.

\bigskip
\noindent
If we work out these lengths we see that

\begin{equation*}
\begin{array}{lllll}
x
&=& \text{OT - DB}   					&\qquad \qquad \qquad \mathrel{\#} \text{See Figure \ref{fig:cycloid_coordinates}} \\
[5pt]
&=& a\theta \; – \; a \sin \theta       &\qquad \qquad \qquad \mathrel{\#} \text{OT = radius times the central angle = $a \theta$ and DB $= a \sin \theta$} \\
[5pt]
&=& a (\theta \; – \; \sin \theta)      &\qquad \qquad \qquad \mathrel{\#} x = a (\theta – \sin \theta) \\
[10pt]
\hspace{-2em} \text{and} \\
[10pt]
y
&=& \text{CT - CB}   					&\qquad \qquad \qquad \mathrel{\#} \text{Figure \ref{fig:cycloid_coordinates}} \\
[5pt]
&=& a \; – \; a \cos \theta             &\qquad \qquad \qquad \mathrel{\#} \text{CT = $a$ and CB $= a \cos \theta$} \\
[5pt]
&=& a (1 \; – \; \cos \theta)           &\qquad \qquad \qquad \mathrel{\#} y = a (1 – \cos \theta)
\end{array}
\end{equation*}

\bigskip
\noindent
So as we saw in Figure \ref{fig:cycloid}, $x = a (\theta – \sin
\theta)$ and $y = a (1 – \cos \theta)$.

\bigskip
\noindent
Ok, back to the history. So almost thirty years later, during the
infancy of the infinitesimal calculus, the Tautochrone problem
was held up as a master example of an “optimal” solution whose
derivation should yield to the much more powerful and elegant
methods of the calculus. Jakob Bernoulli, Johann’s brother,
succeeded in deriving the Tautochrone problem in 1690 using the
calculus, using the term “integral” for the first time in print,
but it was not at first clear what other problems could yield in
a similar way.

\bigskip
\noindent
Then, in 1696, Johann Bernoulli posed the Brachistrochrone
problem in the pages of \emph{Acta Eruditorum}.

\subsection{A (Very) Brief Timeline of Brachistrochrone Related Events}

\bigskip

\begin{equation*}
\begin{array}{lll}
&\bullet \; 1638  & - \;\;\; \text{Galileo proposes arc of a circle as the least time solution (Figure \ref{fig:galileo})}             \\
&\bullet \; 1658  & - \;\;\; \text{Pascal puts forth a challenge involving finding the area under a segment of a cycloid}              \\
&\bullet \; 1659  & - \;\;\; \text{Huygens shows experimentally that the cycloid is the solution to the Tautochrone problem}           \\
&\bullet \; 1662  & - \;\;\; \text{Fermat's proposes the Principle of Least Time \cite{wiki:fermats_principle}}                        \\
&\bullet \; 1696  & - \;\;\; \text{Johann Bernoulli poses the Brachistrochrone problem in \emph{Acta Eruditorum} (edited by Leibniz)}  \\
&\bullet \; 1697  & - \;\;\; \text{Solutions provided by the Bernoulli brothers, L'H\^{o}pital, Leibniz, and Newton (anonymously)}     \\
&\bullet \; 1707  & - \;\;\; \text{Euler is born}                                                                                \\
&\bullet \; 1726  & - \;\;\; \text{Euler completes his Ph.D under Johann Bernoulli}                                                    \\
&\bullet \; 1727  & - \;\;\; \text{Newton dies}                                                                                \\
&\bullet \; 1755  & - \;\;\; \text{Lagrange, at age 19, finds an analytic solution to the Tautochrone problem}                         \\
&\bullet \; 1755  & - \;\;\; \text{Euler drops his own approach in favor of Lagrange's purely analytic approach}                       \\
&\bullet \; 1756  & - \;\;\; \text{Euler renames the subject the "Calculus of Variations" in \emph{Elementa Calculi Variationum} \cite{cov}}
\end{array}
\end{equation*}

\medskip
\section{The Calculus of Variations and the Brachistrochrone Problem}
\label{sec:cov}
Consider a definite integral that depends on an unknown function $y(x)$ and its derivative
$y^{\prime}(x) = \frac{dy}{dx}$

\begin{equation}
I[y(x)] = \int_a^b F(y(x), y^{\prime}(x)) \; dx
\label{eqn:I[y]}
\end{equation}

\bigskip
\noindent
A typical problem in the Calculus of Variations involves finding a particular function $y(x)$ that
maximizes or minimizes the integral $I[y(x)]$ (Equation \ref{eqn:I[y]}), subject to the boundary conditions
$y(a) = A$ and $y(b) = B$. The integral $I[y(x)]$ is an example of a \emph{functional}, which (more generally)
is a mapping from a set of allowable functions to the reals. We say that $I[y(x)]$ has an \emph{extremum}
when $I[y(x)]$ takes its maximum or minimum value. Note that we use square brackets to indicate that $I$
is a functional (rather than a function) and we frequently see $y(t)$ abbreviated as $y$ and $I[y(x)]$
as $I[y]$ (or just $I$).

\subsection{The Euler-Lagrange Equation}
\label{subsec:euler-lagrange}
Let $C^k [a,b]$ denote the set of continuous functions defined on the interval $a \leq x \leq b$ whose
first $k$-derivatives are also continuous on $a \leq x \leq b$. Now, suppose that $I[Y(x)]$ is an
extremum of the functional

\medskip
\begin{equation*}
I[y(x)] = \int_a^b L(y(x), y^{\prime}(x)) \; dx
\end{equation*}

\bigskip
\noindent
that is defined defined on all functions $y(x) \in C^2 [a,b]$ such that $y(a) = A$ and $y(b) = B$.
Then its solution $Y(x)$ satisfies the second order ordinary differential equation

\bigskip
\begin{equation}
\dfrac{d}{dt} \left ( \dfrac{\partial L}{\partial y^{\prime}} \right ) -  \dfrac{\partial L}{\partial y} = 0
\label{eqn:euler-lagrange}
\end{equation}

\bigskip
\noindent
Equation \ref{eqn:euler-lagrange} is known as the Euler-Lagrange equation (see \cite{notes:pola} for a proof of
the Equation \ref{eqn:euler-lagrange}). It is the constraint imposed by the Euler-Lagrange equation
that allows us to find the extremum of the functional.

\bigskip
\noindent
Recall the Brachistochrone problem setup shown in Figure \ref{fig:brachistochrone}.
Here we are trying to find the path that takes the \emph{shortest time} from point
$A$ to $B$. In the language of the calculus of variations \cite{wiki:cov} the
Brachistochrone problem is to minimize the functional

\begin{equation*}
I[y] = \int_{0}^{t} dt
\end{equation*}

\bigskip
\noindent
Consider a curve over in which we call an differential segment of the path $ds$. Then
we know that the velocity of the ball (or bead, in Bernoulli's original problem statement)
is

\begin{equation}
V = \dfrac{ds}{dt}
\label{eqn:v}
\end{equation}

\bigskip
\noindent
Now, treating $\dfrac{ds}{dt}$ as a fraction\footnote{Care needs to be taken when
treating a derivative as a fraction.} we can see from Equation \ref{eqn:v} that

\bigskip
\begin{equation*}
dt = \dfrac{ds}{V}
\end{equation*}

\bigskip
\noindent
and so

\begin{equation}
I[y] = \int_{0}^{t} dt = \int_A^B \dfrac{ds}{V}
\label{eqn:I}
\end{equation}

\bigskip
{\setstretch{1.85}
\noindent
Can we say more about $ds$? Well, if some function
$f(x)$ is differentiable in the neighborhood of $ds$
and if $ds$ is of differential length ($ds \to 0$)
then we can build a right triangle on $f(x)$ with
$ds$ as the hypotenuse. Then from Pythagoras we
know that $ds^2 = dx^2 + dy^2$ and so
$ds = \sqrt{dx^2 + dy^2} = \sqrt{1 + \left ( \dfrac{dy}{dx} \right )^2} \; dx$. This scenario is
depicted in Figure \ref{fig:pythagorean_theorem}.
\par}

\bigskip
%
%	draw Pythagorean theorem stuff
%
\begin{figure}[H]
\centering
  \resizebox{0.75 \textwidth}{!} {																							% resize figure if you want
  \begin{tikzpicture} [every edge quotes/.append style = {anchor=south, sloped},declare function={f(\x)=1 + \x*\x*\x;}]		% define f(x)
       \draw[thick, -latex] (-2,0) -- (8.0,0.0) coordinate [label={[right] $x$}];  											% x axis
       \draw[thick, -latex] (0,-1) -- (0,6.5) coordinate [label={[above] $y$}]; 											% y axis
       \draw [line width=0.25mm, smooth,samples=100,domain=-0.95:1.40] plot (\x,{f(\x)});									% draw f(x)
       \draw [draw=none] (1.40, {f(1.40)}) coordinate [label={[above] $f(x)$}];												% label f(x)
       \coordinate (O) at (0,0); 																							% origin
       \path (O) node[below left] {$O$};																					% draw origin
%
%	various coordinates
%
       \coordinate (x1y1) at (0.5,{f(0.5)});
       \coordinate (x1y0) at (0.5,0.0);
       \coordinate (x2y2) at (1.1,{f(1.1});
       \coordinate (x2y0) at (1.1,0.0);
       \coordinate (cir)  at (0.85,{f(0.85)});
%
       \fill (x1y1) circle (0.05);																					% put a dot on the curve at x1y1
       \fill (x2y2) circle (0.05);																					% put a dot on the curve at x2y2
	   \draw [very thick,blue,smooth,samples=100,domain=0.5:1.1] plot (\x,{f(\x)});									% draw blue segment on f(x)
       \draw[draw=none] (x1y1) -- (x2y2) coordinate [label={[font=\footnotesize,yshift=-0.065cm,xshift=-0.105cm,midway,left] \color{black} $ds$}];
       \draw[dashed] (x1y1) -- (x1y0) coordinate [label={[font=\scriptsize,below] \color{black} $x_1$}];
       \draw[dashed] (x2y2) -- (x2y0) coordinate [label={[font=\scriptsize,below] \color{black} $x_2$}];
       \fill (x1y0) circle (0.05);																					% put a dot on the x axis at x1
       \fill (x2y0) circle (0.05);																					% put a dot on the x axis at x2
       \draw[red, thick] (cir) circle (0.150);																		% put a circle on f(x) where ds is

%
%	draw triangle and circle to show ds
%
%
%
%	set up triangle coordinates
%
	   \coordinate (a) at (3.75,3.00);
	   \coordinate (c) at (6.00,3.00);
	   \coordinate (b) at (6.00,6.00);
%
%	draw the triangle
%
       \draw[] (a) -- (c) coordinate [label={[yshift=-0.1cm,midway,below] ${dx}$}];										% dx
       \draw[] (c) -- (b) coordinate [label={[midway,right] ${dy}$}];													% dy
       \draw[blue,very thick] (a) -- (b) coordinate [label={[xshift=-0.25cm,midway,left] \color{black} ${ds}$}];		% ds
       \draw[red,thick] (5.15,4.29) circle (2.10);																		% big circle
       \draw[draw=none] (0,0) -- (5.15,1.75)  node[draw,rectangle,below] {${\; ds^2 = dx^2 + dy^2 \;}$};				% draw the pythagorean theorem
%
%	draw the angle
%
       \node[xshift=-2mm, yshift=-2mm] at (4.75,3.60) {$\theta$};
       \draw[-stealth,thin] (5.10,3.00) arc (0:45:1.60cm);
%
%	done
%
     \end{tikzpicture}
    }																					 								% end resizebox
 \caption{$f(x)$, $ds$ and the Pythagorean Theorem}
 \label{fig:pythagorean_theorem}
\end{figure}

% \begin{figure}[H]
%  \centering
% \resizebox{0.30 \textwidth}{!} {																				% resize the figure if you want
%    \begin{tikzpicture}[framed]
%      \draw [red] (0,0) coordinate[] (a) --
%            	   (3,0) coordinate[] (c) --
%            	   (3,3) coordinate[] (b) -- cycle;
%       \draw[draw=none] (a) -- (c) coordinate [label={[yshift=-0.1cm,midway,below]					${dx}$}];    % dx
%       \draw[draw=none] (c) -- (b) coordinate [label={[xshift=0.025,midway,right]  				${dy}$}];    % dy
%       \draw[draw=none] (a) -- (b) coordinate [label={[xshift=-0.2cm,yshift=0.025cm,midway,left]	${ds}$}];    % ds
%    \end{tikzpicture}
%  }
% \caption{$ds^2 = dx^2 + dy^2$}
% \label{fig:pythagorean_theorem}
%\end{figure}

\bigskip
\noindent
Armed with this information we can solve for $ds$ in terms of $y^{\prime}$:

\begin{equation*}
\begin{array}{lllll}
ds^2
&=& dx^2 + dy^2   												&\qquad \qquad \qquad \mathrel{\#} \text{Pythagorean Theorem (Figure \ref{fig:pythagorean_theorem})} \\
[10pt]
&=& \left ( 1 + \dfrac{dy^2}{dx^2} \right ) dx^2				&\qquad \qquad \qquad \mathrel{\#} \text{factor out $dx^2$} \\
[12pt]
&=& \left ( 1 + \left (\dfrac{dy}{dx} \right )^2 \right ) dx^2	&\qquad \qquad \qquad \mathrel{\#} \text{group $\dfrac{dy}{dx}$} \\
[12pt]
&=& \left ( 1 + (y^{\prime})^2 \right ) dx^2					&\qquad \qquad \qquad \mathrel{\#} \text{$y^{\prime} = \dfrac{dy}{dx}$; note that here we're treating $\dfrac{dy}{dx}$ as a fraction}\\
[10pt]
&\Rightarrow& ds = \sqrt{1 + (y^{\prime})^2} \; dx				&\qquad \qquad \qquad \mathrel{\#} \text{take the square root of both sides}
\end{array}
\end{equation*}

\bigskip
\noindent
So now we know that

\begin{equation}
ds = \sqrt{1 + (y^{\prime})^2} \; dx
\label{eqn:ds}
\end{equation}

\bigskip
\noindent
Next, we know from the Law of Conservation of Energy \cite{conservation_of_energy}
that

\medskip
\begin{equation*}
\begin{array}{lllll}
\dfrac{1}{2} mV^2 - mgy
&=& 0										&\qquad \mathrel{\#} \text{$\dfrac{1}{2} mV^2$ is the kinetic energy and $mgy$ is the potential energy} \\
[10pt]
&\Rightarrow& \dfrac{1}{2} mV^2 = mgy		&\qquad \mathrel{\#} \text{add $mgy$ to both sides} \\
[10pt]
&\Rightarrow& \dfrac{1}{2} V^2 = gy			&\qquad \mathrel{\#} \text{cancel $m$} \\
[10pt]
&\Rightarrow& V^2 = 2gy						&\qquad \mathrel{\#} \text{multiply both sides by 2} \\
[10pt]
&\Rightarrow& V = \sqrt{2gy}				&\qquad \mathrel{\#} \text{take the square root of both sides}
\end{array}
\end{equation*}

\bigskip
\noindent
So now we know that

\begin{equation}
V = \sqrt{2gy}
\label{eqn:V}
\end{equation}

\bigskip
\noindent
and we can write $I[y]$ as follows

\begin{equation*}
\begin{array}{lllll}
I[y]
&=& {\displaystyle \mathlarger{\mathlarger{\int_A^B}} \dfrac{ds}{V}}
									&\qquad \qquad \mathrel{\#} \text{definition of $I[y]$ (Equation \ref{eqn:I})} \\
[15pt]
&=& {\displaystyle \mathlarger{\mathlarger{\int_A^B}} \dfrac{\sqrt{1 + (y^{\prime})^2}}{\sqrt{2gy}} dx}
									&\qquad \qquad \mathrel{\#} \text{Equations \ref{eqn:ds} and \ref{eqn:V}} \\
[15pt]

&=& {\displaystyle \dfrac{1}{\sqrt{2g}} \mathlarger{\mathlarger{\int_A^B}} \sqrt{\dfrac{1 + (y^{\prime})^2}{y}} \; dx}
									&\qquad \qquad \mathrel{\#} \text{pull $\dfrac{1}{\sqrt{2g}}$ out of integral, consolidate square root}
\end{array}
\end{equation*}

\medskip
\bigskip
\noindent
So now we have this expression for $I[y]$:

\medskip
\bigskip
\begin{equation*}
I[y] = \dfrac{1}{\sqrt{2g}} \mathlarger{\mathlarger{\int_A^B}}  \sqrt{\dfrac{1 + (y^{\prime})^2}{y}} \; dx
\end{equation*}

\medskip
\bigskip
\noindent
The Calculus of Variations is about finding a function (rather than a point) that minimizes or
maximizes some other function. The other function, which takes a function as an argument,
is called a functional. Here $I[y]$ is the functional and we want to find a function
$L(y,y^{\prime})$ that minimizes $I[y]$. $L(y,y^{\prime})$ looks like

\medskip
\begin{equation}
L(y,y^{\prime}) = \sqrt{\dfrac{1 + (y^{\prime})^2}{y}}
\label{eqn:L}
\end{equation}

\bigskip
\noindent
The last piece of machinery we need is the Beltrami Identity \cite{wiki:beltrami_identity},
which is a special case of the Euler–Lagrange equation where
$\dfrac{\partial L}{\partial y} = 0$. In this case the Euler-Lagrange equation reduces
to

\bigskip
\begin{equation}
L - y^{\prime} \dfrac{\partial L}{\partial y^{\prime}} = C
\label{eqn:beltrami}
\end{equation}

\bigskip
\noindent
where $C$ is some constant (see \cite{wiki:beltrami_identity} for a derivation). Plugging
Equation \ref{eqn:L} into Equation \ref{eqn:beltrami} we get

\bigskip
\begin{equation}
\sqrt{\dfrac{1 + (y^{\prime})^2}{y}} - y^{\prime}
\dfrac{\partial}{\partial y^{\prime}}
\mathlarger{\mathlarger{\Bigg [}} \sqrt{\dfrac{1 + (y^{\prime})^2}{y}} \mathlarger{\mathlarger{\Bigg ]}}
= C
\label{eqn:bel0}
\end{equation}

\bigskip
\noindent
Taking the partial derivative in Equation \ref{eqn:bel0} we get

\medskip
\begin{equation*}
\begin{array}{lllll}
\sqrt{\dfrac{1 + (y^{\prime})^2}{y}}  - y^{\prime} \cdot
\Bigg [
  \dfrac{1}{\sqrt{y}} \cdot
  \bigg (
    \dfrac{1}{2} \Big (1 + (y^{\prime})^2 \Big )^{- \frac{1}{2}}
    \cdot 2 y^{\prime}
  \bigg )
\Bigg ]
= C							&\mathrel{\#} \text{power rule: $\dfrac{d}{dx} u^n = n u^{n-1} \dfrac{du}{dx}, \; u= 1 + (y^{\prime})^2, \; n = \dfrac{1}{2}$} \\
[15pt]
\Rightarrow \sqrt{1 + (y^{\prime})^2} - \dfrac{(y^{\prime})^2}{\sqrt{1 + (y^{\prime})^2}} = \sqrt{y} \cdot C
							&\mathrel{\#} \text{multiply both sides by $\sqrt{y}$, cancel 2 and $\dfrac{1}{2}$} \\
[15pt]
\Rightarrow 1 + (y^{\prime})^2 -(y^{\prime})^2 = \sqrt{y (1 + (y^{\prime})^2)} \cdot C
							&\mathrel{\#} \text{multiply both sides by $\sqrt{1 + (y^{\prime})^2}$} \\
[15pt]
\Rightarrow 1 = \sqrt{y (1 + (y^{\prime})^2)} \cdot C
							&\mathrel{\#} (y^{\prime})^2 - (y^{\prime})^2 = 0 \\
[15pt]
\Rightarrow 1 = y (1 + (y^{\prime})^2) \cdot C^2
							&\mathrel{\#} \text{square both sides} \\
[15pt]
\Rightarrow \dfrac{1}{C^2} = y + y (y^{\prime})^2
							&\mathrel{\#} \text{divide both sides by $C^2$, multiply though by $y$} \\
[15pt]
\text{Now, let } \dfrac{1}{C^2} = C_1 = y + y (y^{\prime})^2
							&\mathrel{\#} \text{define constant $C_1$} \\
[15pt]
\Rightarrow (y^{\prime})^2 = \dfrac{C_1 - y}{y}
							&\mathrel{\#} \text{solve for $(y^{\prime})^2$} \\
[15pt]
\Rightarrow y^{\prime} = \sqrt{\dfrac{C_1 - y}{y}}
							&\mathrel{\#} \text{take the square root of both sides} \\
[15pt]
\Rightarrow \dfrac{dy}{dx} = \sqrt{\dfrac{C_1 - y}{y}}
							&\mathrel{\#} y^{\prime} = \dfrac{dy}{dx} \\

\end{array}
\end{equation*}

\bigskip
\noindent
So know we know that

\begin{equation*}
\begin{array}{lllll}
\dfrac{dy}{dx}
&=& \sqrt{\dfrac{C_1 - y}{y}}																		&\qquad \mathrel{\#} \text{previous derivation} \\
[15pt]
&\Rightarrow& dx = \sqrt{\dfrac{y}{C_1 -y}} \; dy													&\qquad \mathrel{\#} \text{solve for $dx$} \\
[15pt]
&\Rightarrow& x + C_2  = \mathlarger{\mathlarger{\mathlarger{\int}}} \sqrt{\dfrac{y}{C_1 -y}} \; dy	&\qquad \mathrel{\#} \text{integrate both sides}
\end{array}
\end{equation*}

\smallskip
\bigskip
\noindent
Now let $y = C_1 \sin^2 \theta$ so that
$dy = 2 \: C_1 \sin \theta \: \cos \theta \: d\theta$.
Then

\medskip
\begin{equation*}
\begin{array}{lllll}
x + C_2
&=& \mathlarger{\mathlarger{\mathlarger{\int}}}
\sqrt{\dfrac{C_1 \sin^2 \theta}{C_1 - C_1 \sin^2 \theta}} \cdot	2 \: C_1 \sin \theta \: \cos \theta \: d\theta
			&\qquad \mathrel{\#} \text{substitute for $y$ and $dy$} \\
[15pt]
&=& \mathlarger {\mathlarger{\mathlarger{\int}}}
\sqrt{\dfrac{C_1 \sin^2 \theta}{C_1 (1 - \sin^2 \theta)}} \cdot	2 \: C_1 \sin \theta \: \cos \theta \: d\theta
			&\qquad \mathrel{\#} \text{factor out $C_1$} \\
[15pt]
&=& \mathlarger{\mathlarger{\mathlarger{\int}}}	\sqrt{\dfrac{C_1 \sin^2 \theta}{C_1 \cos^2 \theta}}
\cdot 2 \: C_1 \sin \theta \: \cos \theta \: d\theta
			&\qquad \mathrel{\#} \cos^2 \theta = 1 - \sin^2 \theta \\
[15pt]
&=& \mathlarger{\mathlarger{\mathlarger{\int}}}	\dfrac{\sin \theta}{\cos \theta}
\cdot 2 \: C_1 \sin \theta \: \cos \theta \: d\theta
			&\qquad \mathrel{\#} \text{take square root, cancel $C_1$}\\
[15pt]
&=& 2 \; C_1 \mathlarger{\mathlarger{\mathlarger{\int}}} \sin^2 \theta \; d\theta
			&\qquad \mathrel{\#} \text{cancel $\cos \theta$, factor out $2 \; C_1$}\\
[15pt]
&=& 2 \; C_1 \mathlarger{\mathlarger{\mathlarger{\int}}} \dfrac{1}{2} (1 - \cos 2\theta) \; d\theta
			&\qquad \mathrel{\#} \text{$\cos 2 \theta = 1 - 2 \sin^2 \theta$ (double-angle trig identity)} \\
[15pt]
&=& C_1 \mathlarger{\mathlarger{\mathlarger{\int}}} (1 - \cos 2\theta) \; d\theta
			&\qquad \mathrel{\#} \text{cancel $2$ and $\dfrac{1}{2}$} \\
[15pt]
&=& C_1 \mathlarger{\mathlarger{\mathlarger{\int}}} \; d\theta -
C_1 \mathlarger{\mathlarger{\mathlarger{\int}}} \cos 2\theta \; d\theta
			&\qquad \mathrel{\#} \text{integration is a linear operator} \\
[15pt]
&=& C_1 \theta - C_1 \left ( \dfrac{1}{2} \sin 2\theta \right )
			&\qquad \mathrel{\#} \text{integrate} \\
[15pt]
&=& C_1 \theta - \dfrac{C_1}{2} \sin 2\theta
			&\qquad \mathrel{\#} \text{simplify} \\
[15pt]
&\Rightarrow&
x + C_2  = C_1 \theta - \dfrac{C_1}{2} \sin 2\theta
			&\qquad \mathrel{\#} x = C_1 \theta - \dfrac{C_1}{2} \sin 2\theta - C_2
\end{array}
\end{equation*}

\bigskip
\noindent
Given these results we can state the solution for the Brachistochrone
problem:

\bigskip
\begin{equation}
x = C_1 \theta -  \dfrac{C_1}{2} \sin 2\theta - C_2
\label{eqn:cycloid_x}
\end{equation}

\bigskip
\noindent
and since $y = C_1 \sin^2 \theta$ (definition) and $\sin^2 \theta = \dfrac{1}{2} (1- \cos 2 \theta)$
(double-angle trig identity)

\bigskip
\begin{equation}
y = \dfrac{C_1}{2} \left ( 1 - \cos 2 \theta \right )
\label{eqn:cycloid_y}
\end{equation}

\smallskip
\bigskip
\noindent
Note that the constants $C_1$ and $C_2$ can be found from the
boundary conditions.

\bigskip
\noindent
Finally, recall the cycloid shown in Figure \ref{fig:cycloid} (reproduced here):

\bigskip
\begin{figure}[H]
\center{\includegraphics[scale=0.40] {images/cycloid1.png}}
\caption{The Cycloid Curve}
\label{fig:cycloid1}
\end{figure}

\bigskip
\noindent
If we set $C_1 = 2 a$, $C_2 = 0$, and $\theta = \dfrac{\phi}{2}$
in Equations \ref{eqn:cycloid_x} and \ref{eqn:cycloid_y} we see
that

\bigskip
\begin{equation*}
\begin{array}{lllll}
x
&=& C_1 \theta -  \dfrac{C_1}{2} \sin 2\theta - C_2
			&\qquad \qquad \qquad \qquad \mathrel{\#} \text{Equation \ref{eqn:cycloid_x}} \\
[10pt]
&=& C_1 \theta -  \dfrac{C_1}{2} \sin 2\theta
			&\qquad \qquad \qquad \qquad \mathrel{\#} C_2 = 0 \\
[10pt]
&=& 2 a \theta -  \dfrac{2 a}{2} \sin 2\theta
			&\qquad \qquad \qquad \qquad \mathrel{\#} C_1 = 2a  \\
[10pt]
&=& 2 a \dfrac{\phi}{2} -  \dfrac{2 a}{2} \sin \Bigg [ 2 \left (\dfrac{\phi}{2} \right ) \Bigg ]
			&\qquad \qquad \qquad \qquad \mathrel{\#} \theta = \dfrac{\phi}{2} \\
[10pt]
&=& a \phi -  a \sin \phi
			&\qquad \qquad \qquad \qquad \mathrel{\#} \text{cancel $\dfrac{1}{2}$ and $2$} \\
[10pt]
&=& a (\phi -  \sin \phi)
			&\qquad \qquad \qquad \qquad \mathrel{\#} x = a (\phi -  \sin \phi) \\
\end{array}
\end{equation*}

\bigskip
\noindent
and

\bigskip
\begin{equation*}
\begin{array}{lllll}
y
&=& \dfrac{C_1}{2} \left ( 1 - \cos 2 \theta \right )
			&\qquad \qquad \qquad \qquad \mathrel{\#} \text{Equation \ref{eqn:cycloid_y}} \\
[10pt]
&=& \dfrac{2a}{2} \left ( 1 - \cos 2 \theta \right )
			&\qquad \qquad \qquad \qquad \mathrel{\#} C_1 = 2 a \\
[10pt]
&=& \dfrac{2a}{2} \Bigg ( 1 - \cos \left [ 2 \left (\dfrac{\phi}{2} \right ) \right ] \Bigg )
			&\qquad \qquad \qquad \qquad \mathrel{\#} \theta = \dfrac{\phi}{2} \\
[10pt]
&=& a \left ( 1 - \cos \phi \right )
			&\qquad \qquad \qquad \qquad \mathrel{\#} y = a \left (1 - \cos \phi \right)
\end{array}
\end{equation*}

\bigskip
\noindent
So we see that Equations \ref{eqn:cycloid_x} and \ref{eqn:cycloid_y}
turn out to be the equations for a cycloid (as we saw in Figures
\ref{fig:cycloid} and \ref{fig:cycloid1}) and hence the "shortest
time curve", that is, the solution to the Brachistochrone
problem, is a cycloid.

\section{Other Path Minimization Problems}
There are many path minimization problems we can solve with the
Calculus of Variations. Here are a few examples (WIP).

\subsection{Minimize the Distance (Pythagoras)}
{\setstretch{1.5}
In Section \ref{subsec:euler-lagrange} we saw that we can
solve for a differential distance $ds$ on some curve using
the Pythagorean Theorem; this is shown in Figure
\ref{fig:pythagorean_theorem}. Note that this approach
requires that we treat $\dfrac{dy}{dx}$ as a fraction
and as mentioned above, care needs to be taken when doing
this. That said, as we saw above \par}

\begin{equation*}
\begin{array}{lllll}
ds^2
&=& dx^2 + dy^2   												&\qquad \qquad \qquad \mathrel{\#} \text{Pythagorean Theorem (Figure \ref{fig:pythagorean_theorem})} \\
[10pt]
&=& \left ( 1 + \dfrac{dy^2}{dx^2} \right ) dx^2				&\qquad \qquad \qquad \mathrel{\#} \text{factor out $dx^2$} \\
[12pt]
&=& \left ( 1 + \left (\dfrac{dy}{dx} \right )^2 \right ) dx^2	&\qquad \qquad \qquad \mathrel{\#} \text{group $\dfrac{dy}{dx}$} \\
[12pt]
&=& \left ( 1 + (y^{\prime})^2 \right ) dx^2					&\qquad \qquad \qquad \mathrel{\#} \text{$y^{\prime} = \dfrac{dy}{dx}$; here we're treating $\dfrac{dy}{dx}$ as a fraction}\\
[10pt]
&\Rightarrow& ds = \sqrt{1 + (y^{\prime})^2} \; dx				&\qquad \qquad \qquad \mathrel{\#} \text{take the square root of both sides}
\end{array}
\end{equation*}

\bigskip
\noindent
So to minimize the distance we want to solve the following integral:

\bigskip
\begin{equation*}
I[y] = \int_{x_1}^{x_2} ds = \int_{x_1}^{x_2} \sqrt{1 + (y^{\prime})^2} \; dx
\end{equation*}

\medskip
\subsection{Minimize the Time (Brachistochrone Problem)}
We saw above that $ds = \sqrt{1 + (y^{\prime})^2} \; dx$
(Equation \ref{eqn:ds}) and $V = \sqrt{2gy}$ (Equation
\ref{eqn:V}) so the integral we want to solve to find the
curve that minimizes the time taken between two points $A$
and $B$ works out to be

\bigskip
\begin{equation*}
I[y]
= \mathlarger{\mathlarger{\int_0^t}} dt = \mathlarger{\mathlarger{\int_A^B}} \dfrac{ds}{V}
= \mathlarger{\mathlarger{\int_A^B}} \sqrt{\dfrac{1 + (y^{\prime})^2}{2gy}} \; dx
= \dfrac{1}{\sqrt{2g}} \mathlarger{\mathlarger{\int_A^B}} \sqrt{\dfrac{1 + (y^{\prime})^2}{y}} \; dx
\end{equation*}

\medskip
\subsection{Minimize the Energy (Catenary Problem)}
The Catenary Problem \cite{wiki:catenary} (also known as
the hanging chain problem) considers the shape of a
chain hanging from two stationary points where the only force
acting on the chain is gravity. In the problem setup $\rho$
is the density of the chain, $g$ is the acceleration of gravity,
and $A$ is the constant cross sectional area of the chain.
Note that since the units of
$\rho$ are $\frac{\text{mass}}{\text{volume}}$ and
${\displaystyle \lim_{ds \to 0} \text{Volume} = \text{Area}}$,
$\rho A = m$  for a differential piece of the chain of
length $ds$.

\bigskip
\noindent
In the Catenary Problem the chain is assumed to be
stationary so the kinetic energy is zero. So the total energy is
the just potential energy $mgy$. So here $I[y]$ is

\bigskip
\begin{equation*}
I[y]
= \int_{x_1}^{x_2} mgy \; ds
= \int_{x_1}^{x_2} (\rho A) g y \: ds
= \rho A g \int_{x_1}^{x_2} y \; ds
= \rho A g\int_{x_1}^{x_2} y \; \sqrt{1 + (y^{\prime})^2} \; dx
\end{equation*}

\medskip
\subsection{Minimize the Lagrangian (Lagrange's Equations)}
This one is discussed in detail in \cite{notes:pola} but briefly
here $I[y]$ is the integral of the Lagrangian:

\bigskip
\begin{equation*}
I[y] = \int_{t_1}^{t_2} L(y(t),y(t)^{\prime}) \; dt
\end{equation*}

\bigskip
\noindent
\bigskip
Recall that for the case of Lagrangian mechanics
the Lagrangian $L$ is defined to be

\begin{equation*}
L \equiv T - V
\end{equation*}

\bigskip
\noindent
where $T$ is an object's kinetic energy and $V$
is its potential energy.

\section{Conclusions}

\section*{Acknowledgements}
%
%	LaTeX source on overleaf.com
%
\section*{\LaTeX Source}
\url{https://www.overleaf.com/read/vpwbqqpnjywj}
%
%	get a bibliography
%
%	Note:.bib files go in ~/Library/texmf/bibtex/bib with TeXShop (MacTeX).
%	You can also use an absolute path, e.g. \bibliography{/Users/dmm/papers/bib/qc}
%
\bibliographystyle{plain}
\bibliography{qc}
%
%	done
%
\end{document}


\chapter{Introduction}
Outlier detection is the process of finding patterns in data that do not adhere to normal behavior. These patterns are known as outliers or anomalies \citep{Survey}. Detection of outliers has applicability in various domains such as fraud detection for credit card and insurance, intrusion detection for cyber security and industrial damage detection.
It is important to detect the outliers in a set of data because they can be translated into actionable information in various applications. In fact, there are different outlier detection techniques that have been developed for specific domains. These techniques can operate in supervised learning, semi-supervised learning and unsupervised learning \citep{Kurukshetra}.
According to the number of data dimensions, outlier detection methods are divided into univariate and multivariate methods \citep{Clark,Knorr}. Of all the different methods available for outlier detection, machine learning methods have the highest success rates for detecting outliers.

This research attempts to test two machine learning outlier detection models \citep{Silvia}: Local Outlier Factor and One Class Support Vector Machine on a real dataset to automatically detect credit card fraud. Models will be discussed in Chapter~\ref{Local_Outlier_Factor} and Chapter~\ref{Support_Vector_Machine} respectively. The models are compared using True Positive Rates and False Positive Rates they obtain. Chapter~\ref{Experiment} contains the research methodology, results and evaluation of the models.

This research will help to understand the performance of outlier detection methods using credit card transactions dataset, where the outliers are fraudulent transaction. Conclusions and suggestions for future work are presented in Chapter~\ref{Conclusion}.

The reminder of this chapter covers the basic terminology and concepts of outlier detection, outlier detection techniques and methods, and relevant machine learning concepts.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Defining Outliers}
An outlier is an observation (or subset of observations) which appears to be
inconsistent with the dataset \citep{Barnett}. Figure~\ref{outlier} illustrates outliers in a 2-dimensional dataset. Clearly, $o_1,o_2$ and $O_3$ are the points that are far from other points $N_1$ and $N_2$ which contain most of the data.
\begin{figure}[!h]
\centering
\includegraphics[width=0.5\textwidth]{images/outlier}
\vspace{-4mm}
\caption{Observations in 2-dimensional space \citep{Kurukshetra}.}
\vspace{-2mm}
\label{outlier}
\end{figure}
\begin{comment}
\begin{figure}[!htb]
\minipage{0.32\textwidth}
  \includegraphics[width=\linewidth]{images/O1-1}
  \caption{A really Awesome Image}\label{fig:awesome_image1}
\endminipage\hfill
\minipage{0.32\textwidth}
  \includegraphics[width=\linewidth,height =4.5cm]{images/O2}
  \caption{A really Awesome Image}\label{fig:awesome_image2}
\endminipage\hfill
\minipage{0.32\textwidth}%
  \includegraphics[width=\linewidth,height =4.5cm]{images/O3}
  \caption{A really Awesome Image}\label{fig:awesome_image3}
\endminipage
\end{figure}
\end{comment}

Outliers are often treated as errors or noise in datasets. Researchers may either reject or delete outliers in the datasets. However, sometimes outliers indicate something scientifically interesting or significant and as such may be useful. %\citep{NIST} %

Outliers may appear in datasets because of manual errors in the transmission or transcription process, changes in the behavior of the system or through deviations in populations. Alternatively, outliers could be caused by a flaw in the theory. Outliers may cause a negative effect on data analyses, so it is very important to define them \citep{Outlier}.

\textbf{Types of Outliers:}
Outliers can be classified into three categories:
\begin{itemize}
\item{Point Outliers:} This is the simplest type of outlier. An outlier is a point when an individual or a group of data patterns can be considered as unusual compared to the rest of the data \citep{Outlier}. Point $o_1$ and group $O_3$ in Figure~\ref{outlier} are examples of point outliers.
\item{Contextual Outliers:}
These are outliers found when observations are unusual in a specific context and are caused by the structure in the dataset. An observation might be an outlier in a certain context, but normal in a different context \citep{Kurukshetra}. There are two sets of attributes that define data patterns: contextual attributes and behavioral attributes. An example of contextual outlier is a sudden drop of temperature (assume 10F) in summer time is an outlier. But it considered as normal during winter.
\item{Collective Outliers:}
These are collected data patterns which are unusual in the dataset. Individual data point in the collective outlier may not be an outlier by itself. In fact, the occurrence of the data points together is unusual. Point outliers can appear in any dataset, whereas collective outliers appear only in datasets where the data patterns are related \citep{Kurukshetra}. The unusual regions on a human electrocardiogram is an example of collective outliers.
\end{itemize}
\vspace{-2mm}
\section{Outlier Detection}
\vspace{-2mm}
Most of the background in this section is a summery of \citep{Kurukshetra}. Where details about outliers applications and techniques may be found.

Outlier detection is the process of identifying the unusual observations in the datasets. However there are many reasons that make detecting outliers challenging. First of all, the unavailability of labeled data to apply in training and validation of the model. Secondly, the encompassing of all normal behavior in the region. In addition, the uncertainty of the boundary between normal and outlier behavior. At the top of that, the difficulty of applying a developed technique from one domain to another because the problem formulation of outlier detection is different.

The input of an outlier detection process is a collection of observations, where each one of them is a set of features of different types. These types can be binary, categorical or continuous.
To distinguish between normal data and outliers in a dataset labels are used. Labels should be accurate and should represent all types of behaviors. It is easier to label normal behavior of the observation, than anomalous data, this is due to the dynamic nature of the anomalous data, for example arrival of new types of outliers which are not labeled \citep{Survey}. Based on the extent to which labels are available, outlier detection techniques can operate in different modes.

\textbf{Modes of Outlier detection techniques}
\begin{itemize}
\item{Supervised Outlier Detection}
technique assumes the availability of labeled training datasets. The labels are either normal or outlier; which helps the predictive model in classification process. This technique faces a major issue which is the unlimited number of unusual data in training sets, as well as the accuracy of the data labels. Some techniques inject artificial outliers in the normal data \citep{Kurukshetra}.
\item{Semi-Supervised Outlier Detection}
technique assumes that training data has labeled patterns for the normal data, and is commonly used. There are some limited techniques that use labeled outliers instead of labeled normal data \citep{Kurukshetra}.
\item{Unsupervised Outlier Detection}
is the most flexible technique due to non-essentiality of training data. The assumption made in this technique is that outliers are far more than normal patterns in the dataset \citep{Kurukshetra}.
\end{itemize}

\textbf{Output of Outlier Detection}

Outlier detection techniques produce the output in one of the following types:
\begin{itemize}
\item{Scores:}
This technique assign scores for all of the observations in the test data based on the degree to which that observation is estimated to be an outlier. Therefore, the output will be a ranked list of all outliers which help the analyst to select the outliers \citep{Kurukshetra}.
\item{Labels:}
This technique assigns the label normal or outlier for each observation in the dataset, in this case, the analyst has no control on the output \citep{Kurukshetra}.
\end{itemize}

\textbf{Outlier Detection Methods}
\begin{itemize}
\item{Univariate Methods:}
There are different statistical theory methods to detect outliers in univariate datasets such as Chauvenet's criterion, Dixon's Q test and Grubb's test for outliers. However, there are major weaknesses of these approaches: The dataset may not follow Gaussian distribution. There are some methods use the mean and the variance to define outliers. However, mean and variance are outlier dependant \citep{Songwon}. These methods may include sample minimum or sample maximum which are not always outliers.
\item{Multivariate Methods:}
There are many methods to detect outliers in multivariate datasets, such as Robust Statistical-based Methods (e.g. Mahalanobis Distance method), Distance-based Methods (e.g. Distance to $k^{th}$ nearest neighbour method), Density-based Methods (e.g. Local Outlier Factor) and Artificial Intelligence Methods (e.g. Artificial Neural Networks and Support Vector Machines).
% \citep{Silvia} %
\end{itemize}

\section{Outlier Detection Applications}
There are several applications of outlier detection in different domains among them being the following:
\begin{itemize}
\item{Fraud Detection:}
These are the fraudulent applications for credit cards, mobile phones and insurance claims \citep{Kurukshetra}. The unauthorized usages may appear in different ways, such as a purchase from outside experienced geographical locations, credit card large amount transaction data and unauthorized and illegal insurance claims. %phd Sudan thesis
\item{Intrusion Detection:}
Is the detection of malicious activity (break-ins, penetrations, and other forms of computer abuse) in a computer related system which is interesting from a computer security perspective \citep{Survey}.
Detection of malicious activity in operating system calls, network traffic, or other unauthorized access activity in the system of host-based or networked computer systems.
\item{Industrial Damage Detection:}
This domain uses recorded sensors data which monitor the performance of industrial components \citep{Kurukshetra}, then these systems classify the defects that might occur due to wear and tear or any other reason \citep{Survey}.
\end{itemize}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Machine Learning}
Machine Learning (ML) is the field of study that gives computers the ability to learn without being explicitly programmed \citep{Andrew}. machine learning is one of the fields of artificial intelligence that focuses on building smart systems such as autonomous robotics, pattern recognition systems, Natural Language Processing(NLP) systems, computer vision and so forth.

ML automatically builds a highly non-linear and general model of the data.
\textbf{Types of ML}
\begin{itemize}
\item \textbf{Supervised Learning:}
Algorithms of this type take the input data as well as the correct output, and it associates the datasets and predicts better answers. The problems of these algorithms could be \textbf{regression} if we want to predict continuous valued output, an example of this is an algorithm that is used to predict the house price based on the size of the house. There is also a \textbf{classification} problem which is focuses on predicting discrete values, for example, if we want to decide whether a tumor type is either malignant or benign using tumor size \citep{Andrew}.
\item \textbf{Unsupervised Learning:}
This type of algorithm finds structure out of the dataset by partitioning/clustering data. Unsupervised learning algorithm is used in clustering genes data, where the algorithm groups the genes based on how they respond to different experiments \citep{Andrew}.
\end{itemize}

\textbf{ML in Outlier detection}

There are various models in machine learning that are used in outlier detection. The supervised detection technique requires a labeled training data which contains the normal observations and the outliers. It uses the testing data to test the model and find the results. Semi-supervised detection technique requires normal labeled training data and it uses the testing data to test the model and find results. While unsupervised detection technique doesn't require neither training data nor testing data \citep{Goldstein}.

This research will focus on the methods of semi-supervised outlier detection technique.
\begin{comment}
\textbf{ML in Outlier detection}

There are various models/methods in machine learning that are used in outlier detection. Figure~\ref{ML-OutlierDetection} describes the approach that is used in supervised, semi-supervised and unsupervised techniques. The first two techniques required data to train the predictive model/method, unlike unsupervised outlier detection technique.

This research will focus on the methods of semi-supervised outlier detection technique.

\begin{figure}[H]
\centering
\includegraphics[width=0.6\textwidth]{images/OutlierDetection-grayscale}
\caption{Outlier detection techniques in machine learning \citep{Goldstein}.}
\label{ML-OutlierDetection}
\end{figure}
\end{comment}


\documentstyle[a4wide,12pt]{article}

\begin{document}

\section*{Chaos in the driven nonlinear pendulum}

The angular equation of motion of the pendulum is given by
Newton's equation and with no external force it reads
\begin{equation}
  ml\frac{d^2\theta}{dt^2}+mgsin(\theta)=0,
\end{equation}
with an angular velocity and acceleration given by
\begin{equation}
     v=l\frac{d\theta}{dt},
\end{equation}
and
\begin{equation}
     a=l\frac{d^2\theta}{dt^2}.
\end{equation}

We do however expect that the motion will gradually come to an end
due a viscous drag torque acting on the pendulum.
In the presence of the drag, the above equation becomes
\begin{equation}
   ml\frac{d^2\theta}{dt^2}+\nu\frac{d\theta}{dt}  +mgsin(\theta)=0,
\label{eq:pend1}
\end{equation}
where $\nu$ is now a positive constant parameterizing the viscosity
of the medium in question. In order to maintain the motion against
viscosity, it is necessary to add some external driving force.
We choose here a periodic driving force. The last equation becomes then
\begin{equation}
   ml\frac{d^2\theta}{dt^2}+\nu\frac{d\theta}{dt}  +mgsin(\theta)=Asin(\omega t),
\label{eq:pend2}
\end{equation}
with $A$ and $\omega$ two constants representing the amplitude and
the angular frequency respectively. The latter is called the driving frequency.

\begin{enumerate}
\item[a)] Rewrite Eqs.~(\ref{eq:pend1}) and (\ref{eq:pend2}) as dimensionless equations. That is, scale the equations.

\item [b)] Write then a code which solves  Eq.~(\ref{eq:pend1}) using the Euler-Cromer method and fourth-order
Runge Kutta method. Perform calculations for at least ten  periods with $N=100$, $N=1000$ and $N=10000$ mesh points
and values of $\nu = 1$, $\nu = 5$ and $\nu =10$.
Set $l=1.0$ m, $g=1$ m/s$^2$ and $m=1$ kg.
Choose as initial conditions $\theta(0) = 0.2$ (radians) and $v(0) = 0$ (radians/s).
Make plots of $\theta$ (in radians) as function of time
and phase space plots of
$\theta$ versus the velocity $v$.
Check the stability of your results as functions of time and number of mesh points.
Which case corresponds to damped, underdamped and overdamped oscillatory motion?  Comment your results.
\item[c)] Now we switch to Eq.~(\ref{eq:pend2}) for the rest of the project. Add an external driving force and
set $l=g=1$, $m=1$, $\nu = 1/2$ and $\omega = 2/3$.
Choose as initial conditions $\theta(0) = 0.2$ and $v(0) = 0$ and $A=0.5$ and $A=1.2$.
Make plots of $\theta$ (in radians) as function of time for at least 300 periods and phase space plots of
$\theta$ versus the velocity $v$. Choose an appropriate time step. Comment and explain the results for the different values of $A$.
\item[d)]  Keep now the constants from the previous exercise fixed but set now $A=1.35$, $A=1.44$ and
$A=1.465$. Plot $\theta$ (in radians) as function of time for at least 300 periods for these values of $A$ and
comment your results.
\item[e)] We want to analyse further these results by making phase space plots of
$\theta$ versus the velocity $v$ using only the points where we have $\omega t=2n\pi$ where $n$ is an integer. These are normally
called the drive periods.
This is an example of what is called a Poincare section and is a very useful way to plot and analyze the behavior of a dynamical
system. Comment your results.
\end{enumerate}

\end{document}


\documentclass[jou]{apa6}

\usepackage[american]{babel}

\usepackage{csquotes}
\usepackage[style=apa,sortcites=true,sorting=nyt,backend=biber]{biblatex}
\DeclareLanguageMapping{american}{american-apa}
\addbibresource{bibliography.bib}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Discrete Structures
%% The start of RBS stuff
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Working internal and external links in PDF
\usepackage{hyperref}
% Extra math symbols in LaTeX
\usepackage{amsmath}
\usepackage{gensymb}
\usepackage{amssymb}
% Enumerations with (a), (b), etc.
\usepackage{enumerate}
\usepackage[framemethod=TikZ]{mdframed}
\usepackage{xcolor}

\let\OLDitemize\itemize
\renewcommand\itemize{\OLDitemize\addtolength{\itemsep}{-6pt}}

\usepackage{etoolbox}
\makeatletter
\preto{\@verbatim}{\topsep=3pt \partopsep=3pt }
\makeatother

% These sizes redefine APA for A4 paper size
\oddsidemargin 0.0in
\evensidemargin 0.0in
\textwidth 6.27in
\headheight 1.0in
\topmargin -24pt
\headheight 12pt
\headsep 12pt
\textheight 9.19in

\title{Sample Quiz 8}
\author{Discrete Structures, Spring 2020}
\affiliation{RBS}

\leftheader{Discrete Sample Quiz 8}

\abstract{%
}

%\keywords{}

\setlength\parindent{0pt}

\begin{document}

%\thispagestyle{empty}

\twocolumn
\section{Quiz 12: Trees}

{\bf Question 1.} The wheel graph $W_4$ has $5$ vertices:
$4$ vertices form a cycle graph $C_4$ \textendash{} a square;
one more vertex sits in the middle and is connected with the remaining $4$ vertices.
($W_4$ is isomorphic to an ordinary Egyptian pyramid.)
Assume that all vertices in the wheel graph are named with letters and are distinguishable.
Find the number of unrooted spanning trees in the $W_4$.\\
Write a positive integer in your answer.

\vspace{10pt}
{\bf Question 2.} It is known that a full $m$-ary tree $T$ has $25$ leaves, but the parameter $m$ is
not known \textendash{} it can take any fixed value: $m = 2,3,4,\ldots$.
How many inner nodes can $T$ have? Find all possible answers.\\
Write an increasing comma-separated list.

\vspace{10pt}
{\bf Question 3.}\\ There is a rooted tree with $111$ vertices and each vertex can have up to $3$ children.
Find the minimum and the maximum height of this tree.\\
Write two comma-separated integers.

\vspace{10pt}
{\bf Question 4.}\\
Assume that there is a rooted tree (with anonymous/unnamed vertices and unordered children) and its vertices have
the following degrees $3, 3, 2, 2, 1, 1, 1, 1$. It is also known that its root vertex has $3$ children.
Find the number of such rooted unordered trees.\\
Write a positive integer.

\vspace{10pt}
{\bf Question 5.}

\begin{figure}[!htb]
\center{\includegraphics[width=3.2in]{quiz-12/huffman-tree.png}}
\caption{\label{fig:huffman-tree} Encoding with a Tree.}
\end{figure}

Figure~\ref{fig:huffman-tree} shows an efficient method to send messages
(like {\tt abracadabra!}) using $6$ characters.
Assume that the characters appear with the following probabilities:\\
{\small
\begin{tabular}{|l|c|c|c|c|c|c|} \hline
Symbol & {\tt a} & {\tt b} & {\tt d} & {\tt r} & {\tt c} & {\tt !} \\ \hline
Probability &  $1/2$ & $1/8$ & $1/8$ & $1/8$ & $1/16$ & $1/16$ \\ \hline
\end{tabular}\\
}
Find the expected number of bits used per character (i.e. $E(X)$ \textendash{} the expected value of
the random variable $X$, which describes the number of bits used per one
character from this random distribution.)\\
Write a real number \textendash{} the number of bits rounded to the nearest thousandth.

%{\em Note.} For the given probabilities, this tree offers the best compression
%(using it for the encoding needs the smallest
%expected number of bits). For such optimal codes we say that the expected number of bits
%for sending one character equals the {\em entropy} of the probability distribution.
%It is fine, if the entropy is a fractional (non-integer) number of bits.

\vspace{10pt}
{\bf Question 6.}
The string
$$\rightarrow\;\rightarrow\;\rightarrow\;\rightarrow\;\rightarrow\;a\;b\;\textcolor{red}{\triangle}\;\neg\;c\;\neg\;d\;c\;e\;\rightarrow\;\neg\;a\;\rightarrow\;d\;e.$$
is the prefix notation for a Boolean logic expression with one symbol replaced by a $\textcolor{red}{\triangle}$.
What can this $\textcolor{red}{\triangle}$ represent?\\
{\bf (A)} It is a propositional variable ($a,b,c$ or similar).\\
{\bf (B)} It is a unary Boolean operator ($\neg$ or similar).\\
{\bf (C)} It is a binary Boolean operator ($\rightarrow$ or similar).\\
{\bf (D)} It cannot be any of these; the expression is invalid in all these cases.

Write the answer letter (A, B, C, or D).

\vspace{10pt}
{\bf Question 7.}
Imagine that you search for all ways how to place $4$ queens on a $4 \times 4$ chessboard so that
they do not attack each other. (A chess queen attacks all squares on its horizontal, on its vertical
and also on both diagonals.)\\
Imagine that you build a tree for this:\\
{\bf (Level 0 to Level 1)} The root of the tree is an empty $4 \times 4$ chess-board; you add to it four children
(all $4$ ways how you can place a queen on the 1st row of the chess-board).\\
{\bf (Level 1 to Level 2)} For any of the vertices you added in the previous step, add children on level $2$ by placing
another queen on the 2nd row so that it does not attack the first one.\\
In general, the vertex on level $L=i$ has queens on rows $1,\ldots,i$ that do not attack each other.
Any vertex on level $L=4$ will be a solution to this ``4 queens problem'' with all four rows containing a queen.

Write the total number of vertices in this tree (that the backtracking algorithm will visit).

\vspace{10pt}
{\bf Question 8.} An undirected graph $G = (V,E)$ has the set of vertices $V$ \textendash{}
the set of all positive divisors of the number $900$ (including $1$ and $900$ itself).
A pair of divisors $(d_1,d_2)$ is an edge in $G$ iff their ratio
$d_1/d_2$ (or $d_2/d_1$) is a prime number.

In the root vertex $v_1=1$ we start the BFS (Breadth-first-search) traversal of the graph $G$. For every vertex
we visit all its adjacent vertices in increasing order (for example edges $(1,2)$, $(1,3)$ and $(1,5)$
are visited in this order. When all the children of vertex $1$ are visited,
we start visiting all the adjacent vertices of $v_2=2$, and so on.). We get the BFS traversal order like this:
$$v_1=1,\,v_2=2,\,v_3=3,\,v_4=5,\,v_5=4,\,\ldots$$
Find the vertices $v_{13},\,v_{14},\,v_{15}$ in this BFS order.

Write three comma-separated numbers.

%\newpage
\vspace{10pt}
{\bf Question 9.}

\begin{figure}[!htb]
\center{\includegraphics[width=2in]{quiz-12/prim-algorithm.png}}
\caption{\label{fig:prim-algorithm} Weighted Graph.}
\end{figure}

Find the weight of a minumum spanning tree (MST) in the tree shown in Figure~\ref{fig:prim-algorithm}.
In order to construct this tree, you can use Prim's algorithm (Rosen2019, p.836).
Start in vertex $a$ - this is your first tree $T_1$.
In every step pick the minimum weight edge that is adjacent to $T_i$ (and does
not create any loop) \textendash{} add it to the tree $T_i$, and obtain the next
tree $T_{i+1}$. Continue adding the minimum weight edges until all vertices are connected.
(In the second step you will have a choice
to add $(b,c)$ or $(a,h)$ as both edges have the same weight $8$. There may be several MSTs
in a graph, but all of them will have the same total weight.)

Write the total MST weight as a number.

\vspace{10pt}
{\bf Question 10.}
The vertices in the directed graph (Figure~\ref{fig:dfs-traversal2}) are visited in the DFS
order. You start with the alphabetically smallest vertex ($q$), order all the vertices connected to it
alphabetically, then build the DFS traversal.

\begin{figure}[!htb]
\center{\includegraphics[width=3in]{quiz-12/dfs-traversal2.png}}
\caption{\label{fig:dfs-traversal2} Graph for DFS traversal.}
\end{figure}

Write the sequence with parentheses and vertices for the graph
on Figure 3 \textendash{} similar to the sequence (\ref{eq:dfs-search-order}) \textendash{} see Appendix.
Each of its $10$ vertices should be mentioned in your
traversal order twice (the first time with an opening parenthesis, the second time
with a closing parenthesis).

\mbox{}
\newpage

{\bf Appendix: DFS}

\begin{figure}[!htb]
\center{\includegraphics[width=1.4in]{quiz-12/dfs-traversal.png}}
\caption{\label{fig:dfs-traversal} Sample DFS traversal.}
\end{figure}

Figure~\ref{fig:dfs-traversal} shows a DFS (Depth-First-Search) traversal
in a directed graph. Vertexes are visited in alphabetical order
(so $u$ is visited first; followed by its alphabetically first
child $v$, followed by $y$, followed by $x$. After that
we visit another tree (unreachable from the first one) \textendash{}
$w$ followed by $z$.)\\
{\bf Tree edges} that belong to the DFS traversal tree are shaded;\\
{\bf Back edges} that point back from a vertex to its ancestor in the tree (or a loop to itself)
are labeled by $B$;\\
{\bf Forward edges} that jump from a vertex to its descendant in the DFS tree (other than a child) are
labeled by $F$;\\
{\bf Cross edges} that jump between two vertices that are not descendants/ancestors of
each other are labeled by $C$.

The following sequence
\begin{equation} \label{eq:dfs-search-order}
\textcolor{blue}{\mathtt{(u\;(v\;(y\;(x\;x)\;y)\;v)\;u)\;(w\;(z\;z)\;w)}}
\end{equation}
denotes the DFS traversal order in the oriented graph.
Every time when we enter some vertex (and its subtree),
we open a parenthesis and write the vertex name; when we leave,
we write the vertex name again and close the parenthesis.\\
This order is also written inside each vertex (for example {\tt 1/8}
for vertex $u$ means that we entered it in Step 1, and left it in
Step 8). For vertex $z$ this pair is {\tt 10/11} (we entered this leaf
of the DFS tree and immediately left it).
%\end{mdframed}

%\begin{mdframed}[roundcorner=6pt]
%\section{Appendix 2: Prim's Algorithm}

%\end{mdframed}

\newpage
\subsection{Answers}

\vspace{4pt}
{\bf Question 1.} Answer: {\tt 45} \\

There are only three kinds of trees with $5$ vertices, if isomorphic trees count as the same.
See {\em isomers of pentane} \textendash{} \url{https://bit.ly/2W4BWiw} (either the vertex degrees are
$1,2,2,2,1$, or $1,2,3,1,1$, or $4,1,1,1,1$).

Let us denote the vertices by $A,B,C,D,E$ ($E$ being the vertex at the top of the pyramid).
The only non-existing edges are $(A,C)$ and $(B,D)$ (otherwise it is almost like $K_5$).
In Figures~\ref{fig:spanning-trees-on-w5-a}, \ref{fig:spanning-trees-on-w5-b},
\ref{fig:spanning-trees-on-w5-c} we show different subcases depending

\begin{figure}[!htb]
\center{\includegraphics[width=2.2in]{quiz-12/spanning-trees-on-w5-a.png}}
\caption{\label{fig:spanning-trees-on-w5-a} 24 trees with all degrees $\leq 2$.}
\end{figure}

\begin{figure}[!htb]
\center{\includegraphics[width=1.8in]{quiz-12/spanning-trees-on-w5-b.png}}
\caption{\label{fig:spanning-trees-on-w5-b} 20 trees with one degree $=3$.}
\end{figure}

\begin{figure}[!htb]
\center{\includegraphics[width=0.9in]{quiz-12/spanning-trees-on-w5-c.png}}
\caption{\label{fig:spanning-trees-on-w5-c} 1 tree with one degree $=4$.}
\end{figure}

$$N = (8 + 8 + 8) + (8 + 8 + 4) + (1) = 45.$$

\vspace{4pt}
{\bf Question 2.} Answer: {\tt 1,2,3,4,6,8,12,24} \\
In a full $m$-ary tree initially there is just one
node (it is the root and also a leaf). After that
the number of leaves can increase by $m-1$ in a single step
(one leaf becomes an internal node and creates exactly $m$ children).
The question now becomes \textendash{} in how many arithmetic
progressions both numbers $1$ and $25$ participate.
We get these variants:

$$\left\{ \begin{array}{l}
1,2,3,\ldots,25 \\
1,3,5,7,9,11,13,15,17,19,21,23,25 \\
1,4,7,10,13,16,19,22,25 \\
1,5,9,13,17,21,25 \\
1,7,13,19,25 \\
1,9,17,25 \\
1,13,25 \\
1,25 \\
\end{array} \right.$$

In each variant we count the number of increments
(how many times $m-1$ was added in order to get $25$). At every such step
an inner node is created.
The number of inner nodes can be computed as $\frac{25-1}{m-1}$.

\begin{tabular}{|l|l|} \hline
$m$ & Inner nodes \\ \hline
$2$ & $24$ \\ \hline
$3$ & $12$ \\ \hline
$4$ & $8$ \\ \hline
$5$ & $6$ \\ \hline
$7$ & $4$ \\ \hline
$9$ & $3$ \\ \hline
$13$ & $2$ \\ \hline
$25$ & $1$ \\ \hline
\end{tabular}

\vspace{4pt}
{\bf Question 3.} Answer: {\tt 4,110}

{\bf Minimum height.} There is one vertex (the root) at depth $d=0$,
at most $3$ vertices at $d=1$, at most $9$ vertices at $d=2$,
at most $27$ vertices at $d=3$ and at most $81$ vertices at $d=4$.
If maximum depth is $4$ (i.e. the height of the tree is also $4$), then
there can be at most $1 + 3 + 9 + 27 + 81 = 121$
vertices in the tree. Which is just fine, since $111 < 121$.

{\bf Maximum height.} In this case each vertex has just one child.
The deepest node has depth $d=110$ (one less than the number of nodes
in the tree).

\vspace{4pt}
{\bf Question 4.} Answer: {\tt 6}

We can sort cases depending on the location of the other vertex $v_1$
with degree $3$ (it is known that another one is root; let us denote the root by $v_0$).
That other vertex can have depth $1$, $2$ or $3$.

{\bf Case 1.} $\text{depth}(v_1) = 1$. There are three possible trees in this case;
the two vertices of degree $2$ can be either both children of $v_0$, or both of $v_1$,
or one can be a child of $v_1$, and another of $v_0$.\\
{\bf Case 2.}  $\text{depth}(v_1) = 2$. There are two possible trees in this case;
one of the vertices of degree $2$ is beteween $v_0$ and $v_1$; but another vertex
of degree $2$ can be a child of $v_0$, or a child of $v_1$.\\
{\bf Case 3.} $\text{depth}(v_1) = 3$. There is just one tree in this case.
Both vertices of degree $2$ are between $v_0$ and $v_1$.

\vspace{4pt}
{\bf Question 5.} Answer: {\tt 2.125} \\
To geet the expected value of the random variable (the length of the code in bits)
multiply the respective code lengths for all six letters of the alphabet
by their respective probabilities:

$$1 \cdot \frac{1}{2} + 3 \cdot \frac{1}{8} + 3 \cdot \frac{1}{8} + 3 \cdot \frac{1}{8} + 4 \cdot \frac{1}{16} + 4 \cdot \frac{1}{16} = 2\frac{1}{8}.$$

{\em Note.} Variable length codes where one code is never a prefix of another and
all the codes can be arranged in a binary tree, where grouping them starts by the two least frequent letters
({\tt !} and {\tt c} in our case) is named {\em Huffman tree}.

\vspace{4pt}
{\bf Question 6.} Answer: {\tt C}\\
In any (infix, prefix or postfix) expression the number of binary operators (in our case the only
such operator is $\rightarrow$) should be one less than the number of operands (in our case
these are the Boolean variables $a$,$b$,$c$,$d$,$e$). Unary operators (in our case $\neg$) do not count.

In our expression so far there are $7$ operators $\rightarrow$. There are also $9$ Boolean variables.
Therefore the triangle should become a binary operator $\rightarrow$.
The expression looks like this:
$$\rightarrow\;\rightarrow\;\rightarrow\;\rightarrow\;\rightarrow\;a\;b\;\textcolor{red}{\rightarrow}\;\neg\;c\;\neg\;d\;c\;e\;\rightarrow\;\neg\;a\;\rightarrow\;d\;e.$$

Here is the regular (infix) representation of the same Boolean expression:

{\footnotesize
$$((((a \rightarrow b) \rightarrow (\neg c \textcolor{red}{\rightarrow} \neg d)) \rightarrow c) \rightarrow e)
  \rightarrow (\neg a \rightarrow (d \rightarrow e)).$$
}

\vspace{4pt}
{\bf Question 7.} Answer: {\tt 17}\\

If we denote the rows/horizontals by numbers $1,2,3,4$ and the columns/verticals by letters $A,B,C,D$
as in ordinary chess, we can denote adding queens in the tree shown in Figure~\ref{fig:backtracking-4-queens}.
The only two solutions are two shaded nodes at the bottom. All other placements of queens are
dead ends.

{\em Note.} Not pursuing these dead ends (not going deeper, if we know that it is useless)
reduces the number of nodes to consider to $17$ (down from $4! = 24$ or even $4^4 = 256$, if we would
apply {\em brute force}). This is not very much, but for larger chess boards the savings are larger.
See \url{https://bit.ly/3aQ1feo}.

\begin{figure}[!htb]
\center{\includegraphics[width=2.4in]{quiz-12/backtracking-4-queens.png}}
\caption{\label{fig:backtracking-4-queens} Backtracking tree: 4 Queens problem.}
\end{figure}

\vspace{4pt}
{\bf Question 8.} Answer: {\tt 18,30,50} \\

You can determine the order of the vertices in the BFS traversal by counting vertices
layer by layer (Figure~\ref{fig:divisors-of-900}).
Note that the number $30 = \sqrt{900}$ is the $14$th vertex in the BFS traversal (it is in the very middle).

\begin{figure}[!htb]
\center{\includegraphics[width=2in]{quiz-12/divisors-of-900.png}}
\caption{\label{fig:divisors-of-900} BFS tree with divisors of 900.}
\end{figure}

\vspace{4pt}
{\bf Question 9.} Answer: {\tt 37} \\

We add edges using Prim's algorithm:

\begin{tabular}{|l|l|l|} \hline
Step 1 & $AB$ & 4 \\ \hline
Step 2 & $BC$ & 8 \\ \hline
Step 3 & $CI$ & 2 \\ \hline
Step 4 & $CF$ & 4 \\ \hline
Step 5 & $FG$ & 2 \\ \hline
Step 6 & $GH$ & 1 \\ \hline
Step 7 & $CD$ & 7 \\ \hline
Step 8 & $DE$ & 9 \\ \hline
\end{tabular}

Total weight:

$$4 + 8 + 2 + 4 + 2 + 1 + 7 + 9 = 37.$$

\vspace{4pt}
{\bf Question 10.} Answer:\\

$$\textcolor{blue}{\mathtt{(q\,(s\,(v\,(w\;w)\,v)\,s)\,(t\,(x\,(z\;z)\,x)\,(y\,y)\,t)\,q)\;\;(r\,(u\;u)\,r)}}$$

The DFS traversal creates two trees
with $q$ and $r$ as their roots respectively.

\end{document}


% Created 2020-12-10 Thu 21:44
% Intended LaTeX compiler: xelatex
\documentclass[12pt]{article}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\usepackage{minted}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{setspace}
\usepackage{subcaption}
\usepackage{mathtools}
\usepackage{xfrac}
\usepackage[margin=1in]{geometry}
\usepackage[utf8]{inputenc}
\usepackage{color}
\usepackage{epsf}
\usepackage{tikz}
\usepackage{graphicx}
\usepackage{pslatex}
\usepackage{hyperref}
\usepackage[adobe-utopia]{mathdesign}
\usepackage{helvet}
\renewcommand{\familydefault}{\sfdefault}
\usepackage{textgreek}
\renewcommand*{\textgreekfontmap}{%
{phv/*/*}{LGR/neohellenic/*/*}%
{*/b/n}{LGR/artemisia/b/n}%
{*/bx/n}{LGR/artemisia/bx/n}%
{*/*/n}{LGR/artemisia/m/n}%
{*/b/it}{LGR/artemisia/b/it}%
{*/bx/it}{LGR/artemisia/bx/it}%
{*/*/it}{LGR/artemisia/m/it}%
{*/b/sl}{LGR/artemisia/b/sl}%
{*/bx/sl}{LGR/artemisia/bx/sl}%
{*/*/sl}{LGR/artemisia/m/sl}%
{*/*/sc}{LGR/artemisia/m/sc}%
{*/*/sco}{LGR/artemisia/m/sco}%
}
\makeatletter
\newcommand*{\rom}[1]{\expandafter\@slowromancap\romannumeral #1@}
\makeatother
\DeclarePairedDelimiterX{\infdivx}[2]{(}{)}{%
#1\;\delimsize\|\;#2%
}
\newcommand{\infdiv}{D\infdivx}
\DeclarePairedDelimiter{\norm}{\lVert}{\rVert}
\def\Z{\mathbb Z}
\def\R{\mathbb R}
\def\C{\mathbb C}
\def\N{\mathbb N}
\def\Q{\mathbb Q}
\def\noi{\noindent}
\onehalfspace
\author{Sandy Urazayev\thanks{University of Kansas (ctu@ku.edu)}}
\date{327; 12020 H.E.}
\title{Final Project Outline\\\medskip
\large MATH 596}
\hypersetup{
 pdfauthor={Sandy Urazayev},
 pdftitle={Final Project Outline},
 pdfkeywords={},
 pdfsubject={},
 pdfcreator={Emacs 28.0.50 (Org mode 9.3)},
 pdflang={English}}
\begin{document}

\maketitle
\section*{Introduction}
\label{sec:org5b6727a}
\begin{itemize}
\item Introduce the problem of data rotting and resolution sharpening
\item Introduce various examples of low-res images (old photos)
\end{itemize}

\section*{Data Overview}
\label{sec:orgee1b949}
\begin{itemize}
\item Talk about the data required for this application
\item Resolve around the quality of data needed for better results
\item Introduce the datasets we are using
\item Talk about data usage limitations
\end{itemize}

\section*{Design Implementation}
\label{sec:org5398d36}
\begin{itemize}
\item Talk about the convolution neural networks, how to design them
\item Discussion on input-output vector sizes
\item Talk about dividing data into training sets and validation sets
\item Discovering ways to record error from the ground-truth
\end{itemize}

\section*{Implementation}
\label{sec:org310c4b6}
\begin{itemize}
\item Actual step-by-step implementation of the problem
\item Include code snippets from Keras
\item Introduce annoying issues with neural networks and fine-tuning
\item More hardships encountered
\end{itemize}

\section*{Mathematical Basis}
\label{sec:org20a6c39}
\begin{itemize}
\item More in-depth look into the system
\item Talk about the math basis for the implemented layers
\item Backpropagation techniques when using convolution
\end{itemize}

\section*{Results}
\label{sec:org6c0b678}
\begin{itemize}
\item Show predictions for our test data
\item Record the errors from the ground truth data
\end{itemize}

\section*{Conclusion}
\label{sec:org520bb78}
\begin{itemize}
\item Conclusion on the problems
\item Areas for improvement
\item Comparing to cutting-edge research
\end{itemize}
\end{document}