%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Short Sectioned Assignment
% LaTeX Template
% Version 1.0 (5/5/12)
%
% This template has been downloaded from:
% http://www.LaTeXTemplates.com
%
% Original author:
% Frits Wenneker (http://www.howtotex.com)
%
% License:
% CC BY-NC-SA 3.0 (http://creativecommons.org/licenses/by-nc-sa/3.0/)
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%----------------------------------------------------------------------------------------
%	PACKAGES AND OTHER DOCUMENT CONFIGURATIONS
%----------------------------------------------------------------------------------------

\documentclass[paper=a4, fontsize=11pt]{scrartcl} % A4 paper and 11pt font size

\usepackage[T1]{fontenc} % Use 8-bit encoding that has 256 glyphs
\usepackage{fourier} % Use the Adobe Utopia font for the document - comment this line to return to the LaTeX default
\usepackage[english]{babel} % English language/hyphenation
\usepackage{amsmath,amsfonts,amsthm} % Math packages
\usepackage{titling}
\usepackage{lipsum} % Used for inserting dummy 'Lorem ipsum' text into the template

\usepackage{multicol}
\setlength{\columnsep}{1cm}

\usepackage{enumitem}

\usepackage{listings}
\usepackage{framed}
\usepackage{setspace} %\begin{doublespacing} \end...

\usepackage{sectsty} % Allows customizing section commands
\allsectionsfont{\centering \normalfont\scshape} % Make all sections centered, the default font and small caps

\usepackage{fancyhdr} % Custom headers and footers
\pagestyle{fancyplain} % Makes all pages in the document conform to the custom headers and footers
\fancyhead{} % No page header - if you want one, create it in the same way as the footers below
\fancyfoot[L]{} % Empty left footer
\fancyfoot[C]{} % Empty center footer
\fancyfoot[R]{\thepage} % Page numbering for right footer
\renewcommand{\headrulewidth}{0pt} % Remove header underlines
\renewcommand{\footrulewidth}{0pt} % Remove footer underlines
\setlength{\headheight}{13.6pt} % Customize the height of the header

\numberwithin{equation}{section} % Number equations within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)
\numberwithin{figure}{section} % Number figures within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)
\numberwithin{table}{section} % Number tables within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)

\setlength\parindent{0pt} % Removes all indentation from paragraphs - comment this line for an assignment with lots of text

%----------------------------------------------------------------------------------------
%	TITLE SECTION
%----------------------------------------------------------------------------------------

\newcommand{\horrule}[1]{\rule{\linewidth}{#1}} % Create horizontal rule command with 1 argument of height

\title{
\normalfont \normalsize
\textsc{Northridge Preparatory School} \\ [25pt] % Your university, school and/or department name(s)
\horrule{0.5pt} \\[0.4cm] % Thin top horizontal rule
\huge Worksheet\#5 : Debugging Exercise \\ % The assignment title
\horrule{2pt} \\[0.5cm] % Thick bottom horizontal rule
}

\preauthor{}
\postauthor{}
\author{} % Your name

\predate{}
\postdate{}
\date{} % Today's date or a custom date

\begin{document}

\maketitle % Print the title

%----------------------------------------------------------------------------------------
%	PROBLEM 1
%----------------------------------------------------------------------------------------

\section{Coding Without Comments -- Just Don't!}

     Get in the habit of adding comments when you write code.  Not only will it help others who read your code to understand what it is supposed to do, but it will also help YOU to remember this when you revisit your programs months after you've written them.

\subsection{No Comments...?}

Read the following faulty code, and try to figure out what this program is trying to do.

\begin{framed}
\begin{lstlisting}
def newfib(x,y):

    a = x + y

    b = x + b

    c = z + b

    d = b * c

    e = c + d

    a,b,x,d,e
\end{lstlisting}
\end{framed}

\begin{enumerate}

\item What does this program do? \rule{9cm}{0.15mm}\\

\rule{13.5cm}{0.15mm}

\pagebreak

\item There are 5 errors in this code.  Try to find them all.  Here it is again:
\begin{framed}
\begin{lstlisting}
def newfib(x,y):

    a = x + y

    b = x + b

    c = z + b

    d = b * c

    e = c + d

    a,b,x,d,e
\end{lstlisting}
\end{framed}

\begin{enumerate}

\item Error 1:\rule{11cm}{0.15mm}\\

\item Error 2:\rule{11cm}{0.15mm}\\

\item Error 3:\rule{11cm}{0.15mm}\\

\item Error 4:\rule{11cm}{0.15mm}\\

\item Error 5:\rule{11cm}{0.15mm}\\

\end{enumerate}

\end{enumerate}

This exercise is intended to be hard, if not impossible.  It's not clear what the program is supposed to do (though the name and general structure gives a small clue...), so you can't really tell what's wrong unless you get an explicit error message.  A few comments would have really helped!

\pagebreak

\subsection{Comments!}

     Try is again with another block of code, this time with comments:

\begin{framed}
\begin{lstlisting}
#Given initial velocity & time, return displacement of freefall

def freef(v_0,t):

    a = -9.0     #gravity acceleration in m/s

    d = v0*t + 0.5g(t^2)  #basic displacement formula

    d
\end{lstlisting}
\end{framed}

\begin{enumerate}

\item What does this program do? \rule{9cm}{0.15mm}\\

\rule{13.5cm}{0.15mm}\\

\item There are 5 errors in this code.  Can you find them all?\\

\begin{enumerate}

\item Error 1:\rule{11cm}{0.15mm}\\

\item Error 2:\rule{11cm}{0.15mm}\\

\item Error 3:\rule{11cm}{0.15mm}\\

\item Error 4:\rule{11cm}{0.15mm}\\

\item Error 5:\rule{11cm}{0.15mm}\\

\end{enumerate}

Technically, you shouldn't clutter up your code with too many comments, since that also makes it hard to read.  But how many comments are too many?  That's something you'll eventually learn through practice.  In the meantime, I suggest erring on the side of over-commenting.  Pound for pound, it will take you as a beginner more time and effort to remember forgotten code than it will to ignore unnecessary comments.

\end{enumerate}

\end{document}


\documentclass{beamer}
\usetheme{Warsaw}
\setbeamertemplate{footline}[frame number]
\setbeamertemplate{headline}{}
\setbeamercovered{transparent}
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}

\title[Anderson’s Mixing]{Anderson’s Mixing for Nonlinear Magnetostatic Poisson Problem}
\author{Alexander Zhiliakov}
\institute[UH] {
	Department of Mathematics \\
	University of Houston
}
\date[\today]{Math\,6376 Numerical Linear Algebra, \today}

% sub figures / grids of pictures
\usepackage{subcaption}
\graphicspath{{../img/}} % includegraphics path
\newcommand{\includegraphicsw}[2][1.]{\includegraphics[width=#1\linewidth]{#2}}
% tables
\let\oldtabular\tabular
\renewcommand{\tabular}[1][1.5]{\def\arraystretch{#1}\oldtabular}
\usepackage{multirow}
\usepackage{colortbl}
% \coloneqq
\usepackage{mathtools}
% math commands for convinience
\DeclareMathOperator{\argmin}{arg\,min}
% bold vectors
\newcommand{\vect}[1]{\boldsymbol{\mathbf{#1}}}

\begin{document}

	\begin{frame}
		\titlepage
	\end{frame}

	\section{Problem Description}

	\begin{frame}{Problem Description}
		\begin{figure}
			\centering
			\includegraphicsw[.6]{magnet.png}
			\caption{$z$ cross-section: iron magnet (gray), copper wires (red \& blue)}
		\end{figure}
		Problem: given current density $\vect J = (0, 0, J_z)$, $J_z \coloneqq \pm j \left[A\,m^{-2}\right]$ in wires, and magnetic permeability $\mu \left[N\,A^{-2}\right]$ of the magnet, find resulting magnetic field $\vect B \left[T\right]$
	\end{frame}

	\begin{frame}{Magnetic Permeability}
		One may write $\mu = \mu_0\,\hat\mu$ with $\mu_0 \coloneqq 4\,\pi\,10^{-7} \left[N\,A^{-2}\right]$ permeability of vacuum. If we assume $\hat\mu_{\text{iron}} = 1000$, resulting problem will be \textbf{linear}. However, in reality $\hat\mu_{\text{iron}}$ depends on $\vect B$:
		\begin{figure}
			\centering
			\begin{subfigure}{.45\linewidth}
				\centering
				\includegraphicsw{interp.pdf}
				\caption{$\hat\mu_{\text{iron}} = \hat{\mu}_{\text{iron}}(||\vect B||)$}
			\end{subfigure}%
			\hfill
			\begin{subfigure}{.45\linewidth}
				\centering
				\includegraphicsw{extrap.pdf}
				\caption{Extrapolated part}
			\end{subfigure}
		\end{figure}
		In this case we end up with \textbf{nonlinear} problem.
	\end{frame}

	\begin{frame}{Mathematical Model}
		\begin{enumerate}
			\item Start with Maxwell’s equations $\nabla\times\frac{1}{\mu} \vect B = \vect J$, $\nabla\cdot\vect B = 0$
			\item Introduce vector potential $\vect A$ via $\vect B = \nabla\times\vect A$ and rewrite (1) as $\nabla\times\frac{1}{\mu}\nabla\times\vect A = \vect J$
			\item Assume that magnet is very long in $z$-direction and note that only $z$-component $J_z$ of $\vect J$ is nonzero and $J_z = J_z(x, y)$; then (2) simplifies to Poisson problem $-\nabla\cdot(\frac{1}{\mu}\nabla A_z) = J_z$
			\item Equip (3) with appropriate boundary conditions and discretize using e.g finite elements; then we have to solve
			\begin{itemize}
				\item Linear system $\vect S\,\vect x = \vect b$ $\left[\mu = \mu(x, y)\right]$ or
				\item Nonlinear system $\vect S(\vect x)\,\vect x = \vect b$ $\left[\mu = \mu(||\vect B|| = ||\nabla A_z||)\right]$
			\end{itemize}
		\end{enumerate}
	\end{frame}

	\begin{frame}{Linear Post–processing}
		\begin{figure}
			\centering
			\caption{$A_z$}
			\begin{subfigure}{.45\linewidth}
				\centering
				\includegraphicsw{l06.png}
				\caption{$|J_z| = 10^6\,A\,m^{-2}$}
			\end{subfigure}%
			\hfill
			\begin{subfigure}{.45\linewidth}
				\centering
				\includegraphicsw{l11.png}
				\caption{$|J_z| = 10^{11}\,A\,m^{-2}$}
			\end{subfigure}
		\end{figure}
		For linear problem ($\hat\mu_{\text{iron}} = 1000$) only scaling changes as we change current density
	\end{frame}

	\section{Nonlinear Solvers}

	\begin{frame}{Nonlinear Solvers (1/2)}
		We need to solve $f(x) = 0$ ($x = g(x)$):
		\vfill
		\begin{itemize}
			\item \textbf{Fixed Point Method}:
			$$
				x^{(k+1)} = g(x^{(k)})
			$$
			\item \textbf{Anderson’s Mixing}:
			$$
				x^{(k+1)} = \sum_{i=0}^m \alpha_i\,g(x^{(k-i)})
			$$
			with $\vect\alpha = \argmin_{\sum_{i=0}^m \alpha_i = 1} || \sum_{i=0}^m \alpha_i\,f(x^{(k-i)}) ||_2$.
		\end{itemize}
		\vfill
		This unconstrained minimization problem may be written as a small least-squares problem. We solve it (e.g using Gaussian elimination or (better) Householder reflections) at each iteration
	\end{frame}

	\begin{frame}{Nonlinear Solvers (2/2)}
		In our case $\vect f(\vect x) \coloneqq \vect x - \vect S(\vect x)\,\vect x$. For the iteration function we may choose
		\begin{itemize}
			\item $\vect g(\vect x) \coloneqq \vect S^{-1}(\vect x)\,\vect b$ (Simplified Newton’s method / Picard’s method) or
			\item $\vect g_\omega(\vect x) \coloneqq \omega\,\vect S^{-1}(\vect x)\,\vect b + (1 - \omega)\,\vect x$ with $0 < \omega \le 1$ (relaxed version).
		\end{itemize}
		In practice, especially for strong nonlinearities, $\vect g$ rarely satisfies contraction property. Thus we need to introduce a relaxation parameter~$\omega$
	\end{frame}

	\section{Numerical Results}

	\begin{frame}{Fixed Point vs. Anderson’s Mixing}
		\begin{table}\centering\small
			\caption{Numb of iterations / execution time for different methods}
			\begin{tabular}[1.3]{ | c | c | c | >{\columncolor[gray]{0.9}}c | >{\columncolor[gray]{0.9}}c | >{\columncolor[gray]{0.9}}c | c | }
				\hline
				$|J_z|$ & $10^6$ & $10^7 $& $10^8$ & $10^9$ &$10^{10}$ & $10^{11}$ \\
				\hline
				\multirow{2}{*}{Picard}
					& 1 & 6 & --- & --- & 203 & 7 \\
					& 0.4\,s & 1.3\,s & --- & --- & 25.2\,s & 1\,s \\
				\hline
				\multirow{2}{*}{Relaxed Picard}
					& 1 & 6 & 84 & 70 & 164 & 7 \\
					& 0.5\,s & 2\,s & 71\,s & 52.4\,s & 54.3\,s & 2.6\,s \\
				\hline
				\multirow{2}{*}{Anderson’s Mixing}
					& 1+0 & 0+4 & 22+20 & 21+21 & 35+6 & 4+2 \\
					& 0.7\,s & 1\,s & 20\,s & 14.5\,s & 11.4\,s & 1.9\,s \\
				\hline
			\end{tabular}
		\end{table}
		We used linear Lagrange elements with a mesh from the first slide (362 DOFs). We stopped when $\vect f(\vect x^{(k)}) < 10^{-8}$. For Anderson’s Mixing $k = i + j$ means $i$ \textit{relaxed} Picard’s iterations ($\vect f(\vect x^{(i)}) < 10^{-4}$) and $j$ \textit{accelerated} Picard’s iterations. We used numb of mixings $m = 10$.
	\end{frame}

	\begin{frame}{Nonlinear Post–processing}
	   	\begin{figure}
	   		\centering
	   		\begin{subfigure}{.42\linewidth}
	   			\centering
	   			\includegraphicsw{a_nonlinear_6.png}
	   		\end{subfigure}%
	   		\hfill
	   		\begin{subfigure}{.42\linewidth}
	   			\centering
	   			\includegraphicsw{a_nonlinear_7.png}
	   		\end{subfigure}%
	   		\par\bigskip
	   		\begin{subfigure}{.42\linewidth}
	   			\centering
	   			\includegraphicsw{a_nonlinear_8.png}
	   		\end{subfigure}%
	   		\hfill
	   		\begin{subfigure}{.42\linewidth}
	   			\centering
	   			\includegraphicsw{a_nonlinear_9.png}
	   		\end{subfigure}%
	   		\par\bigskip
	   		\begin{subfigure}{.42\linewidth}
	   			\centering
	   			\includegraphicsw{a_nonlinear_10.png}
	   		\end{subfigure}%
	   		\hfill
	   		\begin{subfigure}{.42\linewidth}
	   			\centering
	   			\includegraphicsw{a_nonlinear_11.png}
	   		\end{subfigure}
	   		\caption{$|J_z| = 10^6, 10^7, \dots, 10^{11}\,A\,m^{-2}$}
	   	\end{figure}
	\end{frame}

\end{document}

sln/FEM for Magnetostatic Poisson Problem/LaTeX/Anderson Mixing/Anderson Mixing.tex


\documentclass[t]{beamer}
\usetheme{Copenhagen}
\setbeamertemplate{headline}{} % remove toc from headers
\beamertemplatenavigationsymbolsempty

\usepackage{amsmath, tikz, pgfplots, tcolorbox, array, bm, xcolor, colortbl}
\usetikzlibrary{calc}
\pgfplotsset{compat = 1.16}

\tikzstyle{input} = [circle, text centered, radius = 1cm, draw = black]
\tikzstyle{function} = [rectangle, text centered, minimum width = 2cm, minimum height = 1cm, draw = black]

\title{Inverse Functions}
\author{}
\date{}

\AtBeginSection[]
{
  \begin{frame}
    \frametitle{Objectives}
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\begin{frame}{}
    \maketitle
\end{frame}

\section{Find the inverse of a function}

\begin{frame}{Inverse of an Ordered Pair}
The \alert{inverse} of the ordered pair $(x,y)$ is $(y,x)$.
\end{frame}

\begin{frame}{Example 1}
Find the inverse of each.	\newline\\
(a) \quad $(2,-7)$	\quad \pause $(-7,2)$	\newline\\	\pause
(b) \quad $(0,3)$ \quad \pause $(3,0)$
\end{frame}

\begin{frame}{Review of Functions}
Recall that a function is nothing more than a machine that	\newline\\
\begin{enumerate}
	\item Accepts an input, $x$	\newline\\	\pause
	\item Performs some operation(s)	\newline\\	\pause
	\item Gives an output, $y$	\newline\\	\pause
\end{enumerate}
The inverse function is somewhat of an {\color{blue}``undo"} function.	\newline\\	\pause
It allows us to take the output of a function, put it into our inverse function, and get our original input value back.
\end{frame}

\begin{frame}{Visualization}
Suppose we put a value of 10 into the function \[f(x) = x^2\]	\pause
If we put the output (100) into the inverse, we get our 10 back.	\newline\\	\pause
\begin{center}
    \begin{tikzpicture}[node distance = 2cm]
\node (inputVal) [input, color=blue] {\color{blue}10};
\node (func) [function, below of = inputVal] {$f(x)=x^2$};
\node (outputVal) [input, below of = func] {100};
\node (invFunc) [function, right of = func, xshift = 2cm, color=red] {\color{red}$g(x)=\sqrt{x}$};

\draw [->, >=stealth, thick, line width = 0.75mm] (inputVal) -- (func);
\draw [->, >=stealth, thick, line width = 0.75mm] (func) -- (outputVal);
\draw [->, >=stealth, thick, dashed, line width = 1mm] (outputVal) -| (invFunc);
\draw [->, >=stealth, thick, dashed, line width = 1mm, color = red] (invFunc) |- (inputVal);
\end{tikzpicture}
\end{center}
\end{frame}

\begin{frame}{Inverse Notation}
We use the notation \[f^{-1}(x)\] to denote the inverse of $f(x)$.	\newline\\	\pause

\alert{Note:} The notation {\color{blue}\textbf{does not mean}} raise the function to the $-1$ power.
\end{frame}

\begin{frame}{Steps in Finding the Inverse of a Function}
\begin{enumerate}
	\item Rewrite $f(x)=$ as $y=$	\newline\\	\pause
	\item Switch your $x$ and $y$ variables.	\newline\\	\pause
	\item Solve this result for $y$ and rewrite using inverse notation.
\end{enumerate}
\end{frame}

\begin{frame}{Example 2}
Find the inverse of each of the following.	\newline\\
(a)	\quad	$f(x) = 5x$
\begin{align*}
\onslide<2->{y &= 5x} \\[6pt]
\onslide<3->{x &= 5\alert{y}} \\[6pt]
\onslide<4->{\frac{x}{5} &= \alert{y}} \\[8pt]
\onslide<5->{f^{-1}(x) &= \frac{x}{5}}
\end{align*}
\end{frame}

\begin{frame}{Example 2}
(b) \quad $f(x) = 3x+2$
\begin{align*}
\onslide<2->{y &= 3x+2} \\[6pt]
\onslide<3->{x &= 3\alert{y} + 2} \\[6pt]
\onslide<4->{x-2 &= 3\alert{y}} \\[6pt]
\onslide<5->{\frac{x-2}{3} &= \alert{y}} \\[8pt]
\onslide<6->{f^{-1}(x) &= \frac{x-2}{3}}
\end{align*}
\end{frame}

\begin{frame}{Example 2}
(c) \quad $f(x) = \dfrac{x+5}{7}$
\begin{align*}
\onslide<2->{y &= \frac{x+5}{7}} \\[8pt]
\onslide<3->{x &= \frac{\alert{y}+5}{7}} \\[8pt]
\onslide<4->{7x &= \alert{y} + 5} \\[6pt]
\onslide<5->{7x-5 &= \alert{y}} \\[6pt]
\onslide<6->{f^{-1}(x) &= 7x-5}
\end{align*}
\end{frame}

\begin{frame}{Example 2}
(d)	\quad	$g(x) = x^3 + 1$
\begin{align*}
\onslide<2->{y &= x^3 + 1} \\[8pt]
\onslide<3->{x &= \alert{y}^3 + 1} \\[8pt]
\onslide<4->{x-1 &= \alert{y}^3} \\[8pt]
\onslide<5->{\sqrt[3]{x-1} &= \alert{y}} \\[8pt]
\onslide<6->{g^{-1}(x) &= \sqrt[3]{x-1}}
\end{align*}
\end{frame}

\begin{frame}{Example 2}
(e) 	\quad	$h(x) = 4x^5 - 1$
\begin{align*}
\onslide<2->{y &= 4x^5 - 1} \\[8pt]
\onslide<3->{x &= 4\alert{y}^5 - 1} \\[8pt]
\onslide<4->{x+1 &= 4\alert{y}^5} \\[8pt]
\onslide<5->{\frac{x+1}{4} &= \alert{y}^5} \\[8pt]
\onslide<6->{\sqrt[5]{ \frac{x+1}{4} } &= \alert{y}} \\[8pt]
\onslide<7->{h^{-1}(x) &= \sqrt[5]{\frac{x+1}{4}}}
\end{align*}
\end{frame}

\begin{frame}{Example 2}
(f)		\quad	$f(x) = \sqrt{x+3}$
\begin{align*}
\onslide<2->{y &= \sqrt{x+3}} \\[6pt]
\onslide<3->{x &= \sqrt{\alert{y}+3}} \\[6pt]
\onslide<4->{x^2 &= \alert{y}+3} \\[6pt]
\onslide<5->{x^2-3 &= \alert{y}} \\[6pt]
\onslide<6->{f^{-1}(x) &= x^2 - 3}
\end{align*}
\end{frame}

\begin{frame}{Example 2}
(g)	\quad	$g(x) = \dfrac{5}{x}$
\begin{align*}
\onslide<2->{y &= \frac{5}{x}} \\[8pt]
\onslide<3->{x &= \frac{5}{\alert{y}}} \\[8pt]
\onslide<4->{x\alert{y} &= 5} \\[8pt]
\onslide<5->{\alert{y} &= \frac{5}{x}} \\[8pt]
\onslide<6->{g^{-1}(x) &= \frac{5}{x}}
\end{align*}
\end{frame}

\begin{frame}{Visual Interpretation of Inverse Functions}
Visually, when finding the inverse of a function, you are {\color{blue}\textbf{reflecting that function across the line $\bm{y = x}$}}.
\end{frame}

\begin{frame}{Visual Interpretation of Inverse Functions}
Below are the graphs of $f(x) = 3x^3-1$ and $f^{-1}(x) = \sqrt[3]{\frac{x+1}{3}}$ as well as the line $y=x$:	\newline\\
\begin{center}
\begin{tikzpicture}[scale=0.85]
\begin{axis}[
	axis lines = middle,
	 axis line style=<->,
	grid,
	xmin = -10, xmax = 10,
	ymin = -10, ymax = 10
	]
	\addplot[color=red, ultra thick, domain=-1.65:1.65] {3*x^3-1};
	\addplot[color=black, very thick, dashed, domain=-10:10] {x};
	\addplot[color=blue, ultra thick, domain=-1.54:1.54] ({3*x^3-1},{x});
\end{axis}
\end{tikzpicture}
\end{center}
\end{frame}

\section{State the Domain and Range of an Inverse Function}

\begin{frame}{Domain and Range of Inverse Functions}
When switching the $x$ and $y$ in finding the inverse function, you also switch the domain and range of the function and its inverse.		\newline\\	\pause

When you graph a function and its inverse, {\color{blue}\textbf{you'll want to make sure that they are reflections across the line $y=x$}}. This is VERY IMPORTANT for a function such as $y = x^2$.	\newline\\	\pause

This might mean we need to \alert{restrict the domain and/or range} of our original function.
\end{frame}

\begin{frame}[c]{Relationship Between Domain and Range}
\[\text{Domain of }f = \text{Range of }f^{-1}\]
\[\text{and}\]
\[\text{Range of }f = \text{Domain of }f^{-1}\]
\end{frame}

\begin{frame}{Example 3}
Find the domain and range of both the function and its inverse.	\newline\\
(a) \quad $f(x) = 5x$	\quad	\onslide<2->{$f^{-1}(x)=\frac{x}{5}$} \newline\\
\begin{center}
\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c|c}
	&	\textbf{Domain ($\bm{x}$)}	&	\textbf{Range ($\bm{y}$)} \\ \hline
$f(x)$ 			& \onslide<3->{\cellcolor{yellow!75} $\bm{\mathbb{R}}$}	& \onslide<6->{\cellcolor{green!60} $\bm{\mathbb{R}}$}	\\[6pt] \hline
$f^{-1}(x)$	& \onslide<5->{\cellcolor{green!60} $\bm{\mathbb{R}}$}		& \onslide<4->{\cellcolor{yellow!75} $\bm{\mathbb{R}}$} \\[6pt]
\end{tabular}
\end{center}
\end{frame}

\begin{frame}{Example 3}(b) \quad $f(x) = 3x+2$	\quad	\onslide<2->{$f^{-1}(x)=\frac{x-2}{3}$} \newline\\
\begin{center}
\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c|c}
	&	\textbf{Domain ($\bm{x}$)}	&	\textbf{Range ($\bm{y}$)} \\ \hline
$f(x)$ 			& \onslide<3->{\cellcolor{yellow!75} $\bm{\mathbb{R}}$}	& \onslide<6->{\cellcolor{green!60} $\bm{\mathbb{R}}$}	\\[6pt] \hline
$f^{-1}(x)$	& \onslide<5->{\cellcolor{green!60} $\bm{\mathbb{R}}$}		& \onslide<4->{\cellcolor{yellow!75} $\bm{\mathbb{R}}$} \\[6pt]
\end{tabular}
\end{center}
\end{frame}

\begin{frame}{Example 3}
(c)	\quad $g(x) = \sqrt{x+3}$ \quad \onslide<2->{$g^{-1}(x) = x^2 - 3$}	\newline\\
\begin{center}
\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c|c}
					&	\textbf{Domain ($\bm{x}$)}											&	\textbf{Range ($\bm{y}$)} \\ \hline
$g(x)$ 			& \onslide<3->{\cellcolor{yellow!75} $\bm{x \geq -3}$}	& \onslide<5->{\cellcolor{green!60} $\bm{y \geq 0}$}	\\[6pt] \hline
$g^{-1}(x)$	& \onslide<6->{\cellcolor{green!60} $\bm{x \geq 0}$}	& \onslide<4->{\cellcolor{yellow!75} $\bm{y \geq -3}$}  \\[6pt]
\end{tabular}
\end{center}
\end{frame}

\begin{frame}{Example 3}
(d) \quad $f(x) = -\sqrt{x-4}+2$
\begin{align*}
\onslide<2->{y &= -\sqrt{x-4}+2} \\[6pt]
\onslide<3->{x &= -\sqrt{\alert{y}-4}+2} \\[6pt]
\onslide<4->{x-2 &= -\sqrt{\alert{y}-4}} \\[6pt]
\onslide<5->{-x+2 &= \sqrt{\alert{y}-4}} \\[6pt]
\onslide<6->{(-x+2)^2 &= \alert{y} - 4} \\[6pt]
\onslide<7->{(-x+2)^2+4 &= \alert{y}} \\[6pt]
\onslide<8->{f^{-1}(x) &= (-x+2)^2+4}
\end{align*}
\end{frame}

\begin{frame}{Example 3d}
$f(x) = -\sqrt{x-4}+2$	\quad $f^{-1}(x) = (-x+2)^2+4$	\newline\\
\begin{center}
\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c|c}
					&	\textbf{Domain ($\bm{x}$)}											&	\textbf{Range ($\bm{y}$)} \\ \hline
$f(x)$ 			& \onslide<2->{\cellcolor{yellow!75} $\bm{x \geq 4}$}	& \onslide<4->{\cellcolor{green!60} $\bm{y \leq 2}$}	\\[6pt] \hline
$f^{-1}(x)$	& \onslide<5->{\cellcolor{green!60} $\bm{x \leq 2}$}	& \onslide<3->{\cellcolor{yellow!75} $\bm{y \geq 4}$}   \\[6pt]
\end{tabular}
\end{center}
\end{frame}

\begin{frame}{Example 3}
(e)	\quad	$f(x) = x^2$ with $x \geq 0$
\begin{align*}
\onslide<2->{y &= x^2} \\[8pt]
\onslide<3->{x &= \alert{y}^2} \\[8pt]
\onslide<4->{\pm \sqrt{x} &= \alert{y}} \\[8pt]
\onslide<5->{f^{-1}(x) &= \pm \sqrt{x}} \\[8pt]
\end{align*}
\onslide<6->{\emph{Note:} It can NOT be both $\sqrt{x}$ and $-\sqrt{x}$ because it would fail the {\color{blue}\textbf{vertical line test}}.}
\end{frame}

\begin{frame}{Example 3}
$f(x) = x^2$ with $x \geq 0$ \quad $f^{-1}(x) = \pm \sqrt{x}$		\newline\\
\begin{center}
\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c|c}
					&	\textbf{Domain ($\bm{x}$)}											&	\textbf{Range ($\bm{y}$)} \\ \hline
$f(x)$ 			& \onslide<2->{\cellcolor{yellow!75} $\bm{x \geq 0}$}	& \onslide<4->{\cellcolor{green!60} $\bm{y \geq 0}$}	\\[6pt] \hline
$f^{-1}(x)$	& \onslide<5->{\cellcolor{green!60} $\bm{x \geq 0}$}	& \onslide<3->{\cellcolor{yellow!75} $\bm{y \geq 0}$}   \\[6pt]
\end{tabular}
\end{center}

\onslide<6->{\[\bm{f^{-1}(x) = \sqrt{x}}\]}
\end{frame}

%%%%% INSERT 3F HERE
\begin{frame}{Example 3}
(f) \quad   $f(x) = \dfrac{1}{x}-7$
\begin{align*}
    \onslide<2->{y &= \frac{1}{x}-7} \\[10pt]
    \onslide<3->{x &= \frac{1}{\alert{y}}-7} \\[10pt]
    \onslide<4->{x+7 &= \frac{1}{\alert{y}}} \\[10pt]
    \onslide<5->{\alert{y}(x+7)  &= 1} \\[10pt]
    \onslide<6->{\alert{y} &= \frac{1}{x+7}}
\end{align*}
\end{frame}

\begin{frame}{Example 3}
$f(x) = \dfrac{1}{x}-7$ \quad $f^{-1}(x) = \dfrac{1}{x+7}$  \\[12pt]
\begin{center}
\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c|c}
	        & \textbf{Domain ($\bm{x}$)}	                                    &	\textbf{Range ($\bm{y}$)} \\ \hline
$f(x)$ 		& \onslide<2->{\cellcolor{yellow!75} $\bm{x \neq 0}$}	& \onslide<5->{\cellcolor{green!60} $\bm{y \neq -7}$}	\\[6pt] \hline
$f^{-1}(x)$	& \onslide<4->{\cellcolor{green!60} $\bm{x \neq -7}$}	            & \onslide<3->{\cellcolor{yellow!75} $\bm{y \neq 0}$}   \\[6pt]
\end{tabular}
\end{center}
\end{frame}

\begin{frame}{Example 3}
(g) \quad    $f(x)= \dfrac{3}{2x+5}$
\begin{align*}
    \onslide<2->{y &= \frac{3}{2x+5}}   \\[10pt]
    \onslide<3->{x &= \frac{3}{2\alert{y} + 5}} \\[10pt]
    \onslide<4->{x(2\alert{y}+5) &= 3} \\[10pt]
    \onslide<5->{2\alert{y}+5 &= \frac{3}{x}}   \\[10pt]
    \onslide<6->{2\alert{y} &= \frac{3}{x}-5}
\end{align*}
\end{frame}

\begin{frame}{Example 3g}
    \begin{align*}
        2\alert{y} &= \frac{3}{x}-5 \\[12pt]
        \onslide<2->{\alert{y} &= \frac{3}{2x}-\frac{5}{2}} \\[3pt]
    \end{align*}
    \onslide<3->{\[f^{-1}(x) = \frac{3}{2x}-\frac{5}{2}\]}
\end{frame}

\begin{frame}{Example 3g}
$f(x)= \dfrac{3}{2x+5}$   \quad $f^{-1}(x) = \dfrac{3}{2x}-\dfrac{5}{2}$  \\[12pt]
\begin{center}
\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c|c}
	        & \textbf{Domain ($\bm{x}$)}	                                    &	\textbf{Range ($\bm{y}$)} \\ \hline
$f(x)$ 		& \onslide<2->{\cellcolor{yellow!75} $\bm{x \neq -\frac{5}{2}}$}	& \onslide<5->{\cellcolor{green!60} $\bm{y \neq 0}$}	\\[6pt] \hline
$f^{-1}(x)$	& \onslide<4->{\cellcolor{green!60} $\bm{x \neq 0}$}	            & \onslide<3->{\cellcolor{yellow!75} $\bm{y \neq -\frac{5}{2}}$}   \\[6pt]
\end{tabular}
\end{center}
\end{frame}

\end{document}


%%% exercise 01 %%%
\documentclass{article}
\usepackage[margin=1in]{geometry}
\usepackage{amsmath, amssymb, amsfonts, enumitem, fancyhdr, tikz}
% get rid of paragraph indent
\setlength{\parindent}{0 pt}
% allow section.equation numbering
\numberwithin{equation}{section}
% allows you to copy-paste code segments. requires pygments to be installed
% natively, i.e. not just in a Python venv! for example, on WSL Ubuntu you
% need to sudo apt-get install python3-pygments (unfortunately, the version
% will be a bit dated) and cannot rely on your venv Pygments version.
\usepackage{minted}
% makes clickable links to sections
\usepackage{hyperref}
% make the link colors blue, as well as cite colors. urls are magenta
\hypersetup{
    colorlinks, linkcolor = blue, citecolor = blue, urlcolor = magenta
}
% fancy pagestyle so we can use fancyhdr for fancy headers/footers
\pagestyle{fancy}
% add logo in right of header. note that you will have to adjust logo path!
\fancyhead[R]{\includegraphics[scale = 0.15]{../bac_logo1.png}}
% don't show anything in the left and center header
\fancyhead[L, C]{}
% give enough space for logo by reducing top margin height, head separator,
% increasing headerheight. see Figure 1 in the fancyhdr documentation. if
% \topmargin + \headheight + \headsep = 0, original text margins unchanged.
\setlength{\topmargin}{-60 pt}
\setlength{\headheight}{50 pt}
\setlength{\headsep}{10 pt}
% remove decorative line in the fancy header
\renewcommand{\headrulewidth}{0 pt}

% title, author, date
\title{Exercise 1}
\author{Derek Huang\thanks{NYU Stern 2021, BAC Advanced Team.}}
\date{May 20, 2021\thanks{Original version released on February 18, 2021.}}

\begin{document}

\maketitle
% need to include this after making title to undo the automatic
% \thispagestyle{plain} command that is issued.
\thispagestyle{fancy}

\section{Set theory}

\subsection{Partition of $ \mathbb{R} $}

A \textit{partition} $ P $ of a non-empty set $ A $ is a set of non-empty,
disjoint sets $ A_{i_1}, A_{i_2}, \ldots \subset A $ such that
\begin{equation*}
    \bigcup_{k \in \mathcal{I}} A_k = A
\end{equation*}

Here $ \mathcal{I} \triangleq \{i_1, i_2, \ldots \} $ is an \textit{index set}, usually chosen to be a subset of $ \mathbb{N} $. Give a partition of the
half-open unit interval $ [0, 1) $ into an infinite number of half-open
intervals.

\medskip

\textit{Hint.} Consider infinite sequences or series that converge to 1.

\subsection{Slicing the unit circle}

The unit circle on $ \mathbb{R}^2 $ is the set
$ U \triangleq \{\mathbf{x} \in \mathbb{R}^2 :
\Vert\mathbf{x}\Vert_2 \le 1\} $. Define
$ \tilde{U} \triangleq U \setminus A $, $ A \subset U $, as the ``sliced''
unit circle, i.e. the subset of $ U $ where no points that lie on the $ x_1 $
or $ x_2 $ axes are included. Define $ A $.

\subsection{Countably infinite intersection}

Suppose we have a sequence of sets $ \{A_k\}_{k \in \mathbb{N}} $ where
$ A_k \triangleq \left(-e^{-k}, 1 + e^{-k}\right) $. What is
$ \bigcap_{k \in \mathbb{N}} A_k $?

\medskip

\textit{Remark.} $ \bigcap_{k = 1}^\infty A_k $ is \textit{closed} interval
on $ \mathbb{R} $. Try to convince yourself why this is true.

\subsection{$ \ell^p $ norm balls}

Define $ \bar{B}_{n, \ell^p}(\mathbf{x}, r) \triangleq \{\mathbf{x}' \in
\mathbb{R}^n : \Vert\mathbf{x}' - \mathbf{x}\Vert_p \le r\} $ as the closed
$ \ell^p $ norm ball in $ \mathbb{R}^n $ centered at $ \mathbf{x}
\in \mathbb{R}^n $ with radius $ r \in (0, \infty) $, for
$ n \in \mathbb{N} + 1 $. Recall that for
$ \mathbf{x} \triangleq [ \ x_1 \ \ldots \ x_n ]^\top \in \mathbb{R}^n $, its
$ \ell^p $ norm $ \Vert\mathbf{x}\Vert_p $ is such that
\begin{equation*}
    \Vert\mathbf{x}\Vert_p \triangleq
    \left(\sum_{k = 1}^n|x_k|^p\right)^{1 / p}
\end{equation*}

Consider the $ \ell^1 $ norm and the $ \ell^\infty $ norm, where
$ \Vert\mathbf{x}\Vert_\infty = \max\{x_1, \ldots x_n\} $, and define
$ \bar{B}_{2, \ell^1}(\mathbf{0}, 1) $,
$ \bar{B}_{2, \ell^\infty}(\mathbf{0}, 1) $.
Which of these two sets includes the other and
$ \bar{B}_{2, \ell^2}(\mathbf{0}, 1) $?

\medskip

\textit{Hint.} Drawing a picture might help since these norm balls are
defined on $ \mathbb{R}^2 $.

\subsection{Complex numbers}

The complex numbers $ \mathbb{C} $ are numbers of the form $ a + bi $, where
$ a, b \in \mathbb{R} $ and $ i^2 = -1 $. Prove that
$ \mathbb{R} \subset \mathbb{C} $.

\section{Probability}

\subsection{Event unions}

Let $ \Omega $ be an arbitrary sample space and let $ A, B \subseteq \Omega $
be two events. Prove using only Kolmogorov's axioms the well-known fact that
$ \mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B) -
\mathbb{P}(A \cap B) $.

\medskip

\textit{Hint.} Try writing $ A, B $ as unions of disjoint events.

\subsection{Sequences of coin flips}

Consider sequences of $ n \in \mathbb{N} $ independent flips of a fair coin
and note that the sample space $ \Omega \triangleq \{\text{H}, \text{T}\}^n $.
\begin{enumerate}[label = \alph*.]
    \item
    $ \forall \omega \in \Omega $, what is $ \mathbb{P}\{\omega\} $?

    \item
    $ \Omega $ is finite. What is $ |\Omega| $?

    \item
    Define $ H_n : \Omega \rightarrow \mathbb{N} \cup \{0\} $ as the random
    variable where $ \forall \omega \in \Omega $, $ H_n(\omega) $ is the
    number of heads seen during the sequence of coin flips $ \omega $. What
    is $ \mathbb{E}[H_n] $ and $ \operatorname{Var}(H_n) $?

    \medskip

    \textit{Hint.} Note that $ H_n = \sum_{k = 1}^nX_k $, where i.i.d.
    $ X_k \sim \operatorname{Bernoulli}(1 / 2) $.
\end{enumerate}

\subsection{More sequences of coin flips}

Consider sequences of $ n \in \mathbb{N} $ independent flips of an
\textit{unfair} coin, where the probability of getting heads is
$ p \in (0, 1) $. Note the sample space $ \Omega $ is the same as the
$ \Omega $ in the previous question.
\begin{enumerate}[label = \alph*.]
    \item
    Prove that when $ p \ne 1 / 2 $, $ \exists \omega_1, \omega_2 \in
    \Omega $, $ \omega_1 \ne \omega_2 $, that $ \mathbb{P}\{\omega_1\} \ne
    \mathbb{P}\{\omega_2\} $.

    \item
    Define $ U_n \triangleq \sum_{k = 1}^nX_k $, where i.i.d.
    $ X_k $ pays off $ M_1 \in (0, \infty) $ units of cash if the $ k $th
    flip is heads and pays off $ -M_2 \in (-\infty, 0) $ of cash if the
    $ k $th flip is tails. Write $ \mathbb{E}[U_n] $ as a function of $ p $.

    \item
    Suppose for a given value of $ p \in (0, 1) $ we have
    $ \mathbb{E}[U_n] = 0 $. Write $ M_1 $ as a function of $ M_2, p $.
\end{enumerate}

\subsection{The Black-Scholes underlying}

Under the Black-Scholes model, the terminal price $ S_T : \Omega \rightarrow
\mathbb{R} $ of the underlying conditional on its current price $ S_t \in
(0, \infty) $, denoted as $ S_T \mid S_t $ can be written such that for
$ Z \sim \mathcal{N}(0, 1) $, $ T > t $,
\begin{equation*}
    S_T \mid S_t \triangleq S_te^{
        \left(r - \frac{1}{2}\sigma^2\right)(T - t) + \sigma\sqrt{T - t}Z
    }
\end{equation*}
$ r, \sigma \in (0, \infty) $ are the drift and diffusion terms of the
model under $ \mathbb{Q} $, the \textit{risk-neutral probability
measure}\footnotemark\footnotetext{
    If $ \mathbb{P} $ is the real-world probability measure, one can think of
    $ \mathbb{Q} $ as the probability measure corresponding to a world where
    all prices are risk-neutral. In more technical terms, it is a probability
    measure such that the discounted price processes of all risky assets are
    local martingales. For this problem, just replace any uses of
    $ \mathbb{P} $ with $ \mathbb{Q} $.
}.
\begin{enumerate}[label = \alph*.]
    \item
    Compute $ \mathbb{E}[S_T \mid S_t] $, which gives the risk-neutral
    expectation of $ S_T $ conditional on $ S_t $.

    \medskip

    \textit{Hint.} You don't need to derive $ f_{S_T \mid S_t} $, the density
    of $ S_T \mid S_t $, to compute $ \mathbb{E}[S_T \mid  S_t] $.

    \item
    Let $ P_1, P_2 \in (0, \infty) $, $ P_1 < P_2 $, be two price thresholds.
    What is $ \mathbb{Q}\{S_T \in (P_1, P_2) \mid S_t\} $? Write your answer
    as the difference of two terms involving $ \Phi $, the normal cdf.

    \medskip

    \textit{Hint.} Note that $ \mathbb{Q}\{S_T \in (P_1, P_2) \mid S_t\} =
    \mathbb{Q}\{S_T < P_2 \mid S_t\} - \mathbb{Q}\{S_T \le P_1 \mid S_t\} $.
    Use the fact that inequalities are preserved under monotone transforms,
    translations, and positive scaling. Your answer should be of the form
    $ \Phi(\zeta_2) - \Phi(\zeta_1) $, where $ \zeta_k $, $ k \in \{1, 2\} $,
    is a quantity involving $ S_t $, $ P_k $, and constants.
\end{enumerate}

\end{document}


\section{Experimental Study}
\label{sec:experimental}

\subsection{Data Set}
In introduce what data sets you have used in your experimental study. Real dataset first, then synthetic dataset.

\subsection{Approaches and Measurements}
What approaches tested in the experimental study? Introduce each one with one sentence. Your proposed approaches, the compared existing approaches, random approaches, and so on.

Why need this paragraph? Readers may have forgotten your solutions, thus just make them recall your solutions through some simple descriptions.

Then introduce the measures you will compare on, such as running time, memory usage, and so on. Usually, control variate method will be used in the experimental study. Define a default setting, then change one parameter in a set of experiments. Make a table to show the configuration, like the example in Table \ref{tab:settings} \cite{cheng2017utility}.

\begin{table}[t]
	\begin{center}
		{\small\scriptsize % \vspace{-1ex}
			\caption{\small Experimental Settings.} \label{tab:settings}
			\begin{tabular}{l|l}
				{\bf \qquad \qquad \quad Parameters} & {\bf \qquad \qquad \qquad Values} \\ \hline \hline
				the number, $m$, of riders  & 1K,  \textbf{3K}, 5K, 8K, 10K\\
				the number, $n$, of vehicles  & 100, \textbf{200}, 300, 400, 500 \\
				the pickup deadline range $[rt^-_{min}, rt^-_{max}]$  & \textbf{[1, 10]}, [10, 30], [30, 60]\\
				the capacity of vehicles $a_j$ & 2, \textbf{3}, 4, 5\\
				the balancing parameters $(\alpha, \beta)$& (0, 0), (1, 0), (0, 1), \textbf{(0.33, 0.33)}\\
				the flexible factor $\varepsilon$ & 1.2, \textbf{1.5}, 1.7, 2\\
				the length $\delta_j$ of time frame $f_j$ & 30 mins\\
				\hline
			\end{tabular}
		}\vspace{-2ex}
	\end{center}
\end{table}

Finally, introduce the running environment of your experimental study. Running on what kind of PCs or servers? Using what programming languages?

\subsection{Experimental Results}

What do you discover or demonstrated in the experimental study? Show the effects of each parameter one by one. Usually, describe results on real dataset first.

Better to have a summary of the interesting points found in the experimental study at the end of this subsection.

\subsection{Adding Figures}
In this subsection, some example of adding figures are introduces. Usually, put figures on the top of pages. Put figures close to their description.

Below is an example of adding two figures together. Modify the related vspace configuration in head.tex to adjust the global setting. To adjust locally, just add your local spacing command behind them. Parameter ``h'' means ``here''; ``t'' means ``top''; ``!'' means ``mandatory''. In vspace command, I like to use ``ex'', which just a unit and you can use others like ``px''. For other parameters, you can adjust them to see the effects.

\begin{figure}[ht!]\centering \figureTopMargin\vspace{1ex}
	\subfigure[][{\scriptsize Sub Caption1}]{
		\scalebox{0.19}[0.19]{\includegraphics{../figures/example1.eps}}
		\label{subfig:sub_example1}}
	\subfigure[][{\scriptsize Sub Caption2}]{
		\scalebox{0.19}[0.19]{\includegraphics{../figures/example1.eps}}
		\label{subfig:sub_example2}}\figureCaptionMargin
	\caption{\small An Example of Figure.}\figureBelowMargin
	\label{fig:example1}
\end{figure}

Add a bar on top of a group of subfigures, like in Figure \ref{fig:example2}.
\begin{figure}[h!]\centering
	\subfigure{
		\scalebox{0.4}[0.4]{\includegraphics{../figures/bar.eps}}}\hfill\\\vspace{-2ex}
	\addtocounter{subfigure}{-1}
	\subfigure[][{\scriptsize Sub Caption1}]{
		\scalebox{0.19}[0.19]{\includegraphics{../figures/example.eps}}
		\label{subfig:sub_example3}}
	\subfigure[][{\scriptsize Sub Caption1}]{
		\scalebox{0.19}[0.19]{\includegraphics{../figures/example.eps}}
		\label{subfig:sub_example4}}\figureCaptionMargin
	\caption{\small An Example of Figure.}\figureBelowMargin
	\label{fig:example2}
\end{figure}

To draw a figure crossing two columns, like in Figure \ref{fig:example3}.
\begin{figure*}[th!]\centering
	\subfigure{
		\scalebox{0.4}[0.4]{\includegraphics{../figures/bar.eps}}}\hfill\\\vspace{-2ex}
	\addtocounter{subfigure}{-1}
	\subfigure[][{\scriptsize Sub Caption1}]{
		\scalebox{0.18}[0.18]{\includegraphics{../figures/example.eps}}
		\label{subfig:sub_example5}}
	\subfigure[][{\scriptsize Sub Caption2}]{
		\scalebox{0.18}[0.18]{\includegraphics{../figures/example.eps}}
		\label{subfig:sub_example6}}
	\subfigure[][{\scriptsize Sub Caption3}]{
		\scalebox{0.18}[0.18]{\includegraphics{../figures/example.eps}}
		\label{subfig:sub_example7}}
	\subfigure[][{\scriptsize Sub Caption4}]{
		\scalebox{0.18}[0.18]{\includegraphics{../figures/example.eps}}
		\label{subfig:sub_example8}}\figureCaptionMargin
	\caption{\small An Example of Figure.}\figureBelowMargin\vspace{-2ex}
	\label{fig:example3}
\end{figure*}

Some suggestions:
\begin{itemize}[leftmargin=*]
	\item Label sub-figures and figures separately. When you describe the particular figure, your can accurately refer to the one you want refer to.
	\item Use eps files! If you need to convert jpg or png to eps, you can try this website: \url{https://www.online-convert.com}, which is the most stable one I can find.
	\item Put figures on the top of pages for better layout.
	\item Put figures close to their description to ease your readers.
\end{itemize}

Some suggestions/lessons noticed from helping junior PhD students to revise their drafts:
\begin{itemize}[leftmargin=*]
	\item Try to avoid using words like ``naive'', ``obviously'', ``simple'' and so on.
	\item Don't forget leaving a blank space before (. Bad example: abs(e.g., ddd)
	\item Use aaa~\cite{cheng2016task} in stead of aaa \cite{cheng2016task}, as the first one can avoid strange
    \cite{cheng2016task} new line.
	\item Any Tables, Figures, Algorithms or other similar components in your draft should be discussed with some natural language. Don't simply put them in your draft as no one know their purposes unless you describe them clearly.
	\item Put blocks of tables, figures, algorithms on the top of pages for better layout.
	\item Numbered equations should be referred somewhere in your draft. If not, just don't number them. You can add ``$\backslash$notag'' behind an equation to suppress its numbering.
	\item some special commands to use: $log$ $\longrightarrow$ $\log$.
	\item Avoid starting a sentences with words: ``And'', ``So''... You can use ``In addition'', ``Moreover'', ``Thus'', ``Therefore'' ...
	\item Use tools to double check grammar errors, such as Textidote: \url{https://github.com/sylvainhalle/textidote}
	\item Before submitting draft, search: ``??'' in the pdf file to avoid wrong references or citations.
\end{itemize}

Experience from Wangze NI after revising drafts with me:
\begin{itemize}
\item Do not introduce any technical details in the introduction section.
\item Explain the semantic meaning (or usage) of theorems after you prove them.
\item Take a running example for each algorithm.
\item Better to use the same example to explain all concepts/algorithms in the paper.
\item Do not write sentences too long. Try to cut long sentences into several short sentences when you are not masters in English writing.
\item Before writing the paper, it would be helpful to make a presentation to friends. During the preparation of the presentation, organize the content more logically. After presentation, get some feedback from friends.
\item Use the same word for one thing/concept in the whole paper. For the important actions, please keep the verbs on describing them consistently.
\item Conclude the experimental findings at the end of the experimental study section.
\end{itemize}


\documentclass{article}
\usepackage{listings}
\usepackage{upquote}
\usepackage{verbatim}

\lstnewenvironment{likely}{
  \lstset{
    basicstyle=\ttfamily,
    showstringspaces=false
  }}{}

\title{Face Recognition in Likely}
\author{Joshua C. Klontz}
\date{\today}
\begin{document}
\maketitle

\begin{abstract}
This document represents a long-term effort to port the OpenBR face recognition algorithm to \emph{Likely}\footnote{www.liblikely.org}.
As Likely is a literate programming language, this document is both the source code \emph{and} the documentation.
\end{abstract}

\section{Compilation}
\label{sec:compilation}
This document is a valid \LaTeX\ file that can be rendered for reading by humans.

\begin{verbatim}
$ pdflatex face_recognition.tex
\end{verbatim}

It is also a valid Likely file that can be trained/compiled for reading by machines.

\begin{verbatim}
$ likely -p 'data := "path/to/training/data.lm".read-matrix' \
face_recognition.tex face_recognition.o
\end{verbatim}

Note that the outputted \texttt{face\_recognition.o} native object file exposes its functionality through a C-compatible signature.

\begin{verbatim}
#include <likely.h>
likely_mat face_recognition(likely_const_mat src);
\end{verbatim}

\section{Introduction}
We start our journey constructing the face recognition algorithm with an identity function.

\begin{likely}
face-recognition :=
  src :->
    src
\end{likely}

Throughout the remainder of this document we will append additional steps to this function, building up the entire algorithm one transformation at a time.
It is the responsibility of each section in this document to update the \texttt{face-recognition} function and the training \texttt{data}.
It is in this way that the sections maintain independence, and the algorithm can be changed by inserting, removing, or editing sections of this document.
Remember that the \emph{Likely Language Reference}\footnote{https://s3.amazonaws.com/liblikely/latex/standard.pdf} is a good resource to consult if you are uncertain about the meaning of some code.

Note that the remainder of this document assumes that the global variable
\texttt{data} is defined, and contains the appropriate training samples.

\begin{likely}
"Number of training samples:"
data.frames
\end{likely}

\section{...}
This section serves as a placeholder for the unwritten algorithm steps.
There are a lot of steps that haven't been written yet, including face detection, registration and representation!
In fact, only the final few sections of the algorithm have been written.

\section{L2 Normalization}
We wish to normalize every feature vector to a unit L2 norm, ensuring that they live on a high-dimensional unit sphere.
Such a normalization has been observed to improve accuracy when comparing feature vectors using a Euclidean distance.

\begin{likely}
normalize-l2 :=
  src :->
  {
    dst := src.imitate
    len := src.columns
    ((src.channels 1 src.rows src.frames)
     dst src len) :=>
    {
      norm := 0.(cast src).$
      (iter (-> x (<- norm (+ norm src.sq)))
            len)
      norm := (/ 1 (sqrt norm))
      (iter (-> x (<- dst (* src norm)))
            len)
    }
  }
\end{likely}

Now we can re-define \texttt{face-recognition} to append L2 normalization.

\begin{likely}
face-recognition :=
  src :->
    src.face-recognition.normalize-l2
\end{likely}

And project the training data.

\begin{likely}
data := [ data.normalize-l2 ]
\end{likely}

\section{Quantization}
The goal of quantization is to rescale and cast feature vector dimensions into 8-bit unsigned integers.
The resulting feature vectors are smaller in size and faster to compare, with generally negligible loss in representation accuracy.
Training data is required to determine the scaling parameters.

\begin{likely}
train-quantize :=
  (training-samples) :->
  {
    lo    := [ training-samples.min-element  ]
    hi    := [ training-samples.max-element  ]
    scale := [ (/ 255 (- hi lo)) ]

    src :->
    {
      dst := (imitate-size src (imitate-dimensions u8 src.type))
      (dst src) :=>
        dst :<- src.(- lo).(* scale)
    }
  }

quantize := (train-quantize data)
\end{likely}

Now we can re-define \texttt{face-recognition} to append quantization.

\begin{likely}
face-recognition :=
  src :->
    src.face-recognition.quantize
\end{likely}

And project the training data.

\begin{likely}
data := [ data.quantize ]
\end{likely}

And report some statistics.

\begin{likely}
"Quantized feature vector dimensionality:"
data.columns
\end{likely}

\section{Entry Point}
Recall from section \ref{sec:compilation} that the entry point to the outside world is a single C function \texttt{face\_recognition} that takes as input a \texttt{likely\_mat} (the input image) and outputs a new \texttt{likely\_mat} (the feature vector). Now that we have finished defining \texttt{face-recognition} we can finally export this symbol.

\begin{likely}
(extern u8X "face_recognition" f32X face-recognition)
\end{likely}

\end{document}


\input{../../_common/preamble}

\title{Decision trees and random forests}

\begin{document}

\maketitle

\begin{frame}{Contents}
    \tableofcontents[hideallsubsections]
\end{frame}

\section{Decision trees}

\begin{frame}{Should we wait?}
    \only<1>{%
        \begin{block}{Problem}
            You're out with friends and need to decide whether to wait for a
            table at a busy restaurant.
            You have the following information:
            \begin{itemize}
                \small
                \item Whether there's an alternative restaurant nearby
                \item Whether the restaurant has a bar
                \item How busy the restaurant is (empty, some people, packed)
                \item Whether you're hungry (not at all, peckish, starving)
                \item Whether it's raining
                \item Type of restaurant (British, Chinese, Italian or Thai)
                \item Whether it's Friday or Saturday night
            \end{itemize}
        \end{block}}
    \only<2>{%
        \begin{block}{Idea}
            Imagine taking a sequence of decisions:
            \begin{itemize}
                \item If the restaurant is packed\ldots
                      \begin{itemize}
                          \item \ldots and we're starving\ldots
                          \begin{itemize}
                              \item \ldots but there's no alternative in the
                                     area $\rightarrow$ wait
                          \end{itemize}
                      \end{itemize}
            \end{itemize}
        \end{block}
        \vfill
        \begin{block}{Question}
            How `tall' should the decision tree grow?
        \end{block}}
\end{frame}

\begin{frame}{Expressiveness}
    \begin{itemize}
        \item Decision trees can express \alert{any} function of the predictors \\
              (using one leaf per sample) \\[\medskipamount]
        \item We want to find \alert{structure} in the data, not overfit
        \item[$\rightarrow$] Compact trees
    \end{itemize}
    \vfill\pause
    \begin{block}{Idea}
        \begin{itemize}
            \item Choose `most significant' attribute as (sub)root
            \item[$\rightarrow$] Ideally achieving perfect separation of categories \\[\medskipamount]
            \item Repeat (recursively)
        \end{itemize}
    \end{block}
\end{frame}

\begin{frame}{Comparison with logistic regression}
    \begin{itemize}
        \item Logistic regression is a linear model
        \item Each predictor `acts' independently of all others \\
              (its marginal effect is the regression coefficient)
    \end{itemize}
    \vfill\pause
    This doesn't always work:
    \vspace{-0.5em}
    \begin{itemize}
        \item If the restaurant is packed\ldots
              \begin{itemize}
                  \item \ldots and we're starving\ldots
                  \begin{itemize}
                      \item \ldots but \alert{there is} an alternative
                            $\rightarrow$ do we still wait?
                  \end{itemize}
              \end{itemize}
    \end{itemize}
    Decision trees automatically contain \alert{interactions}, since each
    `question' depends on the previous one
\end{frame}

\begin{frame}{Training}
    \begin{itemize}
        \item Start with the entire dataset \\[\bigskipamount]
        \item Find the `question' that best segregates the samples
              $\rightarrow$ \alert{purity} \\[\bigskipamount]
        \item Repeat (recursively) until:
              \begin{itemize}
                  \item You have asked as many questions as you wanted
                  \item The gain in purity of possible splits is negligible
                  \item Leaves are completely pure
              \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Prediction}
    \begin{itemize}
        \item Answer each `question' until you reach a leaf \\[\bigskipamount]
        \item Take the majority label of samples in that leaf
    \end{itemize}
\end{frame}

\begin{frame}{Purity metrics}
    \begin{block}{Gini impurity}
        \begin{itemize}
            \item How often would a randomly chosen sample be labelled
                  incorrectly if it was labelled randomly with class proportions
                  $p_{i}$?
            \item[$\rightarrow$] $\sum_{k} p_{k} \left( 1 - p_{k} \right)$
        \end{itemize}
    \end{block}
    \vfill
    \begin{block}{Information gain}
        \begin{itemize}
            \item What's the reduction in (Shannon) entropy?
            \item[$\rightarrow$] Difference in $- p \log\!\left( p \right)$ from
                                 parent to children
        \end{itemize}
    \end{block}
\end{frame}

\begin{frame}{Overfitting}
    \begin{itemize}
        \item Decision trees can easily `memorise' the data
        \item[$\rightarrow$] Overfitting
    \end{itemize}
    \vfill\pause
    \begin{block}{Solutions}
        Impose a limit on the\ldots\vspace{-1ex}
        \begin{itemize}
            \item Maximum number of questions (depth)
            \item Minimum number of samples in each leaf
        \end{itemize}
    \end{block}
\end{frame}

\begin{frame}{Pros and cons}
    \begin{block}{Pros}
        \begin{itemize}
            \item Can be used for regression or classification
            \item Can be visualised $\rightarrow$ easy to interpret
            \item Correspond to a series of `rules'
            \item Learn interactions and irrelevant predictors
        \end{itemize}
    \end{block}
    \vfill
    \begin{block}{Cons}
        \begin{itemize}
            \item Prone to overfitting and sensitive to small variations
            \item May not be globally optimal because of `greedy' splitting
            \item Don't work well with unbalanced classes or small datasets
        \end{itemize}
    \end{block}
\end{frame}

\section{Random forests}

\begin{frame}{Bagging}
    Imagine a situation where\ldots
    \begin{itemize}
        \item You have many different models
        \item Each predicts your outcome with some accuracy
        \item Each also makes (independent) errors
    \end{itemize}
    How could you improve your prediction?
    \vfill\pause
    \begin{block}{Idea}
        Let all classifiers predict and take the majority vote \\
        (or the mean for continuous outcomes)
    \end{block}
\end{frame}

\begin{frame}{Random forests}
    \begin{itemize}
        \item A collection (\alert{ensemble}) of decision trees \\[\bigskipamount]
        \item Built randomly\ldots
              \begin{itemize}
                  \item On a subset of the data
                  \item Using a subset of predictors
              \end{itemize}
              \ldots to avoid overfitting \\[\bigskipamount]
        \item For prediction, each tree contributes an answer, and the final
              model prediction is the majority vote (or the mean for continuous
              outcomes)
    \end{itemize}
\end{frame}

\begin{frame}{Boosting}
    Imagine a situation where\ldots
    \begin{itemize}
        \item You are training many models of the same type
              \alert{sequentially}
        \item Each predicts your outcome\ldots
              \begin{itemize}
                  \item Correctly for some samples
                  \item Incorrectly for some other samples
              \end{itemize}
    \end{itemize}
    How could you improve your prediction?
    \vfill\pause
    \begin{block}{Idea}
        \begin{itemize}
            \item At each step, `refine' the model by giving more weight to
                  incorrectly predicted samples (the `hard' ones)
            \item For prediction, compute a weighted vote/mean of \alert{all}
                  predictions
        \end{itemize}
    \end{block}
\end{frame}

\end{document}


\documentclass[11pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{graphicx}
%\usepackage[ddmmyyyy]{datetime}
\usepackage[short,nodayofweek,level,12hr]{datetime}
%\usepackage{cite}
%\usepackage{wrapfig}
%\usepackage[left=2cm,right=2cm,top=2cm,bottom=2cm]{geometry}

\newcommand{\e}{\epsilon}
\newcommand{\dl}{\delta}
\newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}}
\newcommand{\vect}[1]{\underline{#1}}
\newcommand{\uvect}[1]{\hat{#1}}
\newcommand{\1}{\vect{1}}
\newcommand{\grad}{\nabla}

\title{Lecture 11: Self Gravitating Equilibria}
\date{\displaydate{date}}
\newdate{date}{10}{09}{2018}
\author{}

\begin{document}
\maketitle
\section*{Overview}

\section{Self-gravitating equilibrium for a spherical planet}
The gravitational potential $\Psi_G$ due to a small mass $dm$ is
\begin{align*}
d\Psi_G &= -\frac{G dm}{|{\vect x - \vect x_0}|}
\end{align*}
And hence for a body of volume $V$, the potential is
\begin{align*}
\Psi_G &= -\int_V \frac{G \rho_0 dV}{|{\vect x - \vect x_0}|}
\end{align*}
For the potential in the exterior of a spherical body, this integral is trivial
\begin{align*}
\Psi_G &= -\int_V \frac{G \rho_0 dV}{|{\vect x - \vect x_0}|}\\
&=-\frac{G \rho_0}{r} \int_V dV \\
&=-\frac{G M}{r}
\end{align*}
This gives us the familiar result that potential (and field) at an external point can be calculated by assuming the entire mass $M$ to be concentrated at the center of the body.

In order to calculate the potential in the interior region of a spherical body, we must perform the integral $\int_V \frac{dV}{r} = \int_V \frac{dV}{|\vect x - \vect x_0|}$ in spherical coordinates. The result we get is
\begin{align*}
\Psi_G &= -2 \pi \rho_0 G (R^2 - \frac{r^2}{3})
\end{align*}
At $r=R$ both interior and exterior regions give the same potential, so the result seems consistent.

Since potential in the interior region is quadratic in $r$, and force is gradient of the potential, the force will be linear in $r$. Since $r = |\vect x - \vect x_0|$, it implies that the force is only a function of $\vect x$ (since the center $\vect x_0$ is fixed). The force is given by
\begin{align*}
F_G = -\frac{4\pi}{3}\rho_0 G\vect x
\end{align*}

This is the integral approach for obtaining the force due to a gravitating sphere. There is also a differential approach that utilises the idea of Green's function of a Laplacian. Recall that the Green's Function of a Laplacian in 3D is given by $G(\vect x) = -\frac{1}{4\pi|\vect x -\vect x_0|}$. Using the equation of the potenial, taking the Laplacian and using the Green's Function, we can obtain the same result as follows:
\begin{align*}
&\Psi_G = -\int_V \frac{G \rho_0 dV}{|{\vect x - \vect x_0}|}\\
&\grad^2 \Psi_G = 4\pi G \rho_0 \int_V \grad^2(\frac{-1}{4\pi|{\vect x - \vect x_0}|}) dV\\
&\grad^2 \Psi_G = 4\pi G \rho_0 \int_V \delta(\vect x - \vect x_0) dV \tag{\footnotesize{by defintion of Green's Function}}\\
&\grad^2 \Psi_G = 4\pi G \rho_0  \tag{\footnotesize{by defintion of Delta Function}}\\
&\frac{1}{r^2}\frac{d}{dr} r^2 \frac{d\Psi_G}{dr} = 4\pi G \rho_0 \\
&r^2 \frac{d\Psi_G}{dr} =\frac{4\pi G \rho_0 r^3}{3} \\
&\frac{d\Psi_G}{dr} =\frac{4\pi G \rho_0 r}{3} \\
&-\vect F =\frac{4\pi}{3}G \rho_0 r \1_r \\
&-\vect F =\frac{4\pi}{3}G \rho_0 (\vect x - \vect x_0) \\
\end{align*}
This is a radial(directed towards the center) harmonic force. The harmonic nature can be easily shown
\begin{align*}
\grad^2 \Psi_G &= 4\pi G \rho_0 \\
\grad (\grad^2 \Psi_G) &= 0  \tag{\footnotesize{taking $\grad$ on both sides}} \\
\grad^2 \grad \Psi_G &= 0  \tag{\footnotesize{$\grad$ and $\grad^2$ commute}}\\
\grad^2 \vect F &= 0  \tag{\footnotesize{$F = \grad \Psi_G$ }}
\end{align*}
To summarize,
\begin{itemize}
\item Interior: $\vect F \propto r \quad V \propto r^2$
\item Exterior: $\vect F \propto 1/r^2 \quad V \propto 1/r$
\end{itemize}

The fact that force has a linear dependence on $\vect x$ can also be seen using the fact that $\vect F$ is harmonic. Since $\grad^2 \vect F = 0$, it implies that $\vect F$ can be represented using the solution of the Laplacian. We know that if $\grad^2 \Phi = 0$, then
\begin{align*}
\Phi &= \underbrace{F(r)}_{\propto r^n, \frac{1}{r^{n+1}}}\quad \underbrace{G(\cos(\theta))}_{P^m_n(\cos(\theta))}\quad \underbrace{H(\phi)}_{e^{\pm i m \phi}}
\end{align*}
where $n$ is the order of tensor involved. The solution constructed using $r^n$ are growing harmonics and the ones constructed using $1/r^{n+1}$ are the decaying harmonics. Since the $(\theta, \phi)$ dependence is same, we can always convert a decaying harmonic to a growing harmonic by multiplying it with $r^{2n+1}$.

Now we know that the vector solution to the Laplace equation is proportional to $\vect x/r^3$. This is a decaying harmonic. So we multiply it by $r^{2(1)+1} = r^3$ to get the growing harmonic $\vect x$. Therefore the force $\vect F$ will be in general proportional to a linear combination of the two solutions. But we want the field inside the gravitating body, so the the force at the center must not be singular. Therefore $\vect x/r^3$ is rejected and we get $\vect F \propto \vect x $. So $\vect F$ must be linear in $\vect x$.

\subsection{Gravitation pressure}

The gravitational field of a planet pulls it inwards. This causes a pressure which keeps it from shrinking to a point. This pressure can be calculated as follows
\begin{align*}
\grad p &= -\rho_0 \grad \Psi_G\\
\Rightarrow p &= p_0 - \rho_0 \Psi_G \\
\Rightarrow p &= p_0 + 2 \pi \rho_0^2 G (R^2 - r^2/3) \\
\Rightarrow p &= \hat{p_0} - 2 \pi \rho_0^2 G r^2/3
\end{align*}
Pressure at the surface ($r=R$) will be extremely small (only due to the atmosphere) compared to pressure in layers inside the planet. We can assume it to be zero. Therefore, $p|_{r=R} = 0$. Using this we get,
\begin{align*}
\hat{p_0} &= 2 \pi \rho_0^2 G R^2/3 \\
\Rightarrow p &= \frac{2 \pi \rho_0^2 G}{3} (R^2 - r^2)
\end{align*}
This is the pressure which keeps a self-gravitating body from shrinking to a point.

\section{Self-gravitating equilibrium with rotation}
The transformation between two first order tensors must be mediated by a second order tensor. For a sphere, we obtained the relation $\vect F = -\grad \Psi = C \dl_{ij}\vect x$. For a spheroid with axis along $\vect p$, this becomes $\grad \Psi = (C_1 p_ip_j + C_2 \dl_{ij})x_j$.

For the case of a rotating self-gravitating body, the eccentricity $(e)$ is a function of ratio of rate of rotation and gravity. Therefore, our aim is to find
\begin{align*}
\frac{\Omega^2 R}{g} &= f(e)
\end{align*}
If we put $g = GM/R^2$ and $M=\rho_0 R^3$, then the equation becomes
\begin{align*}
\frac{\Omega^2}{\rho_0 G} &= f(e)
\end{align*}
This function turns out to be
\begin{align*}
\frac{\Omega^2}{2 \pi \rho_0 G} &= e^2(1-e^2)^{1/2}\int_0^{\infty}\frac{\lambda d\lambda}{(1+\lambda)^2(1-e^2+\lambda)}
\end{align*}

which indicates that for any given rate of rotation, there are two possible equilibrium eccentricities. For small rates of rotation (as for most planets), one of these eccentricities $e \sim 1$ which signifies a nearly flat pancake like body, while the other eccentricity $e \sim 0$ which is indicative of a nearly spherical planet.

Beyond a critical rate of rotation, there do not exist any real roots i.e. centrifugal forces dominate and an equilibrium between centrifugal and gravitational forces is never established.

\section{Appendix}

\subsection{Constant field inside a gouged out sphere}
(figures required)

\subsection{Derivation for Self-gravitating equilibrium with rotation}

\end{document}