%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% University/School Laboratory Report
% LaTeX Template
% Version 3.1 (25/3/14)
%
% This template has been downloaded from:
% http://www.LaTeXTemplates.com
%
% Original author:
% Linux and Unix Users Group at Virginia Tech Wiki
% (https://vtluug.org/wiki/Example_LaTeX_chem_lab_report)
%
% License:
% CC BY-NC-SA 3.0 (http://creativecommons.org/licenses/by-nc-sa/3.0/)
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%----------------------------------------------------------------------------------------
%	PACKAGES AND DOCUMENT CONFIGURATIONS
%----------------------------------------------------------------------------------------

\documentclass{article}

\usepackage[version=3]{mhchem} % Package for chemical equation typesetting
\usepackage{siunitx} % Provides the \SI{}{} and \si{} command for typesetting SI units
\usepackage{graphicx} % Required for the inclusion of images
\usepackage{natbib} % Required to change bibliography style to APA
\usepackage{amsmath} % Required for some math elements
\usepackage[table,xcdraw]{xcolor}
\usepackage{listings}
\usepackage{hyperref}
\usepackage{xcolor}
\lstset { %
    language=C++,
    backgroundcolor=\color{black!5}, % set backgroundcolor
    basicstyle=\footnotesize,% basic font setting
    numbers=left,
    stepnumber=1,
    tabsize = 2
}

\usepackage[utf8]{inputenc} %Türkçe karakterler
\usepackage{floatrow}
\usepackage{geometry}
\usepackage{caption}
\usepackage{subcaption}
\geometry{margin=1in}

\renewcommand{\lstlistingname}{Code Snippet}

%\setlength\parindent{0pt} % Removes all indentation from paragraphs

% Table float box with bottom caption, box width adjusted to content
\newfloatcommand{capbtabbox}{table}[][\FBwidth]

\renewcommand{\labelenumi}{\alph{enumi}.} % Make numbering in the enumerate environment by letter rather than number (e.g. section 6)

%\usepackage{times} % Uncomment to use the Times New Roman font

%----------------------------------------------------------------------------------------
%	DOCUMENT INFORMATION
%----------------------------------------------------------------------------------------

\title{Assignment 1: \\ Image Filtering and Hybrid Images\\ COMP 408} % Title

\author{Ahmet \textsc{Uysal}} % Author name

\date{\today} % Date for the report

\begin{document}

\maketitle % Insert the title, author and date

\begin{center}
\begin{tabular}{l r}
%Date Performed: & October 10, 2018 \\ % Date the experiment was performed
% Partners: & James Smith \\ & Mary Smith \\
Instructor: & Y\"ucel  \textsc{Yemez} % Instructor/supervisor
\end{tabular}
\end{center}

% If you wish to include an abstract, uncomment the lines below
% \begin{abstract}
% Abstract text
% \end{abstract}

%----------------------------------------------------------------------------------------
%	SECTION 1
%----------------------------------------------------------------------------------------

\section{Implementing myFilter Method}

If f[m,n] is an image and h[i,j] is a box filter with size 2k+1 x 2l+1, then we can calculate the resulting image g[m,n] as
\begin{equation}
g[m,n] =\sum\limits_{i=-k}^k \sum\limits_{j = -l }^l h[i+k, j+l] \times f[m+i, n+j]
\end{equation}

So, by iterating over each pixel of the image and element-wise multiplying the corresponding part of the image with filter, we can calculate the resulting filtered image. However, we need to be careful at the boundaries of the image since some of the pixels we are trying to reach will not be in the image. We will solve this problem on later sections according to three different approaches: copying edge ,reflecting across edge, and zero-padding.

In addition, we are using images with three channels. So, we will have to calculate the result for each channel separately.

\subsection{Implementation using C++ and OpenCV}

Mat objects from the assignment are type CV\_64FC3, which means 64-bit floating point (double), three channeled image.

\begin{figure}[!htb]
\centering
\includegraphics[width=0.9\textwidth]{multichannel.png}
\caption{OpenCV array representation in C++ for BGR color system according to official documents.}
\end{figure}

So, instead of iterating every pixel once we need to iterate three times, for each color value. This can be done by separating the image to channels, calculating the result for each channel and then combining them. However, we can also implement it without splitting by carefully calculating indexes when trying to get the neighbor pixels value for the same channel. My implementation uses the second approach to iterate over each color value of the image.

\newpage

Following code snippet is from my submission. However, some parts are modified for the sake of brevity.

\begin{lstlisting}[caption={My implementation of myFilter method.},captionpos=b]
Mat myFilter(Mat im, Mat filter, int borderType = Border_Constant) {
	Mat outI;
	// initilize our result with zeros
	outI = Mat::zeros(im.rows, im.cols, im.type());

	int filterHeight = filter.rows / 2;
	int filterWidth = filter.cols / 2;
	int numRows = im.rows;
	int numCols = im.cols;
	int channels = im.channels();

	// Mat to store the values we will multiply by filter
	Mat frame = Mat::zeros(filter.rows, filter.cols, CV_64F);

	// pointer to relevant row of image
	double* My = NULL;

	int indexX, indexY;

  // Note that we have to separate different channels and combine them together
	// We can handle this by locating neighbor pixels on distance 3 instead of 1
	for (int m = 0; m < numRows; m++) {
		for (int n = 0; n < numCols * channels; n++) {
			for (int j = -filterHeight; j <= filterHeight; j++) {
				indexY = m + j;
				if (indexY < 0 || indexY >= numRows) {
          // Handle boundaries
				}
				My = im.ptr<double>(indexY);
				for (int i = -filterWidth; i <= filterWidth; i++) {
					indexX = n + i * channels;
					if (indexX < 0 || indexX >= numCols * channels) {
						// Handle boundaries
					}
					frame.at<double>(j + filterHeight, i + filterWidth) = My[indexX];
				}
			}
			multiply(filter, frame, frame);
			// need to specify index since method does not assume our input is one channel
			outI.ptr<double>(m)[n] = sum(frame)[0];
		}
	}
	return outI;
}
\end{lstlisting}

I used multiply and sum methods from OpenCV library. Multiply is used for element-wise multiplication of the current frame of the image being iterated and filter. Sum method returns the sum of all elements in the given matrix.

\newpage

\subsection{Boundary Problem}

To solve boundary problem, we will update our index to get value according to given border type. In the case of zero-padding (Border\_Constant), we will use boolean values to indicate whether or not we should put zero. Also, we need to check indicator before getting the pointer for My to avoid null pointer exception.

\begin{lstlisting}[caption={Boundary check for indexX (row index).},captionpos=b]
indicatorY = false;
if (indexY < 0 || indexY >= numRows) {
    if (borderType == Border_Replicate) {
        if (indexY < 0) {
            indexY = 0;
        }
        else {
            indexY = numRows - 1;
        }
    }
    else if (borderType == Border_Reflect) {
        if (indexY < 0) {
            indexY = -indexY - 1;
        }
        else {
            indexY =  2 * numRows - 1  - indexY;
        }
    }
    else {
        // make indicator true to indicate we will put 0 to frame array
        indicatorY = true;
    }
}
if (!indicatorY)
    My = im.ptr<double>(indexY);
\end{lstlisting}

\begin{lstlisting}[caption={Boundary check for indexY (column index).},captionpos=b]
indicatorX = false;
if (indexX < 0 || indexX >= numCols * channels) {
	if (borderType == Border_Replicate) {
		if (indexX < 0) {
			indexX = indexX % channels;
		}
	else {
		indexX = (indexX) % channels + (numCols - 1) * channels;
	}
}
else if (borderType == Border_Reflect) {
	if (indexX < 0) {
		indexX = (indexX % channels) + ((indexX + 1) / -channels) * channels;
	}
	else {
		indexX = (indexX % channels) + (numRows - 1) * channels
				- ((indexX - numRows) / channels) * channels;
	}
else {
	// make indicator true to indicate we will put 0 to frame array
	indicatorX = true;
}
\end{lstlisting}

\begin{lstlisting}[caption={We should also add this check to our code from previous section.},captionpos=b]
if (indicatorX || indicatorY) {
    frame.at<double>(j + filterHeight, i + filterWidth) = 0;
}
else {
    frame.at<double>(j + filterHeight, i + filterWidth) = My[indexX];
}
\end{lstlisting}

\newpage

\subsection{Sample Results}
The results of the filtering tests are below. The order of the images are from left top to right bottom: Identity Image, Blur Image, Large Blur Image, Sobel Image, Laplacian Image, High Pass Image
\begin{figure}[!htb]
\centering
\includegraphics[width=0.75\textwidth]{cat_filtering_results.png}
\caption{Results for the filters on given cat image.}
\end{figure}

\begin{figure}[!htb]
\centering
\includegraphics[width=0.6\textwidth]{yyemez_filtering_results.png}
\caption{Results for the filters on image of Prof. Y\"ucel Yemez (used with his consent).}
\end{figure}

%----------------------------------------------------------------------------------------
%	SECTION 2
%----------------------------------------------------------------------------------------

\newpage
\section{Generating the Hybrid Image }

As described in \href{http://cvcl.mit.edu/publications/OlivaTorralb_Hybrid_Siggraph06.pdf}{SIGGRAPH 2006 paper} we need to filter one of the images with low-pass filter, other image with a high-pass filter and then sum the filtered images to obtain the hybrid image.

For low-pass filtering, we will use a Gaussian filter.

For high-pass filtering, we will also use a Gaussian filter to get the low frequencies, but we will use the filtered image to subtract from the original image to get the high frequencies.

\subsection{Implementation using C++ and OpenCV}
\begin{lstlisting}[caption={Code for generating the hybrid image.},captionpos=b]
int cutoff_frequency = 7;

Mat filter = getGaussianKernel(cutoff_frequency * 4 + 1, cutoff_frequency, CV_64F);
filter = filter*filter.t();

Mat low_freq_img;
low_freq_img = myFilter(image1,filter, Border_Constant);

Mat high_freq_img;
high_freq_img = image2 - myFilter(image2, filter, Border_Constant);

Mat hybrid_image;
hybrid_image = low_freq_img + high_freq_img;

//add a scalar to high frequency image because it is centered around zero and is mostly black
high_freq_img = high_freq_img + Scalar(0.5, 0.5, 0.5) * 255;
//Convert the resulting images type to the 8 bit unsigned integer matrix with 3 channels
high_freq_img.convertTo(high_freq_img, CV_8UC3);
low_freq_img.convertTo(low_freq_img, CV_8UC3);
hybrid_image.convertTo(hybrid_image, CV_8UC3);

Mat vis = hybrid_image_visualize(hybrid_image);

imshow("Low frequencies", low_freq_img); waitKey(0);
imshow("High frequencies", high_freq_img);	waitKey(0);
imshow("Hybrid image", vis); waitKey(0);
\end{lstlisting}

\newpage
\subsection{Sample Results}
\begin{figure}[!htb]
\begin{subfigure}{.35\textwidth}
  \centering
  \includegraphics[width=1\textwidth]{dog.jpg}
  \caption{Original dog image.}
\end{subfigure}%
\begin{subfigure}{.35\textwidth}
  \centering
  \includegraphics[width=1\textwidth]{cat.jpg}
  \caption{Original cat image.}
\end{subfigure}
\begin{subfigure}{.35\textwidth}
  \centering
  \includegraphics[width=1\textwidth]{low_frequencies.jpg}
  \caption{Low frequency component of (a).}
\end{subfigure}%
\begin{subfigure}{.35\textwidth}
  \centering
  \includegraphics[width=1\textwidth]{high_frequencies.jpg}
  \caption{High frequency component of (b).}
\end{subfigure}
\begin{subfigure}{.6\textwidth}
  \centering
  \includegraphics[width=1\textwidth]{hybrid_image.jpg}
  \caption{Hybrid image created by summing low and high frequency images.}
\end{subfigure}
\caption{Input and output images of my implementation.}
\end{figure}

\newpage

\section{DFT Spectrum}

To get the DFT Spectrum in a viewable form, we need to find the complex DFT of the image. Then, we can calculate the magnitude using

\begin{equation}
Magnitude = \sqrt{(Re(DFT(I))^2 + Im(DFT(I))^2)}
\end{equation}

In order to visualize the spectrum, we need to use logarithmic scale. So, we need to switch to logarithmic scale, i.e, log(1 + magnitude).

Finally, we need to modify the image to make origin correspond to image center for visualization purposes.

\subsection{Implementation Using C++ and OpenCV}

\begin{lstlisting}[caption={My implementation of DTF\_Spectrum method.},captionpos=b]
Mat DFT_Spectrum(Mat img) {
	vector<Mat> im_channels(3);
	split(img, im_channels);
	img = im_channels[0];

	//add padding
	int optimalCol = getOptimalDFTSize(img.cols);
	int optimalRow = getOptimalDFTSize(img.rows);

	copyMakeBorder(img, img, 0, optimalRow - img.rows, 0, optimalCol - img.cols, BORDER_CONSTANT);

	//Determine complex DFT of the image.
	dft(img, img, DFT_COMPLEX_OUTPUT);

	Mat magI;
	vector<Mat> im_dims(2);
	split(img, im_dims);
	// calculate the magnitude
	magnitude(im_dims[0], im_dims[1], magI);
	// add one before switching to log scale
	magI += Scalar::all(1);
	log(magI, magI);

	//crop the spectrum, if it has an odd number of rows or columns
	magI = magI(Rect(0, 0, magI.cols & -2, magI.rows & -2));

	// rearrange the quadrant so that the origin is at the image center
	int quadrantCols = magI.cols / 2;
	int quadrantRows = magI.rows / 2;
	Mat topLeftQuadrant(magI, Rect(0, 0, quadrantCols, quadrantRows));
	Mat topRightQuadrant(magI, Rect(quadrantCols, 0, quadrantCols, quadrantRows));
	Mat bottomLeftQuadrant(magI, Rect(0, quadrantRows, quadrantCols, quadrantRows));
	Mat bottomRightQuadrant(magI, Rect(quadrantCols, quadrantRows, quadrantCols, quadrantRows));

	// swap quadrants
	Mat tmp;
	topLeftQuadrant.copyTo(tmp);
	bottomRightQuadrant.copyTo(topLeftQuadrant);
	tmp.copyTo(bottomRightQuadrant);
	topRightQuadrant.copyTo(tmp);
	bottomLeftQuadrant.copyTo(topRightQuadrant);
	tmp.copyTo(bottomLeftQuadrant);

	// Transform the matrix with float values into a viewable image form.
	normalize(magI, magI, 0, 1, CV_MINMAX);
	return magI;
}
\end{lstlisting}

I implemented the method with the help of official \href{https://docs.opencv.org/2.4/doc/tutorials/core/discrete_fourier_transform/discrete_fourier_transform.html}{OpenCV Documentation on Discrete Fourier Transform.}
\newpage

\subsection{Sample Results and Conclusions}
\begin{figure}[!htb]
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=.8\textwidth]{dog_DFT.jpg}
  \caption{DFT of original dog image.}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=.8\textwidth]{cat_DFT.jpg}
  \caption{DFT of original cat image.}
\end{subfigure}
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=.8\textwidth]{dog_filtered_DFT.jpg}
  \caption{DFT of low frequency component of dog.}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=.8\textwidth]{cat_filtered_DFT.jpg}
  \caption{DFT of high frequency component of cat.}
\end{subfigure}
\caption{DFT outputs of my implementation.}
\end{figure}

Before the filtering, both of the images have parts from almost all frequencies. However, after the filtering, DFT of dog image filtered with Gaussian filter has only frequencies below some level, its graph is confined to some small region near center. On the contrary, DFT of high frequency part of cat has lost its parts near the origin. It has only frequencies above some level.

\newpage
\section{Extra Hybrid Image}

On this last part, I wanted to try building a hybrid image myself. I used one of the portrait images of Prof. Y\"ucel with his consent. I combined his portait with one of the portraits of Amedeo Avogadro. I modified Avogadro's portait's background to minimizing unnecessary collusions. Cutoff frequency is used as 5 in the code.

\begin{figure}[!htb]
\begin{subfigure}{.4\textwidth}
  \centering
  \includegraphics[width=.8\textwidth]{yyemez.jpg}
  \caption{Portrait of Prof. Y\"ucel Yemez.}
\end{subfigure}%
\begin{subfigure}{.4\textwidth}
  \centering
  \includegraphics[width=.8\textwidth]{avogadro.png}
  \caption{Modified portraid of Amedeo Avogadro.}
\end{subfigure}
\begin{subfigure}{.4\textwidth}
  \centering
  \includegraphics[width=.8\textwidth]{low_frequencies_yyemez.jpg}
  \caption{Low frequency .}
\end{subfigure}
\begin{subfigure}{.4\textwidth}
  \centering
  \includegraphics[width=.8\textwidth]{high_frequencies_avog.jpg}
  \caption{Modified portraid of Amedeo Avogadro.}
\end{subfigure}
\caption{Inputs to my filtered image with filtered versions.}
\end{figure}

\newpage

\begin{figure}[!htb]
\centering
\includegraphics[width=1\textwidth]{hybrid_image_scales.jpg}
\caption{Resulting hybrid image in different scales.}
\end{figure}

\end{document}


%GiG
\documentclass{beamer}
\usetheme{Copenhagen}
\setbeamertemplate{navigation symbols}{}
\setbeamertemplate{headline}{}
\DeclareMathOperator*{\argmax}{arg\,max}

\usepackage{hyperref}

\definecolor{azure}{rgb}{0.0, 0.5, 1.0}
%\newcommand{\tblue}[1]{\textcolor{blue}{#1}}
\newcommand{\tblue}[1]{{\Large {\textcolor{azure}{#1}}}}
\newcommand{\thblue}[1]{{\Huge {\textcolor{azure}{#1}}}}
\newcommand{\hred}[1]{{\textcolor{red}{#1}}}
\newcommand{\furl}[1]{{\footnote{\url{#1}}}}

\newcommand{\mypause}{\pause}
%\newcommand{\mypause}{}

\title[Saravanan Thirumuruganathan]
{Lecture 9: Naive Bayes, SVM, Kernels}

\author[CSE 5334, Data Mining]
{Instructor: Saravanan Thirumuruganathan}

\date[]

\begin{document}

\begin{frame}
  \titlepage
\end{frame}

%\begin{frame}{Outline}
%  \tableofcontents
%  % You might wish to add the option [pausesections]
%\end{frame}

\section{Outline}

\begin{frame}
\frametitle {Outline}
    \begin{enumerate}
        \item Probability basics
        \item Probabilistic Interpretation of Classification
        \item Bayesian Classifiers, Naive Bayes
        \item Support Vector Machines
    \end{enumerate}
\end{frame}

%\begin{frame}{In-Class Quizzes}
%\begin{itemize}
%\item {\Large {\bf URL:}} {\LARGE \bf \url{http://m.socrative.com/}}
%\item {\Large {\bf Room Name:} {\LARGE \bf 4f2bb99e}}
%\end{itemize}
%\end{frame}

\section{Probabilistic Interpretation of Classification}
\begin{frame}{}
    \begin{center}
        \thblue{Probability Basics}
    \end{center}
\end{frame}

\begin{frame}{Sample Space}
    \begin{itemize}
        \item {\bf Sample Space:} A space of events that we assign probabilities to
        \item Events can be binary, multi-valued, or continuous
        \item Events are mutually exclusive
        \item Examples:
        \begin{itemize}
            \item Coin flip: \{head, tail\}
            \item Die roll: \{1,2,3,4,5,6\}
            \item English words: a dictionary
            \item Temperature: $\mathbb{R}_{+}$ (Kelvin)
        \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Random Variable}
    \begin{itemize}
        \item A variable, $X$, whose domain is the sample space, and whose value is somewhat uncertain
        \item Examples:
        \begin{itemize}
            \item $X$ = coin flip outcome
            \item $X$ = first word in tomorrow's headline news
            \item $X$ = tomorrow's temperature
        \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Probability for Discrete Events}
    \begin{itemize}
        \item Probability $P(X=a)$ is the fraction of times $x$ takes value $a$
        \item Often we write it as P(a)
        \item Examples:
        \begin{itemize}
            \item Fair Coin: P(head)=P(tail)=0.5
            \item Slightly Biased Coin: P(head)=0.51, P(tail)=0.49
            \item Two Face's Coin: P(head)=1, P(tail)=0
            \item Fair Dice: P(getting 1 in a die roll) = 1/6
        \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Probability for Discrete Events}
    \begin{itemize}
        \item P(A=``head or tail in a fair coin'') $$0.5 + 0.5 = 1$$
        \item P(A=``even number in a fair dice roll'') $$1/6 + 1/6 + 1/6 = 0.5$$
        \item P(A=``two dice rolls sum to 2 in a fair dice'') $$1/6 * 1/6 = 1/36$$
    \end{itemize}
\end{frame}

\begin{frame}{Axioms of Probability}
    \begin{center}
        \includegraphics[scale=0.3]{axiomsOfProbability.png}
    \end{center}
\end{frame}

\begin{frame}{Simple Corollaries}
    \begin{itemize}
        \item $P(A') = 1 - P(A)$
        \item If $A$ can take $k$ different values $a_1, \ldots, a_k$, $$P(A=a_1) + \ldots + P(A=a_k) = 1$$
        \item {\bf Law of Total Probability:}
        \begin{itemize}
            \item $P(A) = P(A \cap B) + P(A \cap B')$
            \item $P(A) = \sum_{i=1}^{k} P(A \cap B=b_i)$ if $B$ takes $k$ values $b_1, \ldots, b_k$
        \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Law of Total Probability\furl{http://people.reed.edu/~jones/Courses/P02.pdf}}
    \begin{center}
        \includegraphics[scale=0.6]{lawOfTotalProbability.png}
    \end{center}
\end{frame}

\begin{frame}{Probability Table}
    \begin{center}
        \includegraphics[scale=0.3]{probTbl.png}
    \end{center}
\end{frame}
\begin{frame}{Joint Probability Table}
    \begin{center}
        \includegraphics[scale=0.3]{jointProbTbl.png}
    \end{center}
\end{frame}
\begin{frame}{Marginal Probability Table}
    \begin{center}
        \includegraphics[scale=0.3]{marginalProbTbl1.png}
    \end{center}
\end{frame}
\begin{frame}{Marginal Probability Table}
    \begin{center}
        \includegraphics[scale=0.3]{marginalProbTbl2.png}
    \end{center}
\end{frame}

\begin{frame}{Conditional Probability}
    \begin{itemize}
        \item $P(A=a|B=b)$ = fraction of times when random variable $A$ took a value of $a$, within the region where random variable $B=b$
    \end{itemize}
    \begin{center}
        \includegraphics[scale=0.5]{condnlProb.png}
    \end{center}
\end{frame}

\begin{frame}{Conditional Probability}
    \begin{itemize}
        \item Consider a roll of a fair dice
        \item $A$: it rolled $1$. $P(A) = 1/6$
        \item $B$: it rolled an odd number. $P(B) = 3/6 = 0.5$
        \item Suppose, I knew that $B$ happened. What is the probability that $A$ happened? \pause
        \item $$P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{1/6}{1/2} = \frac{1}{3}$$
    \end{itemize}
\end{frame}

\begin{frame}{Conditional Probability}
    \begin{itemize}
        \item {\bf Conditional Probability:} $P(A|B) = \frac{P(A \cap B)}{P(B)}$
        \item {\bf Multiplication Rule:} $P(A \cap B) = P(A|B) P(B)$
        \item {\bf Chain Rule:}
            \begin{itemize}
                \item $P(A_1, A_2) = P(A_1) P(A_2|A_1)$
                \item
                    \begin{align*}
                        P(A_1, A_2, A_3) &= P(A_3|A_1,A_2) P(A_1,A_2) \\
                                         &= P(A_3|A_1,A_2) P(A_2|A_1) P(A_1)
                    \end{align*}
                \item
                    \begin{align*}
                        P(A_1, A_2, A_3) &= P(A_1|A_2,A_3) P(A_2,A_3) \\
                                         &= P(A_1|A_2,A_3) P(A_2|A_3) P(A_3)
                    \end{align*}
                \item $P\left(\cap_{i=1}^{k} A_i \right) = \prod_{i=1}^{k} P\left( A_i | \cap_{j=1}^{i-1} A_j \right)$
            \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Bayes Theorem}
    \begin{itemize}
        \item $P(A|B) = \frac{P(A \cap B)}{P(B})$
        \item $P(B|A) = \frac{P(A \cap B)}{P(A})$
        \item Proof of Bayes Theorem:
            \begin{align*}
                P(A \cap B) &= P(A|B) P(B) \\
                            &= P(B|A) P(A) \\
                P(A|B) P(B) &= P(B|A) P(A) \\
                P(A|B)      &= \frac{P(B|A) P(A)}{P(B)}
            \end{align*}
    \end{itemize}
\end{frame}

\begin{frame}{Independence}
    \begin{itemize}
        \item Two events A, B are independent, if (all 3 definitions are equivalent)
        \begin{itemize}
            \item $P(A \cap B) = P(A) P(B)$
            \item $P(A|B) = P(A)$
            \item $P(B|A) = P(B)$
        \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Independence Misused}
    \begin{center}
        \includegraphics[scale=0.36]{independenceJoke.png}
    \end{center}
\end{frame}

\begin{frame}{Independence}
    \begin{itemize}
        \item Independence between random variables is typically obtained via domain knowledge
        \item Suppose $A$ and $B$ be two independent random variables that can take $k$ different values $a_1, \ldots, a_k$ and $b_1, \ldots, b_k$
        \item The joint probability table typically has $k^2$ parameters
        \item If random variables are independent, then only $2k-2$ parameters
        \begin{itemize}
            \item $k=2$, $4$ vs $2$
            \item $k=10$, $100$ vs $18$
            \item $k=100$, $10,000$ vs $198$
        \end{itemize}
        \item This is something great for data mining!
    \end{itemize}
\end{frame}

\begin{frame}{Conditional Independence}
    \begin{itemize}
        \item Random variables can be dependent, but {\bf conditionally independent}
        \item Your house has an alarm
        \begin{itemize}
            \item Neighbor John will call when he hears the alarm
            \item Neighbor Mary will call when she hears the alarm
            \item Assume John and Mary don‟t talk to each other
        \end{itemize}
        \item JohnCall independent of MaryCall? \pause
        \begin{itemize}
            \item No - If John called, likely the alarm went off, which increases the probability of Mary calling
            \item $P(MaryCall | JohnCall) \neq  P(MaryCall)$
        \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Conditional Independence}
    \begin{itemize}
        \item If we know the status of the alarm, JohnCall won't affect Mary at all
        \item $P(MaryCall | Alarm, JohnCall) = P(MaryCall | Alarm)$
        \item We say JohnCall and MaryCall are {\bf conditionally independent}, given Alarm
        \item In general A, B are conditionally independent given C
        \begin{itemize}
            \item $P(A | B, C) = P(A | C)$ or
            \item $P(B | A, C) = P(B | C)$ or
            \item $P(A, B | C) = P(A | C) * P(B | C)$
        \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{}
    \begin{center}
        \thblue{Probabilistic Interpretation of Classification}
    \end{center}
\end{frame}

\begin{frame}{Probabilistic Classifiers}
    \begin{itemize}
        \item Type of classifiers that, given an input, produces a {\em probability distribution} over a set of classes
        \begin{itemize}
            \item Probability that this email is spam is X and not spam is Y
            \item Probability that this person has tumour is X and no tumour is Y
            \item Probability that this digit is $0$ is X, $1$ is Y, $\ldots$
        \end{itemize}
        \item Most state of the art classifiers are probabilistic
        \item Even $k$-NN and Decision trees have probabilistic interpretations
    \end{itemize}
\end{frame}

\begin{frame}{Prior Probability}
    \begin{itemize}
        \item $P(A)$: Prior or unconditional probability of $A$
        \item Your belief in $A$ in the absence of additional information
        \item Uninformative priors
        \begin{itemize}
            \item Principle of indifference: Assign equal probabilities to all possibilities
            \item Coin toss, P(head)=P(tail)=$1/2$
        \end{itemize}
        \item Often you get from domain knowledge or from data
        \item P(email is spam) = $0.8$
    \end{itemize}
\end{frame}

\begin{frame}{Conditional Probability as Belief Update}
    \begin{itemize}
        \item $P(A)$: Prior belief in $A$
        \item $P(A|B)$: Belief after obtaining information $B$
        \item $P(A|B,C)$: Belief after obtaining information $B$  and $C$
    \end{itemize}
\end{frame}

\begin{frame}{Conditional Probability as Belief Update\furl{http://www.quora.com/In-laymans-terms-how-does-Naive-Bayes-work}}
    \begin{itemize}
        \item Suppose you work as security guard in Airport
        \item Your job: look at people in security line and choose some for additional screening
        \item You want to pick passengers with high ``risk''
        \item $A$: Passenger is high risk
        \item By experience, you know only 0.1\% of passengers are high risk (Prior probability)
    \end{itemize}
\end{frame}

\begin{frame}{Conditional Probability as Belief Update\furl{http://www.quora.com/In-laymans-terms-how-does-Naive-Bayes-work}}
    \begin{itemize}
        \item Consider a random person:
        \begin{itemize}
            \item The probability that this person is high risk is $A$ is 0.1\%
            \item Suppose you notice that the person is male  \pause
            \begin{itemize}
                \item There are more male criminals than female ones
            \end{itemize}
            \item The passenger is nervous \pause
            \begin{itemize}
                \item Most criminals are nervous but most normal passengers are not
            \end{itemize}
            \item The passenger is a kid \pause
        \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Conditional Probability as Belief Update}
    \begin{center}
        \includegraphics[scale=0.46]{candyEg.png}
    \end{center}
\end{frame}

\begin{frame}{Conditional Probability}
    \begin{itemize}
        \item $X, A =\langle A_1, A_2, \ldots, A_d \rangle$: Input feature vector
        \item $Y, C$: Class value to predict
        \item $P(C|A)$ vs $P(A|C)$ \pause
        \item Key terms
        \begin{itemize}
            \item $P(A), P(C)$: Prior probability
            \item $P(A|C)$: Class conditional probability or likelihood (from training data)
            \item $P(C|A)$: Posterior probability
        \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Likelihood vs Posterior}
    \begin{itemize}
        \item $P(A|C)$: Likelihood, $P(C|A)$: Posterior
        \item Examples:
        \begin{itemize}
            \item P(Viagra$|$Spam) and P(Spam$|$Viagra): \pause Likelihood, Posterior
            \item P(High temperature$|$Flu) and P(Flu$|$High Temperature): \pause Likelihood, Posterior
            \item P(Fever, Headache, Cough$|$Flu) and P(Flu$|$Fever, Headache, Cough)
            \item P(Viagra, Nigeria, Lottery$|$Spam) and P(Spam$|$Viagra, Nigeria, Lottery)
        \end{itemize}
    \end{itemize}
\end{frame}

\begin{frame}{Bayes' Theorem}
    \begin{itemize}
        \item Training data gives us likelihood and prior
        \item Prediction requires Posterior
        \item Bayes rule allows us to do statistical inference
        \item $P(C|A) = \frac{P(A|C) P(C)}{P(A)}$
        \item $P(C|A) \propto P(A|C) P(C)$
        \item Posterior $\propto$ Likelihood $\times$ Prior
    \end{itemize}
\end{frame}

\begin{frame}{Bayes Decision Rule}
    \begin{itemize}
        \item ``When you hear hoofbeats, think of horses not zebras''
        \item When predicting, assign the class with highest posterior probability
        \item To categorize email:
        \begin{itemize}
            \item Compute P(spam$|$email) and P(not spam$|$email)
            \item If P(spam$|$email) $>$ P(not spam$|$email), decide email as spam. Else as not-spam
        \end{itemize}
    \end{itemize}
\end{frame}

\section{Bayesian Classifiers}
\begin{frame}{}
    \begin{center}
        \thblue{Bayesian Classifiers}
    \end{center}
\end{frame}

\begin{frame}{Bayes Classifier}
    \begin{center}
        \includegraphics[scale=0.4]{bayesClassifier1.png}
    \end{center}
\end{frame}
\begin{frame}{Example of Bayes Theorem}
    \begin{center}
        \includegraphics[scale=0.4]{bayesClassifier2.png}
    \end{center}
\end{frame}
\begin{frame}{Bayesian Classifiers}
    \begin{center}
        \includegraphics[scale=0.4]{bayesClassifier3.png}
    \end{center}
\end{frame}
\begin{frame}{Bayesian Classifiers}
    \begin{center}
        \includegraphics[scale=0.4]{bayesClassifier4.png}
    \end{center}
\end{frame}
\begin{frame}{Naive Bayes Classifier}
    \begin{center}
        \includegraphics[scale=0.4]{bayesClassifier5.png}
    \end{center}
\end{frame}
\begin{frame}{How to Estimate Probabilities from Data?}
    \begin{center}
        \includegraphics[scale=0.4]{bayesClassifier6.png}
    \end{center}
\end{frame}
\begin{frame}{How to Estimate Probabilities from Data?}
    \begin{center}
        \includegraphics[scale=0.4]{bayesClassifier7.png}
    \end{center}
\end{frame}
\begin{frame}{How to Estimate Probabilities from Data?}
    \begin{center}
        \includegraphics[scale=0.36]{bayesClassifier8.png}
    \end{center}
\end{frame}
\begin{frame}{Example of Naive Bayes Classifier}
    \begin{center}
        \includegraphics[scale=0.4]{bayesClassifier9.png}
    \end{center}
\end{frame}
\begin{frame}{Naive Bayes Classifier}
    \begin{center}
        \includegraphics[scale=0.4]{bayesClassifier10.png}
    \end{center}
\end{frame}
\begin{frame}{Example of Naive Bayes Classifier}
    \begin{center}
        \includegraphics[scale=0.36]{bayesClassifier11.png}
    \end{center}
\end{frame}
\begin{frame}{Naive Bayes Classifier Summary}
    \begin{center}
        \includegraphics[scale=0.32]{bayesClassifier12.png}
    \end{center}
\end{frame}

\section{Support Vector Machines}
\begin{frame}{}
    \begin{center}
        \thblue{Support Vector Machines}
    \end{center}
\end{frame}

\begin{frame}{HyperPlane}
    \begin{center}
        \includegraphics[scale=0.32]{hyerplane1.png}
    \end{center}
\end{frame}
\begin{frame}{HyperPlane in 2D}
    \begin{center}
        \includegraphics[scale=0.32]{hyerplane2.png}
    \end{center}
\end{frame}
\begin{frame}{Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm1.png}
    \end{center}
\end{frame}
\begin{frame}{Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm2.png}
    \end{center}
\end{frame}
\begin{frame}{Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm3.png}
    \end{center}
\end{frame}
\begin{frame}{Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm4.png}
    \end{center}
\end{frame}
\begin{frame}{Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm5.png}
    \end{center}
\end{frame}
\begin{frame}{Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm6.png}
    \end{center}
\end{frame}
\begin{frame}{Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm7.png}
    \end{center}
\end{frame}
\begin{frame}{Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm8.png}
    \end{center}
\end{frame}
\begin{frame}{Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm9.png}
    \end{center}
\end{frame}
\begin{frame}{Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm10.png}
    \end{center}
\end{frame}
\begin{frame}{Nonlinear Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm11.png}
    \end{center}
\end{frame}
\begin{frame}{Nonlinear Support Vector Machines}
    \begin{center}
        \includegraphics[scale=0.32]{svm12.png}
    \end{center}
\end{frame}
\begin{frame}{Kernels}
    \begin{center}
        \includegraphics[scale=0.32]{kernels1.png}
    \end{center}
\end{frame}
\begin{frame}{Classifying non-linearly separable data}
    \begin{center}
        \includegraphics[scale=0.32]{kernels2.png}
    \end{center}
\end{frame}
\begin{frame}{Classifying non-linearly separable data}
    \begin{center}
        \includegraphics[scale=0.32]{kernels3.png}
    \end{center}
\end{frame}
\begin{frame}{Feature Mapping}
    \begin{center}
        \includegraphics[scale=0.32]{kernels4.png}
    \end{center}
\end{frame}
\begin{frame}{Kernels as High Dimensional Feature Mapping}
    \begin{center}
        \includegraphics[scale=0.32]{kernels5.png}
    \end{center}
\end{frame}
\begin{frame}{Kernels: Formally Defined}
    \begin{center}
        \includegraphics[scale=0.32]{kernels6.png}
    \end{center}
\end{frame}
\begin{frame}{Popular Kernels}
    \begin{center}
        \includegraphics[scale=0.32]{kernels7.png}
    \end{center}
\end{frame}
\begin{frame}{Using Kernels}
    \begin{center}
        \includegraphics[scale=0.32]{kernels8.png}
    \end{center}
\end{frame}

\section{Summary}
\begin{frame}{Summary}

\tblue{Major Concepts:}
\begin{itemize}
    \item Probabilistic interpretation of Classification
    \item Bayesian Classifiers
    \item Naive Bayes Classifier
    \item Support Vector Machines (SVM)
    \item Kernels
\end{itemize}
\end{frame}

\begin{frame}{Slide Material References}
\begin{itemize}
    \item Slides from TSK Book, Chapter 5
    \item Slides from Piyush Rai
    \item See also the footnotes
\end{itemize}
\end{frame}

\end{document}


\documentclass[12pt]{article}
\usepackage[left=2cm, right=2cm, top=2cm, bottom=2cm]{geometry}
\usepackage{graphicx}
\usepackage{float}
\usepackage{adjustbox}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{commath}
\usepackage{pdfpages}

\title{CTA200 Assignment 2}
\author{Sinthushan Sivakumar}
\date{May $8^{th}$, $2021$}

\begin{document}

\maketitle

\section{Question 1}
The numerical approximation of the derivative of a mathematical function $f$ evaluated as $x_{0}$ can be written as:

$$d_{x}f|_{x_{0}} \approx \frac{f_{x_{0}+h} - f_{x_{0}}}{h}$$

Where $h$ is some small "step". This is found by looking at the Taylor series of $f(x)$ and taking $h \rightarrow 0$. A better approximation, when $h$ is finite (rather than infinitesimal) is:

$$d_{x}f|_{x_{0}} \approx \frac{f_{x_{0}+h} - f_{x_{0}-h}}{2h}$$

Make a python function that has the form `def deriv(f, x0, h)` that takes in a python function, and returns the approximation of the function $f(x)=sin(x)$ for $x_{0}=0.1$ using a variety of values of $h<1$. Plot the error compared tot he analytic derivative (ie abs(d\_numerical - d\_analytic)/d\_analytic) as a function of $h$ for each method on the same loglog plot. What do you notice? What does the slope represent?

\subsection{Method}
Created two functions in Jupyter Notebook that take the derivative of a arbitrary function using the infinitesimal way and the finite way. Both functions take in a python function, $f$, a point, $x0$, and step size, $h$, and returns the derivative of the function $f$ at the point $x0$. Next, the error between each of the numerical derivatives and the analytical derivative was calculated using the numpy module for values of $h$ running from $0.001$
 to $0.9999$ in 1000 steps, for the function $sin(x)$ at the point $0.1$. Finally, using the matplotlib.pyplot, the error in each method of determining the derivative was graphed as function $h$ on the same plot.

 \subsection{Analysis}
 Looking at the graph, one can see that for small values of $h$ around $10^{-4}$, deriv2 has errors $3$ orders of magnitude smaller than deriv1. However, for values of $h$ near $1$, both deriv2 and deriv1 have the same magnitude of error (of about $10^{-1}$).
 The slopes of the lines for each of the methods represents the rate at which the error grows. Using np.plotfit, the slope for deriv1 was determined to be $0.7317$ while the slope for deriv2 was determined to be $0.5027$. Therefore, deriv2 is more accurate in determining the derivative at a point.

 \section{Question 2}
 For each point in the complex plane $c = x + iy$, with $−2 < x < 2$ and $−2 < y < 2$, set $z_{0} = 0$ and iterate the equation $z_{i+1} = z^{2}_{i} + c$ Note what happens to the $z_{i}$’s: some points will remain bounded in absolute value $\abs{z}^{2} = \Re(z)^{2} + \Im(z)^{2}$, while others will run off to infinity. Make an image in which your points c that diverge are given one color and those that stay bounded are given another. Make a second image where the points are coloured by a colour scale that indicates the iteration number at which the given point diverged.

 \subsection{Method}

 \subsection{Analysis}

 \section{Question 3}
The SIR model is a simple mathematical model of disease spread in a population. The model divides a fixed population of size $N$ into three groups, which vary as a function of time, $t$:

\begin{itemize}
    \item $S(t)$ is those that are susceptible but not yet infected
    \item $I(t)$ is the number of infected individuals
    \item $R(t)$ is those individuals that have recovered and are now immune
\end{itemize}

The model can be described by a set of 3 first order differential equations for each of the variables as
\begin{align}
    \frac{dS}{dt} &= -\frac{\beta SI}{N} \\
    \frac{dI}{dt} &= \frac{\beta SI}{N} - \gamma I \\
    \frac{dR}{dt} & \gamma I
\end{align}
Using the ODE integrator of your choice (must be callable in Python, we recommend using Scipy as will
be covered in lecture on Friday), integrate the equations with $N = 1000$ from $t = 0$ to $t = 200$ for various values of $\gamma$ and $\beta$ (at least 3-4 values, justify your choices physically).
Use the initial condition $I(0) = 1$, $S(0) = 999$, $R(0) = 0$ (you can also experiment with other initial conditions if you wish). Plot the curves for $S$, $I$, $R$ on the same figure with a legend. make separate plots for each choice of the parameters (hint: use the subplots() command).
Bonus (+0.5 mark): Add a $4$th parameter D for deaths and justify the addition of a 4th differential equation as well as any RHS terms that must be changed on the initial $3$ equations. Integrate the new set of equations
for some choice of parameters and comment on the results compared to the SIR model.

\subsection{Method}
Created a function for the $SIR$ model. Then from scipy.integrate used ode to solve the system of differential equations, for the initial condition of $999$ susceptible individuals, $1$ infected individual, and $0$ recovered individuals at time $t=0$. Finally, the results of the integration was plotted using matplotlib.pyplot. However, trying to run the code produced an error, "'float' object is not iterable".
This error was not solved in time for submissions.

\subsection{Analysis}

\end{document}


\documentclass[10pt,landscape,a4paper]{article}
\usepackage[right=10mm, left=10mm, top=10mm, bottom=10mm]{geometry}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[english]{babel}
\usepackage[rm,light]{roboto}
\usepackage{xcolor}
\usepackage{graphicx}
\graphicspath{{./figures/}}
\usepackage{multicol}
\usepackage{colortbl}
\usepackage{array}
\setlength\parindent{0pt}
\setlength{\tabcolsep}{2pt}
\baselineskip=0pt
\setlength\columnsep{1em}
\definecolor{Gray}{gray}{0.85}

% --- Listing -----------------------------------------------------------------
\usepackage{listings}
\lstset{
  frame=tb, framesep=4pt, framerule=0pt,
  backgroundcolor=\color{black!5},
  basicstyle=\ttfamily,
  commentstyle=\ttfamily\color{black!50},
  breakatwhitespace=false,
  breaklines=true,
  extendedchars=true,
  keepspaces=true,
  language=Python,
  rulecolor=\color{black},
  showspaces=false,
  showstringspaces=false,
  showtabs=false,
  tabsize=2,
  %
  emph = { plot, scatter, imshow, bar, contourf, pie, subplots, show, savefig,
           errorbar, boxplot, hist, set_title, set_xlabel, set_ylabel, suptitle,  },
  emphstyle = {\ttfamily\bfseries}
}

% --- Fonts -------------------------------------------------------------------
\usepackage{fontspec}
\usepackage[babel=true]{microtype}
\defaultfontfeatures{Ligatures = TeX, Mapping = tex-text}
\setsansfont{Roboto} [ Path           = fonts/roboto/Roboto-,
                       Extension      = .ttf,
                       UprightFont    = Light,
                       ItalicFont     = LightItalic,
                       BoldFont       = Regular,
                       BoldItalicFont = Italic ]
\setromanfont{RobotoSlab} [ Path           = fonts/roboto-slab/RobotoSlab-,
                            Extension      = .ttf,
                            UprightFont    = Light,
                            BoldFont       = Bold ]
\setmonofont{RobotoMono} [ Path           = fonts/roboto-mono/RobotoMono-,
                           Extension      = .ttf,
                           Scale          = 0.90,
                           UprightFont    = Light,
                           ItalicFont     = LightItalic,
                           BoldFont       = Regular,
                           BoldItalicFont = Italic ]
\renewcommand{\familydefault}{\sfdefault}

% -----------------------------------------------------------------------------
\begin{document}
\thispagestyle{empty}

\section*{\LARGE \rmfamily
          Matplotlib \textcolor{orange}{\mdseries for beginners}}

\begin{multicols*}{3}

Matplotlib is a library for making 2D plots in Python. It is designed
with the philosophy that you should be able to create simple plots
with just a few commands:\\

\fbox{1} \textbf{Initialize}
\begin{lstlisting}
 import numpy as np
 import matplotlib.pyplot as plt
\end{lstlisting}
%
\fbox{2} \textbf{Prepare}
\begin{lstlisting}
 X = np.linspace(0, 4*np.pi, 1000)
 Y = np.sin(X)
\end{lstlisting}
%
\fbox{3} \textbf{Render}
\begin{lstlisting}
 fig, ax = plt.subplots()
 ax.plot(X, Y)
 fig.show()
\end{lstlisting}
%
\fbox{4} \textbf{Observe} \medskip\\
\includegraphics[width=\linewidth]{sine.pdf}

% -----------------------------------------------------------------------------
\subsection*{\rmfamily Choose}
% -----------------------------------------------------------------------------

Matplotlib offers several kind of plots (see Gallery): \medskip

\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 X = np.random.uniform(0, 1, 100)
 Y = np.random.uniform(0, 1, 100)
 ax.scatter(X, Y)
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{basic-scatter.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 X = np.arange(10)
 Y = np.random.uniform(1, 10, 10)
 ax.bar(X, Y)
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{basic-bar.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 Z = np.random.uniform(0, 1, (8,8))

 ax.imshow(Z)
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{basic-imshow.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 Z = np.random.uniform(0, 1, (8,8))

 ax.contourf(Z)
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{basic-contour.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 Z = np.random.uniform(0, 1, 4)

 ax.pie(Z)
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{basic-pie.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 Z = np.random.normal(0, 1, 100)

 ax.hist(Z)
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{advanced-hist.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 X = np.arange(5)
 Y = np.random.uniform(0, 1, 5)
 ax.errorbar(X, Y, Y/4)
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{advanced-errorbar.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 Z = np.random.normal(0, 1, (100,3))

 ax.boxplot(Z)
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{advanced-boxplot.pdf}}
\end{tabular}

% -----------------------------------------------------------------------------
\subsection*{\rmfamily Tweak}
% -----------------------------------------------------------------------------
You can modify pretty much anything in a plot, including limits,
colors, markers, line width and styles, ticks and ticks labels,
titles, etc. \medskip

% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 X = np.linspace(0, 10, 100)
 Y = np.sin(X)
 ax.plot(X, Y, color="black")
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{plot-color.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 X = np.linspace(0, 10, 100)
 Y = np.sin(X)
 ax.plot(X, Y, linestyle="--")
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{plot-linestyle.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 X = np.linspace(0, 10, 100)
 Y = np.sin(X)
 ax.plot(X, Y, linewidth=5)
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{plot-linewidth.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 X = np.linspace(0, 10, 100)
 Y = np.sin(X)
 ax.plot(X, Y, marker="o")
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{plot-marker.pdf}}
\end{tabular}

% -----------------------------------------------------------------------------
\subsection*{\rmfamily Organize}
% -----------------------------------------------------------------------------

You can plot several data on the the same figure, but you can also
split a figure in several subplots (named {\em Axes}): \medskip

% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 X = np.linspace(0, 10, 100)
 Y1, Y2 = np.sin(X), np.cos(X)
 ax.plot(X, Y1, Y2)
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{plot-multi.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 fig, (ax1, ax2) = plt.subplots((2,1))
 ax1.plot(X, Y1, color="C1")
 ax2.plot(X, Y2, color="C0")
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{plot-vsplit.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 fig, (ax1, ax2) = plt.subplots((1,2))
 ax1.plot(Y1, X, color="C1")
 ax2.plot(Y2, X, color="C0")
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{plot-hsplit.pdf}}
\end{tabular}

% -----------------------------------------------------------------------------
\subsection*{\rmfamily Label \mdseries (everything)}
% -----------------------------------------------------------------------------
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 ax.plot(X, Y)
 fig.suptitle(None)
 ax.set_title("A Sine wave")
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{plot-title.pdf}}
\end{tabular}
% -----------------------------------------------------------------------------
\begin{tabular}{@{}m{.821\linewidth}m{.169\linewidth}}
\begin{lstlisting}[belowskip=-\baselineskip]
 ax.plot(X, Y)
 ax.set_ylabel(None)
 ax.set_xlabel("Time")
\end{lstlisting}
& \raisebox{-0.75em}{\includegraphics[width=\linewidth]{plot-xlabel.pdf}}
\end{tabular}

% -----------------------------------------------------------------------------
\subsection*{\rmfamily Explore}
% -----------------------------------------------------------------------------

Figures are shown with a graphical user interface that allows to zoom
and pan the figure, to navigate between the different views and to
show the value under the mouse.

% -----------------------------------------------------------------------------
\subsection*{\rmfamily Save \mdseries (bitmap or vector format)}
% -----------------------------------------------------------------------------
\begin{lstlisting}[belowskip=-\baselineskip]
 fig.savefig("my-first-figure.png", dpi=300)
 fig.savefig("my-first-figure.pdf")
\end{lstlisting}
%
\vfill
%
{\scriptsize
  Matplotlib 3.4.2 handout for beginners.
  Copyright (c) 2021 Matplotlib Development Team.
  Released under a CC-BY 4.0 International License.
  Supported by NumFOCUS.
\par}

\end{multicols*}
\end{document}


\section{XILINX VIVADO Report}
In this chapter will be presented the results obtained by creating a project with \textbf{Xilin VIVADO} by selecting the Zybo Zynq-7000 (xc7z010clg400-1) as working device. As will be highlighted by the resource utilization paragraph, the \textbf{Implementation phase has not been performed} due to the fact that the \textbf{Zybo-Board} does \textbf{not have enough inputs and outputs ports} (as said during Laboratory classes): in our case we need 10*8 + 11*9 = 179 input pins (even if we fix the weights there is a need of 99 input pins) with only 4 slides switches and 4 push-buttons available to drive the inputs. All things considered the results that will be shown in this chapter are obtained after the \textbf{Synthesis phase} with the timing (clock) constraint.

\subsection{RTL Analysis}
Before heading with the Synthesis a preliminary double-check of the correctness of the system has been made by simply comparing the schemas obtained by the \textbf{Elaborated Design} with the ones in shown in the architecture chapter.\textbf{ No problems has been found at this stage} (in the project folder can be seen all the schemas in pdf format).
\subsection{Timing Report}
After running the Synthesis command after adding the clock constraint the following Timing Report has been displayed:
\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{img/vivado/timing.png}
	\caption{Timing Report}
\end{figure}
As we can see the Worst Negative Slack (WNS) is \textbf{positive}, so we can drive the board at an higher frequency than 125MHz. We can calculate the \textbf{maximum frequency} as:
\begin{equation}
	f_{max} = \frac{1}{T_{clk} - WNS} = 429.0 MHz
\end{equation}
$T_{clk}$ is given by the Zybo Board which operates with 125 MHz and, for this reason, will grant an $T_{clk} = 1/125MHz = 8ns$. The WSN is determined by the \textbf{Critical Path} of the architecture which is shown in the following table (see first row):
\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{img/vivado/critical_path.png}
	\caption{Critical Path Report}
\end{figure}
We can state that the \textbf{Tree Adder} module has the most impact on the \textbf{critical path} so the addition of some registers in between the ripple carry adders and even the addition of the pipelines registers in the latter, during the project phase, were all \textbf{good project choices}.
\subsection{Resource Utilization Report}
The resource utilized by the architecture synthesized are the following:
\begin{figure}[H]
	\centering
	\includegraphics[width=0.6\textwidth]{img/vivado/utilization.png}
	\caption{Critical Path Report}
\end{figure}
As we can see the IO resource utilization is greater than the 100$\%$ of the available one (179 input pins and 16 output pins necessary). So it was not possible to head with the implementation phase with the Zybo Zynq 7000. We can also see that the LUT resource has been utilized by the roughly 10$\%$, so, with this numbers, the LUT optimization done has halved the LUT utilization and it was a \textbf{good project choice}.
\subsection{Power Consumption Report}
The power consumption report is the following:
\begin{figure}[H]
	\centering
	\includegraphics[width=0.8\textwidth]{img/vivado/power.png}
	\caption{Power Consumption Report}
\end{figure}
As we can see, a total of 0.174 W of Power are needed (with the standard settings suggested by Xilinx VIVADO), which is roughly divided equally between dynamic and static power. For the \textbf{Dynamic Power Consumption} the most relevant contributes are from the logic and signals.
\subsection{Warning Messages}
After the synthesis phase the following warning messages were shown:
\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{img/vivado/warnings.png}
	\caption{Warning Messages}
\end{figure}
So, let's analyse them:
\begin{itemize}
	\item \textbf{[Constraints 18-5210] No constraints selected for write}: this warning message was shown even during the laboratory classes and it can be ignored.
	\item \textbf{[Synth 8-3936] Found unconnected internal register 'z$\_$in$\_$lut$\_$reg' and it is trimmed from '21' to '20' bits:} this warning message is due to the fact that to get the proper address on the lut, after the lut optimization, the first bit (the 21th) is the sign one and is used to perform just some additional operation, it has no relevance on the addressing of the lut.
\end{itemize}


\documentclass[main.tex]{subfiles}
\begin{document}

\section*{Fri Nov 08 2019}

One can time a bash command using

\begin{lstlisting}[language=bash]
time [Command]
\end{lstlisting}

\section{Linear systems}

We want to solve a linear system, in the form \(A \vec{x} = \vec{b}\), with unknown \(\vec{x}\).
How do we solve this numerically?
We can transform our system to an equivalent one, by
\begin{enumerate}
    \item exchanging two rows;
    \item multiplying an equation by a nonzero constant;
    \item adding an equation to another.
\end{enumerate}

These allow us to do Gaussian elimination, and LU decomposition. These are \emph{direct methods}.

Another class is that of \emph{indirect methods}: we start with an \emph{ansatz} and refine it. These are easier to implement, more generally applicable, more efficient if the matrix is sparse.
They, however, do not always converge.

An example is the Gauss-Seidel method.

\end{document}


\documentclass[12pt]{article}
\usepackage[margin = 0.9in, top=0.8in]{geometry}
\usepackage{graphicx}
\usepackage{textgreek}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{mathtools}
\usepackage{amssymb}
\usepackage{float}
\usepackage{subcaption}
\usepackage{hyperref}
\usepackage{grffile}
\graphicspath{{./2/images},{./}}

\title{CS 754 - Advanced Image Processing\\Assignment 1 - Report}
\author{Shaan ul Haque - 180070053\\Mantri Krishna Sri Ipsit - 180070032}

\begin{document}

\maketitle

\section*{Question 1}
\subsection*{1.a}
We know that $\delta_{2s}$ is an increasing function of s(sparsity of signal). If we increase s, we are increasing $\delta_{2s}$ which in-turn increases $C_1$ and $C_2$ because they are increasing functions of $\delta_{2s}$ as well. Thus, both have conflicting effects so we can't assume the error bound to arbitrarily decrease as s increases.
\subsection*{1.b}
Though, m(number of measurements) has no direct effect on the bound but it does have few implications on it. The measurement matrix has only m rows. If the sparsity of signal is s which is greater than m then we have:
\begin{equation*}
    \boldsymbol{y} = \sum_{i\epsilon\boldsymbol{S}}\boldsymbol{A_i}x_i
\end{equation*}
where $\boldsymbol{S}$ is the set of index where $\boldsymbol{x}$ has non-zero elements and $\boldsymbol{A_i}$ is the corresponding column. Now since s(cardinality of $\boldsymbol{S}$) is greater than m thus it is linear combination of s m-dimensional vectors which is clearly zero for some $\boldsymbol{x}$ as infinite solutions exists for such under-determined problem. Thus, the null set property of the matrix and hence RIP is lost. Thus, we must have $m>s$, as a necessary condition for RIP. Infact $O(slog(n/s))\leq m$, and thus the value of s cannot be increased beyond the number of rows in $\boldsymbol{A}$.
\subsection*{1.c}
Theorem 3 is better than theorem 3A. This is because a matrix obeying RIP for smaller bound on $\delta_{2s}$ would work fine on signals having more sparsity(smaller values of s) than a matrix whose RIC is larger. Thus, for larger $\delta_{2s}$ we can work with comparatively less sparse signals also and satisfy the BP error limit.
\subsection*{1.d}
If we make the error $\epsilon$ zero in BP, then we want exact relation between y and $\theta$ which is:
\begin{align*}
    y = \boldsymbol{\Phi\Psi\theta}
\end{align*}
But since the measurements are noisy in, the estimated $\theta$ from solving such problem would be incorrect. Thus, we allow some margin so that we do not over-fit our estimated $\theta$ with noisy measurements. Moreover, we see that $\epsilon$ is not some independent parameter but it depends upon the noise power, therefore it is quite unreasonable to take it as zero if noise vector is non-zero in magnitude.

\section*{Question 2}
\subsection*{2.a}
The first $T=3$ frames of the `cars' video are as follows:
\begin{figure}[ht]
	\centering
	\begin{minipage}[bt]{0.3\linewidth}
		\centering
		\includegraphics[scale=0.7]{cars_1.png}
		\caption{$T = 1$}
	\end{minipage}
\begin{minipage}[bt]{0.3\linewidth}
	\centering
	\includegraphics[scale=0.7]{cars_2.png}
	\caption{$T = 2$}
\end{minipage}
\begin{minipage}[bt]{0.3\linewidth}
	\centering
	\includegraphics[scale=0.7]{cars_3.png}
	\caption{$T = 3$}
\end{minipage}
\end{figure}
\subsection*{2.b}
The coded snapshot for $T=3$ is as follows:
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{3_coded_snapshot.png}
	\caption{Coded Snapshot along with Gaussian Noise with standard deviation 2}
\end{figure}
\subsection*{2.c}
We have
$$E_u = \sum \limits_{t=1}^TC_t \cdot F_t$$
Rewriting it as a matrix product and vectorizing the coded snapshot and the image to be measured, we get
$$\mathbf{b =  A\, x}$$
where
$$b = \text{Vectorized } E_u \in \mathbb{R}^{HW}$$
$$x = \text{Vectorized } F_u \in \mathbb{R}^{HWT}$$
$$A = [S_1|S_2|\cdots|S_T], S_t = \text{diag}(C_t(:)) \in \mathbb{R}^{HW \times HWT}$$
\subsection*{2.d}
Rewriting the above equation patchwise, we have
$$\mathbf{A\,x = b}$$
where
$$A = \Phi \Psi \in \mathbb{R}^{64\times 64T}$$
$$\Phi = [S_{i1}|S_{i2}|\cdots|S_{iT}], S_{it} = \text{diag}(C_{it}(:)) \in \mathbb{R}^{64 \times 64T}$$
$$\Psi = I_{T} \:\otimes \psi_{64,64} \: \otimes\: \psi_{64,64}$$
$$\psi_n = \text{2D DCT Basis Matrix of size } n \times n$$
$$x \in \mathbb{R}^{64T}, \text{ sparse}$$
$$b \in \mathbb{R}^{64}$$
Let $\theta^*$ be the optimal solution of the problem P0: $\min \: \left\lVert\theta\right\rVert_0$ such that $\left \lVert y - A \theta\right \rVert_2 \leq \epsilon, A = \Phi \Psi$.
Let the solution obtained using OMP be $\hat{\theta}$. Then we have the squared error as
\begin{eqnarray*}
E^2 &=& \left \lVert \theta^* - \hat{\theta}\right \rVert^2\\
&=& (\theta^* - \hat{\theta})^T\: (\theta^* - \hat{\theta})\\
&=& (A^TA)^{-1} (A\theta^* - A\hat{\theta})^T \: (A\theta^* - A\hat{\theta})\\
&=& (A^TA)^{-1} \bigg((y - A\hat{\theta}) - (y - A\theta^*)\bigg)^T\: \bigg((y - A\hat{\theta}) - (y - A\theta^*)\bigg)\\
&=& (A^TA)^{-1}\: 4\epsilon^2\\
&=& \frac{4\epsilon^2}{\left\lVert A \right \rVert^2}
\end{eqnarray*}
Hence the error will be
$$E = \frac{2\epsilon}{\left\lVert A \right \rVert}$$
\subsection*{2.e}
For $T=3$, the relative mean squared error that we obtained is 0.0112.
The reconstruction for $T=3$ is as follows:
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{3_1.png}
	\caption{$t = 1$, Left - Original, Right - Reconstructed}
\end{figure}
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{3_2.png}
	\caption{$t = 2$, Left - Original, Right - Reconstructed}
\end{figure}
\newpage
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{3_3.png}
	\caption{$t = 3$, Left - Original, Right - Reconstructed}
\end{figure}
\subsection*{2.f}
The relative mean squared error value for $T = 5$ is 0.0194 and for $T = 7$ is 0.0316.
\vspace{2cm}
\begin{center}
\textit{Please turn over}
\end{center}
\newpage
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{5_coded_snapshot.png}
	\caption{Coded snapshot with noise for $T = 5$}
\end{figure}
\begin{figure}[ht]
	\centering
	\begin{minipage}[bt]{0.4\linewidth}
		\centering
			\includegraphics[scale=0.7]{5_1.png}
			\caption{$t = 1$}
			\includegraphics[scale=0.7]{5_2.png}
			\caption{$t = 2$}
			\includegraphics[scale=0.7]{5_3.png}
			\caption{$t = 3$}
	\end{minipage}
\begin{minipage}[bt]{0.5\linewidth}
\centering
	\includegraphics[scale=0.7]{5_4.png}
	\caption{$t = 4$}
	\includegraphics[scale=0.7]{5_5.png}
	\caption{$t = 5$}
\end{minipage}
\caption{Left - Original, Right - Reconstructed}
\end{figure}
\vspace{3cm}
\begin{center}
	\textit{Please turn over}
\end{center}
\newpage
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{7_coded_snapshot.png}
	\caption{Coded snapshot with noise for $T = 7$}
\end{figure}
\begin{figure}[ht]
\centering
\begin{minipage}[bt]{0.4\linewidth}
	\centering
	\includegraphics[scale=0.7]{7_1.png}
	\caption{$t = 1$}
	\includegraphics[scale=0.7]{7_2.png}
	\caption{$t = 2$}
	\includegraphics[scale=0.7]{7_3.png}
	\caption{$t = 3$}
	\includegraphics[scale=0.7]{7_4.png}
	\caption{$t = 4$}
\end{minipage}
\begin{minipage}[bt]{0.5\linewidth}
	\centering
	\includegraphics[scale=0.7]{7_5.png}
	\caption{$t = 5$}
	\includegraphics[scale=0.7]{7_6.png}
	\caption{$t = 6$}
	\includegraphics[scale=0.7]{7_7.png}
	\caption{$t = 7$}
\end{minipage}
\caption{Left - Original, Right - Reconstructed}
\end{figure}
\subsection*{2.g}
We have extracted a portion of about $120 \times 240$ around the lower right corner of every frame to save time. Hence, all our results correspond to this part.
\newpage
\subsection*{2.h}
We have used the first 5 frames of the `flame' video to do the experiment. We got a relative mean squared error of 0.0010.
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{flame_coded_snapshot.png}
	\caption{Coded snapshot for flame with noise for $T = 5$}
\end{figure}
\begin{figure}[ht]
	\centering
		\includegraphics[scale=1]{flame_1.png}
		\caption{$t = 1$, Left - Original, Right - Reconstructed}
\end{figure}
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{flame_2.png}
	\caption{$t = 2$, Left - Original, Right - Reconstructed}
\end{figure}
\newpage
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{flame_3.png}
	\caption{$t = 3$, Left - Original, Right - Reconstructed}
\end{figure}
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{flame_4.png}
	\caption{$t = 4$, Left - Original, Right - Reconstructed}
\end{figure}
\begin{figure}[ht]
	\centering
	\includegraphics[scale=1]{flame_5.png}
	\caption{$t = 5$, Left - Original, Right - Reconstructed}
\end{figure}
\newpage
\section*{Question 3}
For upper limit of the bound, we can use Cauchy-Schwartz inequality:
\begin{equation*}
    |\boldsymbol{\Phi^i\Psi_j}| \leq |\boldsymbol{\Phi^i}||\boldsymbol{\Psi_j}|
\end{equation*}
which is simply dot product of two vectors is always less than product of their magnitude. Since $\boldsymbol{\Psi}$ and $\boldsymbol{\Phi}$ are unit normalized along columns and rows respectively we have:
\begin{equation*}
   \max_{i,j \epsilon \{1,2,3...,n\}}|\boldsymbol{\Phi^i\Psi_j}| \leq 1
\end{equation*}
And thus:
\begin{equation*}
   \max_{i,j \epsilon \{1,2,3...,n\}}\sqrt{n}|\boldsymbol{\Phi^i\Psi_j}| \leq \sqrt{n}
\end{equation*}
For lower limit of the bound, we use the hint by creating a unit vector:
\begin{equation*}
    \boldsymbol{g} = \sum_{k=1}^{n}\alpha_k\boldsymbol{\Psi_k}
\end{equation*}
Since $\boldsymbol{g}$ is a unit vector we have:
\begin{equation*}
    \boldsymbol{g^T}\boldsymbol{g} = (\sum_{k=1}^{n}\alpha_k\boldsymbol{\Psi_k^T})(\sum_{k=1}^{n}\alpha_k\boldsymbol{\Psi_k}) = 1
\end{equation*}
Now, since $\boldsymbol{\Psi}$ is orthonormal, dot product of columns of $\boldsymbol{\Psi}$ is zero when columns are different and equal to 1 if they are same. Thus we get the condition:
\begin{equation*}
     \sum_{k=1}^{n}\alpha_k^2 = 1
\end{equation*}
Now coherence between two matrices $\boldsymbol{\Phi}$ and $\boldsymbol{\Psi}$ is defined as the maximum value of dot product between row vectors of $\boldsymbol{\Phi}$ and column vectors of $\boldsymbol{\Psi}$. If we take $\boldsymbol{\Phi}$ to be a vector we need maximum dot product of the vector with the columns of $\boldsymbol{\Psi}$ which in other words is:
\begin{equation*}
    \mu(\boldsymbol{g}, \boldsymbol{\Psi}) = \max_{j \epsilon \{1,2,3...,n\}}\sqrt{n}\frac{|\boldsymbol{g^T\Psi_j}|}{\boldsymbol{||g||_2||\Psi_j||_2}}
\end{equation*}
Since $\boldsymbol{g}$ is a linear combination of columns of $\boldsymbol{\Psi}$ which is orthonormal also, we get:
\begin{equation*}
    \mu(\boldsymbol{g}, \boldsymbol{\Psi}) =  \max_{j \epsilon \{1,2,3...,n\}}\sqrt{n}\frac{|\alpha_j|}{||\boldsymbol{g||_2}}
\end{equation*}
The equation finally reduces to:
\begin{equation*}
    \mu(\boldsymbol{g}, \boldsymbol{\Psi}) =  \max_{j \epsilon \{1,2,3...,n\}}\sqrt{n}\frac{|\alpha_j|}{\sum_{k=1}^{n}\alpha_k^2} = \max_{j \epsilon \{1,2,3...,n\}}\sqrt{n}|\alpha_j|
\end{equation*}
The minimum value of the above expression can only be achieved when all the elements are equal which gives the lower limit of coherence:
\begin{equation*}
    n\alpha^2 = 1 \implies |\alpha| = 1/\sqrt{n}
\end{equation*}
This is very clear to see, let us assume if we decrease one of the parameters from $1/\sqrt{n}$ a little bit then other parameters will have to increase in order to satisfy the unit magnitude constraint and when max is taken across all elements then we will get some value greater than $1/\sqrt{n}$, whereas if we increase one of the parameters then others will have to decrease and max across all of them will again increase. Thus the minimum value is $\mu(\boldsymbol{g}, \boldsymbol{\Psi}) = 1$ when, $\boldsymbol{g} = 1/\sqrt{n}\sum_{k=1}^{n}\boldsymbol{\Psi_k}$

\section*{Question 4}
\subsection*{4.a}
If $\boldsymbol{x}$ is 1-sparse signal and only one measurement is taken then we won't be able to estimate $\boldsymbol{x}$ as we don't know which index element was non-zero. Whereas, if the index is known then the measurement can be divided by the value at the column of the measurement vector($n\times1$, as only one measurement is taken) corresponding to given index to uniquely determine $\boldsymbol{x}$.
\subsection*{4.b}
If two measurements are taken then also uniqueness is not guaranteed but we might get unique solution in special cases. Let us say some j-th index of $\boldsymbol{x}$ was non-zero while others were zero. Then we will get two measurements whose values are:
\begin{align*}
    m_1 = \Phi_{1j}x_j\\
    m_2 = \Phi_{2j}x_j
\end{align*}
If we divide the measurements we can get ratio of $\Phi_{1j}/\Phi_{2j}$ and check which column of $\boldsymbol{\Phi}$ has elements with this ratio. The problem is, that there might be more than one column satisfying this relation which will lead to many estimations of $x_j$ and hence compromising uniqueness.
\subsection*{4.c}
When $\boldsymbol{x}$ is 2-sparse and three measurements are taken, then let i-th and j-th element of signal have non-zero elements:
\begin{align*}
    m_1 = \Phi_{1i}x_i+\Phi_{1j}x_j\\
    m_2 = \Phi_{2i}x_i+\Phi_{2j}x_j\\
    m_3 = \Phi_{3i}x_i+\Phi_{3j}x_j
\end{align*}
This equation can be re-written in more readable form as:
\begin{align*}
    \begin{bmatrix}
    m_1 \\
    m_2 \\
    m_3 \\
    \end{bmatrix} =
    \begin{bmatrix}
    \Phi_{1i} \\
    \Phi_{2i} \\
    \Phi_{3i} \\
    \end{bmatrix}x_i +
    \begin{bmatrix}
    \Phi_{1j} \\
    \Phi_{2j} \\
    \Phi_{3j} \\
    \end{bmatrix}x_j
\end{align*}
In other words, we need to check which two columns of $\boldsymbol{\Phi}$ are co-planar with the measurement vector. But here we have a problem, which is not such $\boldsymbol{\Phi}$ exists where this condition is satisfied by only two columns because any four 3-D vectors have to be linearly dependent. This is explained below:
\begin{align*}
    \begin{bmatrix}
    \Phi_{1p} \\
    \Phi_{2p} \\
    \Phi_{3p} \\
    \end{bmatrix}x_p +
    \begin{bmatrix}
    \Phi_{1q} \\
    \Phi_{2q} \\
    \Phi_{3q} \\
    \end{bmatrix}x_q +
    \begin{bmatrix}
    \Phi_{1r} \\
    \Phi_{2r} \\
    \Phi_{3r} \\
    \end{bmatrix}x_r +
    \begin{bmatrix}
    \Phi_{1s} \\
    \Phi_{2s} \\
    \Phi_{3s} \\
    \end{bmatrix}x_s = 0
\end{align*}
Thus measurement vector can always be written in two forms for any four set of 3-D vectors which is:
\begin{align*}
    \begin{bmatrix}
    m_1 \\
    m_2 \\
    m_3 \\
    \end{bmatrix} =
    \begin{bmatrix}
    \Phi_{1i} \\
    \Phi_{2i} \\
    \Phi_{3i} \\
    \end{bmatrix}x_i +
    \begin{bmatrix}
    \Phi_{1j} \\
    \Phi_{2j} \\
    \Phi_{3j} \\
    \end{bmatrix}x_j =
    -\begin{bmatrix}
    \Phi_{1k} \\
    \Phi_{2k} \\
    \Phi_{3k} \\
    \end{bmatrix}x_k -
    \begin{bmatrix}
    \Phi_{1l} \\
    \Phi_{2l} \\
    \Phi_{3l} \\
    \end{bmatrix}x_l
\end{align*}
We see for $x_i$ and $x_j$ we can always cook up some $x_k$ and $x_l$ such that the above constraint gets satisfied. Hence, no such instance of $\boldsymbol{\Phi}$ is there where $\boldsymbol{x}$ can be uniquely estimated.

\subsection*{4.d}
When $\boldsymbol{x}$ is 2-sparse and four measurements are taken, then let i-th and j-th element of signal have non-zero elements:
\begin{align*}
    m_1 = \Phi_{1i}x_i+\Phi_{1j}x_j\\
    m_2 = \Phi_{2i}x_i+\Phi_{2j}x_j\\
    m_3 = \Phi_{3i}x_i+\Phi_{3j}x_j\\
    m_4 = \Phi_{4i}x_i+\Phi_{4j}x_j\\
\end{align*}
This equation can be re-written in more readable form as:
\begin{align*}
    \begin{bmatrix}
    m_1 \\
    m_2 \\
    m_3 \\
    m_4 \\
    \end{bmatrix} =
    \begin{bmatrix}
    \Phi_{1i} \\
    \Phi_{2i} \\
    \Phi_{3i} \\
    \Phi_{4i} \\
    \end{bmatrix}x_i +
    \begin{bmatrix}
    \Phi_{1j} \\
    \Phi_{2j} \\
    \Phi_{3j} \\
    \Phi_{4j} \\
    \end{bmatrix}x_j
\end{align*}
In this case previous situation does not arise as four vectors in 4-D can be linearly independent. Hence we need to check which two columns of $\boldsymbol{\Phi}$ are co-planar with measurement vector. But there might be a coincidence that more than two columns of $\boldsymbol{\Phi}$ are co-planar with measurement vector in which unique estimation of $\boldsymbol{x}$ is not possible. Nevertheless, since $\boldsymbol{\Phi}$ is randomly generated matrix the chance of more than two columns of $\boldsymbol{\Phi}$ to satisfy the condition exactly is less but in some cases there might be more than one solution for $\boldsymbol{x}$. Algorithm to determine $\boldsymbol{x}$ is:
\begin{verbatim}
1) Pick any two measurements say 1st and 2nd.
2) Loop over the columns of the matrix \Phi.
3) For any column p, pick any column q such that q>p, for all such q.
4) Solve the two linear equations for x_i and x_j with coefficients from the
1st and 2nd row of \Phi_p and \Phi_q.
5) Put the calculated values in the third equation and fourth equation, i.e.
with coefficients from the third and fouth row and check if the calculated value
is equal to m_3.
\end{verbatim}
If more than two columns of $\boldsymbol{\Phi}$ pass the algorithm then $\boldsymbol{x}$ is not uniquely determinable. Otherwise, calculated $x_i$ and $x_j$ are the values with the column indices as their location in $\boldsymbol{x}$.
\clearpage

\section*{Question 5}
\subsection*{5.a}
Both the papers use sensors to compute coded snapshots. Hitomi's paper changes the code for each frame by sending signals to sensors directly whereas CACTI uses mechanical translation to compute a coded snapshot for several frames. Making uses of mechanics helps in reducing the power consumption and operating bandwidth of the camera. CACTI captures an object's spatiotemporal-multiplexed information. This mechanical translation helps in compressing temporal sampling without substantially increasing system volume or power. \\
The main difference between Hitomi's and CACTI is that Hitomi's paper tries to achieve  high spatial and spectral resolution whereas CACTI tries to achieve high spatial, spectral and temporal resolution by using the fact that the information present in natural images are highly correlated. In Hitomi's, the code/mask is fixed for an integration time whereas in CACTI, the mask changes continuously.
\begin{figure}[ht]
	\centering
	\includegraphics[scale=0.75]{CACTI.png}
	\caption{CACTI}
\end{figure}
\vspace{2cm}
\begin{center}
	\textit{Please turn over}
\end{center}
\newpage
\begin{figure}[ht]
	\centering
	\includegraphics[scale=0.75]{Hitomi.png}
	\caption{Hitomi's}
\end{figure}
\subsection*{5.b}
This paper solves the following unconstrained optimization problem:
$$\arg \min_{\mathbf{f}}\left\lVert\mathbf{g - Hf}\right\rVert^2 + \lambda \mathbf{\Omega}(\mathbf{f}) $$
Here, for an $N$-pixel active sensor area,
\begin{itemize}
\item $\mathbf{g} \in \mathbb{R}^N$ is the sensed data
\item $\mathbf{f} \in \mathbb{R}^{\sqrt{N} \times \sqrt{N} \times N_F}$ is a $(\sqrt{N} \times \sqrt{N}) \times N_F$ voxel spatiotemporal datacube i.e., the scene.
\item $\mathbf{H} \in \mathbb{R}^{N\times NN_F}$ is the system’s discrete forward matrix that accounts for sampling factors
including the optical impulse response, pixel sampling function, and time-varying transmission
function. The forward matrix is a 2-dimensional representation of the 3-dimensional transmission function $\mathbf{T}$
$$\mathbf{H}_k \stackrel{\text{def}}{=} \text{diag} \bigg[T_{1,1,k}, T_{2,1,k}, \cdots, T_{\sqrt{N}, \sqrt{N}, k}\bigg], \:\: k = 1, \ldots, N_F$$
$$\mathbf{H} \stackrel{\text{def}}{=} [\mathbf{H}_1\big| \mathbf{H}_2\big| \cdots \big| \mathbf{H}_{N_F} ]$$
\item $\lambda$ is a regularizer. This is chosen using experimental optimization.
\item $\mathbf{\Omega}(\mathbf{f})$ is the total variation of $\mathbf{f}$
$$\mathbf{\Omega}(\mathbf{f}) = \sum \limits_{k}^{N_F} \sum \limits_{i,j}^N \sqrt{(f_{i+1,j,k} - f_{i,j,k})^2 + (f_{i,j+1,k} - f_{i,j,k})^2}$$
\end{itemize}
\section*{Question 6}
\subsection*{6.a}
\begin{itemize}
    \item Title - \textbf{Sparse imaging for fast electron microscopy}
    \item Authors - \textbf{Hyrum S. Anderson}, \textbf{ Jovana Ilic-Helms}, \textbf{Brandon Rohrer}, \textbf{Jason Wheeler} and \textbf{Kurt Larson}
    \item Place - \textbf{Sandia National Laboratories}
    \item Link - \href{https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/1210508c.pdf}{https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/1210508c.pdf}
\end{itemize}
\subsection*{6.b}
The scanning electron microscope acquires image by raster sampling (images in old phones or video games using dot matrix structure) focused electron beam. At each location, electrons in the incident beam interacts with sample, producing various signals about the composition or topography of the sample’s surface. These signals are  detected and digitally assigned to the image pixel value at the corresponding sample location. The electron probe is then repositioned via electromagnetic or electrostatic deflection to the subsequent pixel location.\\
In order to produce high-quality SEM images, long (on the order of microseconds) integration times per pixel are required to reduce noise. In many non-trivial cases detector response times and other factors necessitate very long integration time. In sum, well-engineered systems are SNR-limited in their data acquisition speed, and can require months to collect millimeters or centimeters of data for some applications.\\
The degrees of freedom of typical electron microscope images are many fewer than the number of image pixels. Hence, they focused the electron beam at random image points and acquired some M measurements, thereby reconstructing the image using BP.
\subsection*{6.c}
The reconstruction technique involves minimizing TVF norm with slight variation. The equations goes as:
\begin{align*}
    \boldsymbol{\textbf{y} = \Phi\textbf{x}+\textbf{n}}
\end{align*}
where \textbf{n} is noise vector $\bold\Phi$ is is a subset of rows of identity \textbf{I} and \textbf{n} is noise with power $\sigma^2$.
The image is obtained by solving regularized basis pursuit:
\begin{align*}
    \min_{x} \boldsymbol{||\Psi^Tx||_1 + ||\nabla x||_1}\\
    s.t. \ \boldsymbol{||y - \Phi x|| \leq  \sigma^2}
\end{align*}
 where  $\boldsymbol{||\nabla x||_1 = \sum_i\sqrt{|\nabla_hx_i|^2+|\nabla_vx_i|^2}}$. By the low mutual coherence
between the DCT basis and image-domain sampling, $\boldsymbol{\Psi}$ is chosen to be a block-DCT basis  with $32 \times 32$ pixel blocks.
\end{document}


\documentclass{article}
\usepackage{parskip}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{listings}

\setlength{\parindent}{0cm}

\begin{document}

\lstset{language=Haskell, frame=single}

\title{CPSC 449 Assignment 2 \\ J. Gallagher T01 MW 1400-1450}
\author{Andrew Helwer}
\date{Winter 2011}
\maketitle

\thispagestyle{myheadings}

\section{Programming: Questions 1-3}

\textbf{Question 1a}

Purpose: to square all odd items in a list of integers

Parameters: a list of integers

Preconditions: none

Returns: a list of integers

\begin{lstlisting}
oddSquaresA :: [Int]->[Int]
oddSquaresA xs = [x*x | x<-xs, (odd x)]
\end{lstlisting}

\textbf{Question 1b}

Purpose: to square all odd items in a list of integers

Parameters: a list of integers

Preconditions: none

Returns: a list of integers

\begin{lstlisting}
oddSquaresB :: [Int]->[Int]
oddSquaresB [] = []
oddSquaresB (x:xs)
    | (odd x)	= (x*x):(oddSquaresB xs)
    | otherwise	= (oddSquaresB xs)
\end{lstlisting}

\newpage

\thispagestyle{myheadings}
\markright{10023875}

\textbf{Question 2 (List Comprehension)}

Purpose: to remove all duplicates from a set of integers

Parameters: a list of integers

Preconditions: none

Returns: a list of unique integers

\begin{lstlisting}
uniqueA :: [Int]->[Int]
uniqueA xs = [x | x<-xs, (isUniqueIn x xs)]
\end{lstlisting}

\textbf{Question 2 (Primitive Recursion)}

Purpose: to remove all duplicates from a set of integers

Parameters: a list of integers

Preconditions: none

Returns: a list of unique integers

\begin{lstlisting}
uniqueB ::[Int]->[Int]
uniqueB [] = []
uniqueB (x:xs)
    | (contains x xs)	= (uniqueB (removeAll x xs))
    | otherwise		= x:(uniqueB xs)
\end{lstlisting}

\textbf{Question 3a}

Purpose: to merge two sorted lists such that the end result is sorted

Parameters: two lists of integers to be merged

Preconditions: both lists of integers are sorted in ascending order

Returns: a sorted list of integers

\begin{lstlisting}
mergeLists :: [Int]->[Int]->[Int]
mergeLists [] [] = []
mergeLists [] ys = ys
mergeLists xs [] = xs
mergeLists (x:xs) (y:ys)
    | x<=y		= x:(mergeLists xs (y:ys))
    | otherwise		= y:(mergeLists (x:xs) ys)
\end{lstlisting}

\newpage

\pagestyle{myheadings}

\textbf{Question 3b}

Purpose: to split a list into two lists, one of even indices one of odd

Parameters: a list of integers to be split

Preconditions: none

Returns: a tuple of two integer lists whose union is the original list

\begin{lstlisting}
splitList :: [Int]->([Int],[Int])
splitList [] = ([],[])
splitList (x:xs)
    | (xs==[])	= ([x],[])
    | otherwise	= (x:(skipOne (tail xs)),(skipOne xs))
\end{lstlisting}

Purpose: to generate a list from an input list by taking every second element

Parameters: a list of integers

Preconditions: none

Returns: a list of integers

\begin{lstlisting}
skipOne :: [Int]->[Int]
skipOne [] = []
skipOne (x:xs)
    | (xs==[])		= [x]
    | otherwise		= x:(skipOne (tail xs))
\end{lstlisting}

\textbf{Question 3c}

Purpose: to sort a list of integers with mergesort

Parameters: a list of integers

Preconditions: none

Returns: a list of integers

\begin{lstlisting}
mSort :: [Int]->[Int]
mSort [] = []
mSort (x:xs)
    | (xs==[])	= [x]
    | otherwise	= mergeLists(mSort(x:(skipOne(tail xs))))
				    (mSort(skipOne xs))
\end{lstlisting}

\newpage

\textbf{Useful functions written by myself:}

Purpose: to check if a list of integers contains a certain element

Parameters: an element to be checked and a list of integers

Preconditions: none

Returns: True or False if the element is or is not in the list

\begin{lstlisting}
contains :: Int->[Int]->Bool
contains x xs
    = case xs of
	   [] -> False
	   (y:ys) -> (x==y) || (contains x ys)
\end{lstlisting}

Purpose: to find the number of instances of an element in a list

Parameters: an element to be checked and a list of integers

Preconditions: none

Returns: an integer specifying number of occurences in the list

\begin{lstlisting}
instancesOf :: Int->[Int]->Int
instancesOf n [] = 0
instancesOf n (x:xs)
    | (n==x)		= 1 + (instancesOf n xs)
    | otherwise		= (instancesOf n xs)
\end{lstlisting}

Purpose: to determine if there is exactly one occurence of an element in a list

Parameters: an element to be checked and a list of integers

Preconditions: none

Returns: True or False if the element is or is not unique

\begin{lstlisting}
isUniqueIn :: Int->[Int]->Bool
isUniqueIn x xs = (instancesOf x xs) == 1
\end{lstlisting}

\newpage

Purpose: to remove all instances of an element in a list

Parameters: an element to be removed and a list of integers

Preconditions: none

Returns: a list with all occurences of the specified element gone

\begin{lstlisting}
removeAll :: Int->[Int]->[Int]
removeAll n [] = []
removeAll n (x:xs)
    | (x==n)		= (removeAll n xs)
    | otherwise		= x:(removeAll n xs)
\end{lstlisting}

\section{Written Proof: Question 4}

\textbf{(a)}

Base Case: $P([\:])$

Induction Step: $P(xs) \Rightarrow P(x:xs)$

\textbf{(b)}

Proving the base case $P([\:])$, $\forall \: n \in \mathbb{Z}$:

\begin{align*}
&\text{ or (match n [ ])} \\
&\rightsquigarrow \text{or ([ ])}			&(match.1) \\
&\rightsquigarrow \text{False}				&(or.1)
\end{align*}

Thus \textbf{or (match x [ ]) = False}.

\begin{align*}
&\text{ elem n [ ]} \\
&\rightsquigarrow \text{False}				&(elem.1)
\end{align*}

Thus \textbf{elem x [ ] = False} as well, and so the base case holds.

\newpage

\textbf{(c)}

Proving the inductive step, $P(xs) \Rightarrow P(x:xs)$:

Assume $P(xs)$ is true. Then \textbf{or (match n xs) = elem n xs}, $\forall \:
n \in \mathbb{Z}$.

Consider $P(x:xs)$:

\begin{align*}
&\text{ or (match n (x:xs))} \\
&\rightsquigarrow \text{ or ((n==x):(match n xs))}	&(match.2) \\
&\rightsquigarrow \text{ (n==x)$||$(or (match n xs))}	&(or.2)
\end{align*}

Thus \textbf{or (match n (x:xs)) = (n==x)$||$(or (match n xs))}.

\begin{align*}
&\text{ elem n (x:xs)} \\
&\rightsquigarrow \text{ (n==x)$||$(elem n xs)}		&(elem.2) \\
\end{align*}

Thus \textbf{elem n (x:xs) = (n==x)$||$(elem n xs)}. So we have:

\begin{align*}
&\text{elem n (x:xs) = (n==x)$||$(elem n xs)} \\
&\text{elem n (x:xs) = (n==x)$||$(or (match n xs))}	&\text{by }P(xs) \\
&\text{elem n (x:xs) = or (match n (x:xs))} \\
\end{align*}

So $P(xs) \Rightarrow P(x:xs)$, and by structural induction the equality
\textbf{or (match n xs) = elem n xs} holds $\forall \: n \in \mathbb{Z}$ and
$\forall$ finite lists xs. $\square$

\end{document}


\documentclass[12pt]{article}

\title{Informática 1A}
\author{Abel Doñate}
\date{}

\usepackage{amsmath}
\usepackage{xcolor}

%Define the comand \code and \codecolor for programming font
\definecolor{light-gray}{gray}{0.95}
\newcommand{\code}[1]{\colorbox{light-gray}{\texttt{\textcolor{blue}{#1}}}}
\newcommand{\codeorange}[1]{\colorbox{light-gray}{\texttt{\textcolor{orange}{#1}}}}

\begin{document}

\maketitle
\tableofcontents
\newpage

\section{Console Shortcuts}
\begin{itemize}
	\item \codeorange{g++ program.cpp} to compile the program
	\item \codeorange{./a.out} to run the program
	\item \codeorange{python3 program.py}
	\item \codeorange{Ctr+c} breaks a \code{while()} loop
	\item \codeorange{Ctr+shift+c} copies information
	\item \codeorange{Ctr+shift+v} pastes information
\end{itemize}

\section{Include and Using}
\subsection{Using}
\code{using namespace std}
\subsection{Include}

To include all the standard libraries \code{\#include$<$bits/stdc++.h$>$} (DO NOT DO THAT)
\begin{itemize}
  \item \code{\#include$<$iostream$>$}
  \item \code{\#include$<$cmath$>$}
  	\begin{itemize}
  		\item \code{sqrt()}
  		\item \code{pow()}
  	\end{itemize}

  \item \code{\#include$<$algorithm$>$}
  	\begin{itemize}
  		\item \code{sort()}
  		\item \code{reverse(v.begin(), v.end())} reverses the string or vector
  	\end{itemize}

  \item \code{\#include$<$iomanip$>$}
  	\begin{itemize}
  		\item \code{cout$<<$setprecision(n)$<<$fixed} fixes n decimal places in doubles.
  	\end{itemize}

  \item \code{\#include$<$string$>$}
  	\begin{itemize}
  		\item \code{push\_back()} adds a char to the string.
  		\item \code{reverse(b.begin(), b.end())} reverses the string or vector (algorithm)
  	\end{itemize}

\end{itemize}

\section{Conversions}
\begin{itemize}
	\item From string to int \\
	\code{(int)s} s is the string
	\item From char to int \\
	\code{(int)c -'0'} c is the char
	\item From int to string \\
	\code{to\_string(n)} n is the int
	\item From int to char \\
	\code{'0' + n} n is the int
	\item From string to char \\
	\code{s[0]} s is the string
\end{itemize}

\section{Short	cuts}
\begin{itemize}
	\item \code{i++}
	\item \code{sum += v[i]}
\end{itemize}

\section{Vectors and matrices}
To define a vector \code{vector<int> v(size, default inputs)} \\
Similar with a matrix \code{vector<vector<int>> v(size x, vector<int>(size y))} \\
Usefull functions:
\begin{itemize}
	\item \code{v.size()} returns the size of the vector
	\item \code{v.push\_back()} adds a term at the end of the vector
	\item \code{v.pop\_back()}  removes the last term of the vector
	\item \code{v.clear()} cleans the vector
	\item \code{sort(v.begin, v.end)} sorts the vector
	\item \code{reverse(v.begin, v.end)} reverses the vector
	\item \code{swap(v[i], v[j])} swaps i and j term
\end{itemize}
To define a new type of valiable \code{typedef vector<int> vec} \\ or \code{using Row = vector<int>; using Matrix = vector<Row>;}

\section{Fundamental algorithms}
	\subsection{MergeSort}

	\subsection{structs}
	We can define our struct of Person as follows:\\
	\code{struct Person\{ string name; int age \};} \\ \\
	\code{bool comp(Person a, Person b)\{return a.age < b.age\} }\\
	\code{vector<Person> p;}\\
	\\
	Now we can sort \code{p} by ages: \\
	\code{sort(p1.begin(), p1.end(), comp)}

\section{Python}
Remember to construct the function \code{def main()} and call it later.
\subsection{Input and output}
\subsubsection{Output}
For the output \code{print()}
\subsubsection{Input}
For the input first \code{from easyinput import read}. Then \code{read(\textit{type})} reads the input of the given type.\\
If you want to stop if no parameter is read use \code{while input != None}

\subsection{Fix Decimals}
To format the float a with n decimals
\code{a.formatted = "\{:.nf\}".format(a)}

\subsection{Conversions}
\begin{itemize}
	\item From int to string \\
	\code{str(n)} n is the int
	\item From string to int \\
	\code{int(s)} s is the string
\end{itemize}

\section{Tips for the Exam}
Here are some tips for the exam
\begin{itemize}
	\item Local valiables
	\item Minimum posible cases
	\item Do not use \code{break}. Instead, use a \code{bool}
	\item One line \code{if()}
\end{itemize}

\end{document}