%---Define your document class, packages, and new commands

\documentclass[11pt]{article}
\usepackage[top=2.54cm, left=2.54cm, right=2.54cm, bottom=2.54cm]{geometry}

\usepackage{amsfonts}   % for statements to be commented out
\usepackage{amssymb}
\usepackage{amsmath}    % for subequations
\usepackage{setspace}
\usepackage{graphicx}

%------------Document-Begins-Here----------------------%

\begin{document}

\title{\textbf{Orbital Patterns of Martian Moons}}

\author{Jake Mathews
\thanks{mathewsj2@wit.edu}\\
Computer Science Undergraduate Program, Wentworth Institute of Technology\\
550 Huntington Avenue, Boston, MA 02115\\
\and Ian Brown
\thanks{browni1@wit.edu}\\
Computer Engineering Undergraduate Program, Wentworth Institute of Technology\\
550 Huntington Avenue, Boston, MA 02115}

\maketitle

\begin{abstract}

This project's goal was to to create a top-down 2-dimentional model
of the positional path of the moons of Mars, Phobos and Deimos.
We created a Fortran 90 program to simulate the orbit of the two moons
and generate data files for later analysis.
The simulation will be provided some initial conditions of a given moon,
and simulate the orbit over the course of one orbital period, as defined
by NASA~\cite{nasa}

\end{abstract}

% Write your introduction here, include pictures, tables, equations, etc...
\section{Introduction}
An object in orbit is a simple kinematics problem illustrated in Fig.~\ref{fig:centripetal}

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.6\textwidth, angle =0]{../images/centripetal-force}
  \caption{Centripetal force on an object with a velocity vector}
  \label{fig:centripetal}
\end{figure}
\noindent
In Fig.~\ref{fig:centripetal} there is an object with an intial velocity vector
and a centripetal force pulling it towards the center. The black circle represents
the objects path over time. In the case of an orbit, the velocity vector is always
perpendicular to the centripetal force vector. This is what the program is attempting to simulate,
where the orbiting object is each of the Mars' moons, the center object is Mars
iteself, and the centripetal force providing the acceleration to change each moon's
velocity vector is gravity.

\vspace{\baselineskip} \noindent
The program will need some initial conditions to be set before simulating
the motion of Phobos and Deimos such as the initial position, the initial
velocity, moon mass, Mars mass, gravitational constants, as well as time steps and a maximum
simulation period. Initial velocity and gravitational constants are calculated
by the program based of the properties in Table~\ref{table:moon-properties}.

\begin{table}[ht]
  \centering
  \begin{tabular}{|r|c|c|}
  \hline
  Moon Orbit Properties                & Phobos  & Deimos  \\ \hline
  Semi-Major Axis ($km$)                 & 9378    & 23459   \\ \hline
  Mass ($10^{15} kg$) & 1015    & 2.4     \\ \hline
  Orbital Period ($Earth\ days$)          & 0.31891 & 1.26244 \\ \hline
  \end{tabular}
  \label{table:moon-properties}
  \caption{Properties of Phobos and Deimos as described by NASA~\cite{nasa}}
  \end{table}

\noindent
The initial $x-y$ position is defined by each moons semi-major axis.
The semi-major axis is half the length of the major axis of an ellipse.
This is illustrated in Fig.~\ref{fig:semi-major-axis} with both
Phobos and Deimos labled.

\begin{figure}[ht]
  \centering
  \includegraphics[width=1.0\textwidth, angle =0]{../images/semi-major-axis}
  \caption{A depiction of the semi-major axis of both Phobos and Demimos}
  \label{fig:semi-major-axis}
\end{figure}

% Do ALL math derivations here...label each equation and reference the
% equations accordingly... For example:  From Eq. (2), we see that Eqs. (3) - (7)
% can be simplified, blah, blah...
\section{Theory}

\noindent The first step in solving for the system's X and Y coordinates involves calculating a relative Gravitational Constant for each moon.

\begin{equation}
\label{eq:gravConstant}
 G = 4\pi ^{2}*\frac{R^{3}}{Year_{Moon}*M_{System}}~.
\end{equation}

\noindent Where $R$ is the semi-major axis, $Year_{Moon}$ is a time in seconds, and $M_{System}$ is the total mass of the Mars and the considered Moon, the following values may be plugged in and calculated:

\begin{table}[ht]
  \centering
  \begin{tabular}{|r|c|c|c|c|}
  \hline
                             & $R_{Moon}  (m)$  &  $Year_{Moon} (s)$ & $Mass (kg)$ & $Gravity  (N*m^{2}*kg^{-2})$ \\ \hline
  Phobos                 & $9.378*10^{6}$  & $27553.82$ &  $10.6*10^{15}$ & $4.2887*10^{13}$ \\ \hline
  Deimos 		     & $23.459*10^{6}$  & $109074.81$ &  $2.4*10^{15}$ & $4.283910*10^{13}$   \\ \hline
  Mars		     &                             &                    & $6.4171*10^{23}$  &\\ \hline
  \end{tabular}
  \label{table:lunar-properties}
  \caption{More Lunar Properties of The Moons as described by NASA~\cite{nasa}}
\end{table}

\noindent While the initial position doesn't need to be calculated due to our known semi-major axis. The initial velocity can be solved by equating the following two equations:

\begin{equation}
\label{eq:netForce}
 F_{Net}=\frac{M_{Moon}*v^{2}}{R}
\end{equation}

\begin{equation}
\label{eq:gravForce}
 F_{Gravity}=\frac{G_{Moon}}{R^{2}}
\end{equation}

\noindent Solving for velocity gives you the equation:

\begin{equation}
\label{eq:velocity}
V_{Moon} = \sqrt{\frac{Gravity_{Moon}}{R_{Moon}}}
\end{equation}

\noindent For every equation, you need to explain each variable, for example:

\begin{equation}
\label{eq:force}
F=\dfrac{mv^2}{r}~.
\end{equation}
\noindent In Eq.~\eqref{eq:force}, $F$ is the force measured in Newtons, $m$ is the
mass in kilograms, $v$ is the velocity measured in meters per second, and $r$ is
the radius of the curved path.  Equation~\eqref{eq:force} was obtain from~\cite{uni}.

\section{Computational Methods \& Techniques}
\noindent Once Parameter Calculations are completed the $(x, y)$ coordinates of the satelites must then be calculated using Runge Kutta systems ~\eqref{eq:rk2} and ~\eqref{eq:rk4}. This is a simple step summation used to simulate integration by properties of the right hand rule described as:

\begin{equation}
\label{eq:rhr}
\sum_{t=0}^{year}y(t)*dt
\end{equation}

\noindent The steps of Runge Kutta RK1-RK4 are as follows:

\begin{equation}
\label{eq:rk1}
k_{1}= \Delta t\vec{f}(\vec{y}^{n},t)
\end{equation}

\begin{equation}
\label{eq:rk2}
k_{2}= \Delta t\vec{f}(\vec{y}^{n} + \frac{1}{2}k_{1},t + \frac{1}{2}\Delta t)
\end{equation}

\begin{equation}
\label{eq:rk3}
k_{3}= \Delta t\vec{f}(\vec{y}^{n} + \frac{1}{2}k_{2},t + \frac{1}{2}\Delta t)
\end{equation}

\begin{equation}
\label{eq:rk4}
k_{4}= \Delta t\vec{f}(\vec{y}^{n} + k_{3}, t + \Delta t)
\end{equation}

\noindent In these differential systems the included calculation of the previous level show each step of the approximation system. From a basic vector in ~\eqref{eq:rk1} from the given condition, you then have a continuation off of the trajectory as ~\eqref{eq:rk2}. A better midpoint may then be calculated using ~\eqref{eq:rk3} (usually falling on the other side of the curve as ~\eqref{eq:rk2}) and ~\eqref{eq:rk4} shows a significant increase in accurace compared to just ~\eqref{eq:rk2}.

\section{Results}
\noindent While the primary printout of the results can be found in a data file containing approximately 100,000 data points, the full orbits can be seen visually in Fig. ~\ref{fig:orbit-diagram}.
\begin{figure}[ht]
\centering
\includegraphics[width=1.0\textwidth, angle =0]{../images/orbits}
\caption{Orbital period in the $x-y$ plane for one full orbit of Phobos and Deimos orbiting Mars.}
\label{fig:orbit-diagram}
\end{figure}

\noindent With Mars at position $(0,0)$ in this system both orbits can be seen to have quite clean elliptical patterns. They look so perfect that you would assume that there would be more interference between the two of them having a two large satelites orbiting the same planet.

\section{Conclusions}
\noindent  The first check we can do is to make sure the energy in the system is conserved. By adding the Kinetic and Potential energy at each point for both systems we can get a figure of total energy over time.

\begin{figure}[ht]
\centering
\includegraphics[width=1.0\textwidth, angle =0]{../images/energy}
\caption{Total Energy of Phobos and Deimos over time.}
\label{fig:energy}
\end{figure}

\noindent What we find is that our systems must be correct as the total energy in the systems remain constant; following a standard gravitational constant formula with the masses removed such as:

\begin{equation}
\label{relGravity}
acceleration = \frac{G}{R^2}
\end{equation}

\noindent where $G$ is the universal gravitational constant $6.67*10^-11 (N^{-11}*m^{3}*kg^{-1}*s^{-2}$, and the distance between them being their closest point at the semi-major axis where they are less affected by the planet $R = Semi-Major_{Deimos}-Semi-Major_{Phobos}$ the acceleration developed is still only about $4.73*10^{-15}$ Which explains why energy is conserved in the system.

\begin{thebibliography}{99}

\bibitem{uni}
Douglas C. Giancoli
\textit{Physics for Scientists and Engineers}.
Pearson Education Inc., Upper Saddle River, New Jersey, 2009.

\bibitem{nasa}
NASA
\textit{Mars Fact Sheet}.
NASA, 2016
nssdc.gsfc.nasa.gov/planetary/factsheet/marsfact.html

\end{thebibliography}

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{\pandas}
\label{pandas}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Basic Commands}
\label{pandas:basic}

\begin{lstlisting}[language=Python]
# IO
dfp = pd.read_csv('file.csv', header=None)
dfp['col'] = dfp['col'].astype(int)
dfp.to_csv('out.csv')

# descriptive commands
dfp.describe(); dfp.columns; dfp.shape;

# aggregation commands
dfp.sum(); dfp.cumsum();
dfp.min(); dfp.max(); dfp.idxmin(); dfp.idxmax();
dfp.mean(); dfp.std(); dfp.median(); dfp.mode();

# extract all rows from one column as df, not series
dfp_y = dfp.loc[:, ['y']]

# select by value
dfp_selection = dfp.loc[( ((dfp['x'] == x_value) & (dfp['y'] == y_value)) | (dfp['z'] < z_value))]

# select by value and assign new value
dfp.loc[(dfp['x'] == x_value), 'y'] = y_value

# select with query
dfp_selection = dfp.query('0 < x')

# select with isin
dfp_selection = dfp.isin({'x': list2, 'y': list2})

# select row by index
series = dfp.iloc[0]

# apply an arbitrary function
def func(x, y):
	return x*np.sin(y)
dfp['z'] = np.vectorize(func)(dfp['x'], dfp['y'])

# iterate through rows - slow!
for index, row in dfp.iterrows():
	x_value = row['x']

# construct df from rows
rows_list = []
for nrow in range(nrows):
	rows_list.append({'x':x_value, 'y':y_value})
dfp = pd.DataFrame(rows_list)
dfp = dfp[['x', 'y']]

# rename columns
dfp = dfp.rename({'old': 'new'}, axis='columns')

# sort
dfp = dfp.sort_values(by=['x', 'y'], ascending=[True, False]).reset_index(drop=True)

# group by, while dropping new count column and duplicates
dfp = dfp.groupby(['x', 'y', 'z']).size().to_frame(name = 'count').reset_index().drop(['count'], axis=1).drop_duplicates()

# return duplicate rows
columns_to_check_for_duplicates = ['x', 'y']
dfp_duplicates = dfp[dfp.duplicated(subset=columns_to_check_for_duplicates, keep=False)]

# shuffle rows
dfp = dfp.sample(frac=1., replace=False, random_state=rnd_seed).reset_index(drop=True)

# drop columns
dfp = dfp.drop(['col_to_drop1', 'col_to_drop2'], axis=1)

# union dataframes
dfp = pd.concat([dfp_1, dfp_2], ignore_index=True)

# fill nans, for all columns and per column
dfp = dfp.fillna(0.0)
dfp = dfp.fillna(value={'x': x_nan_val, 'z': y_nan_val})
\end{lstlisting}

\clearpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Joining}
\label{pandas:join}

\noindent See the \href{https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html#database-style-dataframe-or-named-series-joining-merging}{documentation}
and this
\href{https://chrisalbon.com/code/python/data_wrangling/pandas_join_merge_dataframe/}{useful guide}.

\begin{lstlisting}[language=Python]
dfp = pd.merge(dfp_l, dfp_r, left_on='id_left', right_on='id_right', how='left')
\end{lstlisting}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Pivoting}
\label{pandas:pivoting}

\subsubsection{pivot}
\label{pandas:pivoting:pivot}

\noindent \href{http://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.pivot.html}{\texttt{pivot} documentation}.

\begin{lstlisting}[language=Python]
dfp.pivot(index=None, columns=None, values=None)
\end{lstlisting}

\begin{figure}[H]
\centering
\includegraphics[width=0.85\textwidth]{figures/pandas/reshaping_pivot.png}
\caption{
Example \pandas \texttt{pivot} operation, from the package \href{http://pandas.pydata.org/pandas-docs/stable/user_guide/reshaping.html}{documentation}.
}
\label{fig:pandas:pivot}
\end{figure}

\subsubsection{pivot\_table}
\label{pandas:pivoting:pivot_table}

\noindent \href{https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.pivot_table.html}{\texttt{pivot\_table} documentation}.

\begin{lstlisting}[language=Python]
pd.pivot_table(dfp, values=None, index=None, columns=None, aggfunc='mean', fill_value=None, margins=False, dropna=True, margins_name='All')
\end{lstlisting}

\begin{figure}[H]
\centering
\includegraphics[width=0.95\textwidth]{figures/pandas/pivot-table-datasheet.png}
\caption{
Example \pandas \texttt{pivot\_table} operation, by \href{http://pbpython.com/pandas-pivot-table-explained.html}{Chris Moffitt}.
}
\label{fig:pandas:pivot_table}
\end{figure}


\documentclass[a4paper,11pt]{article}

\usepackage[T1]{fontenc} \usepackage{lmodern} \usepackage[utf8]{inputenc}
\usepackage[english]{babel} \usepackage{csquotes}
\usepackage{float} \usepackage{graphicx,subfig}
\usepackage{amssymb,amsmath} %\usepackage{siunitx}
\usepackage[nodayofweek]{datetime}
\usepackage[top=3.5cm,bottom=2.5cm,left=3cm,right=3cm,headheight=30pt]{geometry}
\usepackage[style=numeric,backend=biber]{biblatex} \bibliography{refs}
\usepackage{fancyhdr} \pagestyle{fancy} \usepackage{lastpage}
\usepackage{parskip} \setlength{\parskip}{.5em} \setlength{\parindent}{1em}
\usepackage[colorlinks=true,allcolors=blue]{hyperref} \hypersetup{
	pdfauthor={Michaël Defferrard, Soroosh Shafiee},
	pdftitle={Incremental Gradient Methods},
	pdfsubject={Project proposal}
}
\lhead{Advanced Topics in Data Sciences\\ Project proposal}
\chead{\hspace{2cm}EPFL\\ \hspace{2cm}\shortdate\today}
\rhead{Michaël \textsc{Defferrard}\\ Soroosh \textsc{Shafiee}}
\cfoot{}

\newcommand{\R}{\mathbb{R}}
\newcommand{\eqnref}[1]{(\ref{eqn:#1})}

\begin{document}

\begin{center}
	\Large{\textbf{\textsc{Incremental Gradient Methods}}}
\end{center}

This project is aimed to be a way for us to better understand and thinker with
the recent advances in the Stochastic Gradient Descent algorithms, specifically
some of the newest Incremental Gradient Methods such as SAG
\cite{schmidt_minimizing_2013}, SVRG \cite{johnson_accelerating_2013} and SAGA
\cite{defazio_saga_2014}. This class of algorithms have been developed to solve
problems of the form
\begin{equation} \label{eqn:problem}
	\min_{x \in \R^d} \frac{1}{n} \sum_{i=1}^n f_i(x) + h(x),
\end{equation}
where each $f_i$ is convex and has Libschitz continuous derivatives with
constant $L$ or is strongly convex with constant $\mu$; and $h$ is a convex but
potentially non-differentiable function (his proximal operator is however easy
to compute). While computing the full gradient would be prohibitive due to large
$d$ and $n$, these iterative stochastic algorithms reduce the computational cost
of optimization by only computing the gradient of a subset of the functions
$f_i$ at each step.

Many machine learning problems can be cast in \eqnref{problem}, such as
(constrained) Least-Square or Logistic Regressions with $\ell_1$ or $\ell_2$
regularization; where $x$ would represent the model parameters, $f_i$ the data
fidelity term applied to a particular sample $i$, and $h$ a regularization or
indicator function of a convex set. As such, these methods are of use in our
respective domains of expertise: Signal Processing on Graphs and Risk Analytics.

With the general setting in mind, we identify four directions relevant to our
research in which we could contribute:
\begin{enumerate}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item Play with the trade-off between the computational efficiency of SAGA
		and the memory efficiency of SVRG, especially relevant when working with
		large datasets, e.g. for $n > 10^6$ which is not uncommon in these days
		of Big Data. A first approach to compromise on the memory requirement of
		SAGA would be to store averaged gradients over mini-batches instead of
		the full gradient matrix. This task will involve the implementation and
		empirical testing of the devised scheme. A novel proof of convergence
		can be envisioned. This work is related to \cite{nitanda2014stochastic}.
	\item A distributed implementation of one of those algorithms. This would be
		useful to diminish the clock time needed to solve a given problem or to
		solve large-scale optimizations where the memory of one computer is not
		sufficient anymore. This goal will require the analysis of the
		inter-nodes communication cost as well as the design of a merging or
		synchronization scheme. Novel proofs of convergence could be required.
		It could be inspired by \cite{bianchi2014coordinate}.
	\item Explore the application of these algorithms to minimax problems which
		aim at finding saddle points \cite{nemirovski2009robust}. The min-max
		formulation appears in the context of zero-sum games and robust
		optimization. Traditionally, robust optimization problems focus on
		converting the minimax problem to a minimization problem by leveraging
		duality theory. Instead, we aim to find the saddle points using
		incremental methods.
	\item Use these methods to fit statistical models. In particular, we are
		interested to fit a Gaussian Mixture Model (GMM) viewed as a manifold
		optimization problem. Our goal would be to adapt one of the incremental
		methods to fit GMMs \cite{reshad_matrix_2015}.
\end{enumerate}
We do not expect to complete all of the above objectives. We plan to discuss
with experts in the domain\footnote{Such as the first author of
\cite{reshad_matrix_2015}, whom Soroosh met during his master studies. Or
someone from the EPFL LIONS lab.} and will then choose two of them to focus on two of them only.

\paragraph{Roles.}
Each of us will pursue one of the mentioned goals from beginning to end; which
includes any necessary theory, implementation, testing, writing and
presentation. Our work (code, report and presentation) will be tracked by
\textit{git}, such that individual contributions can easily be spotted.

\paragraph{Milestones.}
Following are the milestones we envision for the completion of the
aforementioned project.
\begin{itemize}
	\setlength{\itemsep}{0pt} \setlength{\parskip}{0pt}
	\item 2016-03-24 Proposal submitted.
	\item 2016-04-01 Proposal approved.
	\item 2016-04-08 Two directions chosen.
	\item 2016-04-22 Problems stated and solutions formulated.
	\item 2016-05-06 Solutions implemented (Jupyter notebooks, Python).
	\item 2016-05-20 Tested on real or synthetic data.
	\item 2016-05-27 Report written.
	\item 2016-06-03 Project presented.
\end{itemize}

\printbibliography

\end{document}


\documentclass[12pt,b5paper]{book}
\usepackage[toc,page]{appendix}
\usepackage[utf8]{inputenc}
\usepackage{csquotes}
\usepackage{fullpage}
\usepackage[english]{babel}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{listings}
\usepackage{paralist}
\usepackage{tikzsymbols}
\lstset{frame=lrbt}
\lstset{basicstyle=\footnotesize}

\lstset
{
	basicstyle=\scriptsize\ttfamily,
	breaklines=true,
	frame=single,
	showstringspaces=false,
	tabsize=2
}

\theoremstyle{definition}
\newtheorem{example}{Example}

\title{Friendly F\# - fun with logic.}
\author{Dr. Giuseppe Maggiore \\
Dr. Giulia Costantini \\
Francesco Di Giacomo \\
Gerard van Kruiningen}

\begin{document}
\maketitle
\tableofcontents

\chapter{Introduction}
\label{sec:introduction}
\input{tex/introduction}

\section{An informal introduction to logic}
\label{sec:informalIntroduction}
\input{tex/informal_introduction}

\section{Inference systems}
\label{sec:inferenceSystems}
\input{tex/inference_systems}

\section{A first example: boolean expressions}
\label{sec:booleanExpressions}
\input{tex/examples/boolean_expressions}

\chapter{Numbers}
\label{chap:numbers}
In order to investigate the power of a logical framework such as the one we are working with, it may be interesting to see what we can do about the lack of direct support for numbers in the language. At a first glance, there is no relationship between our ability to define symbols and rules, and for example natural numbers, and as such we may be tempted to just ``add primitive support'' to manipulate such quantities directly.

Interestingly enough, natural numbers are not strictly needed as a primitive concept. Even though for practical reasons\footnote{They are fast, and time is money. Thus, machine integers are money. QED. You never realized that your computer was indeed full of gold!} a modern programming language should always offer the possibility of directly interfacing with machine integers, they can be easily re-built outside of hardware support.

In this chapter we will begin with formalizing and implementing the concept of \textit{counting} in Section \ref{sec:peanoNumbers}. We will then implement a higher performance version of numbers that uses the positional system in Section \ref{sec:binaryNumbers}.

After this chapter we will have seen that logic is powerful enough to define apparently primitive concepts such as natural numbers. Of course it would be possible to go much further, into the definition of rational numbers, and then many other concepts of traditional arithmetics. In any case for practical reasons we will not use this in later chapters as it would have a dramatic impact on performance.

\section{Peano numbers}
\label{sec:peanoNumbers}
\input{tex/examples/peano_numbers}

\section{Binary numbers}
\label{sec:binaryNumbers}
\input{tex/examples/binary_numbers}

\chapter{Data structures}
\label{chap:dataStructures}
It is one thing to implement numbers, but of course in order to tackle computationally interesting problems we do need to define data structures. This should, again at a first glance, give an impression of an impossible task. After all, how could we define something as complex as a self-balancing binary tree with something as fine-grained such as logical inference?

Thanks to data structures like lists, trees, etc. defined in this chapter, we will be able to implement most of the well-known algorithms that are found in collections such as \cite{CLRS}. Moreover, these data structures will become fundamental helpers for the meta-programming that we will see in Chapter \ref{chap:aPL}.

\section{List manipulation}
\label{sec:lists}
\input{tex/examples/lists}

\section{Binary search trees}
\label{sec:binarySearchTrees}
\input{tex/examples/binary_search_trees}

\section{Balanced binary trees}
\label{sec:balancedBinaryTrees}
\input{tex/examples/balanced_binary_search_trees}

\chapter{Syntax and semantics of a programming language}
\label{chap:aPL}
As our ``crowning achievement'', we will push our logic language yet further. We will define the semantics of a small, imperative programming language within our logic language. Thanks to this we will see that even the most complex scenario, that of building a whole programming language, is not too much for our programming language.

\section{Evaluating expressions}
\label{sec:evaluatingExpressions}
\input{tex/examples/evaluating_expressions}

\chapter{Closing remarks}
\label{chap:closingRemarks}
As we have seen so far, we have shown that we can build a multitude of different things with our logic language:
\begin{inparaenum}[\itshape i\upshape)]
\item basic concepts from mathematics such as numbers in various formats;
\item various concepts from programming such as data structures and even interpreters for programming languages.
\end{inparaenum}

It is quite evident that any mathematical procedure can be defined as a logic program, well beyond the simplicity of implementing numbers.
Moreover we could build a modern assembler and use our logical language as a ``universal'' interpreter, therefore running any existing software in it (albeit at a snail's pace). If we can implement the whole of mathematical procedures and the whole of programming, then what is left that we cannot compute? In this chapter we draw some closing remarks on the field of computability and what consequences this has for programming languages and logical thinking in general.

\section{Fragments of computability}
\label{sec:computability}
\input{tex/computability}

\chapter{Conclusions}
\input{tex/conclusions}

\begin{appendices}
\chapter{A not so small imperative language}
\input{tex/examples/ImperativeLanguage/intro}
\input{tex/examples/ImperativeLanguage/semantics}
\input{tex/examples/ImperativeLanguage/conclusions}
\end{appendices}

\end{document}

Course materials/Dictaat/Inference systems - dictaat.tex


\documentclass[010-intro.tex]{subfiles}

\begin{document}

    \subsection{Common Tools in Data Science History: R}
    \label{ss:intro-r-history}

        At the time of writing (November 2021) R ranks \#15 on the TIOBE index,
        a measure of the popularity of a language
        \cite{IndexTIOBESoftware}.
        R is one of the premier languages used for statistical computing and graphics,
        that was the successor to the S programming language
        \cite{ProjectStatisticalComputing}.

        John Chambers, Rick Becker, Doug Dunn, Jean McRae, and Judy Schilling
        implement the S language in the statistics research department at Bell Laboratories in 1976.
        It was the first implemented statistical computing language
        \cite{beckerBriefHistory1994}.
        The need for a statistical computing language grew from the necessity of
        interacting with data using Exploratory Data Analysis (EDA)
        techniques and graphical output
        to work with larger data sets and iterate faster
        \cite{beckerBriefHistory1994}.
        In 1988, a commercial version of S was implemented, S-PLUS, and was
        succeeded by R, which was in development in 1991 by
        Ross Ihaka and Robert Gentleman at the University of Auckland, New Zealand,
        and later open sourced in 1995
        \cite{beckerBriefHistory1994}.
        The Comprehensive R Archive Network (CRAN) was founded in 1997 by the R Core team
        as a means to serve as a software repository for users to submit packages that can be
        used by other R users
        \cite{hornikAnnounceCRAN1997}.
        R's first stable v1.0 version was released in 2000
        \cite{smith16YearsProject2016, ihakaProjectBriefHistory}.

        The release of additional
        data manipulation
        (\code{reshape} v0.4 in 2005 and \code{plyr} v0.1.1 in 2008)
        and visualization
        (\code{ggplot} v0.2.2 in 2006 and its successor, \code{ggplot2} v0.5.1 in 2007)
        tools in CRAN by Hadley Wickham
        focused R in exploratory data analysis
        \cite{wickhamPracticalToolsExploring2008, tukeyExploratoryDataAnalysis1977}.
        Joseph J. Allaire founds RStudio in 2009
        \cite{allaireRStudioPBC2020, rstudioRStudio}.

        Wickham believed a lot of the work needed to explore and understand data was being overlooked by statisticians
        \cite{smithHadleyWickhamWhy2015}:
        \begin{displayquote}
            % There are definitely some academic statisticians who just don't understand why what I do is statistics,
            % but basically I think they are all wrong.
            % What I do is fundamentally statistics.
            ... The fact that data science exists as a field is a colossal failure of statistics.
            To me, that is what statistics is all about.
            It is gaining insight from data using modelling [sic] and visualization.
            Data munging and manipulation is hard and statistics has just said that's not our domain.
        \end{displayquote}
        He continued to improve and create a domain-specific language (DSL) in R focused on the data science workflow
        (\code{reshape2} v1.0 in 2010, \code{dplyr} v0.1.1 and \code{tidyr} v0.1 2014).
        During this time,
        RStudio Inc. works on developing the RStudio integrated development environment (IDE)
        and is released in 2011 to make interacting and programming
        in the R programming language much easier for users and also provides tools to make exploratory data analysis more readily available.
        In 2012, RStudio Inc. releases \code{shiny},
        a dashboard framework that can be written in R which
        lowers the barrier of entry to create, interact, analyze, and publish graphics and data on the web.
        These ideas around ``tidy data'' principles for exploratory data analysis
        \cite{wickhamTidyData2014}
        cumulated in the release of the \code{tidyverse} in 2016,
        which hosts a multitude of libraries for the data science DSL in R.
        For this work and ``influential work in statistical computing, visualization, graphics, and data analysi'' and
        ``making statistical thinking and computing accessible to a large audience'',
        Wickham was awarded the international COPSS Presidents' Award in 2019
        \cite{InstituteMathematicalStatistics2019}.

        Separately,
        Max Kuhn began to unify the various modeling packages in R with the \code{caret}
        (first release in CRAN in 2007 as v2.27) package
        and was to be superseded by
        the \code{tidymodels} package
        (first released in CRAN in 2018 as v0.0.1)
        for its consideration of Tidyverse principles that where objects share a common philosophy, grammar, and data structure
        to make things more interoperable in data and analysis pipelines
        \cite{wickhamR4ds, kuhnTidyModeling2021}.
        In 2020, RStudio Inc announced that they have become a Public Benefit Corporation (PBC)
        \cite{allaireRStudioPBC2020, rstudioRStudio},
        and RStudio PBC employees develop, maintain, and advocate for a plethora of R packages.
        The Tidyverse set of packages are not dependencies of many other R packages in the ecosystem.
        These packages are also one of the main ways R is taught to new learners
        \cite{wickhamR4ds}.

\end{document}


\documentclass[12pt,twocolumn,a4paper]{article}
\usepackage[top=0.5in, bottom=1in, left=1in, right=1in]{geometry}
\usepackage{listings}
\usepackage{hyperref}
\usepackage{natbib}

\begin{document}
\author{Dionisio Perez-Mavrogenis}
\title{Machine Learning Assignment 1}
\maketitle

\section{Introduction}
To complete this assignment I used the WEKA\footnote{WEKA website : \href{http://www.cs.waikato.ac.nz/ml/weka/}{http://www.cs.waikato.ac.nz/ml/weka/}} tool. WEKA is a collection of tools for data preprocessing, learning algorithms for data mining and visualization of the results. The main source that I used in order to learn how WEKA works and its various capabilities is \citep{Witten2011}.

For this assignment I will be comparing and evaluating the performance of different classifiers and describe the data preprocessing that I performed in order to boost that performance.

The dataset provided described 16 classes, however several classes had no instances at all (classes 11, 12 and 13), while others had a very small number of instances that were not enough for training and testing performance (classes 7, 8, 9, 14, 15). Classes 1, 2 and 10 had plenty of instances, resulting in better classifier training and performance while testing those. Classes 3, 4, 5, 6 and 16 had some examples (few compared to classes 1, 2 and 10) and including them in the training worsened the performance of the classifiers. For this reason, I will be using classes 1, 2 and 10 in order to compare the classifiers' performances.

\section{Data Preprocessing}
The first part of data preprocessing was to generate the input arff file for WEKA. I wrote a perl script to do that and Appendix \ref{sec:arff} describes the file.

Instances of the dataset provided had 280 attributes each. However, not all attributes were present and certainly not all attributes were useful, as they experienced very little or no variation between instance of the same or different class.

Retaining all 280 attributes would be unnecessary and would increase the training time required by each classifier. Thankfully WEKA offers a great selection of filtering algorithms for attribute selection, both supervised and unsupervised.

Table \ref{tab:filtcomp} shows a comparison of various filters. The results in \ref{tab:filtcomp} were derived by measuring the performance of an MLP in an attempt for the performance to be a slight indicator towards which is the best filter to use.

The filter that I chose to finally use in order to reduce the number of attributed used for classification was the \texttt{AttributeSelection} filter with the \texttt{GeneticSearch} search algorithm, an algorithm selecting the most useful attributes by using the genetic algorithm (\cite[p~41]{Goldberg1989}) with the final set of attributes consisting of 14 attributes and a relatively good performance, both in correct classification and running time of the classifiers. In addition to AttributeSelection, I used the \texttt{Randomize} filter which randomizes the order in which instances appear in the dataset. That also boosted performance, but not significantly.

\begin{table*}
\center
    \begin{tabular}{c | c | c | c | c }
    ~                      & \textbf{PCA}   & \textbf{AS-BF} & \textbf{AS-GS} & \textbf{Random subset}\\ \hline
    \textbf{Attributes after}       & 94    & 23    & 14    & 85 \\
    \textbf{Correctly classified(\%)}   & 73.53 & 85.29 & 86.76 & 83.82 \\
    \textbf{Incorrectly classified(\%)} & 26.47 & 14.71 & 13.24 & 16.18\\
    \textbf{Mean absolute error}    & 0.20  & 0.10  & 0.094 & 0.11 \\
    \textbf{RMS error} &  0.39 & 0.28  & 0.27  & 0.32 \\
    \end{tabular}
\caption{Filter performance of a default MLP. Comparison of PCA, AttributeSelection with the default BestFirst search algorithm, AttributeSelection with GeneticAlgorithm search and a filter selecting a random subset of the attributes.}
\label{tab:filtcomp}
\end{table*}

\section{Testing Classifiers}
I will be examining the performance of three learning algorithms, in their default settings, in an attempt to pick the most suitable for tweaking. Furthermore, I will also be looking at the best performing algorithm from the meta-learning algorithms that WEKA offers in order to examine whether further performance gains are possible.

\subsection{Learning algorithm selection} The learning algorithms that I will be comparing are \textit{RBFNetwork}, \textit{SimpleLogistic}, \textit{Logistic} and \textit{MLP} \footnote{Information on what each function is : \href{http://weka.sourceforge.net/doc.dev/}{http://weka.sourceforge.net/doc.dev/}}.

\begin{table*}
\centering
\begin{tabular}{c | c | c | c | c }
    & \textbf{RBFNetwork} & \textbf{Logistic} & \textbf{SimpleLogistic} & \textbf{MLP}   \\ \hline
    \textbf{Correctly classified (\%)} & 85.84      & 85.55    & 86.43          & 84.66 \\
    \textbf{Mean abs. error}           & 0.14       & 0.12     & 0.12           & 0.11  \\
    \textbf{Root mean squared error}   & 0.29       & 0.27     & 0.26           & 0.28  \\
    \end{tabular}
 \label{tab:class}
 \caption{Performance statistics of various learning algorithms.}
\end{table*}

The performance of these algorithms is given in Table \ref{tab:class}, with \textit{SimpleLogistic} being the chosen function for further exploration.

For evaluating the algorithms I used 100-fold cross validation rather than a percentage split on the available data because there was not an equal number of instances from each class and, while performing percentage split, classes with fewer instances had no instances left for testing. Furthermore, a high number of folds for cross validations was possible thanks to the reduced dimensionality of the input data.

\subsection{Tweaking SimpleLogistic}
\label{sub:tweak}
The available parameters that one can tweak on the algorithm are:
\begin{itemize}
\item \textbf{errorOnProbabilities}
\item \textbf{heuristicStop}
\item \textbf{maxBoostingIterations}
\item \textbf{numBoostingIterations}
\item \textbf{useAIC}
\item \textbf{useCrossValidation}
\item \textbf{weightTrimBeta}
\end{itemize}
Detailed information on what each parameter is are given in Appendix \ref{sec:det}.

In order to observe the impact on the classification performance of the classifier I altered the parameters one at a time, starting always from the default values. Table \ref{tab:opt} gives an outline of the optimal values.

\begin{table}
\begin{tabular}{c | c }
\textbf{Parameter} & \textbf{Value} \\ \hline
\textbf{errorOnProbabilities} & False \\
\textbf{heuristicStop} & 10 \\
\textbf{maxBoostingIterations} & 8 \\
\textbf{numBoostingIterations} & 8 \\
\textbf{useAIC} & True \\
\textbf{useCrossValidation} & True \\
\textbf{weightTrimBeta} & 0.0 \\
\end{tabular}
\caption{Optimal performance settings.}
\label{tab:opt}
\end{table}

\begin{table*}
\centering
\begin{tabular}{c | c | c | c | c }
    \textbf{Settings(Folds)} & \textbf{Correct (\%)} & \textbf{Incorrect(\%)} & \textbf{Mean abs. error} & \textbf{RMS error} \\ \hline
    Default (10) & 86.43 & 13.57 & 0.12 & 0.26 \\
    Optimal (10) & 87.61 & 12.39 & 0.13 & 0.25 \\
    Optimal (50) & 87.91 & 12.9 & 0.13 & 0.25 \\
\end{tabular}
\caption{Comparison of classification performance of optimal vs. default settings for the SimpleLogistic classifier.}
\label{tab:comp}
\end{table*}

SimpleLogistic with its default parameters, on 10-fold cross-validation, performed worse than the version with the optimal settings. Details of the comparison are given in Table \ref{tab:comp}.

The final parameter that I altered in order to improve performance was the number of folds. I found that the optimal is 50, resulting in a minuscule performance increase (Table \ref{tab:comp}).

In addition to classification performance, the optimal settings also decrease the classifier's training time, as the time taken to build the model with the optimal settings is 0.01 seconds and 0.27 seconds for the default settings, as measured by WEKA.

\section{Training a meta-learner}
Another interesting category of classifiers that WEKA offers are the meta-learners.

The classifier that I chose to test was the AdaBoostM1 boosting classifier, as boosting was the technique that seemed most appropriate for this type of problem. The learning algorithm that I used with AdaBoostM1 was again SimpleLogistic with the optimal settings described in Section \ref{sub:tweak}.

The performance of the meta-learner was the same as that of SimpleLogistic, however the errors , shown in table \ref{tab:err}, were a bit worse.

\begin{table}
\centering
\begin{tabular}{c | c | c }
    & \textbf{Mean abs. error} & \textbf{RMS error} \\ \hline
    ABM1 & 0.14 & 0.27 \\
    SL & 0.13 & 0.25 \\
\end{tabular}
\caption{Comparison of errors for AdaBoostM1 and SimpleLogistic for 50-fold cross validation on optimal settings.}
\label{tab:comp}
\end{table}

\section{Conclusion}
The combinations of the settings of the various algorithms offered in WEKA was too large to explore to its entirety. However, the algorithms presented here have an acceptable performance.

An issue that worsens algorithm generalization performance, although counter intuitive, was over-fitting on the training data. SimpleLogistic and AdaBoostM1 experienced a decline in performance as the number of internal boosting iterations performed by the algorithms increased.

Another factor that affected classifier performance was the number of folds while cross-validating. If the number of folds was too small(10) or too large(200) the classification performance suffered. The optimal number of folds was found by trial and error to be 50 folds.

\bibliographystyle{plainnat}
\bibliography{bibl}

\pagebreak
\onecolumn

\appendix
\section{WEKA arff File}
These are the headers of the WEKA arff file. All attributes are real valued numbers, except for \textit{sex} and \textit{class} which are nominal, i.e. can only take certain values.\\

Because the arff file was too big to include here, a link is provided for downloading the file. \\

Download from Dropbox : \href{https://dl.dropboxusercontent.com/u/23023752/popular.arff}{https://dl.dropboxusercontent.com/u/23023752/popular.arff} .
\label{sec:arff}
%\lstinputlisting[breaklines=true,numbers=left,numberstyle=\tiny]{appendices/wekatitles}

\section{SimpleLogistinc Parameters}
\label{sec:det}
The information presented here is quoted directly from the WEKA information on the SimpleLogistic algorithm.
\centering
\begin{itemize}
\item[errorOnProbabilities]\hfill \\  Use error on the probabilties as error measure when determining the best number of LogitBoost iterations. If set, the number of LogitBoost iterations is chosen that minimizes the root mean squared error (either on the training set or in the cross-validation, depending on useCrossValidation).

\item[heuristicStop]\hfill \\  If heuristicStop > 0, the heuristic for greedy stopping while cross-validating the number of LogitBoost iterations is enabled. This means LogitBoost is stopped if no new error minimum has been reached in the last heuristicStop iterations. It is recommended to use this heuristic, it gives a large speed-up especially on small datasets. The default value is 50.

\item[maxBoostingIterations] \hfill \\ Sets the maximum number of iterations for LogitBoost. Default value is 500, for very small/large datasets a lower/higher value might be preferable.

\item[numBoostingIterations] \hfill \\ Set fixed number of iterations for LogitBoost. If >= 0, this sets the number of LogitBoost iterations to perform. If < 0, the number is cross-validated or a stopping criterion on the training set is used (depending on the value of useCrossValidation).

\item[useAIC]\hfill \\  The AIC is used to determine when to stop LogitBoost iterations (instead of cross-validation or training error).

\item[useCrossValidation]\hfill \\  Sets whether the number of LogitBoost iterations is to be cross-validated or the stopping criterion on the training set should be used. If not set (and no fixed number of iterations was given), the number of LogitBoost iterations is used that minimizes the error on the training set (misclassification error or error on probabilities depending on errorOnProbabilities).

\item[weightTrimBeta]\hfill \\  Set the beta value used for weight trimming in LogitBoost. Only instances carrying (1 - beta)\% of the weight from previous iteration are used in the next iteration. Set to 0 for no weight trimming. The default value is 0.
\end{itemize}
\end{document}


\documentclass[13pt,onlymath]{beamer}
\usefonttheme{serif}
\usepackage{graphicx,amsmath,amssymb,tikz,psfrag,epstopdf,fancyvrb}
\usepackage[lighttt]{lmodern}
%\usepackage{graphicx,psfrag}

\input defs.tex

%% formatting

\mode<presentation>
{
\usetheme{default}
}
\setbeamertemplate{navigation symbols}{}
\usecolortheme[rgb={0.13,0.28,0.59}]{structure}
\setbeamertemplate{itemize subitem}{--}
\setbeamertemplate{frametitle} {
    \begin{center}
      {\large\bf \insertframetitle}
    \end{center}
}

\newcommand\footlineon{
  \setbeamertemplate{footline} {
    \begin{beamercolorbox}[ht=2.5ex,dp=1.125ex,leftskip=.8cm,rightskip=.6cm]{structure}
      \footnotesize \insertsection
      \hfill
      {\insertframenumber}
    \end{beamercolorbox}
    \vskip 0.45cm
  }
}
\footlineon

\AtBeginSection[]
{
    \begin{frame}<beamer>
        \frametitle{Outline}
        \tableofcontents[currentsection,currentsubsection]
    \end{frame}
}

%% begin presentation

\title{\large \bfseries Dynamic Programming}

\author{Jaehyun Park\\[3ex]
CS 97SI\\
Stanford University}

\date{\today}

\begin{document}

\frame{
\thispagestyle{empty}
\titlepage
}

\section{Dynamic Programming}

\begin{frame}{What is DP?}
\BIT
\item Wikipedia definition: ``method for solving complex problems by breaking them down into simpler subproblems''
\vfill
\item This definition will make sense once we see some examples
\BIT
\item Actually, we'll only see problem solving examples today
\EIT
\EIT
\end{frame}

\begin{frame}{Steps for Solving DP Problems}
\begin{enumerate}
\item Define subproblems
\item Write down the recurrence that relates subproblems
\item Recognize and solve the base cases
\end{enumerate}
\vfill
\BIT
\item Each step is very important!
\EIT
\end{frame}

\section{1-dimensional DP}

\begin{frame}{1-dimensional DP Example}
\BIT
\item Problem: given $n$, find the number of different ways to write $n$ as the sum of 1, 3, 4
\item Example: for $n=5$, the answer is 6
\BEAS
5 &=& 1+1+1+1+1 \\ &=& 1+1+3 \\ &=& 1+3+1 \\ &=& 3+1+1 \\ &=& 1+4 \\ &=& 4+1
\EEAS
\EIT
\end{frame}

\begin{frame}{1-dimensional DP Example}
\BIT
\item Define subproblems
\BIT
\item Let $D_n$ be the number of ways to write $n$ as the sum of 1, 3, 4
\EIT
\item Find the recurrence
\BIT
\item Consider one possible solution $n = x_1 + x_2 + \cdots + x_m$
\item If $x_m = 1$, the rest of the terms must sum to $n-1$
\item Thus, the number of sums that end with $x_m=1$ is equal to $D_{n-1}$
\item Take other cases into account ($x_m=3$, $x_m=4$)
\EIT
\EIT
\end{frame}

\begin{frame}{1-dimensional DP Example}
\BIT
\item Recurrence is then
\[
D_n = D_{n-1} + D_{n-3} + D_{n-4}
\]
\item Solve the base cases
\BIT
\item $D_0 = 1$
\item $D_n = 0$ for all negative $n$
\item Alternatively, can set: $D_0 = D_1 = D_2 = 1$, and $D_3 = 2$
\EIT
\vfill
\item We're basically done!
\EIT
\end{frame}

\begin{frame}[fragile]{Implementation}
\begin{Verbatim}[xleftmargin=25pt]
D[0] = D[1] = D[2] = 1; D[3] = 2;
for(i = 4; i <= n; i++)
    D[i] = D[i-1] + D[i-3] + D[i-4];
\end{Verbatim}
\BIT
\item Very short!
\item Extension: solving this for huge $n$, say $n \approx 10^{12}$
\BIT
\item Recall the matrix form of Fibonacci numbers
\EIT
\EIT
\end{frame}

\begin{frame}{POJ 2663: Tri Tiling}
\BIT
\item Given $n$, find the number of ways to fill a $3 \times n$ board with dominoes
\vfill
\item Here is one possible solution for $n=12$
\begin{center}
\includegraphics[height=0.3\textheight]{figures/tritiling}
\end{center}
\EIT
\end{frame}

\begin{frame}{POJ 2663: Tri Tiling}
\BIT
\item Define subproblems
\BIT
\item Define $D_n$ as the number of ways to tile a $3 \times n$ board
\EIT
\vfill
\item Find recurrence
\BIT
\item Uuuhhhhh...
\EIT
\EIT
\end{frame}

\begin{frame}{Troll Tiling}
\begin{center}
\includegraphics[height=0.7\textheight]{figures/trolltiling}
\end{center}
\end{frame}

\begin{frame}{Defining Subproblems}
\BIT
\item Obviously, the previous definition didn't work very well
\item $D_n$'s don't relate in simple terms
\vfill
\item What if we introduce more subproblems?
\EIT
\end{frame}

\begin{frame}{Defining Subproblems}
\begin{center}
\includegraphics[height=0.7\textheight]{figures/tritiling_sub1}
\end{center}
\end{frame}

\begin{frame}{Finding Recurrences}
\begin{center}
\includegraphics[height=0.7\textheight]{figures/tritiling_sub2}
\end{center}
\end{frame}

\begin{frame}{Finding Recurrences}
\BIT
\item Consider different ways to fill the $n$th column
\BIT \item And see what the remaining shape is \EIT
\item Exercise:
\BIT
\item Finding recurrences for $A_n$, $B_n$, $C_n$
\item Just for fun, why is $B_n$ and $E_n$ always zero?
\EIT
\vfill
\item Extension: solving the problem for $n \times m$ grids, where $n$ is small, say $n \le 10$
\BIT \item How many subproblems should we consider? \EIT
\EIT
\end{frame}

\section{2-dimensional DP}

\begin{frame}[fragile]{2-dimensional DP Example}
\BIT
\item Problem: given two strings $x$ and $y$, find the longest common subsequence (LCS) and print its length
\item Example:
\BIT
\item $x$: {\ttfamily A{\bfseries BC}BD{\bfseries AB}}
\item $y$: {\ttfamily {\bfseries B}D{\bfseries CAB}C}
\item ``\texttt{BCAB}'' is the longest subsequence found in both sequences, so the answer is 4
\EIT
\EIT
\end{frame}

\begin{frame}{Solving the LCS Problem}
\BIT
\item Define subproblems
\BIT
\item Let $D_{ij}$ be the length of the LCS of $x_{1\ldots i}$ and $y_{1\ldots j}$
\EIT
\item Find the recurrence
\BIT
\item If $x_i = y_j$, they both contribute to the LCS
\BIT
\item $D_{ij} = D_{i-1, j-1} + 1$
\EIT
\item Otherwise, either $x_i$ or $y_j$ does not contribute to the LCS, so one can be dropped
\BIT
\item $D_{ij} = \max\{D_{i-1,j}, D_{i, j-1}\}$
\EIT
\item Find and solve the base cases: $D_{i0} = D_{0j} = 0$
\EIT
\EIT
\end{frame}

\begin{frame}[fragile]{Implementation}
\begin{Verbatim}[xleftmargin=25pt]
for(i = 0; i <= n; i++) D[i][0] = 0;
for(j = 0; j <= m; j++) D[0][j] = 0;
for(i = 1; i <= n; i++) {
    for(j = 1; j <= m; j++) {
        if(x[i] == y[j])
            D[i][j] = D[i-1][j-1] + 1;
        else
            D[i][j] = max(D[i-1][j], D[i][j-1]);
    }
}
\end{Verbatim}
\end{frame}

\section{Interval DP}

\begin{frame}{Interval DP Example}
\BIT
\item Problem: given a string $x = x_{1\ldots n}$, find the minimum number of characters that need to be inserted to make it a palindrome
\vfill
\item Example:
\BIT
\item $x$: \texttt{Ab3bd}
\item Can get ``\texttt{dAb3bAd}'' or ``\texttt{Adb3bdA}'' by inserting 2 characters (one `\texttt d', one `\texttt A')
\EIT
\EIT
\end{frame}

\begin{frame}{Interval DP Example}
\BIT
\item Define subproblems
\BIT
\item Let $D_{ij}$ be the minimum number of characters that need to be inserted to make $x_{i\ldots j}$ into a palindrome
\EIT
\item Find the recurrence
\BIT
\item Consider a shortest palindrome $y_{1\ldots k}$ containing $x_{i\ldots j}$
\item Either $y_1 = x_i$ or $y_k = x_j$ (why?)
\item $y_{2 \ldots k-1}$ is then an optimal solution for $x_{i+1 \ldots j}$ or $x_{i \ldots j-1}$ or $x_{i+1 \ldots j-1}$
\BIT \item Last case possible only if $y_1 = y_k = x_i = x_j$ \EIT
\EIT
\EIT
\end{frame}

\begin{frame}{Interval DP Example}
\BIT
\item Find the recurrence
\[
D_{ij} = \begin{cases}
1 + \min\{D_{i+1, j}, D_{i, j-1}\} & x_i \ne x_j \\
D_{i+1, j-1} & x_i = x_j
\end{cases}
\]
\item Find and solve the base cases: $D_{ii} = D_{i, i-1} = 0$ for all $i$
\vfill
\item The entries of $D$ must be filled in increasing order of $j-i$
\EIT
\end{frame}

\begin{frame}[fragile]{Interval DP Example}
\begin{Verbatim}[xleftmargin=25pt]
// fill in base cases here
for(t = 2; t <= n; t++)
    for(i = 1, j = t; j <= n; i++, j++)
        // fill in D[i][j] here
\end{Verbatim}
\vfill
\BIT
\item Note how we use an additional variable \verb.t. to fill the table in correct order
\item And yes, for loops can work with multiple variables
\EIT
\end{frame}

\begin{frame}{An Alternate Solution}
\BIT
\item Reverse $x$ to get $x^R$
\item The answer is $n-L$, where $L$ is the length of the LCS of $x$ and $x^R$
\vfill
\item Exercise: Think about why this works
\EIT
\end{frame}

\section{Tree DP}

\begin{frame}{Tree DP Example}
\BIT
\item Problem: given a tree, color nodes black as many as possible without coloring two adjacent nodes
\vfill
\item Subproblems:
\BIT
\item First, we arbitrarily decide the root node $r$
\item $B_v$: the optimal solution for a subtree having $v$ as the root, where we color $v$ black
\item $W_v$: the optimal solution for a subtree having $v$ as the root, where we don't color $v$
\item Answer is $\max\{B_r, W_r\}$
\EIT
\EIT
\end{frame}

\begin{frame}{Tree DP Example}
\BIT
\item Find the recurrence
\BIT
\item Crucial observation: once $v$'s color is determined, subtrees can be solved independently
\item If $v$ is colored, its children must not be colored
\[
B_v = 1 + \sum_{u \in \mathrm{children}(v)} W_u
\]
\item If $v$ is not colored, its children can have any color
\[
W_v = 1 + \sum_{u \in \mathrm{children}(v)} \max\{B_u, W_u\}
\]
\EIT
\vfill
\item Base cases: leaf nodes
\EIT
\end{frame}

\section{Subset DP}

\begin{frame}{Subset DP Example}
\BIT
\item Problem: given a weighted graph with $n$ nodes, find the shortest path that visits every node exactly once (Traveling Salesman Problem)
\vfill
\item Wait, isn't this an NP-hard problem?
\BIT
\item Yes, but we can solve it in $O(n^2 2^n)$ time
\item Note: brute force algorithm takes $O(n!)$ time
\EIT
\EIT
\end{frame}

\begin{frame}{Subset DP Example}
\BIT
\item Define subproblems
\BIT
\item $D_{S, v}$: the length of the optimal path that visits every node in the set $S$ exactly once and ends at $v$
\item There are approximately $n 2^n$ subproblems
\item Answer is $\min_{v \in V} D_{V, v}$, where $V$ is the given set of nodes
\EIT
\vfill
\item Let's solve the base cases first
\BIT
\item For each node $v$, $D_{\{v\}, v} = 0$
\EIT
\EIT
\end{frame}

\begin{frame}{Subset DP Example}
\BIT
\item Find the recurrence
\BIT
\item Consider a path that visits all nodes in $S$ exactly once and ends at $v$
\item Right before arriving $v$, the path comes from some $u$ in $S - \{v\}$
\item And that subpath has to be the optimal one that covers $S-\{v\}$, ending at $u$
\item We just try all possible candidates for $u$
\EIT
\EIT
\vfill
\[
D_{S, v} = \min_{u \in S-\{v\}} \left(D_{S-\{v\}, u} + \mathrm{cost}(u, v) \right)
\]
\end{frame}

\begin{frame}{Working with Subsets}
\BIT
\item When working with subsets, it's good to have a nice representation of sets
\item Idea: Use an integer to represent a set
\BIT
\item Concise representation of subsets of small integers $\{0, 1, \ldots\}$
\item If the $i$th (least significant) digit is 1, $i$ is in the set
\item If the $i$th digit is 0, $i$ is not in the set
\item \eg, $19 = \mathtt{010011}_{(2)}$ in binary represent a set $\{0, 1, 4\}$
\EIT
\EIT
\end{frame}

\begin{frame}[fragile]{Using Bitmasks}
\BIT
\item Union of two sets \verb,x, and \verb,y,: \verb,x | y,
\item Intersection: \verb,x & y,
\item Symmetric difference: \verb,x ^ y,
\item Singleton set $\{i\}$: \verb,1 << i,
\item Membership test: \verb,x & (1 << i) != 0,
\EIT
\end{frame}

\begin{frame}{Conclusion}
\BIT
\item Wikipedia definition: ``a method for solving complex problems by breaking them down into simpler subproblems''
\BIT
\item Does this make sense now?
\EIT
\vfill
\item Remember the three steps!
\begin{enumerate}
\item Defining subproblems
\item Finding recurrences
\item Solving the base cases
\end{enumerate}
\EIT
\end{frame}

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS240: Programming in C
% Copyright 2016 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% https://github.com/ghorbanzade/UMB-CS240-2016S/blob/master/LICENSE
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\def \topDirectory {.}
\def \resDirectory {\topDirectory/src/c/main/ls16}
\def \texDirectory {\topDirectory/src/tex}
\def \styDirectory {\texDirectory/sty}
\def \cfgDirectory {\texDirectory/cfg}
\def \imgDirectory {\texDirectory/img}

\documentclass[compress]{beamer}
%\mode<presentation>
%\usetheme{default}

\usepackage{\styDirectory/directives}
\input{\cfgDirectory/config}
\usepackage{\styDirectory/beamerthemePejman}
\doc{number}{16}
%\setbeamertemplate{footline}[text line]{}

\usepackage{booktabs}

\begin{document}

\prepareCover

\section{Structures}

\subsection{Definition}

\begin{slide}
	\begin{block}{Definition}

	A structure is a user-defined data type that defines a physically grouped list of variables under one name in a block of memory.

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Syntax}

	\begin{terminal}
	struct @*\textit{datatype\_name}*@ {
	        @*\textit{type1 var1}*@;
	        @*\textit{type2 var2}*@;
	};
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Declaring a \texttt{struct}}

	\inputminted[
		fontsize=\scriptsize,
		firstline=10,
		linenos
	]{c}{\resDirectory/student.h}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Accessing \texttt{struct} Members}

	To access any member of a structure, we use the member access operator (\texttt{.}).

	\begin{terminal}
	struct student std;
	std.age = 10;
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Initializing a \texttt{struct}}

	\inputminted[
		fontsize=\scriptsize,
		firstline=10,
		linenos
	]{c}{\resDirectory/student1.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Accessing \texttt{struct} Members}

	Given a pointer to a structure, any member of that structure can be accessed using the member access operator (\texttt{->}).

	\begin{terminal}
	struct student std;
	ptr = &std;
	ptr->age = 10;
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Initializing a \texttt{struct}}

	\inputminted[
		fontsize=\scriptsize,
		firstline=10,
		lastline=20,
		linenos
	]{c}{\resDirectory/student2.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Initializing a \texttt{struct}}

	\inputminted[
		fontsize=\scriptsize,
		firstline=22,
		linenos
	]{c}{\resDirectory/student2.c}

	\end{block}
\end{slide}

\subsection{Example}

\begin{slide}
	\begin{figure}
	\includegraphics[width=0.7\textwidth]{\imgDirectory/daltons.png}
	\end{figure}
\end{slide}

\begin{slide}
	\begin{block}{Objective}

	Write a program \texttt{daltons.h} that takes the height of the Dalton brothers, Joe, William, Jack and Averell, and outputs their names in ascending order of their heights.

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=10,
		linenos
	]{c}{\resDirectory/daltons.h}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=16,
		lastline=33,
		linenos
	]{c}{\resDirectory/daltons.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=38,
		lastline=55,
		linenos
	]{c}{\resDirectory/daltons.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=60,
		lastline=71,
		linenos
	]{c}{\resDirectory/daltons.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=76,
		lastline=81,
		linenos
	]{c}{\resDirectory/daltons.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=86,
		lastline=93,
		linenos
	]{c}{\resDirectory/daltons.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=98,
		lastline=104,
		linenos
	]{c}{\resDirectory/daltons.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{columns}
	\column{0.5\textwidth}
	\begin{quotation} \scriptsize \normalfont

A programmer was walking out of door for work when his wife said: ``while you are out, buy some milk.''

He never came home.

	\end{quotation}
	\column{0.5\textwidth}
	\begin{figure}
	\includegraphics[width=\textwidth]{\imgDirectory/obama.jpg}
	\end{figure}
	\end{columns}
\end{slide}

\end{document}


\input ../setup

\usepackage{hyperref}
\usepackage{listings}
\usepackage{color}
\usepackage{enumerate}
\usepackage{graphicx}
\usepackage{tikz-qtree}
\usepackage{amsmath}
\usepackage{amssymb}

% \renewcommand{\labelenumi}{\alph{enumi}.}
% \renewcommand{\labelenumii}{(\roman{enumii})}

\definecolor{mygreen}{rgb}{0,0.6,0}
\definecolor{mygray}{rgb}{0.5,0.5,0.5}
\definecolor{mymauve}{rgb}{0.58,0,0.82}

\lstdefinestyle{normalPy}{
language=Python,				% the language of the code
basicstyle=\footnotesize,			% the size of the fonts that are used for the code
numbers=left,				% where to put the line-numbers; possible values are (none, left, right)
numberstyle=\color{mygray},		% the style that is used for the line-numbers
stepnumber=1,				% the step between two line-numbers. If it's 1, each line will be numbered
numbersep=5pt,				% how far the line-numbers are from the code
backgroundcolor=\color{white},		% choose the background color; you must add \usepackage{color} or \usepackage{xcolor}
showstringspaces=false,			% underline spaces within strings only
showspaces=false,				% show spaces everywhere adding particular underscores; it overrides 'showstringspaces'
showtabs=false,				% show tabs within strings adding particular underscores
frame=shadowbox,				% adds a frame around the code
tabsize=4,					% sets default tabsize to 4 spaces
captionpos=t,				% sets the caption-position to bottom
breaklines=true,				% sets automatic line breaking
breakatwhitespace=false,			% sets if automatic breaks should only happen at whitespace
commentstyle=\color{mygreen},    	% comment style
keepspaces=true,                 		% keeps spaces in text, useful for keeping indentation of code (possibly needs columns=flexible)
keywordstyle=\color{blue},       		% keyword style
}

\lstdefinestyle{consolePy}{
stepnumber=0,
}
\tikzset{every tree node/.style={minimum width=2em,draw,circle},
         blank/.style={draw=none},
         edge from parent/.style=
         {draw, edge from parent path={(\tikzparentnode) -- (\tikzchildnode)}},
         level distance=1.5cm}

\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\psetheader{Extra Practice 4 Solutions}{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\medskip

\section{Help}
Yes, we put these functions here so that you can refer to them easily.
\begin{python}
def sum(term, a, next, b):
    if a > b:
        return 0
    else:
        return term(a) + sum(term, next(a), next, b)

def fold(op, f, n):
    if n == 0:
        return f(0)
    else:
        return op(f(n), fold(op, f, n-1))

def compose(f, g):
    return lambda x: f(g(x))

def thrice(f):
    return compose(compose(f, f), f)
\end{python}

\section{Question 1}
This question tests you about the left-right evaluation.
\begin{python}
def new_if(predicate, then, otherwise):
    if predicate:
        then
    else:
        otherwise

def p(x):
    new_if(x > 5, print(x), p(x+1))

p(1) # Infinite print; 'otherwise' part always evaluates first.
\end{python}

\newpage

\section{Question 2}
It's time for a simple function nesting! What's the output of the code below?
\begin{python}
print((lambda x: lambda y: 2*x)(3)(4))
# (lambda x: lambda y: 2*x)(3)(4)
# = (lambda y: 6)(4) = 6
\end{python}

\section{Question 3}
Let's apply the same idea from Mission 3 :D
\begin{python}
print(thrice(thrice)(lambda x: x-1)(27))
# thrice(thrice)(lambda x: x-1)(27)
# = thrice(thrice(thrice(lambda x: x-1)))(27)
# = thrice(thrice(lambda x: x-3))(27)
# = thrice(lambda x: x-9)(27)
# = (lambda x: x-27)(27) = 0
\end{python}

\section{Question 4}
Nested function calls. Can you do it?
\begin{python}
foo = lambda y: lambda x: x(y)
add1 = lambda x: x+1
# This means foo(y)(x) = x(y) and add1(x) = x+1

print(foo(add1(2))(foo)(add1))
# foo(add1(2))(foo)(add1)
# = foo(3)(foo)(add1)
# = foo(3)(add1)
# = add1(3) = 4
\end{python}

\section{Question 5}
Nested variable scopes. Can you do it?
\begin{python}
def foo(x):
    def bar(x, y):
        return lambda y: y(x)
    return lambda y: bar(x, y)
# This means foo(x)(y) = bar(x, y) = lambda y: y(x)
#                                  = lambda z: z(x) <- y is a dummy variable
# In conclusion, foo(x)(y)(z) = z(x)

print(foo(lambda x: x**3)(lambda x: x**2)(lambda x: x)(4))
# foo(lambda x: x**3)(lambda x: x**2)(lambda x: x)(4)
# = (lambda x: x)(lambda x: x**3)(4)
# = (lambda x: x**3)(4) = 64
\end{python}

\section{Question 6}
Try to do each subquestion within \textbf{5 minutes}. \underline{You are to use the \texttt{\bfseries fold}
function} in all of the following questions.
\begin{enumerate}
\item Define a function \texttt{\bfseries between} that takes in a word as an input and returns a new
string with "*" placed in between all consecutive words that are identical. \\
\textbf{Sample Tests:}
\begin{python}
>>> between("happy")
'hap*py'
>>> between("oookayy")
'o*o*okay*y'
\end{python}
\textbf{Solution:}
\begin{python}
def between(word):
    # Define a function that takes in the index and see
    # if word[idx] == word[idx+1] then add a star
    def add_star(idx):
        if word[idx] == word[idx+1]:
            return word[idx] + "*"
        return word[idx]

    return fold(lambda x, y: x + y,                 # op
                lambda x: add_star(len(word)-1-x) \
                          if x != 0 else word[-1],  # f
                len(word)-1)                        # n
\end{python}

\item Define a function \texttt{\bfseries check\_vowel} that takes in a word as an input and returns
\texttt{\bfseries True} if there is at least one vowel in the word or \texttt{\bfseries False} otherwise. This is
case-sensitive. \\
\textbf{Sample Tests:}
\begin{python}
>>> check_vowel("qwertyjkl")
True
>>> check_vowel("482jfn")
False
\end{python}
\textbf{Solution:}
\begin{python}
def check_vowel(word):
    return fold(lambda x, y: x or y,            # op
                lambda x: word[x] in "aeiou",   # f
                len(word) - 1)                  # n
\end{python}

\item We have a tuple that contains some integers. Define a function \texttt{\bfseries largest} that
takes in a tuple and returns the largest integer. \\
\textbf{Sample Tests:}
\begin{python}
>>> largest((4, 2, 6, 2, 1))
6
>>> largest((0, 0, 0, 0, 0))
0
\end{python}
\textbf{Solution:}
\begin{python}
def largest(tup):
    return fold(lambda x, y: max(x, y), lambda x: tup[x], len(tup) - 1)
\end{python}
\end{enumerate}

\section{Question 7}
Given a tuple containing numbers, find the number of combination of numbers that
you can use from the tuple that can add up to a target number. Define this function
\texttt{\bfseries no\_of\_ways} that takes in a tuple and a target number, and return the number of ways
you can hit this target. You can only use each number once. You do not need to use
any higher-order functions for this. \\ \\
\textbf{Sample Tests:}
\begin{python}
>>> no_of_ways((4, 2, 5), 8)
0                               # no way you can make the number 8
>>> no_of_ways((4, 2, 5, 3), 7)
2                               # either 2 + 5 or 4 + 3 gives 7
>>> no_of_ways((4, 2, 5, 3, 5, 1), 5)
4                               # either 4 + 1, 2 + 3, 5 or 5 gives 5
\end{python}
\textbf{Solution:}
\begin{python}
# Very similar to the counting change problem
def no_of_ways(tup, num):
    if num <= 0 or tup == ():
        return 0
    elif len(tup) == 1 and tup[0] == num:
        return 1
    return int(tup[0] == num) + no_of_ways(tup[1:], num - tup[0]) \
           + no_of_ways(tup[1:], num)
\end{python}

\section{Question 8}
Define a function that returns the sum of cubes for integers from 1 to n, using \texttt{\bfseries accumulate}.
Recall the definition of \texttt{\bfseries accumulate} below.
\[a_1 = a, a_n \le b\]
\[\text{accumulate}(\oplus, \text{base}, f, a, \text{next}, b): (f(a_1) \oplus (f(a_2) \oplus (... \oplus (f(a_n) \oplus \text{base})...)))\]
\textbf{Solution:}
\begin{python}
def sum_of_cubes(n):
    return accumulate(lambda x, y: x + y,       # op
                      0, lambda x: x**3,        # base, f
                      1, lambda x: x + 1, n)    # a, next, b
\end{python}

\newpage
\section{Question 9}
Define the double factorial function using \texttt{\bfseries accumulate}.
\[
    n!! =
    \begin{cases}
        (n)(n-2)(n-4)\cdots(4)(2) & \text{if } \frac{n}{2} \in \mathbb{Z^+} \\
        (n)(n-2)(n-4)\cdots(3)(1) & \text{if } \frac{n+1}{2} \in \mathbb{Z^+}
    \end{cases}
\]
\textbf{Solution:}
\begin{python}
def double_factorial(n):
    return accumulate(lambda x, y: x*y,                 # op
                      1, lambda x: x,                   # base, f
                      2 - (n % 2), lambda x: x + 2, n)  # a, next, b

# Idea:
# base if definitely 1 when we want to set this is a product
# of a sequence.
# Note that next should be an increment by 2, but now it leaves
# us the variable a, which might seem unintuitive, but the idea
# is just that we want a = 1 when n is odd and a = 2 when n is even.
\end{python}

\begin{flushright}
\vspace{2 cm}\textbf{\textit{Solution compiled by Russell Saerang.}}
\end{flushright}
\end{document}


\documentclass{article}
\usepackage{mathrsfs}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{color}
%\include{macros}
%\usepackage{floatflt}
%\usepackage{graphics}
%\usepackage{epsfig}

\newcommand{\reals}{{\mathbb{R}}}
\newcommand{\dom}{{\bf{dom}}}
\newcommand{\symm}{{\bf{S}}}

\theoremstyle{definition}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{corollary}[theorem]{Corollary}

\theoremstyle{definition}
\newtheorem*{defition}{Definition}
\newtheorem*{example}{Example}

\theoremstyle{remark}
\newtheorem*{remark}{Remark}
\newtheorem*{note}{Note}
\newtheorem*{exercise}{Exercise}

\setlength{\oddsidemargin}{-0.25 in}
\setlength{\evensidemargin}{-0.25 in} \setlength{\topmargin}{-0.25
in} \setlength{\textwidth}{7 in} \setlength{\textheight}{8.5 in}
\setlength{\headsep}{0.25 in} \setlength{\parindent}{0 in}
\setlength{\parskip}{0.1 in}

\newcommand{\homework}[4]{
\pagestyle{myheadings} \thispagestyle{plain}
\newpage
\setcounter{page}{1} \setcounter{section}{#4} \noindent
\begin{center}
\framebox{ \vbox{\vspace{2mm} \hbox to 6.28in { {\bf
VE485,~Optimization~in~Machine~Learning (Summer 2020) \hfill Homework: #1} }
\vspace{6mm} \hbox to 6.28in { {\Large \hfill #1 \hfill} }
\vspace{6mm} \hbox to 6.28in { {\it Lecturer: #2 \hfill} }
\vspace{2mm} \hbox to 6.28in { {\it Student: #3 \hfill} }
\vspace{2mm} } }
\end{center}
\markboth{#1}{#1} \vspace*{4mm} }

\begin{document}

\homework{1. Convex Set}{Xiaolin Huang \hspace{5mm} {\tt
xiaolinhuang@sjtu.edu.cn}}{Chongdan Pan
\hspace{5mm} {\tt panddddda@sjtu.edu.cn } }{9}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Section 2.  Problem
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Problem 1} \label{ex-midpoint-cvx}
\emph{Midpoint convexity.}
A set $C$ is \emph{midpoint convex} if whenever two points $a,b$ are in $C$, the average or midpoint $(a+b)/2$ is in $C$.  Obviously a convex set is midpoint convex. It can be proved that under mild conditions midpoint convexity implies convexity. As a simple case, prove that if $C$ is closed and midpoint convex, then $C$ is convex.

\bf{Answer.
\\\\Assume \emph{C} is a closed midpoint convex.
\\If $C$ only has one boundary point $a$, then $C$ also has only one element $a=2a/2$.
\\\\When $C$ has two boundary points $a,b$, then $(a+b)/2, (3a+b)/4, (a+3b)/4, (7a+b)/8, (3a+5b)/8\cdots\in\emph{C}$
\\Therefore, $\forall x\in\emph{C},x=\sigma/2^n\cdot a+(1-\sigma/2^n)b\in\emph{C}$ where $\sigma\in N, 0\leq\sigma\leq2^n$
\\\\Let $z=\theta x_1+(1-\theta)x_2$ where $x_1,x_2\in\emph{C}, 0\leq\theta\leq1, \theta\in R$
\\Then $z=[\theta\sigma_1/2^{n_1}+(1-\theta)\sigma_2/2^{n_2}]a+[\theta(1-\sigma_1/2^{n_1})+(1-\theta)(1-\sigma_2/2^{n_2})]b$
\\Therefore, $z=\gamma a+(1-\gamma)b$ where $\gamma=\theta\sigma_1/2^{n_1}+(1-\theta)\sigma_2/2^{n_2}$
\\Since $0\leq\gamma\leq1$, $\gamma$ can be represented in the form of $\sigma_{\gamma}/2^{n_\gamma}$ where $n_\gamma\in N,\sigma_{\gamma}\in N,0\leq\sigma_{\gamma}\leq2^{n_\gamma}$
\\$z=\sigma_{\gamma}/2^{n_\gamma}\cdot a+(1-\sigma_{\gamma}/2^{n_\gamma})b$
\\\\As a result, $z\in C$ and \emph{C} is a convex. It still applies when $C$ has more than 2 boundary points.
}

\section*{Problem 2}\label{e-lin-frac-image}%
\emph{Linear-fractional functions and convex sets.}
Let $f:\reals^m \rightarrow \reals^n$ be the linear-fractional function
\[
f(x) = (Ax+b)/(c^T x+ d), \qquad \dom f = \{x \;|\; c^Tx+d >0\}.
\]
In this problem we study the inverse image of a convex set $C$ under $f$, i.e.,
\[
f^{-1} (C) = \{ x \in \dom f \;|\; f(x) \in C \}.
\]
For each of the following sets $C\subseteq \reals^n$, give a simple
description of $f^{-1}(C)$.

\begin{enumerate}
\item The halfspace $C = \{ y \; | \; g^Ty \leq h\}$ (with $g\neq 0$).

\item The polyhedron $C = \{y \; | \; Gy \preceq h\}$.

\item The ellipsoid $\{ y \;|\; y^T P^{-1} y \leq 1 \}$ (where $P \in \symm^n_{++}$).

\item The solution set of a linear matrix inequality, $C = \{y\; |\; y_1 A_1+ \cdots + y_n A_n \preceq B\}$, where $A_1$, \ldots, $A_n$, $B \in \symm^p$.
\end{enumerate}

\bf{Answer.
\\\\
$f^{-1}(x)=P[RP^{-1}(x)]$ where $P(x)$ is a perspective function and $R:\reals^{n+1} \rightarrow \reals^{m+1}$
\\\\Let $Q=   \left(
                \begin{array}{cc}
                    A & b\\
                    c^T & d\\
                \end{array}
            \right)\in\reals^{(m+1)\times(n+1)}$, then if $m>n$ let $R=(Q^TQ)^{-1}Q^T$
\\\\Then $f^{-1}(x)$ is also a linear-fractional function, which conserves convexity.
\\\\ \begin{enumerate}
    \item $$f^{-1}(C)=\{x\in\dom f|g^T f(x)\leq h\}$$
    $$f^{-1}(C)=\{x\in\dom f|g^T(Ax+b)/(c^T x+d)\leq h\}$$
    $$f^{-1}(C)=\{x\in\dom f|(g^T A-hc^T)x\leq hd-g^T b\}$$
    $$f^{-1}(C)=\{x\in\dom f|(A^Tg-ch^T)^Tx\leq hd-g^T b\}$$
    Let $g'=A^Tg-ch^T,h'=hd-g^T$
    \\The inverse image is the intersection of a halfspace and domain: $\{x|g'^Tf(x)\leq h'\}\cap\dom f$.
    \item $$f^{-1}(C)=\{x\in\dom f|Gf(x)\preceq h\}$$
    $$f^{-1}(C)=\{x\in\dom f|G(Ax+b)/(c^T x+d)\preceq h\}$$
    $$f^{-1}(C)=\{x\in\dom f|(GA-hc^T)x\preceq hd-Gb\}$$
    Let $G'=GA-hc^T,h'=hd-Gb$
    \\The inverse image is the intersection of a polyhedron and domain: $\{x|G'x\preceq h'\}\cap\dom f$.
    \item $$f^{-1}(C)=\{x\in\dom f|f(x)^T P^{-1}f(x)\leq1\}$$
    $$f^{-1}(C)=\{x\in\dom f|(\frac{(Ax+b)^T}{c^T x+d}) P^{-1}\frac{Ax+b}{c^T x+d}\leq1\}$$
    $$f^{-1}(C)=\{x\in\dom f|(Ax+b)^T P^{-1}(Ax+b)\leq(c^T x+d)^2\}$$
    $$f^{-1}(C)=\{x\in\dom f|[x^T(A^T P^{-1}A)x+b^TP^{-1}Ax+x^TA^T P^{-1}b+b^TP^{-1}b]\leq(x^Tcc^Tx+dc^Tx+x^Tdc+d^2\}$$
    $$f^{-1}(C)=\{x\in\dom f|[x^T(A^T P^{-1}A-cc^T)x+(b^TP^{-1}A-dc^T)x+x^T(A^T P^{-1}b-dc)]\leq d^2-b^T P^{-1}b\}$$
    $$f^{-1}(C)=\{x\in\dom f|(x+x_c)^TP'^{-1}(x+x_c)\leq d^2-b^T P^{-1}b+x_c^2\}$$
    where $P'^{-1}=A^T P^{-1}A-cc^T,x_c=P'(A^T P^{-1}b-dc)$
    \\Then $f^{-1}(C)=\{x\in\dom f|(x+x_c)^TM^{-1}(x+x_c)\leq 1\}$
    \\where $M^{-1}=\frac{P'^{-1}}{d^2-b^T P^{-1}b+x_c^2}$
    \\\\The inverse image is the intersection of a ellipsoid and domain: $\{x|(x+x_c)^TM^{-1}(x+x_c)\leq 1\}\cap\dom f$ (where $M \in \symm^n_{++}$).
    \item $$f^{-1}(C)=\{x_1 \cdots x_n\in\dom f|(a_1x+b_1)A_1+ \cdots + (a_nx+b_n)A_n \preceq B(c^T x+d)\}$$
    \\ where $a_i$ represents for $A$ in $(Ax+b)/(c^Tx+d)$
    $$f^{-1}(C)=\{x_1 \cdots x_m\in\dom f|x_1 A'_1+ \cdots + x_m A'_m \preceq B'\}$$
    \\The inverse image is the solution set of a linear matrix inequality and domain: $\{x|x_1 A'_1+ \cdots + x_m A'_m \preceq B'\}\cap\dom f$ (where $A'_i\in \symm^p$, $B'=Bd-b_1A_1-\cdots-b_nA_n)$
\end{enumerate}
}

\section*{Problem 3}\label{exe-sep-hyp-strict-counterexample}
Give an example of two closed convex sets that are disjoint but cannot be strictly separated.

\bf{$\{(x,y)|y\geq2^x\}$ and $\{(x,y)|y\leq0\}$}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Reference
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}


\documentclass[12pt]{article}

\usepackage[noend]{algpseudocode}
\usepackage{algorithm}
\usepackage{algorithmicx}
\usepackage{float}
\usepackage{graphicx}
\usepackage[margin=0.75in]{geometry}
\usepackage{amsmath,amsthm,amssymb}
\usepackage{dsfont}
\usepackage{subcaption}
\usepackage{amsthm}
\usepackage{mathtools,amssymb}
\usepackage{wrapfig}
\usepackage[font=small,skip=-20pt]{caption}
\allowdisplaybreaks

\newtheorem*{definition}{Definition}
\newtheorem*{question}{Question}
\newtheorem{theorem}{Theorem}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{conjecture}[theorem]{Conjecture}

\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\Rgz}{\mathbb{R}_{\ge 0}}

\newcommand{\ip}[2]{\left\langle{#1},{#2}\right\rangle}

\newcommand{\woloss}{without loss of generality }

\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\cone}{cone}
\DeclareMathOperator*{\hull}{hull}

\newcommand{\1}[1]{\mathds{1}[{#1}]}
\renewcommand{\P}[1]{\mathds{P}\left[{#1}\right]}
\newcommand{\E}[1]{\mathds{E}\left[{#1}\right]}
\newcommand{\Var}[1]{\mathrm{Var}[{#1}]}

\newcommand{\unit}{\mathds{1}}

% \renewcommand{\thesubsection}{\thesection.\alph{subsection}}
% \renewcommand\thesubsection{\ \ (\alph{subsection})}

\begin{document}

% \renewcommand{\qedsymbol}{\filledbox}

\title{
  \vspace{-0.5in}
  Voting With $d$-Dimensional Preferences
}
\date{}
\author{
  Clay Thomas\\
  claytont@princeton.edu
  \and
  Yufei Zheng\\
  yufei@cs.princeton.edu
}

\maketitle

  \begin{wrapfigure}{r}{0.2\textwidth}
    \begin{center}
      \vspace{-0.5in}
      \begin{align*}
        1 > 2 > 3 \\
        2 > 3 > 1 \\
        3 > 1 > 2 \\
      \end{align*}
      \caption{\textsc{Cycle}}
      \vspace{-0.25in}
      \begin{align*}
        1 > 2 > 3 \\
        1 > 3 > 2 \\
        3 > 2 > 1 \\
        2 > 3 > 1 \\
      \end{align*}
      \caption{\textsc{Sandwich}}
      \vspace{-0.25in}
      \begin{align*}
        1 > 2 > 3 \\
        1 > 3 > 2 \\
        3 > 2 > 1 \\
      \end{align*}
      \caption{\textsc{LeastFav}}
      \vspace{-0.25in}
      \begin{align*}
        1 > 2 > 3 > 4 \\
        1 > 2 > 4 > 3 \\
        2 > 1 > 3 > 4 \\
        2 > 1 > 4 > 3 \\
      \end{align*}
      \caption{\textsc{FlipFlop}}
      \vspace{-0.25in}
      \begin{align*}
        1 > 2 > 3 > 4 \\
        4 > 3 > 2 > 1 \\
      \end{align*}
      \caption{\textsc{Reverse}}
    \end{center}
  \end{wrapfigure}

  \emph{Social choice theory} is the field of study
  concerned with making collective decisions. A typical formulation
  is to study ``social choice functions'', i.e.
  functions $f$ which take as input a list of \emph{linear
  preferences} on $[n]$ (that is, total orders on $[n]$) and outputs a
  ``group decision'' $f(>_1,\ldots,>_m) \in [n]$.
  When $n = 2$, majority rule constitutes an excellent choice of $f$.
  Unfortunately when $n>2$, social choice functions are
  plagued by impossibility results,
  such as Arrow's Impossibility Theorem (every ``unanimous'' voting scheme
  satisfying ``independence of irrelevant alternatives'' is a ``dictatorship'')
  and the Gibbard-Satterthwaite Theorem (every surjective
  ``strategy-proof'' voting scheme is a dictatorship)
  \cite{AgtBookMechDesignInto}.

  To get around the issues with general voting schemes, theorists
  consider \emph{restricted classes of preferences},
  for example \emph{single peaked preferences}.
  Consider a set $P = \{>_i\}_{i=1}^K$ of linear preferences on $[n]$.
  We say $P$ is single peaked when there exists a set
  $Y = \{y_1,\ldots,y_n\} \subseteq [0,1]$ of distinct numbers such that,
  for each $>_i \in P$, there exists a ``peak'' $p_i \in [0,1]$
  where: if $p_i < y_k < y_\ell$, then $k >_i \ell$,
  and if $y_k < y_\ell < p_i$, then $k <_i \ell$.
  This corresponds to voters having a ``favorite point'' (i.e.
  their peak), and their preference for an alternative decreases
  as you move away from the peak.
  Single peaked preferences are dramatically better behaved
  than general preferences.
  If $P$ is single peaked, then there exist voting schemes
  $f : P^m \to [n]$ which are ``strategy-proof'',
  ``Pareto-optimal'' (a stronger
  version of unanimity), and ``anonymous'' (a stronger version
  of not being a dictatorship) \cite{AgtBookNoMoney}.
  This voting scheme essentially takes the medians of the ``peaks''.

  We introduce the notion of \emph{$d$-dimensional} preferences.
  We say $P$ is $d$-dimensional if there exists a set
  $X = \{x_1,\ldots,x_n\} \subseteq \R_{\ge 0}^d$ such that, for each $>_i \in P$,
  there exists a ``preference weight'' $a_i\in \R_{\ge 0}^d$ such that
  $k >_i \ell$ if and only if $\ip{a_i}{x_k} > \ip{a_i}{x_\ell}$.
  This corresponds to voters choosing options based on a simple weighted
  sum of the attributes of these options.

\section*{$2$-dimensional voting}

  When $d=1$, the $1$-dimensional preference sets are trivial,
  i.e. they contain only one preference.
  The first interesting case, $d=2$, is the main focus and application
  of our project.
  The class of single peaked preferences and $2$-dimensional preferences
  seem vaguely related, simply by looking at what preference lists
  are possible and impossible.
  For example, both preference sets \textsc{Cycle} (a common counterexample
  for certain voting schemes having nice properties) and \textsc{Sandwich},
  shown to the right, are neither single peaked nor $2$-dimensional.
  However, we note that neither class is a subset of the other:
  for example, \textsc{LeastFav} is $2$-dimensional but not single peaked,
  and \textsc{FlipFlop} is single peaked but not two dimensional.
  Finally, \textsc{Reverse} gives a nontrivial preference set which
  is both single peaked and $2$-dimensional.

  Moreover, these one very compelling similarity between $2$-dimensional
  preferences and single-peaked preferences.
  Both correspond to a \emph{one} dimensional set of options:
  the peaks all lie in the one-dimensional interval $[0,1]$,
  and the preference weights can without loss of generality
  all be taken to lie on the \emph{unit circle},
  because $>_a$ is independent of $\|a\|$.
  This gives a natural generalization of the ``median peaks''
  voting algorithm that worked for single peaked preferences.
  Fix some set of ``candidates'' $X\subseteq \Rgz^2$.
  Then our voting scheme is:

  \begin{algorithm}
    Poll a preference weight $a_i\in\Rgz^d$ from each voter $i$ \\
    Select the $a^*$ with median angle (from the $x$ axis) \\
    Return the favorite option of $a^*$.
  \end{algorithm}

  Because this scheme explicity picks some voter's favorite candidate,
  it is Pareto-optimal (and hence unanimous).
  Moreover, if the number of voters is odd, the order of the voters
  does not matter, so the voting scheme is formally called anonymous
  (and hence not a dictatorship).
  \textbf{The main conjecture of our project is that this voting
  scheme is also strategyproof, and satisfies independence
  of irrelevent alternatives}.

  Strategyproofness was easy to prove for the single-peaked
  medians algorith, but the proof does not carry over to our case automatically.
  Strategyproofness is equivalent to a simple combinatorial condidion
  called ``monotonicity''. We have checked for examples
  of non-monotonicity by randomly picking points and preference
  vectors many times, and no examples have turned up.
  Our goal is thus to prove that this rule is strategyproof,
  and if we can, replicate the discussion of \cite{AgtBookNoMoney}
  which fully characterizes strategy-proof single-peaked voting schemes.
  We were not able to test independence of irrelevent alternatives experimentally,
  and that think it may be harder to prove anyway.
  However, we would like to try!

\section*{Higher-dimensional preferences}

  For $d\ge 3$, the situation is less nice.
  Any set of preferences on $3$ options is $3$-dimensional,
  and $3$ options is already sufficient to prove negative
  results such as Arrow's Theorem.
  Thus, good voting schemes for $d > 2$ would require new notions
  of good voting scheme (such as using randomness \cite{AgtBookNoMoney}).
  We do not plan to investigate these.

  However, the proofs that certain preference sets fail to be $d$-dimensional
  are interesting and novel, and we regard the combinatorial classification
  of $d$-dimensional preference sets to be of independent interest.
  So far, we have proofs for all the examples mentioned in the figures,
  and that the generalization of {\sc Cycle} to $d+1$ options
  is impossible in $d$ dimensions.
  The proofs involve considering the convex hulls of the option points,
  and convex cones of the preference weights.
  We do not have a concrete conjecture for a classification of $d$-dimensional
  preferences, but we think that the maximal number of $d$-dimensional
  preferences may be something like $O(n^{d-1})$
  (in particular, we would like to show that the number is linear
  when $d=2$).

  We are attempting to use SMT solvers to automatically check
  if a given list is $2$-dimensional, and we would ideally like
  an efficient algorithm for finding the minimal dimension $d$
  needed such that a set of preferences is $d$-dimensional.
  Moreover, we plan to consider how voting schemes should work,
  if voters are know to be $2$ dimensional, but the underlying space
  $X$ is unknown and voters must submit a ranked preference list $<_a$
  (instead of their preference weight vector $a$).
  We believe we can adapt our medians algorithm to work in this case.

  \bibliography{weightedPref}{}
  \bibliographystyle{alpha}

\end{document}


\subsection{Hotelling competition}

Compete on quality, features, on an axis. For example 2 shops in equilibrium are next to each other on road, to capture both on either side.


In this part, we walk you through some longer analyses using MARSS models for a variety of different applications.  Most of these are analyses of ecological data, but the same models are used in many other fields.  These longer examples will take you through both the conceptual steps (with pencil and paper) and a \R step which translates the conceptual model into code.

\section*{Set-up}
\begin{itemize}
\item If you haven't already, install the MARSS package.  See directions on the CRAN webpage (http://cran.r-project.org/) for instructions on installing packages.  You will need write permissions for your \R program directories to install packages.  See the help pages on CRAN for workarounds if you don't have write permission.
\item Type in \texttt{library(MARSS)} at the \R command line to load the package after you install it.
\end{itemize}

\section*{Tips}
\begin{itemize}
\item \verb@summary(foo$model)@, where \verb@foo@ is a fitted model object, will print detailed information on the structure of the MARSS model that was fit in the call \verb@foo = MARSS(logdata)@. This allows you to double check the model you fit.  \verb@print(foo)@ will print a `English' version of the model structure along with the parameter estimates.
\item When you run \verb@MARSS()@, it will output the number of iterations used.  If you reached the maximum, re-run with \texttt{control=list(maxit=...)} set higher than the default.
\item If you mis-specify the model, \texttt{MARSS()} will post an error that should give you an idea of the problem (make sure \verb@silent=FALSE@ to see full error reports).  Remember, the number of rows in your data is $n$, time is across the columns, and the length of the vector of factors passed in for \verb@model$Z@ must be $n$ while the number of unique factors must be $m$, the number of $x$ hidden state trajectories in your model.
\item The missing value indicator is NA.
\item Running \texttt{MARSS(data)}, with no arguments except your data, will fit a MARSS model with $m=n$, a diagonal $\QQ$ matrix with $m$ variances, and i.i.d. observation errors.
\item Try \texttt{MARSSinfo()} at the command line if you get errors or warnings you don't understand.  You might find insight there.  Or look at the warnings and errors notes in the appendix of this user guide.
\end{itemize}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% UMB-CS114-2015F: Introduction to Programming in Java
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/UMB-CS114-2015F
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\def \topDirectory {../..}
\def \texDirectory {\topDirectory/src/main/tex}

\documentclass[10pt, compress]{beamer}

\usepackage{\texDirectory/template/style/directives}
\input{\texDirectory/template/config}
\usepackage{\texDirectory/template/style/beamerthemeUmassLecture}
\doc{number}{8}
%\setbeamertemplate{footline}[text line]{}

\usepackage{wrapfig}
\usepackage{verbatimbox}

\begin{document}
\prepareCover

\section{Course Administration}

\begin{slide}
	\begin{itemize}
		\item[] Assignment 1 Due October 5, 2015 at 17:30 PM.
		\item[] Assignment 2 Released. Due October 19, 2015 at 17:30 PM.
	\end{itemize}
\end{slide}

\begin{slide}
	\begin{block}{Overview}
		\begin{itemize}
			\item[] Loops
		\end{itemize}
	\end{block}
\end{slide}

\section{Loops}

\begin{slide}
	\begin{block}{\texttt{HelloWorld.java}}
		\begin{minted}[fontsize=\small,tabsize=8, linenos, firstnumber=1]{java}
			public class HelloWorld {
			    public static void main(String[] args) {
			        System.out.println("Hello World!");
			    }
			}
		\end{minted}
	\end{block}
	\begin{block}{Objective}
		Write a program \texttt{HelloWorld100.java} so that it prints \texttt{Hello World!} a hundred times.
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{HelloWorld100.java}}
		\begin{minted}[fontsize=\small,tabsize=8, linenos, firstnumber=1]{java}
			public class HelloWorld {
			    public static void main(String[] args) {
			        int i = 0;
			        while (i < 100) {
			            System.out.println("Hello World!");
			            i++;
			        }
			    }
			}
		\end{minted}
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{while} Loop}
		The \texttt{while} statement continually executes a block of statements while a particular condition is \texttt{true}.
		\begin{minted}[fontsize=\small,tabsize=8]{text}
			while (condition) {
			    statement(s);
			}
		\end{minted}
	\end{block}
	\begin{block}{Remember}
		\begin{itemize}
			\item[] Condition is always evaluated first.
			\item[] Statements are executed \textbf{while} condition is evaluated as \texttt{true}.
			\item[] Condition will be reevaluated once \textbf{all} statements are executed.
		\end{itemize}
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Objective}
		Write a program \texttt{twoSix.java} that keeps rolling two dice until you get two six.
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{TwoSix.java} (v1.0)}
		\begin{minted}[fontsize=\small,tabsize=8, linenos, firstnumber=1]{java}
			public class TwoSix {
			    public static void main(String[] args) {
			        boolean roll = true;
			        int dice1, dice2;
			        while (roll) {
			            dice1 = (int) Math.ceil( Math.random() * 6 );
			            dice2 = (int) Math.ceil( Math.random() * 6 );
			            System.out.printf("Dice 1: %d\tDice2: %d\n",
			                dice1, dice2);
			            if (dice1 + dice2 == 12)
			                roll = false;
			        }
			    }
			}
		\end{minted}
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{TwoSix.java} (v2.0)}
		\begin{minted}[fontsize=\small,tabsize=8, linenos, firstnumber=1]{java}
			public class TwoSix {
			    public static void main(String[] args) {
			        int dice1, dice2;
			        do {
			            dice1 = (int) Math.ceil( Math.random() * 6 );
			            dice2 = (int) Math.ceil( Math.random() * 6 );
			            System.out.printf("Dice 1: %d\tDice2: %d\n",
			                    dice1, dice2);
			        } while (dice1 + dice2 < 12);
			    }
			}
		\end{minted}
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{do-while} Loop}
		The \texttt{do-while} loop is a variation of \texttt{while} loop which evaluates iteration condition after execution of block of statements.
		\begin{minted}[fontsize=\small,tabsize=8]{text}
			do {
			    statement(s);
			} while (condition);
		\end{minted}
	\end{block}
	\begin{block}{Remember}
		\begin{itemize}
			\item[] Condition is evaluated once \textbf{all} statements are executed.
			\item[] Statements are reexecuted \textbf{while} condition is evaluated as \texttt{true}.
			\item[] Condition will be reevaluated once \textbf{all} statements are reexecuted.
		\end{itemize}
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{HelloWorld100.java} (v1.0)}
		\begin{minted}[fontsize=\small,tabsize=8, linenos, firstnumber=3]{java}
			int i = 0;
			while (i < 100) {
			    System.out.println("Hello World!");
			    i++;
			}
		\end{minted}
	\end{block}
	\begin{block}{Anathomy of a Code}
		\begin{minted}[fontsize=\small,tabsize=8]{text}
			intitialize iteration counter;
			while (iteration counter has not reached threshold) {
			    execute block of statements;
			    increment iteration counter;
			}
		\end{minted}
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{HelloWorld100.java} (v2.0)}
		\begin{minted}[fontsize=\small,tabsize=8, linenos, firstnumber=1]{java}
			public class HelloWorld {
			    public static void main(String[] args) {
			        for (int i = 0; i < 100; i++)
			            System.out.println("Hello World!");
			    }
			}
		\end{minted}
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{for} Loop}
		The \texttt{for} loop is a special case of a \texttt{while} loop that executes a block of statements for a specified number of times. Any \texttt{for} loop can be converted to a \texttt{while} loop.
		\begin{minted}[fontsize=\small,tabsize=8]{text}
			for (initialization, condition, afterthought) {
			    statement(s);
			}
		\end{minted}
		\begin{enumerate}
			\item Initialization\\Intitializes iteration counter
			\item Condition\\Specifies iteration condition
			\item Afterthought\\Increments/Decrements iteration counter
		\end{enumerate}
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Objective}
		Write a program \texttt{Triangle.java} that takes number $n$ from user and prints a traingle in $n$ lines using star characters, such that there are $k$ star characters in $k^{th}$ line.
		\begin{verbbox}
		*
		**
		***
		****
		*****
		\end{verbbox}
		\begin{figure}[H]\centering
		\theverbbox
		\caption{Sample riangle generated for $n = 5$}
		\end{figure}
	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{\texttt{Triangle.java}}
		\begin{minted}[fontsize=\small,tabsize=8, linenos, firstnumber=1]{java}
			public class Triangle {
			    public static void main(String[] args) {
			        Scanner input = new Scanner(System.in);
			        int numRow = input.nextInt();
			        input.close();
			        for (int i = 0; i < numRow; i++) {
			            for (int j = 0; j <= i; j++)
			                System.out.print("*");
			            System.out.printf("\n");
			        }
			    }
			}
		\end{minted}
	\end{block}
\end{slide}

\plain{}{Keep Calm\\and\\Practice}

\end{document}