\documentclass{article}

% Lecture notes theme files
\usepackage{../../Layout/lecturenotes}

\usepackage{import}
\import{../../Layout/}{environments.tex}
\import{../../Layout/}{definitions.tex}

%%%% Title
\title{Lecture 1 - Introduction}
\author{Fabricio Oliveira}

\begin{document}

%%%% Setting cover page
\begin{titlepage}
\singlespacing
\thispagestyle{empty}

\maketitle

\begin{abstract} In this lecture, we introduce the concept of nonlinear optimisation problems and the difference between mathematical programming and optimisation. We also discuss a few important examples to motivate the future developments in the course.

\end{abstract}

\tableofcontents
\end{titlepage}

\onehalfspacing

%%%% Content starts from here

\section{What is optimisation?}

An optimisation is one of these words that has many meanings, depending on the context you take as a reference. In the context of this course, optimisation refers to \emph{mathematical optimisation}, which is a discipline of applied mathematics.

In mathematical optimisation, we build upon concepts and techniques from calculus, analysis, linear algebra, and other domains of mathematics to develop methods that allow us finding values for variables within a given domain that maximise (or minimise) the value of a function. In specific, we are trying to solve the following general problem:
%
\begin{align}
    \mini &f(x) \label{eq:opt_prob} \\
    \st   &x \in X. \nonumber
\end{align}
%
where $x \in \reals^n$ is a vector of $n$ \emph{variables}, $f:\reals^n \mapsto \reals$ is a \emph{function} to be optimised (minimised) and $X \subseteq \reals^n$ is a \emph{domain} containing acceptable values for $x$.

In a general sense, these problems can be solved by employing the following strategy:
%
\begin{enumerate}
    \item Analysing properties of functions under specific domains and deriving the conditions that must be satisfied such that a point $x$ is a candidate optimal point.
    \item Applying numerical methods that iteratively searches for points satisfying these conditions.
\end{enumerate}
%
This idea is central in several domains of knowledge, and very often are defined under area-specific nomenclature. Fields such as economics, engineering, statistics, machine learning and, perhaps more broadly, operations research, are intensive users and developers of optimisation theory and applications.

\subsection{Mathematical programming and optimisation}

Operations research and mathematical optimisation are somewhat intertwined, as they both were born around a similar circumstance. %(Include something on the history of OR)

I like to separate \emph{mathematical programming} from (mathematical) \emph{optimisation}. Mathematical programming is a modelling paradigm, in which we rely on (very powerful, I might add) analogies to model \emph{real-world} problems. In that, we look at a given decision problem considering that
%
\begin{itemize}
    \item \emph{variables} represent \emph{decisions}, as in a business decision or a course of action. Examples include setting the parameter of (e.g., prediction) model, production systems layouts, geometries of structures, topologies of networks, and so forth;
    \item \emph{domain} represents business rules or \emph{constraints}, representing logic relations, design or engineering limitations, requirements, and such;
    \item function is an \emph{objective function} that provides a measure of solution quality.
\end{itemize}
%
With these in mind, we can represent the decision problem as a \emph{mathematical programming model} of the form of \eqref{eq:opt_prob} that can be solved using \emph{optimisation} methods. From now on, we will refer to this specific class of models as mathematical optimisation models, or optimisation models for short. We will also use the term to \emph{solve the problem} to refer to the task of finding optimal solutions to optimisation models.

This course is mostly focused on the optimisation techniques employed to find optimal solutions for these models. As we will see, depending on the nature of the functions $f$ and $g$ that are used to formulate the model, some methods might be more or less appropriate. Further complicating the issue, for models of a given nature, there might be alternative algorithms that can be employed and with no generalised consense whether one method is generally better performing than another.

\subsection{Types of mathematical optimisation models}

In general, the simpler the assumptions on the parts forming the optimisation model, the more efficient are the methods to solve such problems.

Let us define some additional notation that we will use from now on. Consider a model in the general form
%
\begin{align*}
	\mini & f(x) \\
	\st   & g_i(x) \leq 0, i = 1, \dots, m \\
	      & h_i(x) = 0, i = 1, \dots, l \\
	      & x \in X,
\end{align*}
%
where $f: \reals^n \mapsto \reals$ is the objective function, $g:\reals^m \mapsto \reals^m$ is a collection of $m$ inequality constraints and $h: \reals^n \mapsto \reals^l$ is a collection of $l$ equality constraints.

{\bf Remark:} in fact, every inequality constraint can be represented by an equality constraint by making $h_i(x) = g_i(x) + x_{n+1}$ and augmenting the decision variable vector $x \in \reals^n$ to include the slack variable $x_{n+1}$. However, since these constraints are of very different nature, we will explicitly represent both whenever necessary.

The most general types of models are the following. We also use this as an opportunity to define some (admittedly confusing) nomenclature from the field of operations research that we will be using in these notes.
%
\begin{enumerate}
    \item \emph{Unconstrained models:} in these, the set $X = \reals^n$ and $m=l=0$. These are prominent in, e.g., machine learning and statistics applications, where $f$ represents a measure of model fitness or prediction error.
    \item \emph{Linear programming (LP):} presumes linear objective function. $f(x) = c^\top x$ and constraints $g$ and $h$ affine, i.e., of the form $a_i^\top x - b_i$, with $a_i \in \reals^n$ and $b \in \reals$. Normally, $X = \braces{x \in \reals^n \mid x_j \geq 0, j = 1,\dots, n}$ enforce that decision variables are constrained to be the nonnegative orthant.
    \item \emph{Nonlinear programming (NLP):} some or all of the functions $f$, $g$, and $h$ are nonlinear.
    \item \emph{Mixed-integer (linear) programming (MIP):} consists of an LP in which some (or all) of the variables are constrained to be integers. In other words, $X \subseteq \reals^k \times \integers^{n-k}$. Very frequently, the integer variables are binary terms, i.e., $x_i \in \braces {0,1}$, for $i = 1,\dots, n-k$ and are meant to represent true-or-false or yes-or-no conditions.
    \item \emph{Mixed-integer nonlinear programming (MINLP):} are the intersection of MIPs and NLPs.
\end{enumerate}

{\bf Remark:} notice that we use the vector notation $c^\top x = \sum_{j \in J} c_j x_j$, with $J = \braces{1,\dots,N}$. This is just a convenience for keeping the notation compact.

\section{Examples of applications}

We now discuss a few examples to illustrate the nature of the problems to which we will develop solution methods and their applicability to real-world contexts.

\subsection{Resource allocation and portfolio optimisation} \label{sec:resource_allocation}

In a general sense, any mathematical optimisation model is an instantiation of the \emph{resource allocation problem}. A resource allocation problem consists of how to design an optimal allocation of resources to tasks, such that a given outcome is optimised.

Classical examples typically include production planning settings, in which raw materials or labour resources are inputted into a system and a collection of products, a production plan, results from this allocation. The objective is to find the best production plan, that is, a plan with the maximum profit or minimum cost. Resource allocation problems can also appear in a less obvious setting, where the resources can be the capacity of transmission lines in an energy generation planning setting, for example.

Let $i \in I = \braces{1,\dots, M}$ be a collection of resources and $j \in J = \braces{1,\dots,N}$ be a collection of products. Suppose that, to produce one unit of product $j$, a quantity $a_{ij}$ of resource $i$ is required. Assume that the total availability of resource $i$ is $b_i$ and that the return per unit of product $j$ is $c_j$.

Let $x_j$ be the decision variable representing total of product $j$ produced. The resource allocation problem can be modelled as
%
\begin{align}
\maxi \ & \sum_{j \in J} c_j x_j \label{ex1:obj} \\
\st & \sum_{j \in J}a_{ij} x_j \leq b_i, \ \forall i \in I \label{ex1:const1} \\
& x_j \geq 0, \ \forall j \in J. \label{ex1:const2}
\end{align}
%
Equation \eqref{ex1:obj} represents the objective function, in which we maximise the total return obtained from a given production plan. Equation \eqref{ex1:const1} quantify the resource requirements for a given production plan and enforce that such a requirement does not exceed the resource availability. Finally, constraint \eqref{ex1:const2} defines the domain of the decision variables.

Notice that, as posed, the resource allocation problem is linear. This is perhaps the most basic, and also most diffused setting for optimisation models for which very reliable and mature technology is available. In this course, we will concentrate on methods that can solve variants of this model in which the objective function and/or the constraints are required to include nonlinear terms.

One classic variant of resource allocation that include nonlinear terms is the \emph{portfolio optimisation problem}. In this, we assume that a collection of assets $j \in J = \braces{1,\dots, N}$ are available for investment. In this case, capital is the single (actual) resource to be considered. Each asset has random return $R_j$, with an expected value $\mathbb{E}[R_j] = \mu_j$. Also, the covariance between two assets $i,j \in J$ is given by $\sigma_{ij} = \mathbb{E}[(R_i - \mu_i)(R_j - \mu_j)]$, which can be denoted as the covariance matrix
%
\begin{align*}
\Sigma = \begin{bmatrix}
 \sigma_{11} & \dots & \sigma_{1N} \\
 \vdots      & \ddots & \vdots \\
 \sigma_{N1}  & \dots & \sigma_{NN}
\end{bmatrix}
\end{align*}
%
Markowitz (1952) proposed using $x^\top\Sigma x$ as a risk measure that captures the variability in the return of the assets. Given the above, the optimisation model that provides the investment portfolio with the least risk, given a minimum requirement $\epsilon$ in terms of expected returns is given by
%
\begin{align}
\mini \ &  x^\top\Sigma x \label{ex2:obj} \\
\st & \mu^\top x  \geq \epsilon \label{ex2:const1}\\
& 0 \leq x_j \leq 1, \ \forall j \in J. \label{ex2:const2}
\end{align}
%
Objective function \eqref{ex2:obj} represents the portfolio risk to be minimised, while constraint \eqref{ex2:const1} enforces that the expected return must be at least $\epsilon$. Notice that $\epsilon$ can be seen as a resource that has to be (at least) completely depleted, if one wants to do a parallel with the resource allocation structure discussed early. Constraint \eqref{ex2:const2} defined the domain of the decision variables. Notice how the problem is posed in a scaled form, where $x_j \in [0,1]$ represents a percentage of a hypothetical available capital for investment.

In this example, the problem is nonlinear due to the quadratic nature of the objective function $x^\top\Sigma x = \sum_{i,j \in J} \sigma_{ij}x_ix_j$. As we will see later on, there are efficient methods that can be employed to solve quadratic problems like this.

\subsection{The pooling problem: refinery operations planning}

The \emph{pooling problem} is another example of a resource allocation problem that naturally presents nonlinear constraints. In this case, the production depends on \emph{mixing operations}, known as pooling, to obtain certain product specification for a given property.

As an illustration, suppose that products $j \in J = \braces{1,\dots,N}$ are produced by mixing byproducts $i \in I_j \subseteq I = \braces{1,\dots,M}$. Assume that the qualities of byproducts $q_i$ are known and that there is no reaction between byproducts. Each product is required to have a property value $q_j$ within an acceptable range $[\underline{q}_j, \overline{q}_j]$ to be classified as product $j$. In this case, mass and property balances are calculated as
%
\begin{align}
& x_j  = \sum_{i \in I_j}{x_i}, \ \forall j \in J \\
& q_j = \frac{\sum_{i \in I_j}q_ix_i}{x_j}, \ \forall j \in J \label{ex3:const2}.
\end{align}
%
These can then incorporated into the resource allocation problem accordingly. One key aspect associated with pooling problem formulations is that the property balances represented by \eqref{ex3:const2} define \emph{nonconvex} feasibility regions. As we will see later, convexity is a powerful property that allows for developing efficient solution methods and its absence typically compromises computational performance and tractability in general.

\subsection{Robust optimisation}

Robust optimisation is a subarea of mathematical programming concerned with models that support decision-making under \emph{uncertainty}. In specific, the idea is to devise a formulation mechanism that can guarantee feasibility of the optimal solution in face of variability, ultimately taking a risk-averse standpoint.

Consider the resource allocation problem from Section \ref{sec:resource_allocation}. Now, suppose that the parameters $\tilde{a}_i \in \reals^N $ associated with a given constraint $i \in I = \braces{1,\dots,M}$ are uncertain with a unknown probability distribution. The resource allocation problem can then be formulated as
%
\begin{align*}
\maxi \ &  c^\top x  \\
\st & \tilde{a}_{i}^\top x \leq b_i, \ \forall i \in I  \\
& x_j \geq 0, \ \forall j \in J.
\end{align*}
%
Let us assume that the only information available are observations $\hat{a}_i$, from which we can estimate a nominal value $\overline{a}_i$. This is illustrated in Figure \ref{fig:random_obsservations}, in which 100 random observations are generated for $\tilde{a_i} = [\tilde{a}_{i1}, \tilde{a}_{i2}]$ with $\tilde{a}_{i1} \sim \text{Normal}(10,2)$ and $\tilde{a}_{i2} \sim \text{Normal}(5,3)$ for a single constraint $i \in I$. The nominal values are assumed to have coordinates given by the average values used in the Normal distributions.
%
\begin{figure}[h]
\includegraphics[width=0.8\textwidth]{Figures/data_no_ellipsoid.pdf}
\caption{One hundred random realisations for $\tilde{a}_i$.} \label{fig:random_obsservations}
\end{figure}
%
Our objective is to develop a model that incorporates a given level of protection in terms of feasibility guarantees. That is, we would like to develop a model that provides solutions that are guaranteed to remain feasible if the realisation of $\tilde{a}_i$ falls within an \emph{uncertainty set} $\epsilon_i$ of size controlled by the parameter $\Gamma_i$. The idea is that the bigger the uncertainty set $\epsilon_i$, the more robust is the solution, which typically comes at the expense of accepting solutions with expected worse performance.

The tractability of robust optimisation models depends on the geometry of the uncertainty set employed. Let us assume in what follows that
%
\begin{align}
	\epsilon_i = \braces{\overline{a}_i + P_iu \mid ||u||_2 \leq \Gamma_i} \label{eq:ellipsoid}
\end{align}
%
is an ellipsoid with the characteristic matrix $P_i$ (i.e., its eigenvalues show how the ellipsoid extends in every direction from $\overline{a}_i$) and $\Gamma_i$ employs a scaling of the ellipsoid size.

{\bf Remark:} an alternative (perhaps more frequent) characterisation of an ellipsoid $\epsilon \subset \reals^n$ centred at $\overline{x}$ is given by $\epsilon = \braces{x \in \reals^n \mid (x - \overline{x})^\top A(x - \overline{x}) = 1}$. By making $A = P^{1/2}$, we recover the representation in \eqref{eq:ellipsoid}.

We can now formulate the \emph{robust counterpart}, which consists of a risk-averse version of the original resource allocation problem. In that, we try to anticipate the worst possible outcome and make decisions that are both optimal and guarantee feasibility in this worst-case sense. This standpoint translates into the following optimisation model.
%
\begin{align}
\maxi \ &  c^\top x \nonumber \\
\st & \maxi_{a_{i} \in \epsilon_i}\braces{a_i^\top x} \leq b_i, \ \forall i \in I \label{ex3:robust_const}\\
& x_j \geq 0, \forall j \in J. \nonumber
\end{align}
%
Notice how the constraint \eqref{ex3:robust_const} has an embedded optimisation problem, turning into a \emph{bi-level optimisation} problem. This highlights the issue associated with tractability, since solving the whole problem strongly depends on deriving tractable equivalent reformulations.

Assuming that the uncertainty set $\epsilon_i$ is an ellipsoid, the following result holds.
%
\begin{align}
\max_{a_{i} \in \epsilon_i}\braces{a_i^\top x}  & = \overline{a}_i^\top x + \max_u\braces{u^\top P_i x : ||u||_2 \leq \Gamma_i} \label{eq:robust_set1}\\
& = \overline{a}_i^\top x + \Gamma_i||P_i x||_2. \label{eq:robust_set2}
\end{align}
%
In \eqref{eq:robust_set1}, we recast the inner problem in terms of the ellipsoidal uncertainty set, ultimately meaning that we recast the inner maximisation problem in terms of variable $u$. Since the only constraint is $||u||_2 \leq \Gamma_i$, in \eqref{eq:robust_set2} we can derive a closed form for the inner optimisation problem.

With the closed form derived in \eqref{eq:robust_set2}, we can reformulate the original bi-level problem as a tractable single-level problem of the following form
%
\begin{align}
\maxi \ &  c^\top x \nonumber \\
\st & \overline{a}_i^\top x + \Gamma_i||P_i x||_2 \leq b_i, \ \forall i \in I \label{eq:robust_counter1}\\
& x_j \geq 0, \ \forall j \in J. \nonumber
\end{align}
%
Notice how the term $\Gamma_i||P_i^\top x||_2$ creates a buffer for constraint \eqref{eq:robust_counter1}, ultimately preventing the complete depletion of the resource. Clearly, this will lead to a suboptimal solution when compared to the original deterministic at the expense of providing protection against deviations in coefficients $a_i$. This difference is often referred to as the \emph{price of robustness}.

In Figure \ref{fig:ellipsoids}, we show the ellipsoidal sets for two levels of $\Gamma_i$ for a single constraint $i$. We define
%
\begin{align}
\epsilon_i = \braces{\begin{bmatrix} 10 \\ 5 \end{bmatrix} + \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix} \begin{bmatrix} u_1 \\ u_2 \end{bmatrix}}
\end{align}
%
using the average and standard deviation of the original distributions that generated the observations. We plot the ellipsoids for $\Gamma_1 = 1$ and $\Gamma_2 = 1.5$, illustrating how the protection level increases as $\Gamma$ increases. This can be inferred since the uncertainty set covers more of the observations and the formulation is such that feasibility is guaranteed for any observation within the uncertainty set.
%
\begin{figure}
\includegraphics[width=0.8\textwidth]{Figures/data_with_ellipsoid.pdf}
\caption{One hundred random realisations for $\tilde{a}_i$.} \label{fig:ellipsoids}
\end{figure}
%

\subsection{Classification: support-vector machines}

This is an example in which the resource allocation structure within the optimisation model is not as obvious. Suppose we are given a data set $D \in \reals^{n}$ with $|D| = N + M$ that can be divided into two disjunct sets $I^- = \braces{x_1, \dots, x_N}$ and $I^+ = \braces{x_1,\dots, x_M}$.

Each element in $D$ is an observation of a given set of $n$ features with values represented by a $x \in \reals^n$ that has been classified as belonging to set $I^-$ and $I^+$. Because of the availability of labelled data, classification is said to be ane xample of supervised learning in the field of machine learning.

Figure \ref{fig:classified_observations} illustrates this situation for $n = 2$, in which the orange dots represent points classified as belonging to $I^-$ (negative observations) and the blue dots represent points classified as belonging to $I^+$ (positive observations).

\begin{figure}
    \includegraphics[width=0.8\textwidth]{Figures/classes_no_classifier.pdf}
    \caption{Two hundred observations for $x_i$ classified to belong to $I^-$ (orange) or $I^+$ (blue).}
    \label{fig:classified_observations}
\end{figure}

Our task is to obtain a function $f:\reals^n \mapsto \reals$ from a given family of functions that is capable to, given an observed set of features $\hat{x}$, classify whether it belongs to $I^-$ or $I^+$. In other words, we want to calibrate $f$ such that
%
\begin{align}
f(x_i) < 0, \ \forall x_i \in I^-, \text{ and } f(x_i) > 0, \ \forall x_i \in I^+.
\end{align}
%
This function would then act as a classifier that could be employed to any new observation $\hat{x}$ made. If $f$ is presumed to be an affine function of the form $f(x) = a^\top x - b$, then we obtain a \emph{linear classifier}.

Our objective is to obtain $a \in \reals^n$ and $b \in \reals$ such that misclassification error is minimised. Let us define the error measure as
%
\begin{align*}
& e^-(x_i \in I^-; a, b) :=
    \begin{cases} 0, \text{ if } a^\top x_i - b \leq 0, \\
        a^\top x_i - b, \text{ if } a^\top x_i - b > 0.
    \end{cases} \\
& e^+(x_i \in I^+; a, b) :=
    \begin{cases} 0, \text{ if } a^\top x_i - b \geq 0, \\
        b -  a^\top x_i, \text{ if } a^\top x_i - b < 0.
    \end{cases}
\end{align*}
%
Using this error measure, we can define constraints that capture deviation on each measure by means of nonnegative slack variables. Let $u_i \geq 0$ for $i = 1, \dots, N$ and $v_i \geq 0$ for $i = 1,\dots, M$ be slack variables that measure the \emph{misclassification error} for $x_i \in I^-$ and $x_i \in I^+$, respectively.

The optimisation problem that finds optimal parameters $a$ and $b$ can be stated as
%
\begin{align}
\mini \ & \sum_{i=1}^M u_i + \sum_{i=1}^N v_i \label{ex4:obj}\\
\st & a^\top x_i - b - u_i \leq 0, i = 1, \dots, M \label{ex4:const1} \\
    & a^\top x_i - b + v_i \geq 0, i = 1,\dots,N \label{ex4:const2} \\
    & ||a||_2 = 1 \label{ex4:const3} \\
    & u_i \geq 0, i = 1, \dots, N \\
    & v_i \geq 0, i = 1, \dots, M \\
    & a \in \reals^n, b \in \reals. \label{ex4:end}
\end{align}
%
The objective function \eqref{ex4:obj} accumulates the total misclassification error. Constraint \eqref{ex4:const1} allows for capturing the misclassification error for each $x_i \in I^-$. Notice that $u_i = \max\braces{0, a^\top x_i -b} = e^-(x_i \in I^-; a, b)$. Likewise, constraint \eqref{ex4:const2} guarantees that $v_i = e^+(x_i \in I^+; a, b)$.
To avoid trivial solutions in which $(a,b) = (0, 0)$, the normalisation constraint $|| a ||_2 = 1$ is imposed in constraint \eqref{ex4:const3}, which turns the model nonlinear.

Solving the model \eqref{ex4:obj}--\eqref{ex4:end} provides optimal $(a,b)$ which translates into the classifier represented as the green line in Figure \ref{fig:observations_with_classifier}.

\begin{figure}[h]
    \includegraphics[width=0.8\textwidth]{Figures/classes_with_classifier.pdf}
    \caption{Two hundred observations for $x_i$ classified to belong to $I^-$ (orange) or $I^+$ (blue) with a classifier (green).}
    \label{fig:observations_with_classifier}
\end{figure}

A variant referred to as \emph{robust classifier} penalises not only the the misclassification error, but also the observations within a given slab $S = \braces{x \in \reals^n \mid -1 \leq a^\top x - b \leq 1}$. Notice that, being the two lines defined by $f^-(x) : a^\top x - b = -1$ and $f^+(x) : a^\top x - b = +1$, the distance between the two hyperplanes is given by $\frac{2}{||a||_2}$.

Accordingly, we redefine our error measures as follows.
%
\begin{align*}
& e^-(x_i \in I^-; a, b) :=
    \begin{cases} 0, \text{ if } a^\top x_i - b \leq -1, \\
        a^\top x_i - b, \text{ if } a^\top x_i - b > -1.
    \end{cases} \\
& e^+(x_i \in I^+; a, b) :=
    \begin{cases} 0, \text{ if } a^\top x_i - b \geq 1, \\
        b -  a^\top x_i, \text{ if } a^\top x_i - b < 1.
    \end{cases}
\end{align*}
%
By doing so, a penalty is applied not only to those points that were misclassified but also to those points correctly classified that happen to be inside the slab $S$. To define an optimal robust classifier, one must trade off the size of the slab, which is inversely proportional to $||a||$, and the total of observations that fall in the slab $S$. The formulation for the robust classifier then becomes
%
\begin{align}
\mini \ & \sum_{i=1}^M u_i + \sum_{i=1}^N v_i + \gamma||a||_2^2 \label{ex5:obj}\\
\st & a^\top x_i - b - u_i \leq -1, \ i = 1,\dots,M \label{ex5:const1} \\
    & a^\top x_i - b + v_i \geq -1, \ i = 1,\dots,N \label{ex5:const2} \\
    & u_i \geq 0, i = 1, \dots, N \\
    & v_i \geq 0, i = 1, \dots, M \\
    & a \in \reals^n, b \in \reals.
\end{align}
%
In objective function \eqref{ex5:obj}, the errors accumulated in variables $u_i$, $i=1,\dots,N$ and $v_i$, $i = 1,\dots,M$ and the squared norm $||a||_2^2$ are considered simultaneously. The term $\gamma$ is a scalar used to impose an emphasis on minimising the norm $||a||_2$ and incentivising a larger slab $S$ (recall that the slab is large for smaller $||a||_2$). The squared norm  $||a||_2^2$ is considered instead as a means to recover differentiability, as the norm  $||a||_2$ is not differentiable. Later on, we will see how beneficial it is for optimisation methods to be able to assume differentiability. Moreover, note how in constraints \eqref{ex5:const1} and \eqref{ex5:const2} $u$ and $v$ also accumulate penalties for correctly classified $x_i$ that happen to be between the slab $S$, that is, that have term $a^\top x - b$ larger/ smaller than -1/ +1. Figure \ref{fig:observations_with_rob_classifier} shows a robust classifier an arbitrary value of $\gamma$.

\begin{figure}[H]
    \includegraphics[width=0.8\textwidth]{Figures/classes_with_robust_classifier.pdf}
    \caption{Two hundred observations for $x_i$ classified to belong to $I^-$ (orange) or $I^+$ (blue).}
    \label{fig:observations_with_rob_classifier}
\end{figure}

{\bf Remark:} robust classifiers are known in the machine learning literature as \emph{support vector machines}, where the support vectors are the observations that support the slab.

\end{document}


% !TeX root = ./document.tex
%%% Main file. General parameters are defined here as well as other parts are referenced.

%% One-sided print version:
% Margins: left 40mm, right 25mm, top and bottom 25mm.
% Warning: LaTeX adds 1in by itself.
\documentclass[12pt,a4paper]{report}
\setlength\textwidth{145mm}
\setlength\textheight{247mm}
\setlength\oddsidemargin{15mm}
\setlength\evensidemargin{15mm}
\setlength\topmargin{0mm}
\setlength\headsep{0mm}
\setlength\headheight{0mm}
% \openright makes the next text to begin on the right side of the book.
\let\openright=\clearpage

\usepackage[utf8]{inputenc}
\usepackage[czech]{babel}
%% Other packages
\usepackage{graphicx}
\usepackage{enumerate}  % http://ctan.org/pkg/enumerate
\usepackage{amssymb}  % for \varnothing
\usepackage{amsmath}
\usepackage{amsfonts}  % Number sets (naturals etc.).
\usepackage{amsthm}  % Theorems
\usepackage{thmtools}  % Theorem list
\usepackage{mathtools}  % Norm/Abs definitions & "xRightarrow".
\usepackage{setspace}  % Allows for easy 1.5 spacing
\usepackage{placeins}  % Adds command \FloatBarrier, which draws all images in the queue.
\usepackage{array}   % for \newcolumntype macro !!!
\usepackage{verbatim}  % \begin{comment}
\usepackage{tocloft}  % Table of Contents

%% Package hyperref creates clickable pdf hyperlinks, but mainly we are
%% going to use it to save metadata to pdf (including table of contents).
%% WARNING, do not forget to fill in title and author.
\usepackage[pdfpagelabels,unicode,colorlinks]{hyperref}   % Must be after all other packages [ps2pdf,unicode].
\hypersetup{pdftitle=Matematická analýza}
\hypersetup{pdfauthor=Martin Brajer}

%%% Little changes to the style.

% Those macros use nasty trick to force LaTeX to typeset chapter headers more
% decently without leaving so much space above them.
\makeatletter
\def\@makechapterhead#1{
    {\parindent \z@ \raggedright \normalfont
    \Huge\bfseries \thechapter. kapitola: #1
    \par\nobreak
    \vskip 20\p@
}}
\def\@makeschapterhead#1{
    {\parindent \z@ \raggedright \normalfont
    \Huge\bfseries #1
    \par\nobreak
    \vskip 20\p@
}}
\makeatother

% This macro defines chapter which is not numbered, but is listed in the content.
\def\chapwithtoc#1{
    \chapter*{#1}
    \addcontentsline{toc}{chapter}{#1}
}

\renewcommand{\figurename}{Obr.}  % Picture label change from "Obrázek" to "Obr". !!!
\onehalfspacing  % Line spacing 1.5.
%\renewcommand{\baselinestretch}{2}
\setlength{\parindent}{0cm}  % To look more like handwritten notes.
\newcolumntype{C}{>{$}c<{$}} % math-mode version of "c" column type, using "array" package !!!
\renewcommand{\deg}{^{\circ}}  % Degree sign for math mode (\deg).
\newcommand{\mylongrightarrow}[2]{\xrightarrow{\hspace*{#2mm} #1 \hspace*{#2mm}}}

%% Absolute value (and norm).
\DeclarePairedDelimiter\abs{\lvert}{\rvert}%
\DeclarePairedDelimiter\norm{\lVert}{\rVert}%
% Swap the definition of \abs* and \norm*, so that \abs
% and \norm resizes the size of the brackets, and the
% starred version does not.
\makeatletter
\let\oldabs\abs
\def\abs{\@ifstar{\oldabs}{\oldabs*}}
%
\let\oldnorm\norm
\def\norm{\@ifstar{\oldnorm}{\oldnorm*}}
\makeatother

%% ToC settings.
\renewcommand{\listtheoremname}{Věty a definice}
% Semesters. Add by "\semester".
\newcommand{\listsemestername}{Semestry}
\newlistof{semester}{sem}{\listsemestername}
\newcommand{\semester}{%
% \newcommand{\semester}[1]{%
\refstepcounter{semester}
\par\noindent\textbf{\textit{Semestr \thesemester}}
%   \par\noindent\textbf{Semestr \thesemester. #1}
\addcontentsline{sem}{semester}
{\protect\numberline{{\thesemester}}}\par
}
% \renewcommand*{\thesemester}{\Alph{semester}}

%% Theorem settings.
% Theorems numbered by letters are unnumbered in my lecture notes.
\declaretheorem[numberwithin=chapter, name=Definice]{definition}
\declaretheorem[name=Definice]{definitionAi}
\renewcommand*{\thedefinitionAi}{A\arabic{definitionAi}}

\declaretheorem[numberwithin=chapter, name=Lemma]{lemma}
\declaretheorem[name=Lemma]{lemmaAlph}
\renewcommand*{\thelemmaAlph}{\Alph{lemmaAlph}}

\declaretheorem[numberwithin=chapter, name=Věta]{theorem}
\declaretheorem[name=Věta]{theoremAlph}
\renewcommand*{\thetheoremAlph}{\Alph{theoremAlph}}

\declaretheorem[numberwithin=chapter, name=Příklad]{example}

\renewcommand\qedsymbol{QED}
% \renewcommand\qedsymbol{$\bot$}

%% Change how \autoref prints the references.
\addto\extrasczech{  % Same as babel package language.
\def\definitionautorefname{D}
\def\definitionAiautorefname{D}
%
\def\lemmaautorefname{L}
\def\lemmaAlphautorefname{L}
%
\def\theoremautorefname{V}
\def\theoremAlphautorefname{V}
%
\def\exampleautorefname{P}
%
\def\itemautorefname{bod}
\def\equationautorefname{Eq}
}

%% Itemize settings.
\renewcommand{\labelitemi}{$\bullet$}
\renewcommand{\labelitemii}{$\circ$}
\renewcommand{\labelitemiii}{*}

%%% Individual chapters are placed into their respective files for convenience.
\begin{document}
% Different page numbering style, so it doesn't override the following one (1/2).
\pagenumbering{gobble}

\include{mainPage}
\openright

\tableofcontents
\newpage

\listoftheorems%[ignoreall,show={theorem}]
\newpage

\listofsemester
\newpage

\pagestyle{plain}  % Start numbering again.
\pagenumbering{arabic}  % Different page numbering style, so it doesn't override the following one (2/2).
\setcounter{page}{1}

\include{introduction}
\include{chapter1}  % Basic concepts. CHECK E1.46
\include{chapter2}  % Limits.
%\chapwithtoc{Seznam použitých zkratek}	% Abbreviations.

\openright
\end{document}


\documentclass{article}
\usepackage{listings}
% \usepackage{ctex}
\usepackage{amsmath}
\usepackage[a4paper, left = 2cm, right = 2cm]{geometry}
\usepackage[colorlinks, urlcolor=blue]{hyperref}

\begin{document}

\begin{titlepage}
    \title{Chapter 3}
    \author{\href{https://github.com/ZacBi}{Zac Bi}}
    \date{December 9, 2019}
    \maketitle
    \thispagestyle{empty}
\end{titlepage}

\section*{Solutions}

\subsection*{3.1}
\paragraph{1.}
Equation for trigram prob estimation:

$$P(w_n|w_{n-2}^{n-1}) =  \displaystyle\frac{C(w_{n-2}^{n})}{\sum_{w} C(w_{n-2} w_{n-1} w)} = \displaystyle\frac{C(w_{n-2}^{n})}{C(w_{n-2} w_{n-1})}$$

\paragraph{2.}
Add '<s> <s>' and '</s> </s>' at the start and end  of each sequence for counting trigrams.
The resulting corpus:
\begin{align*}
     & \texttt{<s> <s> I am Sam </s> </s>}                         \\
     & \texttt{<s> <s> Sam am I </s> </s>}                         \\
     & \texttt{<s> <s> I do not like green eggs and ham </s> </s>}
\end{align*}

\paragraph{}
So the trigrams probabilities of \texttt{I am Same} is:
\begin{align*}
     & P(\texttt{I|<s> <s>}) = \frac{2}{3} = 0.67 \quad
    P(\texttt{am|<s> I}) = \frac{1}{2} = 0.5 \quad
    P(\texttt{Sam|I am}) = \frac{1}{1} = 1              \\
     & P(\texttt{</s>|am Sam}) = \frac{1}{1} = 1 \quad
    P(\texttt{</s>|Sam </s>}) = \frac{1}{1} = 1 \quad
\end{align*}

\subsection*{3.2}
\paragraph{1.}
With Fig. 3.1(no smoothing)
\begin{align*}
    P(\texttt{<s> i want chinese food </s>})
     & = P(\texttt{i|<s>}) P(\texttt{want|i}) P(\texttt{chinese|want}) \\
     & \qquad P(\texttt{food|chinese}) P(\texttt{</s>|food})           \\
     & = 0.25 \times 0.33 \times 0.0065 \times 0.52 \times 0.68        \\
     & = 0.0001896
\end{align*}

\paragraph{2.}
Use add-1 smoothing table in Fig. 3.6
\begin{align*}
    P(\texttt{<s> i want chinese food </s>})
     & = P(\texttt{i|<s>}) P(\texttt{want|i})P(\texttt{chinese|want}) \\
     & \qquad P(\texttt{food|chinese}) P(\texttt{</s>|food})          \\
     & = 0.19 \times 0.21 \times 0.0029 \times 0.052 \times 0.40      \\
     & = 2.4 \times 10^{-6}
\end{align*}

\subsection*{3.3}
\paragraph{}
Probabilities using add-1 smoothing is lower since it saves probabilities
mass for unseen words in intuition. Now we prove it using the equation:

$$P_{add-1}(c_i) = \displaystyle\frac{C(c_i) + 1}{C(c_{i-1}) + |V|}$$

\paragraph{}
For each bigram whose count is greater than 1
is multiplied by a normalized factor $\displaystyle\frac{C(c_{i-1})}{C(c_{i-1}) + V}$.
As mentioned in this chapter before, the probabilities for those n-grams are discounted.

\subsection*{3.4}
\paragraph{}
\begin{align*}
    P_{add-1}(\texttt{Sam|am}) & = \frac{C(\texttt{am Sam}) + 1}{C(\texttt{am}) + |V|} \\
                               & = \frac{3}{3 + 11}                                    \\
                               & = 0.2143
\end{align*}

\subsection*{3.5}
\paragraph{}
Use the \textbf{chain rule of probability}:
\begin{align*}
    P(w_1^n) & = P(w_1)P(w_2|w_1) \dots P(w-n|w_1^{n-1}) \\
             & = \prod_{k=1}^{n} P(w_k|w_1^{k-1})
\end{align*}

\paragraph{1.}
As for bigram, the final formula should be $\prod_{k=1}^{n} P(w_k|w_{k-1})$.
I mark four sentence as $s_1, s_2, s_3, s_4$.
So the sum of four 2 word sentences is below, I ignored $P(\texttt{<s>})$
here since every sentence is started with \texttt{<s>}.
\begin{align*}
    \sum_{i=1}^{4}P(w_1^2) & = P(\texttt{a b}) + P(\texttt{a a}) + P(\texttt{b b}) + P(\texttt{b a}) \\
                           & = P(\texttt{a|<s>})P(\texttt{b|a}) + P(\texttt{b|<s>})P(\texttt{b|b})
    + P(\texttt{b|<s>})P(\texttt{a|b}) + P(\texttt{a|<s>})P(\texttt{b|a})                            \\
                           & = \frac{1}{2} \times \frac{1}{2} \times 4                               \\
                           & = 1
\end{align*}

\paragraph{2.}
There is eight 3 word sentences: \texttt{aaa, aab, aba, abb, baa, bab, bba, bbb}.
It's obvious the sum of probabilities of all these sentences is 1.
Without loss of generality, for a bigram model the sum of probabilities of all k(k=1\dots n).
Taking a easy example assuming k=2, the sum of probabilities could be written as:
\begin{align*}
    P(\texttt{<s>})\sum_v \sum_w P(v|\texttt{<s>} P(w|v) & = \sum_v P(v|\texttt{<s>}) \sum_w P(w|v) \\
                                                         & = \sum_v P(v|\texttt{<s>})               \\
                                                         & = 1
\end{align*}
\paragraph{}
For more longer sentence, the same is ture. After adding \texttt{<s>}, all sentence lengths will be assigned
a single probability distribution since $\sum_w P(\texttt{</s>}|w)$ is obviously less than 1.0.

\subsection*{3.6}
\paragraph{}
\begin{align*}
    P(w_3|w_1, w_2) & = \frac{C(w_1 w_2 w_3) + 1}{\sum_w (c(w_1 w_2 w) + 1)} \\
                    & =  \frac{C(w_1 w_2 w_3) + 1}{\sum_w c(w_1 w_2 w) + |V|}
\end{align*}

\subsection*{3.7}
\paragraph{1.}
Recall the backoff interpolation smoothing:
\begin{align*}
    \hat{P}(\texttt{Sam|am}) & = \lambda(w_1^2)P(\texttt{Sam|am}) + \lambda(w_1)P(\texttt{Sam})   \\
                             & = \frac{1}{2} \times \frac{C(\texttt{am Sam})}{C(\texttt{am})}
    + \frac{1}{2} \times \frac{C(\texttt{Sam})}{\sum_w C(w)}                                      \\
                             & = \frac{1}{2} \times \frac{2}{3} + \frac{1}{2} \times \frac{4}{25} \\
                             & = \frac{31}{75}                                                    \\
                             & = 0.413
\end{align*}

\subsection*{3.8-3.11}
\paragraph{}
Get the implementation of algorithms in \href{https://github.com/ZacBi/learn_NLP_again/tree/master/lna/algorithms}{ZacBi/learn\_NLP\_again/lna/algorithms}

\subsection*{3.12}
\paragraph{}
As described in the exercise, the probability for 0 is $P(0) = 0.91$ while the probability for i(i=1\dots9) is $P(i) = 0.01$.
So the probability for \tt{0 0 0 0 0 3 0 0 0 0} should be:
\begin{align*}
    PP(test set) & = P(\tt{0 0 0 0 0 3 0 0 0 0})^{-\frac{1}{10}}    \\
                 & = \sqrt[10]{\prod_{i=1}^{9}\frac{1}{\tt{P(0)}} \
        \times \frac{1}{P(\tt{3})}}                                 \\
        & = \sqrt[10]{\frac{1}{0.91^9 \times 0.01}} \\
        & = 1.37
\end{align*}

\end{document}


\documentclass[bigger]{beamer}

\input{header-beam} % change to header-handout for handouts

% ====================
\title[Lecture 10]{Logic I F13 Lecture 10}
\date{October 10, 2013}
% ====================

\include{header}

\setlength{\fitchprfwidth}{5em}

\section{Conditionals}

\subsec{Indicative Conditionals}{

\bit
\item Conditionals = ``if---then\dots'' sentences
\item \emph{Indicative} conditionals: in indicative mood
\item ``If P, then Q'' or ``Q, if P'' with P, Q indicative sentences
\item P: \emph{antecedent}; Q: \emph{consequent}
\eit
}

\subsec{Truth Conditions of Conditionals}{

\[\text{If\ } \underbrace{\text{X is drinking alcohol}}_{P},
\text{then\ }\underbrace{\text{X is over 18}}_{Q}\]

\bit
\item ``If P, then Q'' can only be \emph{false} if:
\bit
\item P is \emph{true}: we check age if X is drinking beer (P~true), not if
 drinking pop; \emph{and}
\item Q is \emph{false}: we check drink if X under age (Q false),\\ not
if over 18
\eit
\item ``If P, then Q'' is true if:
\bit
\item P is \emph{false:} we don't check people drinking pop; \emph{or}
\item Q is \emph{true:} we don't card if X is over 18;
\item (or both)
\eit
\eit }

\subsec{The Material Conditional $\to$}{
\[
\begin{array}{cc|c}
P & Q & (P \to Q)\\
\hline
\T & \T & \T\\
\T & \F & \F\\
\F & \T & \T\\
\F & \F & \T
\end{array}
\]
}

\subsec{Other Indicative Conditionals}{

\bit \item The following all get translated by $P \to Q$:
\bit
\item If P, then Q
\item Q if P
\item Q provided P
\item P only if Q
\eit
\eit
}

\subsec{If and Only If}{

\bit
\item Sue drinks beer only if she is over 18
\[
\sf Drinks(sue, beer) \to Over18(sue)
\]
\item Sue drinks beer if she is over 18.
\[
\sf Over18(sue) \to Drinks(sue, beer)
\]
\eit
}

\subsec{Subjunctive Conditionals}{

\bit
\item Subjunctive conditionals = if---then statements in \emph{subjunctive} mood
\item ``If P were true, then Q would be true.''
\item Indicative conditional is \emph{truth-functional}: truth value of ``If P, then Q'' depends
  \emph{only on truth values} of P and Q.
\item Not truth functional, e.g., consider:
\ben
\item If the world were just, no evil deed would go unpunished.\\
$P_1$ = the world is just\\
$Q_1$ = no evil deed goes unpunished
\item If the world were flat, no evil deed would go unpunished.\\
$P_2$ = the world is flat\\
$Q_2$ = no evil deed goes unpunished
\een
\item $P_1$, $Q_1$ both false; $P_2$, $Q_2$ both false, but
\item (1) is true, but (2) is false
\eit
}

\subsec{The Biconditional}{

\bit
\item ``P if and only if Q'' short for:\\
``P if Q and P only if Q'' and\\
``P just in case Q''
\item Translated as: $P \iff Q$
\item Truth table:
\[
\begin{array}{cc|c}
P & Q & (P \iff Q)\\
\hline
\T & \T & \T\\
\T & \F & \F\\
\F & \T & \F\\
\F & \F & \T
\end{array}
\]
\eit

}

\subsec{Unless}{

Which of these people do you have to check (age or drink) to ensure that:
\begin{quote}
People are drinking pop unless they are over 18.
\end{quote}

\begin{tabular}{cccc}
\begin{beamerboxesrounded}[width=4em]{}
\vskip 2ex
\Large 22 years\\
\end{beamerboxesrounded} &
\begin{beamerboxesrounded}[width=4em]{}
\vskip 2ex
\Large 16 years\\
\end{beamerboxesrounded} &
\begin{beamerboxesrounded}[width=4em]{}
\vskip 2ex
\Large drinks pop\\
\end{beamerboxesrounded} &
\begin{beamerboxesrounded}[width=4em]{}
\vskip 2ex
\Large drinks beer\\
\end{beamerboxesrounded} \\
(1) & (2) & (3) & (4)
\end{tabular}

}

\subsec{Unless}{

\[\underbrace{\text{X is drinking pop}}_{P},
\text{unless\ }\underbrace{\text{X is over 18}}_{Q}\]

\bit
\item ``P unless Q'' can only be \emph{false} if:
\bit
\item P is \emph{false}\\
(we check age if person is drinking beer), \emph{and}
\item Q is \emph{false}\\
(we check drink if person not at least 18)
\eit
\item ``P unless Q'' is true (test ok) if:
\bit
\item P is false, and Q is true;
\item Q is false, and P is true;
\item P is true and Q is true
\eit
\item ``P unless Q'' is equivalent to
\bit
\item ``P if not Q'' ($\sf \lnot Q \to P$)
\item ``P or Q'' ($\sf P \lor Q$)
\eit
\eit

}

\section{Proof Methods for Conditionals}

\subsec{Modus Ponens}{
\setlength{\fitchprfwidth}{1.2in}

\fitchprf{\tline{If P, then Q}\\ \tline{P}}{\tline{Q}}
}

\subsec{Conditional Proof}{

If on the assumption that P is true, you can prove Q, then you have
given a proof of the conditional ``If P, then Q''

}

\subsec{Example}{

\setlength{\fitchprfwidth}{1.2in}

\begin{columns}
\begin{column}{3.5cm}
\fitchprf{
\nline{Cube(a) \lor Small(a)}}{\nline{\lnot Cube(a) \to Small(a)}}
\end{column}
\begin{column}{7cm}
\bens
\item Suppose a is not a cube.
\item We are told (premise) that a is a cube or a is small.
\item By disjunctive syllogism, a is small.
\item So by conditional proof, if a is not a cube, it is small. QED
\een
\end{column}
\end{columns}
}

\subsec{Example}{

\setlength{\fitchprfwidth}{1.2in}

\begin{columns}
\begin{column}{3.5cm}
\fitchprf{
\nline{\lnot Cube(a) \to Small(a)}}{\nline{Cube(a) \lor Small(a)}}
\end{column}
\begin{column}{7cm}
\bens
\item We proceed indirectly: Assume $Cube(a) \lor Small(a)$ were false.
\item Then a is not a cube, and a is also not small.
\item In particular, a is not a cube.
\item We are told (premise) that if a is not a cube, it is small.
\item By modus ponens, a is small.
\item Contradiction, since from (2) we have that a isn't small.
\item So by indirect proof, a is either a cube or it is small. QED
\een
\end{column}
\end{columns}
}

\section{Formal Proofs with $\to$}

\subsec{Formal Rules for $\to$}{

\noindent\setlength{\fitchprfwidth}{.9in}
\begin{tabular}{@{}l@{}l@{}}
\fitchctx{
\subproof{\nline[$k.$]{P}}{\dots\\ \nline[$l.$]{Q}}
\fpline[\quad]{P \to Q}[\lifi{$k$--$l$}]}
&
\fitchctx{
\nline[$n.$]{P \to Q}\\
\nline[$m.$]{P}\\
\fpline[\quad]{Q}[\life{n}{m}]}
\end{tabular}
}

\subsec{Formal Rules for $\iff$}{

\noindent\setlength{\fitchprfwidth}{.9in}

\fitchctx{
\subproof{\nline[$k.$]{P}}{\dots\\ \nline[$l.$]{Q}}\\
\subproof{\nline[$m.$]{Q}}{\dots\\ \nline[$n.$]{P}}\\
\fpline[\quad]{P \iff Q}[\liffi{$k$--$l$}{$m$--$n$}]}}

\subsec{Formal Rules for $\iff$}{

\noindent\setlength{\fitchprfwidth}{.9in}

\noindent\fitchctx{
\nline[$n.$]{P \iff Q}\\
\nline[$m.$]{P}\\
\fpline[\quad]{Q}[\liffe{n}{m}]}\\[2ex]
\fitchctx{
\nline[$n.$]{P \iff Q}\\
\nline[$m.$]{Q}\\
\fpline[\quad]{P}[\liffe{n}{m}]}

}

\subsec{Strategies}{

\bit
\item If your goal sentence is of the form $P \to Q$:
\bit
\item Start subproof
\item Put P in assumption line
\item Put Q as last line
\item Justify goal sentence with $\to$Intro
\item Try to prove Q inside subproof
\item This strategy usually works, so use it sooner rather than later
\eit
\item If you have $P \to Q$ available:
\bit
\item You can use it to get Q via $\to$Elim
\item For that, you need to prove P first
\item Wait with applying this strategy until you need Q
\eit
\eit

}

\subsec{Strategies}{

\bit
\item If your goal sentence is of the form $P \iff Q$:
\bit
\item Start 2 subproofs
\item In one, put P in assumption line and Q as last line
\item In one, put Q in assumption line and P as last line
\item Justify goal sentence with $\iff$Intro
\item Try to prove Q/P inside subproofs
\item This strategy usually works, so use it sooner rather than later
\eit
\item If you have $P \iff Q$ available:
\bit
\item You can use it to get P or Q via $\to$Elim
\item For that, you need to prove Q or P, resp., first
\item Wait with applying this strategy until you need Q or P
\eit
\eit

}

\subsec{Example}{

\fitchprf{\tline{\ }}{\nline{((A \lor B) \to C) \iff ((A \to C) \land (B \to C))}}

}
\subsec{Example}{

\fitchprf{\tline{\ }}{\nline{(\lnot A \iff B) \to (A \iff \lnot B)}}

}

\subsec{Example}{

\fitchprf{\tline{\ }}{\nline{(A\to B) \lor (B \to A)}}

}

\end{document}


\documentclass[a4paper]{article}

\usepackage[utf8]{inputenc}

\usepackage{url}
\usepackage[]{hyperref}

\usepackage{caption}

\usepackage{listings}

\usepackage{color}

\usepackage{amsmath}

\usepackage{pythonhighlight}

% *** GRAPHICS RELATED PACKAGES ***
%\usepackage[pdftex]{graphicx}
\usepackage{graphicx}
%\usepackage[dvips]{graphicx}
% to place figures on a fixed position
\usepackage{float}

\usepackage[margin=1in]{geometry}

\title{Traffic analysis study – Queuing Theory I. Home Assignment I.}
\author{Ferenc Nandor Janky - OA8AT9}
\date{}

\begin{document}

\maketitle

\tableofcontents

\section{Introduction}

The goal of the 1st home assignment is to perform a similar traffic analysis study that is presented in \cite{MolnarOnburst} with an arbitrarily chosen time series.
For the analysis a capture file was selected from \url{http://ita.ee.lbl.gov/}. The link to the actual capture file archive is \url{ftp://ita.ee.lbl.gov/traces/dec-pkt-1.tar.Z}. This capture contains 3.3 million packets from 1995. The packets are further filtered into multiple files based on packet types (TCP,UDP,etc.). This analysis deals with \verb!dec-pkt-1.tcp! from the archive mentioned before that is around 62 MBytes ands contains around 2.2 million TCP packets.

\section{Implementation}

The calculation and plotting of the resulting graphs have been implemented in the Python programming language.The software is available on \url{https://github.com/fecjanky/QT_home_assignment/blob/master/ha1/toki1.py}.
For the implementation the following Python libraries have been utilized:

\begin{itemize}
\item \verb!matplotlib! , for plotting various statistical data
\item \verb!numpy! , for effective numerical calculations
\end{itemize}

\section{Results}

\subsection{General metrics}

The values of Peak-to-Mean Ration (PMR), Squared Coefficient of Variation (SCV) and Skewness (S) of the inter-arrival data set  -- that was calculated from the arrival time series using the code in Listing~\ref{lst:interarr} --
 were the following:

 \begin{align*}
& PMR = & 250.4374 & \\
& SCV = & 2.2048  & \\
& S   = & 6.2460 &
\end{align*}

\begin{lstlisting}[style=mypython,caption={The functions used for calculating IDI and IDC},label={lst:interarr}]
def calc_interarrivals(arrivals):
    if len(arrivals) < 2:
        return None
    return np.subtract(np.array(arrivals[1:]), arrivals[0:-2])
\end{lstlisting}

\subsection{Intensity graph}

Figure~\ref{fig:intensity} shows the intensity graph of the analyzed capture. The intensity was aggregated using a 1 second basis based on the arrival times.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.9\textwidth]{figures/intensity.png}
    \caption{Intensity graph of the analyzed packet capture}
    \label{fig:intensity}
\end{figure}

\subsection{Inter-arrival KDE}

Figure~\ref{fig:interarrival} shows the kernel density estimation of the packet inter-arrival times.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.9\textwidth]{figures/interarrival.png}
    \caption{KDE graph of the inter-arrival times}
    \label{fig:interarrival}
\end{figure}

\subsection{Packet count and inter-arrival auto-correlation}

The following two Figures \ref{fig:countcorr} and \ref{fig:interarrcorr} shows the auto-correlation of the intensity and inter-arrival data as a function of lag.
Figure~\ref{fig:countcorr} shows signs of auto-correlation levels that could be an indicator of traffic bursts. Figure~\ref{fig:interarrcorr} shows slight auto-correlation also.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.9\textwidth]{figures/packet_count_correlation.png}
    \caption{Intensity auto-correlation as a function of lag}
    \label{fig:countcorr}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.9\textwidth]{figures/interarrival_correlation.png}
    \caption{Inter-arrival auto-correlation as a function of lag}
    \label{fig:interarrcorr}
\end{figure}

\subsection{IDI and IDC}

The \emph{Index of dispersion for Indices} (IDI) and \emph{Index of Dispersion for Counts} (IDC) were computationally intensive to calculate for the chosen data file.
For reducing the calculation times the code in Listing~\ref{lst:python} has been used that basically memoizes the windowed sum
calculation for a given data set that can be reused when calculating IDI or IDC as a function of lag. The resulting plots can be seen of Figures \ref{fig:idi} and \ref{fig:idc}.

\begin{lstlisting}[style=mypython,caption={The functions used for calculating IDI and IDC},label={lst:python}]
windowed_sum_cache = {}

def get_windowed_sum(array, size):
    assert size > 0
    if (array.ctypes.data, size) not in windowed_sum_cache:
        if size == 1:
            windowed_sum_cache[(array.ctypes.data, size)] = array
        else:
            prev = get_windowed_sum(array, size - 1)
            wsum = np.fromiter((prev[i] + array[i + size - 1] for i in range(0, len(prev) - 1)), np.float64,
                               len(prev) - 1)
            windowed_sum_cache[(array.ctypes.data, size)] = wsum
    return windowed_sum_cache[(array.ctypes.data, size)]

def idi(interarrirval_times, lag):
    assert lag > 0
    return np.var(get_windowed_sum(interarrirval_times, lag)) / (
            lag * np.mean(interarrirval_times) * np.mean(interarrirval_times))

def idc(packet_counts, t):
    assert t > 0
    return np.var(get_windowed_sum(packet_counts, t)) / (t * np.mean(packet_counts))
\end{lstlisting}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.9\textwidth]{figures/idi.png}
    \caption{Index of dispersion for Indices as function of lag}
    \label{fig:idi}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.9\textwidth]{figures/idc.png}
    \caption{Index of Dispersion for Counts as a function of lag}
    \label{fig:idc}
\end{figure}

\section{Conclusion}

As a part of this homework an arbitrarily chosen times series had to be analyzed similarly as presented in \cite{MolnarOnburst}.
The selected data file contained data about 2.2 million TCP packets. The related statistics and statistical metrics and their plots and analysis have been presented in this paper.

\bibliographystyle{unsrt}
\bibliography{references}

\end{document}


\documentclass[12pt]{article}

\usepackage{tikz}
\usepackage{hyperref}

\usetikzlibrary{shapes.geometric, arrows}

\begin{document}

\begin{tikzpicture}
  % Make a grid
  \draw[step=1,gray,very thin, dashed] (-4,-4) grid (4,4);
  % Make a rectangle (`cycle` makes a line between the first and last points)
  \draw (0,0) -- (3,3) -- (3,0) -- cycle;
  % Make a circle at origin with radius 1
  \draw (0,0) circle (1);
  \draw (3,3) circle (0.4);
  % Make an arc with the starting point at origin
  \draw[red] (0,0) arc (0:-90:2); % from 0 to -90 degrees, with radius=2
  \draw[blue] (0,0) arc (-90:-180:2);
  % Fill a rectangle with 50% red and 50% green
  \fill[red!50!green] (-4,4) rectangle (-3,3);
  % Fill and draw at the same time
  \filldraw[fill=red, draw=black] (-4,-4) rectangle (-3,-3);
  % Make a line
  \draw (-4,-5) -- (-3,-6) -- (-3,-6) -- (3,-6) -- (4,-5) ;
  % A few arrow styles
  \draw[thick,|->] (-4,-7) -- (-1,-7) ;
  \draw[thick,<-|] (1,-7) -- (4,-7) ;
  \draw[thick,|-<] (-4,-8) -- (-1,-8) ;
  \draw[thick,|>-|] (1,-8) -- (4,-8) ;
  % Add an annotation
  \draw (3.4,3.4) node[anchor=south west] {YOLO} ;
  % Write grid numbers
  \foreach \x in {-4,-3,-2,-1,0,1,2,3,4}
    \draw (\x,-4) node[anchor=north] {\x} ;
\end{tikzpicture}

\section{Overleaf Tutorial Workflow}

This example is taken from: \href{https://www.overleaf.com/learn/latex/LaTeX\_Graphics\_using\_TikZ:\_A\_Tutorial\_for\_Beginners\_(Part\_3)\%E2\%80\%94Creating\_Flowcharts}{here}

\tikzstyle{startstop} = [rectangle, rounded corners, minimum width=3cm, minimum height=1cm,text centered, draw=black, fill=red!30, text width=3cm]
\tikzstyle{io} = [trapezium, trapezium left angle=70, trapezium right angle=110, minimum width=3cm, minimum height=1cm, text centered, draw=black, fill=blue!30, text width=3cm]
% `text width=3cm` forces wrapping of long lines
\tikzstyle{process} = [rectangle, minimum width=3cm, minimum height=1cm, text centered, draw=black, fill=orange!30, text width=3cm]
\tikzstyle{decision} = [diamond, minimum width=3cm, minimum height=1cm, text centered, draw=black, fill=green!30]
\tikzstyle{arrow} = [thick,->,>=stealth]

\begin{tikzpicture}[node distance=2cm]
\node (start) [startstop] {Start};
\node (in1) [io, below of=start] {Input};
\node (pro1) [process, below of=in1] {Process 1};
\node (dec1) [decision, below of=pro1, yshift=-0.5cm] {Decision 1};
\node (pro2a) [process, below of=dec1, yshift=-1.5cm] {Process 2a text text text text text text text text text text};
\node (pro2b) [process, right of=dec1, xshift=2cm] {Process 2b};
\node (out1) [io, below of=pro2a, yshift=-0.3cm] {Output};
\node (stop) [startstop, below of=out1] {Stop};
\draw [arrow] (start) -- (in1);
\draw [arrow] (in1)   -- (pro1);
\draw [arrow] (pro1)  -- (dec1);
\draw [arrow] (dec1)  -- node[anchor=east] {yes} (pro2a);
\draw [arrow] (dec1)  -- node[anchor=south] {no} (pro2b);
\draw [arrow] (pro2b) |- (pro1);
\draw [arrow] (pro2a) -- (out1);
\draw [arrow] (out1)  -- (stop);
\end{tikzpicture}

\section{Synteny map}

%                      [-------]
% =====   =====   =====         =====
%   |       |       |             |
%   |       |       |             |
% =====   =====   =====  <--->  =====
% <--->  query interval
% [---]  search interval

\begin{tikzpicture}
  \foreach \x in {0,1,2,4} {
      \fill[black] (2*\x,1) rectangle (2*\x+1,1.2);
      \draw[dashed] (2*\x+0.5,1.2) -- (2*\x+0.5,2.2);
      \fill[black] (2*\x,2) rectangle (2*\x+1,2.2);
    }
  \draw[<->] (6,1.1) -- (7,1.1);
  \draw[|-|,dashed] (5,2.3) -- (8,2.3);
  % legend
  \draw[|-|,dashed] (0,0.5) -- (1,0.5);
  \draw (1,0.5) node[anchor=west] {search interval};
  \draw[<->] (0,0) -- (1,0);
  \draw (1,0) node[anchor=west] {query interval};
\end{tikzpicture}

\section{Interval tree}

\begin{figure}
  \centering
\begin{tikzpicture}
  \draw (1,-1) -- (0,0) -- (1,1) -- (6,1) -- (6.5,0.5);
  \draw (0.5,-0.5) -- (0,-1);
  \draw (0,0) -- (-0.5, -0.5);
  %
  \draw (-0.5,-0.5) node[anchor=north east] {...} ;
  \draw ( 0.0,-1.0) node[anchor=north east] {...} ;
  \draw ( 6.5, 0.5) node[anchor=north west] {...} ;
  \draw ( 5.3, 1.5) node[anchor=south west] {...} ;
  %
  \draw[dotted] (1,-1) -- (1,-2.2);
  \filldraw[fill=white] (1,-1) circle (0.15);
  \draw (1,-1) node {\tiny{A}};
  %
  \draw[dotted] (4.8,1) -- (4.8,-0.4);
  \draw (4.8,1) -- (5.3,1.5);
  \filldraw[fill=white] (4.8,1) circle (0.15);
  \draw (4.8,1) node {\tiny{B}};
  %
  \draw[dotted] (2,1) -- (2,-2.2);
  \draw[dotted] (4,1) -- (4,-2.2);
  %
  \draw[|-|] (4.2,  0.5) -- (5.2,  0.5);
  \draw[|-|] (4.4,  0.2) -- (5.4,  0.2);
  \draw[|-|] (4.6, -0.1) -- (5.6, -0.1);
  \draw (5.2, 0.5) node[anchor=west] {\tiny{$b_1$}};
  \draw (5.4, 0.2) node[anchor=west] {\tiny{$b_2$}};
  \draw (5.6,-0.1) node[anchor=west] {\tiny{$b_3$}};
  %
  \draw[|-|] (0.5, -1.7) -- (1.5, -1.7);
  \draw[|-|] (0.5, -2.0) -- (1.3, -2.0);
  \draw (0.5, -1.7) node[anchor=east] {\tiny{$a_1$}};
  \draw (0.5, -2.0) node[anchor=east] {\tiny{$a_2$}};
  %
  \draw[|-|,thick] (2.5, -1.7) -- (3.5, -1.7);
  %
  \draw (3,-1.75) node[anchor=north] {\tiny{query}} ;
\end{tikzpicture}
\caption{This is a diagram in a float}
\label{dia:tree}
\end{figure}

\end{document}


\input /zu/6001-devel/fall94/6001mac

\begin{document}

\psetheader{Sample programming assignment}{Prisoner's Dilemma}

\section{The Prisoner's Dilemma: A Fable}

In the mid-1920's, the Nebraska State Police achieved what may still
be their finest moment. After a 400-mile car chase over dirt roads and
corn fields, they finally caught up with the notorious bank robbers
Bonnie and Clyde. The two criminals were brought back to the police
station in Omaha for further interrogation.

Bonnie and Clyde were questioned in separate rooms, and each was
offered the same deal by the police. The deal went as follows (since
both are the same, we need only describe the version presented to
Bonnie):

``Bonnie, here's the offer that we are making to both you and Clyde.
If you both hold out on us, and don't confess to bank robbery, then we
admit that we don't have enough proof to convict you. However, we {\it
will} be be able to jail you both for one year, for reckless driving
and endangerment of corn. If you turn state's witness and help us
convict Clyde (assuming he doesn't confess), then you will go free,
and Clyde will get twenty years in prison. On the other hand, if you
don't confess and Clyde does, then {\it he} will go free and {\it you}
will get twenty years.''

``What happens if both Clyde and I confess?'' asked Bonnie.

``Then you both get ten years,'' said the interrogator.

Bonnie, who had been a math major at Cal Tech before turning to crime,
reasoned this way: ``Suppose Clyde intends to confess.  Then if I
don't confess, I'll get twenty years, but if I do confess, I'll only
get ten years. On the other hand, suppose Clyde intends to hold out on
the cops. Then if I don't confess, I'll go to jail for a year, but if
I do confess, I'll go free.  So no matter what Clyde intends to do, I
am better off confessing than holding out. So I'd better confess.''

Naturally, Clyde employed the very same reasoning. Both criminals
confessed, and both went to jail for ten years.\footnote{Actually,
they didn't go to jail. When they were in court, and heard that they
had both turned state's witness, they strangled each other.  But
that's another story.} The police, of course, were triumphant, since
the criminals would have been free in a year had both remained silent.

\section{The Prisoner's Dilemma}

The Bonnie and Clyde story is an example of a situation known in
mathematical game theory as the ``prisoner's dilemma.'' A prisoner's
dilemma always involves two ``game players,'' and each has a choice
between ``cooperating'' and ``defecting.'' If the two players
cooperate, they each do moderately well; if they both defect, they
each do moderately poorly. If one player cooperates and the other
defects, then the defector does extremely well and the cooperator does
extremely poorly. (In the case of the Bonnie and Clyde story,
``cooperating'' means cooperating with one's partner --- i.e., holding
out on the police --- and ``defecting'' means confessing to bank
robbery.) Before formalizing the prisoner's dilemma situation, we need
to introduce some basic game theory notation.

\subsection{A Crash Course in Game Theory}

In game theory, a {\it two-person binary-choice game} is represented
as a two-by-two matrix. Figure 1 shows a hypothetical game matrix.
The two players in this case are called {\bf A} and {\bf B}, and
the choices are called ``cooperate'' and ``defect.''

\begin{center}
\begin{tabular}{c|c|c|}
             & B cooperates   & B defects  \\
\cline{2-3}
A cooperates & A gets 5       & A gets 2   \\
             & B gets 5       & B gets 3   \\
\cline{2-3}
A defects    & A gets 3       & A gets 1 \\
             & B gets 2       & B gets 1 \\
\cline{2-3}
\end{tabular}
\end{center}

\centerline{{\it Figure 1:\/} A hypothetical game matrix}

Players {\bf A} and {\bf B} can play a single game by separately (and
secretly) choosing to either cooperate or defect. Once each player has
made a choice, he announces it to the other player; and the two then
look up their respective scores in the game matrix.  Each entry in the
matrix is a pair of numbers indicating a score for each player,
depending on their choices. Thus, in Figure 1, if Player {\bf A}
chooses to cooperate while Player {\bf B} defects, then {\bf A} gets 2
points and {\bf B} gets 3 points. If both players defect, they each
get 1 point. Note, by the way, that the game matrix is a matter of
public knowledge; for instance, Player {\bf A} knows before the game
even starts that if he and {\bf B} both choose to defect, they will
each get 1 point.

In an {\it iterated game}, the two players play repeatedly: thus,
after finishing one game, {\bf A} and {\bf B} may play another.
(Admittedly, there is a little confusion in the terminology here: you
can think of each individual game as a single ``round'' of the larger,
iterated game.) There are a number of ways in which iterated games may
be played; in the simplest situation, {\bf A} and {\bf B} play for
some fixed number of rounds (say, 200), and before each round they are
able to look at the record of all previous rounds. For instance,
before playing the tenth round of their iterated game, both {\bf A}
and {\bf B} are able to study the results of the previous nine rounds.

\subsection{An Analysis of a Simple Game Matrix}

The game depicted in Figure 1 is a particularly easy one to analyze.
Let's examine the situation from Player {\bf A}'s point of view
(Player {\bf B}'s point of view is identical):

``Suppose {\bf B} cooperates. Then I do better by cooperating
myself (I receive five points instead of three). On the other
hand, suppose {\bf B} defects. I still do better by cooperating
(since I get two points instead of one). So no matter what
{\bf B} does, I am better off cooperating.''

Player {\bf B} will, of course, reason the same way, and both will
choose to cooperate. In the terminology of game theory, both {\bf A}
and {\bf B} have a {\it dominant} choice --- i.e., a choice that gives
a preferred outcome no matter what the other player chooses to do.
Figure 1, by the way, does {\it not} represent a prisoner's dilemma
situation, since when both players make their dominant choice, they
also both achieve their highest personal scores.  We'll see an example
of a prisoner's dilemma game very shortly.

To recap: in any particular game using the matrix of Figure 1, we
would expect both players to cooperate; and in an iterated game, we would
expect both players to cooperate repeatedly, on every round.

\subsection{The Prisoner's Dilemma Game Matrix}

Now consider the game matrix shown in Figure 2.

\begin{center}
\begin{tabular}{c|c|c|}
             & B cooperates   & B defects  \\
\cline{2-3}
A cooperates & A gets 3       & A gets 0   \\
             & B gets 3       & B gets 5   \\
\cline{2-3}
A defects    & A gets 5       & A gets 1 \\
             & B gets 0       & B gets 1 \\
\cline{2-3}
\end{tabular}
\end{center}

\centerline{{\it Figure 2:\/} Prisoner's Dilemma Game Matrix}

In this case, Players {\bf A} and {\bf B} both have a dominant
choice---namely, defection. No matter what Player {\bf B} does,
Player {\bf A} improves his own score by defecting, and vice versa.

However, there is something odd about this game. It seems as though
the two players would benefit by choosing to cooperate. Instead of
winning only one point each, they could win three points each.  So the
``rational'' choice of mutual defection has a puzzling
self-destructive flavor.

The matrix of Figure 2 is an example of a prisoner's dilemma game
situation. Just to formalize the situation, let {\it CC} be the number
of points won by each player when they both cooperate; let {\it DD} be
the number of points won when both defect; let {\it CD} be the number
of points won by the cooperating party when the other defects; and let
{\it DC} be the number of points won by the defecting party when the
other cooperates. Then the prisoner's dilemma situation is
characterized by the following conditions:

\begin{eqnarray*}
DC > CC > DD > CD \\
CC > (DC + CD) / 2
\end{eqnarray*}

In the game matrix of Figure 2, we have:

\begin{eqnarray*}
DC = 5 \\
CC = 3 \\
DD = 1 \\
CD = 0
\end{eqnarray*}

\noindent so both conditions are met. In the Bonnie and Clyde story,
by the way, you can verify that:

\begin{eqnarray*}
DC = 0 \\
CC = -1 \\
DD = -10 \\
CD = -20
\end{eqnarray*}

\noindent Again, these values satisfy the prisoner's dilemma conditions.

\section{Axelrod's Tournament}

In the late 1970's, political scientist Robert Axelrod held a computer
tournament designed to investigate the prisoner's dilemma
situation.\footnote{Actually, there were two tournaments. Their rules
and results are described in Axelrod's book {\sl The Evolution of
Cooperation}.} Contestants in the tournament submitted computer
programs that would compete in an iterated prisoner's dilemma game of
approximately two hundred rounds, using the same matrix shown in
Figure 2. Each contestant's program played five iterated games against
each of the other programs submitted, and after all games had been
played the scores were tallied.

The contestants in Axelrod's tournament included professors of
political science, mathematics, psychology, computer science, and
economics. The winning program --- the program with the highest
average score --- was submitted by Anatol Rapoport, a professor of
psychology at the University of Toronto. In this problem set, we will
pursue Axelrod's investigations and make up our own Scheme programs to
play the iterated prisoner's dilemma game. The second (optional) part
of this problem set is itself an experiment in the spirit of Axelrod's
tournament: a contest of programs that play a {\it three-person}
prisoner's dilemma game.

Before we look at the two-player program, it is worth speculating on
what possible strategies might be employed in the iterated prisoner's
dilemma game. Here are some examples:

\begin{description}

\item[All-Defect]
A program using the {\bf all-defect} strategy simply defects on
every round of every game.

\item[Poor-Trusting-Fool]
A program using the {\bf poor-trusting-fool} strategy cooperates
on every round of every game.

\item[Random]
This program cooperates or defects on a random basis.

\item[Go-by-Majority]
This program cooperates on the first round. On all subsequent rounds,
{\bf go-by-majority} examines the history of the other player's
actions, counting the total number of defections and cooperations by
the other player. If the other player's defections outnumber her
cooperations, {\bf go-by-majority} will defect; otherwise this
strategy will cooperate.

\item[Tit-for-Tat]
This program cooperates on the first round, and then on every subsequent
round it mimics the other player's previous move. Thus, if the other
player cooperates (defects) on the $n$th round, then {\bf tit-for-tat}
will cooperate (defect) on the $(n + 1)$th round.
\end{description}

All of these strategies are extremely simple. (Indeed, the first three
do not even pay any attention to the other player; their responses are
uninfluenced by the previous rounds of the game.)  Nevertheless,
simplicity is not necessarily a disadvantage.  Rapoport's first-prize
program employed the {\bf tit-for-tat} strategy, and achieved the
highest average score in a field of far more complicated programs.

\section{The Two-Player Prisoner's Dilemma Program}

A Scheme program for an iterated prisoner's dilemma game is shown at
the end of this problem set. The procedure {\tt play-loop} pits two
players (or, to be more precise, two ``strategies'') against one another
for approximately 100 games, then prints out the scores for each of
the two players.

Player strategies are represented as procedures. Each strategy takes
two inputs --- its own ``history'' (that is, a list of all of its
previous ``plays'') and its opponent's ``history.'' The strategy
returns either the symbol {\tt c} (for ``cooperate'') or the symbol
{\tt d} (for ``defect'').

At the beginning of an iterated game, each history is an empty list.
As the game progresses, the histories grow (via {\tt extend-history})
into lists of {\tt c}'s and {\tt d}'s. Note how each strategy must
have its {\it own} history as its first input. So in {\tt
play-loop-iter}, {\tt strat0} has {\tt history0} as its first input,
and {\tt strat1} has {\tt history1} as its first input.

The values from the game matrix are stored in a list named
{\tt *game-association-list*}. This list is used to calculate the scores at
the end of the iterated game.

Some sample strategies are given at the end of the program. {\tt
All-defect} and {\tt poor-trusting-fool} are particularly simple; each
returns a constant value regardless of the histories. {\tt
Random-strategy} also ignores the histories and chooses randomly
between cooperation and defection. You should study {\tt
go-by-majority} and {\tt tit-for-tat} to see that their behavior is
consistent with the descriptions in the previous section.

\paragraph{Problem 0 (no write-up necessary)}
Use {\tt play-loop} to play games among the five defined strategies.
Notice how a strategy's performance varies sharply depending on its
opponent.  For example, {\tt poor-trusting-fool} does quite well
against {\tt tit-for-tat} or against another {\tt poor-trusting-fool},
but it loses badly to {\tt all-defect}.  Pay special attention to {\tt
tit-for-tat}. Notice how it never beats its opponent --- but it never
loses badly.

\paragraph{Problem 1}
{\it a.} Games involving {\tt go-by-majority} tend to be slower than
other games. Why is that so? Use order-of-growth notation to explain
your answer.

{\it b.} Alyssa P. Hacker, upon seeing the code for {\tt go-by-majority},
suggested the following iterative version of the procedure:

\beginlisp
(define (go-by-majority my-history other-history)
  (define (majority-loop cs ds hist)
    (cond ((empty-history? hist) (if (> ds cs) 'd 'c))
          ((eq? (most-recent-play hist) 'c)
           (majority-loop (1+ cs) ds (rest-of-plays hist)))
          (else
           (majority-loop cs (1+ ds) (rest-of-plays hist)))))
  (majority-loop 0 0 other-history))
\endlisp

Compare this procedure with the original version. Do the orders of
growth (in time) for the two procedures differ? Is the newer version
faster?

\paragraph{Problem 2}
Write a new strategy {\tt tit-for-two-tats}. The strategy should always
cooperate unless the opponent defected on both of the previous two
rounds. (Looked at another way: {\tt tit-for-two-tats} should cooperate if
the opponent cooperated on either of the previous two rounds.) Play
{\tt tit-for-two-tats} against other strategies.

\paragraph{Problem 3}
Write a procedure {\tt make-tit-for-n-tats}. This procedure should
take a number as input and return the appropriate {\tt
tit-for-tat}-like strategy.  For example, {\tt (make-tit-for-n-tats
2)} should return a strategy equivalent to {\tt tit-for-two-tats}.

\paragraph{Problem 4}
{\it a.} Write a procedure {\tt make-dual-strategy} which takes as
input two strategies (say, {\tt strat0} and {\tt strat1}) and an
integer (say, {\tt switch-point}). {\tt Make-dual-strategy} should
return a strategy which plays {\tt strat0} for the first {\tt
switch-point} rounds in the iterated game, then switches to {\tt
strat1} for the remaining rounds.

{\it b.} Use {\tt make-dual-strategy} to define a procedure {\tt
make-triple-strategy} which takes as input three strategies and two
switch points.

\paragraph{Problem 5}
Write a procedure {\tt niceify}, which takes as input a strategy (say,
{\tt strat}) and a number between 0 and 1 (call it {\tt
niceness-factor}). The {\tt niceify} procedure should return a
strategy that plays the same as {\tt strat} except: when {\tt strat}
defects, the new strategy should have a {\tt niceness-factor}
chance of cooperating.  (If {\tt niceness-factor} is 0, the returned
strategy is exactly the same as {\tt strat}; if {\tt niceness-factor}
is 1, the returned strategy is the same as {\tt poor-trusting-fool}.)

Use {\tt niceify} with a low value for {\tt niceness-factor} --- say,
0.1 --- to create two new strategies: {\tt slightly-nice-all-defect}
and {\tt slightly-nice-tit-for-tat}.

\section{The Three-Player Prisoner's Dilemma}

So far, all of our prisoner's dilemma examples have involved two
players (and, indeed, most game-theory research on the prisoner's
dilemma has focused on two-player games). But it is possible to create
a prisoner's dilemma game involving three --- or even more ---
players.

Strategies from the two-player game do not necessarily extend to a
three-person game in a natural way. For example, what does {\tt
tit-for-tat} mean? Should the player defect if {\it either} of the
opponents defected on the previous round? Or only if {\it both}
opponents defected? And are either of these strategies nearly as
effective in the three-player game as {\tt tit-for-tat} is in the
two-player game?

Before we analyze the three-player game more closely, we must
introduce some notation for representing the payoffs. We use a
notation similar to that used for the two-player game. For example, we
let DCC represent the payoff to a defecting player if both opponents
cooperate. Note that the first position represents the player under
consideration. The second and third positions represent the opponents.

Another example: CCD represents the payoff to a cooperating player if
one opponent cooperates and the other opponent defects.  Since we
assume a symmetric game matrix, CCD could be written as CDC. The
choice is arbitrary.

Now we are ready to discuss the payoffs for the three-player game. We
impose three rules:\footnote{Actually, there is no universal
definition for the multi-player prisoner's dilemma. The constraints
used here represent one possible version of the three-player
prisoner's dilemma.}

\noindent 1) Defection should be the dominant choice for each player.
In other words, it should always better for a player to defect,
regardless what the opponents do. This rule gives three constraints:

\begin{eqnarray*}
DCC > CCC  & {\rm (both\ opponents\ cooperate)}\\
DDD > CDD  & {\rm (both\ opponents\ defect)}\\
DCD > CCD  & {\rm (one\ opponent\ cooperates,\ one\ defects)}
\end{eqnarray*}

\noindent 2) A player should always be better off if more of his
opponents choose to cooperate. This rule gives:

\begin{eqnarray*}
DCC > DCD > DDD \\
CCC > CCD > CDD
\end{eqnarray*}

\noindent 3) If one player's choice is fixed, the other two players should be
left in a two-player prisoner's dilemma. This rule gives the following
constraints:

\begin{eqnarray*}
CCD > DDD \\
CCC > DCD \\
CCD > (CDD + DCD) / 2 \\
CCC > (CCD + DCC) / 2
\end{eqnarray*}

\begin{eqnarray*}
CDD = 0 \\
DDD = 1 \\
CCD = 3 \\
DCD = 5 \\
CCC = 7 \\
DCC = 9
\end{eqnarray*}

\paragraph{Problem 6}
Revise the Scheme code for the two-player game to make a three-player
iterated game. The program should take three strategies as input, keep
track of three histories, and print out results for three players. You
need to change only three procedures: {\tt play-loop}, {\tt
print-out-results}, and {\tt get-scores}. You also need to change
{\tt *game-association-list*} as follows:

\beginlisp
(define *game-association-list*
  '(((c c c) (7 7 7))
    ((c c d) (3 3 9))
    ((c d c) (3 9 3))
    ((d c c) (9 3 3))
    ((c d d) (0 5 5))
    ((d c d) (5 0 5))
    ((d d c) (5 5 0))
    ((d d d) (1 1 1))))
\endlisp

\paragraph{Problem 7}
{\it a.} Write strategies {\tt poor-trusting-fool-3}, {\tt all-defect-3}, and
{\tt random-strategy-3} that will work in a three-player game. Try
them out to make sure your code is working.

{\it b.} Write two new strategies: {\tt tough-tit-for-tat} and {\tt
soft-tit-for-tat}. {\tt tough-tit-for-tat} should defect if {\it
either} of the opponents defected on the previous round.  {\tt
soft-tit-for-tat} should defect only if {\it both} opponents defected
on the previous round. Play some games using these two new strategies.

\paragraph{Problem 8}
A natural idea in creating a prisoner's dilemma strategy is to
try and deduce what kind of strategies the {\it other} players might
be using. In this problem, we will implement a simple version of
this idea.

First, we need a procedure that takes three histories as arguments:
call them {\tt hist-0}, {\tt hist-1}, and {\tt hist-2}. The idea is
that we wish to characterize the strategy of the player responsible
for {\tt hist-0}. Our procedure will return a list of three numbers:
the probability that the {\tt hist-0} player cooperates given that the
other two players cooperated on the previous round, the probability
that the {\tt hist-0} player cooperates given that only one other
player cooperated on the previous round, and the probability that the
{\tt hist-0} player cooperates given that both others defected on the
previous round. To fill out some details in this picture, let's look
at a couple of examples. We will call our procedure {\tt
get-probability-of-c}; here are a couple of sample calls.

\beginlisp
==> (get-probability-of-c '(c c c c) '(d d d c) '(d d c c))
(1 1 1)
\null
==> (get-probability-of-c '(c c c d c) '(d c d d c) '(d c c c c))
(0.5 1 ())
\endlisp

In the top example, the returned list indicates that the first player
cooperates with probability 1 no matter what the other two players do.
In the bottom example, the first player cooperates with probability
0.5 when the other two players cooperate; the first player cooperates
with probability 1 when one of the other two players defects; and
since we have no data regarding what happens when both other players
defect, our procedure returns {\tt ()} for that case.

Write the {\tt get-probability-of-c} procedure. Using this procedure,
you should be able to write some predicate procedures that help
in deciphering another player's strategy. For instance, here are
two possibilities:

\beginlisp
(define (is-he-a-fool? hist0 hist1 hist2)
  (equal? '(1 1 1) (get-probability-of-c hist0 hist1 hist2)))
\null
(define (could-he-be-a-fool? hist0 hist1 hist2)
  (equal? (list true true true)
          (map (lambda (elt) (or (null? elt) (eqv? elt 1)))
               (get-probability-of-c hist0 hist1 hist2))))
\endlisp

Use the {\tt get-probability-of-c} procedure to write a predicate that
tests whether another player is using the {\tt soft-tit-for-tat}
strategy from Problem 7. Also, write a new strategy named {\tt
dont-tolerate-fools}.  This strategy should cooperate for the first
ten rounds; on subsequent rounds it checks (on each round) to see
whether the other players might both be playing {\tt
poor-trusting-fool}.  If our strategy finds that both other players
seem to be cooperating uniformly, it defects; otherwise, it
cooperates.

\paragraph{Problem 9}
Write a procedure {\tt make-combined-strategies} which takes as input
two {\it two-player} strategies and a ``combining'' procedure.  {\tt
Make-combined-strategies} should return a {\it three-player} strategy
that plays one of the two-player strategies against one of the
opponents, and the other two-player strategy against the other
opponent, then calls the ``combining'' procedure on the two two-player
results. Here's an example: this call to {\tt
make-combined-strategies} returns a strategy equivalent to {\tt
tough-tit-for-tat} in Problem 7.

\beginlisp
(make-combined-strategies
  tit-for-tat tit-for-tat
  (lambda (r1 r2) (if (or (eq? r1 'd) (eq? r2 'd)) 'd 'c)))
\endlisp

The resulting strategy plays {\tt tit-for-tat} against each
opponent, and then calls the combining procedure on the two results.
If either of the two two-player strategies has returned {\tt d}, then
the three-player strategy will also return {\tt d}.

Here's another example. This call to {\tt make-combined-strategies}
returns a three-player strategy that plays {\tt tit-for-tat} against
one opponent, {\tt go-by-majority} against another, and chooses randomly
between the two results:

\beginlisp
(make-combined-strategies
  tit-for-tat go-by-majority
  (lambda (r1 r2) (if (= (random 2) 0) r1 r2)))
\endlisp

\section{Extra Credit: The Three-Player Prisoner's Dilemma Tournament}

As decribed earlier, Axelrod held two computer tournaments to
investigate the two-player prisoner's dilemma. Last semester, 6.001
students made game theory history by participating in the world's
first THREE-player prisoner's dilemma tournament. Now, in a return
engagement, the second-ever three-player tournament will be held.

Last semester's results indicated that basically cooperative
strategies do very well. The winning strategy, for instance,
defected only if both opponents had each defected twice in a row. And
the third place strategy (out of more than fifty entries) was simply
{\tt poor-trusting-fool-3}!

You can participate this year by designing a strategy for the
tournament. You might submit one of the strategies developed in the
problem set, or develop a new one. The only restriction is that the
strategy must work against any other legitimate entry. Any strategies
that cause the tournament software to crash will be disqualified.

Instructions for entering your strategy in the tournament will be
provided on a separate handout and posted in the laboratory.

\end{document}