\documentclass{article}

\title{Pig Analysis}
\author{Andrew Pennebaker}
\date{December 14, 2010}

\usepackage[
    pdfauthor={Andrew Pennebaker},
    pdftitle={Pig Analysis},
    pdfborder={0 0 0},
    urlbordercolor={1 1 1}
]{hyperref}

\begin{document}

\maketitle
\begin{center}
\url{https://github.com/mcandre/pig}
\end{center}

\section*{Overview}
Pig is an old dice game for two or more players. They need not be human, as in an automated monte carlo simulation. Pig's simplicity lends itself as an educational tool for studying statistics, game theory, and numerical analysis.

\subsection*{Game Requirements}
\begin{itemize}
\item A six-sided die (1d6)
\item A scoring mechanism (e.g.\ pencil and paper)
\end{itemize}

\subsection*{Objective}
Be the first player to score 100 points.

\subsection*{Rules}
On a turn, a player continually rolls 1d6, recording the pips as runs, until either
\begin{itemize}
\item The player rolls 1 (pigging)
\item The player decides to stop rolling
\end{itemize}
If the player pigs, the turn ends and the next player gets to roll.\\
Otherwise, the sum of the run is added to the player's score.

\section*{Strategies}
There are several strategies for playing Pig, each with their advantages and disadvantages.

\subsection*{Always Hold}
$points/turn = 0$
\\\\
$E[turns] = \infty$
\\\\
This strategy never wins. Fortunately, skipping affords more time to other players.

\subsection*{Always Roll}
$p(pigging) = \frac{1}{6}$
\\\\
$p(scoringRoll) = 1 - p(pigging) = \frac{5}{6}$
\\\\
$0.5 = 1 - p(scoringRoll)^{E[runLength]}$
\\\\
$E[runLength] = \frac{ln 0.5}{ln \frac{5}{6}} > 3.5$
\\\\
$E[turns] = \infty$
\\\\
As with Always Hold, Always Roll never wins, insanely pigging even when a lucky run would total 100+ and win the game. Additionally, Always Roll requires an average of 4 rolls before pigging and passing the turn. Always Roll can halt the game for arbitrarily long turns.

\subsection*{100 or Bust}
A minor variation on Always Roll, this strategy attempts to win by rolling a 100+ point run in a single turn.
\\\\
$E[rollScore] = \frac{0 + 2 + 3 + 4 + 5 + 6}{6} = \frac{10}{3} > 3$
\\\\
$E[winningRunLength] = \frac{100}{E[rollScore]} = 30$
\\\\
$p(winningRun) = p(scoringRoll)^{30} = (\frac{5}{6})^{30} < 1\%$
\\\\
$0.5 = 1 - p(winningRun)^{E[turns]}$
\\\\
$E[turns] = ln_{(\frac{5}{6})^{30}} 0.5 < 8$
\\\\
As unlikely as 100 or Bust is to win on any given turn, the chance of winning in $n$ turns dramatically increases. 100 or Bust is not the best strategy, but it does win games every now and then.

\subsection*{Roll Once}
$E[turns] = \frac{100}{E[rollScore]} = 30$
\\\\
This strategy is simple to implement. Intuitively, it seems like it would win more often than 100 or Bust. However, Roll Once is actually too conservative. $\frac{10}{3}$ points per turn is too few, and 100 or Bust will tend to reach 100 more quickly and therefore win more often.
\subsection*{Roll $n$}
Rolling only once per turn is sub-optimal because $p(pigging) = \frac{1}{6}$. Rolling more times per turn will gain more points, balancing against increasing odds of pigging.
\\\\
$E[scorePerTurn] = 4n \cdot p(scoringRoll)^n$
\\\\
$E[scorePerTurn; n = 1] = \frac{10}{3} > 3$
\\\\
$E[scorePerTurn; n = 2] = \frac{50}{9} > 5$
\\\\
$E[scorePerTurn; n = 3] = \frac{125}{18} > 6$
\\\\
$E[scorePerTurn; n = 4] = \frac{625}{81} > 7$
\\\\
$E[scorePerTurn; n = 5] = \frac{15625}{1944} > 8$
\\\\
$E[scorePerTurn; n = 6] = \frac{15625}{1944} > 8$
\\\\
$E[scorePerTurn; n = 7] = \frac{546875}{69984} < 8$
\\\\
\ldots
\\\\
$E[scorePerTurn; n = 26] < 1$
\\\\
$n$ is optimal at 5 or 6. Since the average score per turn for these is the same, the player might as well choose 5 rolls instead of 6 to save time.
\\\\
A strategy of rolling 26 times per turn would average less than 1 point per turn because the odds of pigging in 26 rolls is so high, canceling out any run points.
\\\\
\subsection*{Roll $k$}
This strategy defines $k$ as a value relative to the state of the game. For example, $k$ could be proportional to the current player scores.
\\\\
$maxPlayerScore = 98$
\\\\
$p(maxPlayerWinsNextTurn) = p(scoringRoll) = \frac{5}{6}$
\\\\
In this situation, some players have greater odds of winning--they could win next turn in one roll. In Pig, the only way to win is to win before they win. In other words, to defeat an opponent with a score of 98, another player must win first. Therefore, it is optimal for losing players to roll until they win, employing the 100 or Bust strategy. As long as an opponent player will likely win on their next roll, players have nothing to lose.
\\\\
$k$ could be relative to the max player score. At the beginning of the game, when players start with score 0, $k$ might be 5, the optimal value from the Roll $n$ strategy. Towards the end, when other players near 100, $k$ increases, approaching $\infty$, the 100 or Bust strategy.
\\\\
Conversely, when a player is winning and close to 100 points, $k$ decreases to 1, the Roll Once strategy. At $score = 98$, the next roll will either win the game or pig.

\section*{Monte Carlo}
An automated simulation for evaluating Pig strategies was implemented in Haskell. As a pure, functional, type-safe language, Haskell may seem unequipped to handle rolling dice, a necessarily non-pure computation (either stateful for pseudorandom number generators or I/O intensive for natural random number generation). In fact, Haskell is up to the task. A custom $roll$ function returns a random number from 1 to 6.
\begin{verbatim}
roll :: IO Int
roll = runRVar (choice [1..6]) DevRandom
\end{verbatim}
Some strategies are harder to implement than others, but a Pig strategy is nonetheless a function which takes a subset of the information in the game state and decides the next move. In reality, game state includes various and sundry information, such as whether lunch is nearly over or whether there's a cute girl nearby. Metastrategies such as substituting trick dice or bullying players into skipping turns would significantly add to simulation complexity. For the purposes of the monte carlo, the game information reduces to the player's current scores and the status of the current run. Pig is such a simple game that there are only two moves: roll or hold.
\begin{verbatim}
data Move = Roll | Hold

type Strategy = [Player] -> [Int] -> Move
\end{verbatim}
A player can be modeled with three variables: name, strategy, and current score. With extra effort, these could be treated as two variables, where a strategy encodes its own name programmatically for reference purposes.
\begin{verbatim}
data Player = Player {
    name :: String,
    strategy :: Strategy,
    score :: Int
    }
\end{verbatim}
In detail, the information available to a player consists of a rotating list of players, where the first player in the list is the current player, and a list of the current player's run (an empty list if the player has not yet rolled).
\\\\
The Always Hold strategy disregards game state and always holds; a constant function. Haskell uses an underscore (\_) to denote conditions irrelevant to a computation. Both player list and run list are irrelevant, so they may be underscored from the function inputs. Haskell uses double colons (::) to specify types for robustness, clarity, and code generation. Many of them can be omitted (Haskell has a powerful type inference system), but the signatures shall remain here for readability.
\begin{verbatim}
alwaysHold :: Strategy
alwaysHold _ _ = Hold
\end{verbatim}
Similarly, the Always Roll strategy disregards game state.
\begin{verbatim}
alwaysRoll :: Strategy
alwaysRoll _ _ = Roll
\end{verbatim}
The 100 or Bust strategy is implemented using list pattern matching (:) to extract the current player and guards (\textbar) as syntactic sugar for conditionals. There is a subtle bug: Should a simulation run with no players, the $hundredOrBust$ strategy will return an error as it is not defined for a list without at least one players. The simulation can gain robustness by adding a clause after the signature to handle the case where the player list is empty ([]).
\begin{verbatim}
hundredOrBust :: Strategy
hundredOrBust (p:ps) rs
    | score p + sum rs >= 100 = Hold
    | otherwise = Roll
\end{verbatim}
Arbitrary strategies can be implemented once Haskell proficiency is achieved.
\\\\
Creating players and their strategies is boilerplate coding, but boilerplate can be minimized with the use of default constructors (and further minimized with Lisp macros).
\begin{verbatim}
defaultPlayer :: Player
defaultPlayer = Player {
    name = "Player",
    strategy = roll5,
    score = 0
    }

rb = defaultPlayer { name = "Roll Bad K", strategy = rollBadK }
\end{verbatim}
A default player is assumed to be using the Roll 5 strategy, and all players start the game with 0 points. Here, a player employs a particularly sub-optimal strategy, Bad $k$, the opposite of Roll $k$. The player will begin by rolling too conservatively, staying behind his opponents' scores, and in the unlikely event his score surpasses his peers', he will roll recklessly when he is winning, handing the game back to them.
\\\\
Statistical analysis of a simple folk game may be overkill. In any case, the rest of the code is tedium: shuffling players and running $n = 10,000$ simulated games. $n$ appears to be large enough that running $n+$ simulations would produce similar results, $\pm 1\%$.
\begin{verbatim}
$ shake
$ ./pig
Running 10000 games...
Totaling wins...

Roll K Times    30%
Roll Six    30%
Roll Five 27%
100 or Bust 8%
Roll Bad K    3%
Always Hold 0%
Always Roll 0%
Roll Once 0%
\end{verbatim}
\end{document}


%
% API Documentation for Peach - Computational Intelligence for Python
% Module peach.nn.base
%
% Generated by epydoc 3.0beta1
% [Mon Dec 21 08:51:37 2009]
%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%                          Module Description                           %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}|(}
\section{Module peach.nn.base}

    \label{peach:nn:base}

Basic definitions for layers of neurons.

This subpackage implements the basic classes used with neural networks. A neural
network is basically implemented as a layer of neurons. To speed things up, a
layer is implemented as a array, where each line represents the weight vector
of a neuron. Further definitions and algorithms are based on this definition.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%                               Variables                               %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

  \subsection{Variables}

\begin{longtable}{|p{.30\textwidth}|p{.62\textwidth}|l}
\cline{1-2}
\cline{1-2} \centering \textbf{Name} & \centering \textbf{Description}& \\
\cline{1-2}
\endhead\cline{1-2}\multicolumn{3}{r}{\small\textit{continued on next page}}\\\endfoot\cline{1-2}
\endlastfoot\raggedright \_\-\_\-d\-o\-c\-\_\-\_\- & \raggedright \textbf{Value:}
{\tt \texttt{...}}&\\
\cline{1-2}
\raggedright \_\-B\-I\-A\-S\- & \raggedright This constant vector is defined to implement in a fast way the bias of a
neuron, as an input of value 1, stacked over the real input to the neuron.

\textbf{Value:}
{\tt array([[ 1.]])}&\\
\cline{1-2}
\end{longtable}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%                           Class Description                           %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}|(}
\subsection{Class Layer}

    \label{peach:nn:base:Layer}
\begin{tabular}{cccccc}
% Line for object, linespec=[False]
\multicolumn{2}{r}{\settowidth{\BCL}{object}\multirow{2}{\BCL}{object}}
&&
  \\\cline{3-3}
  &&\multicolumn{1}{c|}{}
&&
  \\
&&\multicolumn{2}{l}{\textbf{peach.nn.base.Layer}}
\end{tabular}

\textbf{Known Subclasses:} peach.nn.nn.SOM

Base class for neural networks.

This class implements a layer of neurons. It is represented by a array of
real values. Each line of the array represents the weight vector of a
single neuron. If the neurons on the layer are biased, then the first
element of the weight vector is the bias weight, and the bias input is
always valued 1. Also, to each layer is associated an activation function,
that determines if the neuron is fired or not. Please, consult the module
\texttt{af} to see more about activation functions.

In general, this class shoulb be subclassed if you want to use neural nets.
But, as neural nets are very different one from the other, check carefully
the documentation to see if the attributes, properties and methods are
suited to your task.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%                                Methods                                %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

  \subsubsection{Methods}

    \label{peach:nn:base:Layer:__call__}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_call\_\_ \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_call\_\_}(\textit{self}, \textit{x})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

The feedforward method to the layer.

The \texttt{{\_}{\_}call{\_}{\_}} interface should be called if the answer of the neuron
to a given input vector \texttt{x} is desired. \emph{This method has collateral
effects}, so beware. After the calling of this method, the \texttt{v} and
\texttt{y} properties are set with the activation potential and the answer of
the neurons, respectivelly.
    \vspace{1ex}

      \textbf{Parameters}
      \begin{quote}
        \begin{Ventry}{x}

          \item[x]

The input vector to the layer.
        \end{Ventry}

      \end{quote}

    \vspace{1ex}

      \textbf{Return Value}
      \begin{quote}

The vector containing the answer of every neuron in the layer, in the
respective order.
      \end{quote}

    \vspace{1ex}

    \end{boxedminipage}

    \label{object:__delattr__}
    \index{object.\_\_delattr\_\_ \textit{(function)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_delattr\_\_}(\textit{...})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

x.{\_}{\_}delattr{\_}{\_}('name') {\textless}=={\textgreater} del x.name
    \vspace{1ex}

    \end{boxedminipage}

    \label{object:__getattribute__}
    \index{object.\_\_getattribute\_\_ \textit{(function)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_getattribute\_\_}(\textit{...})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

x.{\_}{\_}getattribute{\_}{\_}('name') {\textless}=={\textgreater} x.name
    \vspace{1ex}

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__getbias}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_getbias \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_getbias}(\textit{self})

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__getinputs}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_getinputs \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_getinputs}(\textit{self})

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__getitem__}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_getitem\_\_ \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_getitem\_\_}(\textit{self}, \textit{n})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

The \texttt{{[} {]}} get interface.

The input to this method is forwarded to the \texttt{weights} property. That
means that it will return the respective line/element of the weight
array.
    \vspace{1ex}

      \textbf{Parameters}
      \begin{quote}
        \begin{Ventry}{x}

          \item[n]

A slice object containing the elements referenced. Since it is
forwarded to an array, it behaves exactly as one.
        \end{Ventry}

      \end{quote}

    \vspace{1ex}

      \textbf{Return Value}
      \begin{quote}

The element or elements in the referenced indices.
      \end{quote}

    \vspace{1ex}

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__getphi}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_getphi \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_getphi}(\textit{self})

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__getshape}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_getshape \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_getshape}(\textit{self})

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__getsize}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_getsize \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_getsize}(\textit{self})

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__getv}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_getv \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_getv}(\textit{self})

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__getweights}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_getweights \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_getweights}(\textit{self})

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__gety}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_gety \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_gety}(\textit{self})

    \end{boxedminipage}

    \label{object:__hash__}
    \index{object.\_\_hash\_\_ \textit{(function)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_hash\_\_}(\textit{x})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

hash(x)
    \vspace{1ex}

    \end{boxedminipage}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_init\_\_}(\textit{self}, \textit{shape}, \textit{phi}=\texttt{{\textless}class 'peach.nn.af.Linear'{\textgreater}}, \textit{bias}=\texttt{False})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

Initializes the layer.

A layer is represented by a array where each line is the weight vector
of a single neuron. The first element of the vector is the bias weight,
in case the neuron is biased. Associated with the layer is an activation
function defined in an appropriate way.
    \vspace{1ex}

      \textbf{Parameters}
      \begin{quote}
        \begin{Ventry}{xxxxx}

          \item[shape]

Stablishes the size of the layer. It must be a two-tuple of the
format \texttt{(m, n)}, where \texttt{m} is the number of neurons in the
layer, and \texttt{n} is the number of inputs of each neuron. The neurons
in the layer all have the same number of inputs.
          \item[phi]

The activation function. It can be an \texttt{Activation} object (please,
consult the \texttt{af} module) or a standard Python function. In this
case, it must receive a single real value and return a single real
value which determines if the neuron is activated or not. Defaults
to \texttt{Linear}.
          \item[bias]

If \texttt{True}, then the neurons on the layer are biased. That means
that an additional weight is added to each neuron to represent the
bias. If \texttt{False}, no modification is made.
        \end{Ventry}

      \end{quote}

    \vspace{1ex}

      Overrides: object.\_\_init\_\_

    \end{boxedminipage}

    \label{object:__new__}
    \index{object.\_\_new\_\_ \textit{(function)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_new\_\_}(\textit{T}, \textit{S}, \textit{...})

      \textbf{Return Value}
      \begin{quote}
\begin{alltt}
a new object with type S, a subtype of T
\end{alltt}

      \end{quote}

    \vspace{1ex}

    \end{boxedminipage}

    \label{object:__reduce__}
    \index{object.\_\_reduce\_\_ \textit{(function)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_reduce\_\_}(\textit{...})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

helper for pickle
    \vspace{1ex}

    \end{boxedminipage}

    \label{object:__reduce_ex__}
    \index{object.\_\_reduce\_ex\_\_ \textit{(function)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_reduce\_ex\_\_}(\textit{...})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

helper for pickle
    \vspace{1ex}

    \end{boxedminipage}

    \label{object:__repr__}
    \index{object.\_\_repr\_\_ \textit{(function)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_repr\_\_}(\textit{x})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

repr(x)
    \vspace{1ex}

    \end{boxedminipage}

    \label{object:__setattr__}
    \index{object.\_\_setattr\_\_ \textit{(function)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_setattr\_\_}(\textit{...})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

x.{\_}{\_}setattr{\_}{\_}('name', value) {\textless}=={\textgreater} x.name = value
    \vspace{1ex}

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__setitem__}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_setitem\_\_ \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_setitem\_\_}(\textit{self}, \textit{n}, \textit{w})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

The \texttt{{[} {]}} set interface.

The inputs to this method are forwarded to the \texttt{weights} property.
That means that it will set the respective line/element of the weight
array.
    \vspace{1ex}

      \textbf{Parameters}
      \begin{quote}
        \begin{Ventry}{x}

          \item[n]

A slice object containing the elements referenced. Since it is
forwarded to an array, it behaves exactly as one.
          \item[w]

A value or array of values to be set in the given indices.
        \end{Ventry}

      \end{quote}

    \vspace{1ex}

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__setphi}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_setphi \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_setphi}(\textit{self}, \textit{phi})

    \end{boxedminipage}

    \label{peach:nn:base:Layer:__setweights}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}!peach.nn.base.Layer.\_\_setweights \textit{(method)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_setweights}(\textit{self}, \textit{m})

    \end{boxedminipage}

    \label{object:__str__}
    \index{object.\_\_str\_\_ \textit{(function)}}

    \vspace{0.5ex}

    \begin{boxedminipage}{\textwidth}

    \raggedright \textbf{\_\_str\_\_}(\textit{x})

    \vspace{-1.5ex}

    \rule{\textwidth}{0.5\fboxrule}

str(x)
    \vspace{1ex}

    \end{boxedminipage}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%                              Properties                               %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

  \subsubsection{Properties}

\begin{longtable}{|p{.30\textwidth}|p{.62\textwidth}|l}
\cline{1-2}
\cline{1-2} \centering \textbf{Name} & \centering \textbf{Description}& \\
\cline{1-2}
\endhead\cline{1-2}\multicolumn{3}{r}{\small\textit{continued on next page}}\\\endfoot\cline{1-2}
\endlastfoot\raggedright \_\-\_\-c\-l\-a\-s\-s\-\_\-\_\- & \raggedright \textbf{Value:}
{\tt {\textless}attribute '\_\_class\_\_' of 'object' objects{\textgreater}}&\\
\cline{1-2}
\raggedright b\-i\-a\-s\- & \raggedright True if the neuron is biased. Not writable.

\textbf{Value:}
{\tt {\textless}property object at 0x8cf98ec{\textgreater}}&\\
\cline{1-2}
\raggedright i\-n\-p\-u\-t\-s\- & \raggedright Number of inputs for each neuron in the layer. Not writable.

\textbf{Value:}
{\tt {\textless}property object at 0x8cf989c{\textgreater}}&\\
\cline{1-2}
\raggedright p\-h\-i\- & \raggedright The activation function. It can be set with an \texttt{Activation} instance or
a standard Python function. If a standard function is given, it must receive
a real value and return a real value that is the activation value of the
neuron. In that case, it is adjusted to work accordingly with the internals
of the layer.

\textbf{Value:}
{\tt {\textless}property object at 0x8cf993c{\textgreater}}&\\
\cline{1-2}
\raggedright s\-h\-a\-p\-e\- & \raggedright Shape of the layer, given in the format of a tuple \texttt{(m, n)}, where
\texttt{m} is the number of neurons in the layer, and \texttt{n} is the number of
inputs in each neuron. Not writable.

\textbf{Value:}
{\tt {\textless}property object at 0x8cf98c4{\textgreater}}&\\
\cline{1-2}
\raggedright s\-i\-z\-e\- & \raggedright Number of neurons in the layer. Not writable.

\textbf{Value:}
{\tt {\textless}property object at 0x8cf9874{\textgreater}}&\\
\cline{1-2}
\raggedright v\- & \raggedright The activation potential of the neuron. Not writable, and only available
after the neuron is fed some input.

\textbf{Value:}
{\tt {\textless}property object at 0x8cf9964{\textgreater}}&\\
\cline{1-2}
\raggedright w\-e\-i\-g\-h\-t\-s\- & \raggedright A \texttt{numpy} array containing the synaptic weights of the network. Each
line is the weight vector of a neuron. It is writable, but the new weight
array must be the same shape of the neuron, or an exception is raised.

\textbf{Value:}
{\tt {\textless}property object at 0x8cf9914{\textgreater}}&\\
\cline{1-2}
\raggedright y\- & \raggedright The activation value of the neuron. Not writable, and only available
after the neuron is fed some input.

\textbf{Value:}
{\tt {\textless}property object at 0x8cf998c{\textgreater}}&\\
\cline{1-2}
\end{longtable}

    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}!peach.nn.base.Layer \textit{(class)}|)}
    \index{peach \textit{(package)}!peach.nn \textit{(package)}!peach.nn.base \textit{(module)}|)}


\documentclass{article}
\usepackage{fullpage}
\usepackage{nopageno}
\usepackage{amsmath}
\usepackage{amssymb}
\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
\title{Notes}
\date{March 3, 2014}
\maketitle

\section*{lesson 13 laplace transform}
\subsection*{sample problem}
Get a transform $U(s,t)$ for a solution (last page of lesson 13) and text gives $u(x,t)$ and says ``follows from tables''. Turns out it's not in the texts, tables. Not available in Mathematica either.

We will take a couple of days on approaches to finding inverse laplace transforms. Started talking about this on the 28th.
\subsection*{elementary inversion based}
Find laplace transforms for $t^p$ and $t^ne^{at}\begin{cases}\cos(bt)\\\sin(bt)\end{cases}$ that occur in circuit analysis

the general answer is
\begin{align*}
  f(t)&=\frac{1}{2\pi i}\int_{c-i\infty}^{c+i\infty}{e^{st}F(x)\,\mathrm{d}s}
\end{align*}
this is the general inversion formula for laplace transforms. Requires essentialuse of complex analysis. the typical result is an infinite series of an integral representation for $f(t)$
\subsection*{approach 1}
expand $F(s)$ in reciprocal powers and invert termwise. $\frac{F(p+1)}{s^{p+1}}\to t^p$
\subsubsection*{example}
\begin{align*}
  F(s)&=(s^2+1)^{-1/2} \text{ in text table}\\
  \text{Find }f(t)\\
  \text{Found }f(t)&=\sum\limits_{n=0}^\infty{\frac{(-t^2/4)^n}{(n!)^2}}
\end{align*}
approach 1 give series for $f(t)$.

\emph{note} this happens to be $J_0(t)$
\subsection*{approach 2}
using $F(s)$, try to find an equation (typically differential equatin) for $f(t)$.
\subsubsection*{example}
\begin{align*}
  F(s)&=(s^2+1)^{-1/2}&&=\mathcal{L}\{f(t)\}\\
  F'(s)&=-\frac{1}{2}(s^2+1)^{-3/2}(2s)&&=\mathcal{L}\{-t\cdot f(t)\}\\
  &=-\frac{1}{s^2+1}(s^2+1)^{-1/2}\\
  &=-\frac{s}{s^2+1}F(s)\\
  (s^2+1)F'(s)+sF(s)&=0\\
  sF(s)-f(0)&&&=\mathcal{L}\{f'(t)\}\\
  F'(s)&&&=\mathcal{L}\{-t\cdot f(t)\}\\
  s^2F'(s)-s(-t\cdot f(t))_{t=0}-\left(\frac{\mathrm{d}}{\mathrm{d}t}(-t\cdot f(t))\right)&&&=\mathcal{L}\left\{\frac{\mathrm{d}^2}{\mathrm{d}t^2}(-t\cdot f(t)\right\}\\
  s^2G(s)-sg(0)-g'(0)&&&=\mathcal{L}\{g''(t)\}\\
  \mathcal{L}\{f'(t)-tf(t)-\frac{\mathrm{d}^2}{\mathrm{d}t^2}(-tf(t))\}&\\
  +f(0)\underbrace{-s(tf(t))_{t=0}}_{=0}-\frac{\mathrm{d}}{\mathrm{d}t}(tf(t))_{t=0}-(f(t)+tf'(t))_{t=0}&=0\\
  -tf'(t)\text{ as }t\to0
\end{align*}
we assume $f(0)$ exists, and that $\lim\limits_{t\to0^+}tf'(t)=0$. \emph{AFTER} solving for $f(t)$ wee can check that these conditions hold.
\begin{align*}
  \mathcal{L}\{f'(t)-tf(t)-\frac{\mathrm{d}^2}{\mathrm{d}t^2}(tf(t))-(tf''(t)+2f'(t)\}&=0\\
  tf''(t)+f'(t)+tf(t)&=0
\end{align*}
bessel de
\begin{align*}
  z^2\frac{\mathrm{d}^2w}{\mathrm{d}z^2}+z\frac{\mathrm{d}w}{\mathrm{d}z}+(z^2+\mu^2)w&=0\\
  \mu &=\text{order}\\
  J_{\mu }(z)&\text{ Bessel function of first kind (order $\mu $)}\\
  Y_{\mu }(z)&\text{ Bessel function of second kind (order $\mu $)}\\
\end{align*}

\emph{note} dlmf.nist.gov is reference for standard functions.
\end{document}


\documentclass{beamer}
\input{../util/beamerpreamble.tex}

\usepackage{tikz}
\usetikzlibrary{arrows,shapes}

\title[GB13604]{GB13604 - Maths for Computer Science}
\subtitle[]{Lecture 9 -- Probability, Part II}
\author[Claus Aranha]{Claus Aranha\\{\footnotesize caranha@cs.tsukuba.ac.jp}}
\institute[COINS]{College of Information Science}
\date[2018-12-12]{2018-12-12\\{\tiny Last updated \today}}

\tikzstyle{vertex}=[circle,fill=black!25,minimum size=10pt,inner sep=0pt]
\tikzstyle{blue vertex}=[circle,fill=blue!100,minimum size=10pt,inner sep=0pt]
\tikzstyle{red vertex}=[circle,fill=red!100,minimum size=10pt,inner sep=0pt]
\tikzstyle{yellow vertex}=[circle,fill=yellow!100,minimum size=10pt,inner sep=0pt]
\tikzstyle{edge} = [draw,thick,-]
\tikzstyle{pedge} = [draw,thick,.]
\tikzstyle{red edge} = [draw, thick,-,red!50]
\tikzstyle{black edge} = [draw, line width=2pt,-,black!20]
\tikzstyle{weight} = [font=\smaller]

\begin{document}

\begin{frame}
  \maketitle

  \begin{center}
    {\smaller This course is based on Mathematics for Computer Science, Spring
    2015, by Albert Meyer and Adam Chlipala, Massachusetts Institute
    of Technology OpenCourseWare.}

    \includegraphics[width=0.2\textwidth]{../img/by-nc-sa}
  \end{center}
\end{frame}

\begin{frame}
  \frametitle{Introduction}
  \begin{itemize}
  \item Independence and Causality
  \item Random Variables
  \item Expectation
  \end{itemize}
\end{frame}

\section{Independence and Causality}

\begin{frame}

  \begin{center}
    {\huge
      Independence and Causality
    }
  \end{center}
\end{frame}

\subsection{Basics}

\begin{frame}
  \frametitle{Independent Events}

  \begin{itemize}
  \item If it is raining, the probability that people will take the
    bus from tsukuba station is higher than if it is sunny.

    \begin{equation*}
      \text{Pr(Take the bus | raining)} \neq \text{Pr(Take the bus | sunny)}
    \end{equation*}
    \bigskip

  \item If your father and mother have type A blood, the probability
    that you have type A blood is higher.
    \begin{equation*}
      \text{Pr}(A_{me} | A_{P} \& A_{M}) \geq \text{Pr}(A_{me})
    \end{equation*}
    \bigskip

  \item If you flip two separate coins, the probability of the second
    coin being heads is not influenced by the result of the first
    coin.
    \begin{equation*}
      \text{Pr}(H_2 | H_1) = \text{Pr}(H_2 | \bar{H_1}) = \text{Pr}(H_2)
    \end{equation*}
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Independent Events}

  Two events, \structure{A} and \structure{B} are \structure{independent} if the
  probability of A \structure{does not depend} on the occurrence of B.

  \bigskip

  {\bf Mathematical Definitions:}
  \begin{itemize}
  \item {\bf Definition 1:} Events \structure{A} and \structure{B} are
    \structure{independent} {\bf iff}:
    \begin{equation}
      \text{Pr}(A) = \text{Pr}(A|B)
    \end{equation}
  \item {\bf Definition 2:} Events \structure{A} and \structure{B} are
    \structure{independent} {\bf iff}:
    \begin{equation}
      \text{Pr}(A)\cdot\text{Pr}(B) = \text{Pr}(A\cap B)
    \end{equation}
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Proof of Equivalence of Definitions 1 and 2}

  Definition 1:
  \begin{equation*}
    \text{Pr}(A) = \text{Pr}(A|B) \text{ {\bf iff}}
  \end{equation*}

  \bigskip

  Conditional Prob Definition:
  \begin{equation*}
    \text{Pr}(A) = \frac{\text{Pr}(A \cap B)}{\text{Pr}(B)} \text{ {\bf iff}}
  \end{equation*}

  \bigskip

  Algebra:
  \begin{equation*}
    \text{Pr}(A)\cdot\text{Pr}(B) = \text{Pr}(A \cap B).
  \end{equation*}
\end{frame}

\begin{frame}
  \frametitle{Independence: Characteristics}

  {\bf Simmetry:}
  \begin{equation*}
    \text{Pr}(A)\cdot\text{Pr}(B) = \text{Pr}(A \cap B).
  \end{equation*}
  If $A$ is independent from $B$, \alert{then} $B$ is independent from $A$.

  \vfill

  {\bf Zero probability event:}\\
  If Pr$(B) = 0$ then:
  \begin{itemize}
  \item {\bf Definition 1:} Does not work
  \item {\bf Definition 2:} $\text{Pr}(A)\cdot0 = \text{Pr}(A \cap
    \emptyset) = 0$.
  \end{itemize}
  So, $B$ is \alert{independent from all events}!
\end{frame}

\begin{frame}
  \frametitle{Independence: Complement}

  \structure{A independent of B} means probability of A does not
  change \alert{if B happens or not}. {\bf Therefore} A should also be
  independent of {\bf Not B}.

  \vfill
  {\bf Lemma:} A independent of B {\bf iff} A independent of NOT B.

  \vfill

  {\bf Proof:} Proof can be derived using algebra from:
  \begin{equation*}
    \text{Pr}(A-B) = \text{Pr}(A)-\text{Pr}(A\cap B)
  \end{equation*}
\end{frame}

\subsection{Mutual Independence}

\begin{frame}
  \frametitle{Mutual Independence: Idea}

  {\bf Experiment:} Two coins are thrown
  \begin{itemize}
  \item {\bf Event 1}: Coin 1 is heads: Pr$(H_1) = (0,0),(0,1),(1,0),(1,1) = 0.5$
  \item {\bf Event 2}: Coin 2 is heads: Pr$(H_2) = (0,0),(0,1),(1,0),(1,1) = 0.5$
  \item {\bf Event 3}: \# of heads is odd: Pr$(O) = (0,0),(0,1),(1,0),(1,1) = 0.5$
  \end{itemize}

  \vfill

  \begin{onlyenv}<2>
  Events $H_1,H_2$, $H_1,O$, $H_2,O$ are, \alert{2-by-2}, independent:
  \begin{itemize}
  \item Pr$(O|H_1) = (1,0), (1,1) = 0.5$
  \item Pr$(O|H_2) = (0,1), (1,1) = 0.5$
  \item Pr$(H_1|H_2) = (0,1), (1,1) = 0.5$
  \end{itemize}

  \bigskip

  These 3 events are \structure{2-way independent}.
  \end{onlyenv}
\end{frame}

\begin{frame}
  \frametitle{Mutually Independent Events -- Definition}

  Events $A_1, A_2, A_3, \ldots, A_n$ are \structure{Mutually
    Independent} when:

  \vfill

  The probability that $A_i$ occurs is not changed by the occurence of
  the other events.
\end{frame}

\begin{frame}
  \frametitle{Mutually Independent Events -- Example}

  {\bf Experiment:} You throw the coin $n$ times\\
  {\bf Event:} $H_i$ the $i$-th throw is \structure{heads}

  \vfill

  What happens in the 5th flip is independent from the 1st flip, or
  the 7th flip, etc.

  \bigskip

  \begin{equation*}
    \text{Pr}(H_5) = \text{Pr}(H_5|H_1 \cap H_3 \cap H_7 \cap \ldots)
  \end{equation*}
\end{frame}

\begin{frame}
  \frametitle{Mutiually Independent Events -- Math Def}

  Events $A_1, A_2, \ldots A_n$ are \structure{mutually independent} when:

  \vfill

  {\bf Definition 1:}
  \begin{equation}
    \text{Pr}(A_i) = \text{Pr}(A_i|A_1 \cap A_2 \cap \ldots \cap A_j \cap  \ldots \cap A_n); j \neq i
  \end{equation}

  \bigskip

  {\bf Definition 2:}
  \begin{equation}
    \text{Pr}(A_1 \cap A_2 \cap A_3 \cap \ldots \cap A_n) = \text{Pr}(A_1)\cdot\text{Pr}(A_2)\cdot\text{Pr}(A_3)\cdot\ldots\cdot\text{Pr}(A_n)
  \end{equation}
\end{frame}

\begin{frame}
  \frametitle{K-wise independence}

  {\bf Experiment:} Two coins are thrown
  \begin{itemize}
  \item {\bf Event 1}: Coin 1 is heads: Pr$(H_1) = (0,0),(0,1),(1,0),(1,1) = 0.5$
  \item {\bf Event 2}: Coin 2 is heads: Pr$(H_2) = (0,0),(0,1),(1,0),(1,1) = 0.5$
  \item {\bf Event 3}: \# of heads is odd: Pr$(O) = (0,0),(0,1),(1,0),(1,1) = 0.5$
  \end{itemize}

  \vfill

  \begin{itemize}
  \item Each pair $(H_1, H_2),(H_1,O),(H_2,O)$ is mutually independent.
  \item However, $(H_1, H_2, O)$ are not mutually independent:
    Pr$(O|H_1\cap H_2) = 0$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{K-wise independence}
  {\larger
  {\bf Experiment:} $n$ coins are thrown
  \begin{itemize}
  \item {\bf Event i}: Coin i is heads: Pr$(H_i) = 0.5$
  \item {\bf Event $i+1$}: \# of heads is odd: Pr$(O) = 0.5$
  \end{itemize}

  \vfill

  \structure{Any subset of $i$ events} is mutually independent.\\
  However, \alert{the set of all $i+1$} events is {\bf not}.
  }
\end{frame}

\begin{frame}
  \frametitle{K-wise independence}

  {\larger
    A set of $m$ events is $k$-wise independent if:

    \bigskip

    $A_1, A_2, \ldots, A_m$

    \bigskip

    Any subset of $k$ events is mutually independent.
  }
\end{frame}

\section{Random Variables}

\begin{frame}
  \begin{centering}
    {\huge
      Random Variables
    }
  \end{centering}
\end{frame}

\subsection{Intuitive Example}
\begin{frame}
  \frametitle{Playing tRPGs}

  \begin{columns}
    \column{0.7\textwidth}
    In the game \structure{3:16}, you \alert{win a space battle} if you
    roll a d10 dice, and the result is below the value \structure{FA},
    which you decide before the game.

    \bigskip

    \begin{itemize}
    \item If you lose one battle \alert{you lose your armor}
    \item If you lose another battle \alert{you are wounded}
    \item If you lose a third battle \alert{you die}
    \end{itemize}
    \column{0.3\textwidth}
    \includegraphics[width=1\textwidth]{../img/316}
  \end{columns}
\end{frame}

\begin{frame}
  \frametitle{Events and Numbers}

  Until now, we approached probability from a \structure{Set} and
  \structure{Events} point of view:
  \begin{itemize}
  \item {\bf Event 1:} Result of the dice: $\{1,2,3,4,5,6,7,8,9,10\}$
  \item {\bf Event 2:} Value of {\bf FA}: $\{1,2,3,4,5,6,7,8,9,10\}$
  \item {\bf Event 3:} You win a combat with FA = 6:\\
    $\{1,6\}, \{2,6\}, \{3,6\}, \ldots$
  \item {\bf Event 4:} 3 victories in 5 battles with FA = 6:\\ $\{E_3,
    E_3, E_3, \bar{E_3}, \bar{E_3}\}, \{E_3, E_3, \bar{E_3}, E_3,
    \bar{E_3}\}, \ldots$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Events and Numbers}

  However, many times we want to see probability in terms of
  \structure{Numbers}:
  \begin{itemize}
  \item What is the probability that I can \alert{survive 7 battles}
    if my FA is 6?
  \item What is the \alert{minimum FA} I need to have more than .8
    chance to survive 5 battles?
  \item \alert{How many battles in a row} I expect to win if my FA is 6?
  \end{itemize}

  \vfill

  Intuitively, a \structure{Random Variable} is a number that comes from
  a \structure{Random Process}
  \begin{itemize}
  \item \# of hours to the next \structure{System Crash}
  \item \# \structure{faulty pixels} in the monitor
  \item \# heads in a \structure{coin flip}
  \end{itemize}
\end{frame}

\subsection{Random Variable: Definition}
\begin{frame}
  \frametitle{Random Variable: Formal Definition}

  A random variable is a \structure{total function} from a
  \structure{sample space} to a \structure{numeric domain}.

  \bigskip
  \begin{equation}
    R: S \to \mathbb{R} \text{ or } R: S \to \mathbb{N}, \text{ etc...}
  \end{equation}

  \vfill

  So a random \emph{variable} is really a \alert{function}...\\
  \hfill (... maybe that is a bad name)
\end{frame}

\begin{frame}
  \frametitle{Random Variable: Formal Example}

  {\bf Experiment}: We throw three \emph{fair} coins independently.
  \begin{itemize}
  \item Variable $C$ ::= Number of heads ({\bf C}ount)
  \item Variable $M$ ::= 1 if all coins {\bf M}atch, 0 if they don't {\bf M}atch
  \end{itemize}

  \vfill

  \begin{itemize}
  \item $M = 1$ when $\{H,H,H\}$ or $\{T,T,T\}$, \\
    Pr$(M=1) = 2/8$
    \bigskip

  \item $C = 1$ when $\{H,T,T\},\{T,H,T\},\{T,T,H\}$, \\
    Pr$(C=1) = 3/8$
    \bigskip

  \item Pr$(C\cdot M > 0) = \text{Pr}(C>0 \cap M>0) = 1/8$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{``Indicator'' random variables}

  An {\bf indicator random variable} \structure{takes a value of 0 or 1} for
  a subset of the sample space.
  \begin{itemize}
  \item Variable $M$ ::= 1 if all coins {\bf M}atch, 0 if they don't {\bf M}atch
    \bigskip

  \item Variable $O$ ::= 1 if number of heads is {\bf O}dd.
  \end{itemize}

  \vfill

  Other \structure{integer} random variables \structure{partition} the
  sample space:
  \begin{itemize}
  \item $C = 0 \to \{T,T,T\}$
  \item $C = 1 \to \{H,T,T\}, \{T,H,T\}, \{T,T,H\}$
  \item $C = 2 \to \{H,T,H\}, \{H,H,T\}, \{T,H,H\}$
  \item $C = 3 \to \{H,H,H\}$
  \end{itemize}
\end{frame}

\subsection{Independence}
\begin{frame}
  \frametitle{Independence and Random Variables}

  So, a \structure{Random Variable} is a \alert{function} that assigns
  a value to a set of outcomes.

  \bigskip

  Two or more \structure{Random Variables} $R_1, R_2, \ldots, R_n$ are
  mutually independent {\bf iff} $(R_1 = a_1), (R_2 = a_2), \ldots,
  (R_n = a_n)$ are all \structure{mutually indepedent events}.

  \bigskip

  {\bf Alternatively:}\\
  \begin{equation*}
    \text{Pr}(R_1 = a_1 \cap R_2 = a_2 \cap R_3 = a_3 \cap \ldots) =
    \text{Pr}(R_1 = a_1)\cdot\text{Pr}(R_2 = a_2)\ldots
  \end{equation*}
  \hfill For all values of $a_1$
\end{frame}

\begin{frame}
  \frametitle{Independence and Random Variables}

  Are R.V.s $C$ and $M$ independent?
  \bigskip

  \begin{itemize}
  \item Variable $M$ ::= 1 if all coins {\bf M}atch, 0 if they don't {\bf M}atch
  \item Variable $C$ ::= number of heads ({\bf C}ount).
  \end{itemize}

  \bigskip

  {\bf Answer:} \alert{NO!}
  \begin{itemize}
  \item Pr$(M=1)\cdot$Pr$(C=1) > 0$
  \item Pr$(M=1 \cap C=1) = 0$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Independence and Random Variables}

  Are R.V.s $O$ and $M$ independent?
  \bigskip

  \begin{itemize}
  \item Variable $M$ ::= 1 if all coins {\bf M}atch, 0 if they don't {\bf M}atch
  \item Variable $O$ ::= 1 if \# heads is {\bf O}dd, 0 if it is even.
  \end{itemize}

  \bigskip

  {\bf Answer:} \structure{YES!}
  \begin{itemize}
  \item Pr$(M=1)\cdot$Pr$(O=1) = \frac{2}{8}\cdot\frac{1}{2} = \frac{1}{8}$
  \item Pr$(M=1 \cap C=1) = \frac{1}{8}$
    \bigskip

  \item Don't forget to test for the other combinations: $(M = 0, C = 1), (M = 1, C = 0), \ldots$

    \bigskip

  \item To prove \alert{NOT} independence, you just need one counterexample.
  \end{itemize}
\end{frame}

\subsection{PDF and CDF}

\begin{frame}
  \frametitle{Uniform Random Variables}

  A \structure{Uniform} Random Variable is one where the \structure{all values are equally likely}.

  \vfill

  \begin{itemize}
  \item $D_n$ ::= result of a \structure{fair} dice with $n$ sides.
    \begin{equation*}
      \text{Pr}(D_6 = 1) = \text{Pr}(D_6 = 2) = \ldots = \text{Pr}(D_6 = 6) =
      \frac{1}{6}
    \end{equation*}
    \bigskip

  \item $S_4$ ::= lottery number with 4 digits.
    \begin{equation*}
      \text{Pr}(S_4 = 0000) = \text{Pr}(S_4 = 1234) = \ldots =
      \text{Pr}(S_4 = 9998) = 0.0001
    \end{equation*}
  \end{itemize}

\end{frame}

\begin{frame}
  \frametitle{Binomial Random Variable}

  {\bf Experiment:} We throw a biased coin $n$ times, each time with
  $p$ chance to turn heads.
  \bigskip

  {\bf Event:} $B_{n,p}$::= \# of heads in $n$ mutually independent
  flips.

  \vfill

  For $n = 5, p = 2/3$, Pr$(HHTTH)$:
  \begin{itemize}
  \item<2-> = Pr$(H)\cdot$Pr$(H)\cdot$Pr$(T)\cdot$Pr$(T)\cdot$Pr$(H)$
  \item<3-> $=
    \frac{2}{3}\cdot\frac{2}{3}\cdot\frac{1}{3}\cdot\frac{1}{3}\cdot\frac{2}{3}$
  \item<4-> $= \frac{2}{3}^3\cdot\frac{1}{3}^2$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Binomial Random Variable}

  {\bf Experiment:} We throw a biased coin $n$ times, each time with
  $p$ chance to turn heads.
  \bigskip

  {\bf Event:} $B_{n,p}$::= \# of heads in $n$ mutually independent
  flips.

  \vfill

  For $n = 5, p = 2/3$:
  \begin{itemize}
  \item Pr(\structure{ONE seq with i $H$ and (n-i) $T$}):\\
    $p^i(1-p)^{n-i}$
    \bigskip

  \item Pr(\structure{ANY seq with i $H$ and (n-i) $T$}):\\
    $\binom{n}{i}p^i(1-p)^{n-i}$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Probability Density Function}

  The \structure{Probability Density Function (PDF)} of a random variable $R$
  defines the probability that $R$ will have a given value $i$
  \bigskip

  \begin{equation*}
    \text{PDF}_R(i) ::= \text{Pr}(R = i)
  \end{equation*}
  \bigskip

  {\bf Example 1:} for the binomial random variable:
  \begin{equation*}
    \text{PDF}_{B_{n,p}}(i) = \binom{n}{i}p^i(1-p)^{n-i}
  \end{equation*}
  \bigskip

  {\bf Example 2:} for the uniform random variable:
  \begin{equation*}
    \text{PDF}_{U}(v) = \text{constant}
  \end{equation*}
  \hfill (for $v$ in range of $U$)
\end{frame}

\begin{frame}
  \frametitle{Cumulative Distribution}

  The \structure{Cumulative Distribution Function} is the function that
  measures the probability that random variable $R$ will have a value
  \structure{smaller or equal} than $i$
  \bigskip

  \begin{equation*}
    \text{CDF}_R(i) ::= \text{Pr}(R \leq i)
  \end{equation*}
  \vfill

  {\bf Important points}
  \begin{itemize}
  \item PDF and CDF capture similar information.
  \item PDF and CDF \structure{do not involve} the actual sample space.
  \item Many different {\bf experiment} have \structure{very similar PDFs}
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{PDF and CDF example: 2d6}
  \begin{center}
    \includegraphics[height=0.4\textheight]{../img/pdf_2d6}
    \bigskip

    \includegraphics[height=0.4\textheight]{../img/cdf_2d6}
  \end{center}
\end{frame}

\section{Expectation}
\begin{frame}
  \begin{center}
    {\huge
      Expectation
    }
  \end{center}
\end{frame}

\begin{frame}
  \frametitle{Game: Carnival Dice}

  Consider the following game: \structure{Choose a number of 1 to 6}
  and roll \alert{3 dice}. Each dice that \structure{rolls the same number}
  you win \$1. If \alert{no dice} roll that number, you lose \$1.

  \bigskip

  {\bf Example:} You choose \alert{number 5}:
  \begin{itemize}
  \item<2-> 2, 3, 4: \alert{Lose \$1}
  \item<3-> {\bf 5}, 4, 6: \structure{Win \$1}
  \item<4-> {\bf 5}, {\bf 5}, 1: \structure{Win \$2}
  \item<5-> {\bf 5}, {\bf 5}, {\bf 5}: \structure{Win \$3}
    \bigskip

  \item<6> Is this a fair game?
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Game: Carnival Dice Analysis}

  {\larger
  \begin{itemize}
  \item Pr(zero 5s) = $\frac{5}{6}^3 = \frac{125}{216}$ = \alert{Lose \$1}
    \bigskip

  \item Pr(one 5) = $\binom{3}{1}\frac{5}{6}^2\frac{1}{6} =
    \frac{75}{216}$ = \structure{Win \$1}
    \bigskip

  \item Pr(two 5) = $\binom{3}{2}\frac{5}{6}\frac{1}{6}^2 =
    \frac{15}{216}$ = \structure{Win \$2}
    \bigskip

  \item Pr(three 5) = $\frac{1}{6}^3 = \frac{1}{216}$ = \structure{Win
    \$3}
  \end{itemize}
  }
\end{frame}

\begin{frame}
  \frametitle{Carnival Game: Law of Averages}

  Every \alert{216 Games}, you expect \alert{125 games} with 0
  matches, \alert{75 games} with 1 match, \alert{15 games} with 2
  matches, and only \alert{one game} with 3 matches!

  \bigskip

  \begin{equation*}
    \frac{216\times(-1)+75\times(1)+15\times(2)+1\times(3)}{216} = \frac{-17}{216} = -0.08
  \end{equation*}

  \bigskip

  So on average, you expect to \alert{lose \$0.08} per game. Not a fair game!
\end{frame}

\begin{frame}
  \frametitle{Carnival Game: Impossible expectations}

  Note that, for the carnival game, you \alert{expect to lose 0.08 per
    game} but {\bf the result -0.08 does not exist!}

  \vfill

  {\bf Another example:} Rolling one 6-sided die:
  \bigskip

  Pr(1) = 1/6, Pr(2) = 1/6, Pr(3) = 1/6, $\ldots$
  \bigskip

  \begin{equation*}
    \text{Expected value } = \frac{1\cdot1 + 1\cdot2 + 1\cdot3 + 1\cdot4 + 1\cdot5 + 1\cdot6}{6} = \frac{21}{6} = 3.5
  \end{equation*}
  You expect an \structure{average result} of 3.5, but the 3.5 result
  does not exist!
\end{frame}

\begin{frame}
  \frametitle{Expectation: Definitions}

  The \structure{Expected Value} of a variable $R$ is the
  \structure{Average Value} of $R$, weighted by the probabilities.

  \bigskip

  \begin{equation}
    E[R] ::= \sum_{v \in \text{range}(R)}v\cdot\text{Pr}[R = v]
  \end{equation}

  \bigskip
  So, $E[\$ \text{ at carnival game}] = -0.08$

  \vfill

  {\bf Note 1:} We can use sums in this definition because the
  \structure{probability space is countable}.\\
  {\bf Note 2:} For \structure{continuous probability spaces} we need
  to use integrals.
\end{frame}

\begin{frame}
  \frametitle{Total Expectation}

  \structure{Law of Total Expectation} is useful for reasoning by cases:
  \bigskip

  \begin{itemize}
  \item {\bf Conditional Expectation:} $E[R|A] = \sum v\cdot\text{Pr}[R = v|A]$
    \bigskip

  \item {\bf Total Expectation:} $E[R] = E[R|A]\cdot\text{Pr}[A] +
    E[R|\bar{A}]\cdot\text{Pr}[\bar{A}]$
  \end{itemize}

  \bigskip

  In other words: The total expectation is \structure{The expectation
    when $A$ happens} plus \structure{The expectation when $A$ does
    not happen}.
\end{frame}

\begin{frame}
  \frametitle{General Total Expectation}

  More generally:
  \bigskip

  \begin{equation}
    E[R] = E[R|A_1]\cdot\text{Pr}[A_1] + E[R|A_2]\cdot\text{Pr}[A_2] +
    \ldots + E[R|A_n]\cdot\text{Pr}[A_n]
  \end{equation}
  \bigskip

  when $\text{Pr}[A_1] + \text{Pr}[A_2] + \ldots + \text{Pr}[A_n] = 1$
\end{frame}

\begin{frame}
  \frametitle{Total Expectation: How many heads?}

  What is \structure{expected number of Heads} when we flip a coin $n$
  times with \alert{bias p}?

  \bigskip
  A coin with \alert{bias p}:
  \begin{itemize}
  \item Will turn {\bf Heads} with probability $p$
  \item Will turn {\bf Tails} with probability $q = 1-p$
  \end{itemize}

  \vfill

  $E[n]$ expectation of heads after $n$ flips - \structure{study by cases}:
  \begin{itemize}
  \item {\bf Case 1:} First coin is Heads: $E[n|H_1] = 1 + E[n-1]$
  \item {\bf Case 2:} First coin is Tails: $E[n|T_1] = E[n-1]$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Total Expectation: How many heads?}

  $E[n]$ expectation of heads after $n$ flips - \structure{study by cases}:
  \begin{itemize}
  \item {\bf Case 1:} First coin is Heads: $E[n|H_1] = 1 + E[n-1]$
  \item {\bf Case 2:} First coin is Tails: $E[n|T_1] = E[n-1]$
  \end{itemize}

  \bigskip

  By \structure{total expectation}:
  \begin{itemize}
  \item<2-> $E[n] = E[n|H_1]\cdot p + E[n|T_1]\cdot q$
  \item<3-> $E[n] = (1+E[n-1])\cdot p + (E[n-1])\cdot q$
  \item<4-> $E[n] = p + E[n-1](p+q) = p + E[n-1]$
  \item<5-> $E[n] = p + E[n-1] = p + p + E[n-2] = 3p + E[n-3] = np + E[0] = np$
  \end{itemize}
\end{frame}

\subsection{Mean Time To Failure}
\begin{frame}
  \frametitle{Mean Time To Failure}

  Suppose that a computer has a \structure{chance to fail} of $p$ every hour.

  \bigskip

  \begin{center}
    \includegraphics[width=0.7\textwidth]{../img/bsod}
  \end{center}

  \bigskip

  \alert{How many hours} does it take for the computer to fail on average?
\end{frame}

\begin{frame}
  \frametitle{Mean Time To Failure}

  \begin{itemize}
  \item Chance to fail \alert{in the first hour} = $p$
  \item Chance to fail \alert{in the second hour} = $qp$
  \item Chance to fail \alert{in the third hour}  = $q^2p$
  \item $\ldots$
  \item Chance to fail \alert{in the n-th hour} = $q^{n-1}p$
  \end{itemize}

  \vfill

  \begin{center}
    {\bf Geometric Distribution / Geometric Sum}
  \end{center}

\end{frame}

\begin{frame}
  \frametitle{Mean Time To Failure}

  \alert{How many hours} does it take for the computer to fail on
  average?

  \bigskip

  \begin{itemize}
  \item $E[F] = \sum_{n>0}n\cdot\text{Pr}[F = n]$
    \bigskip

  \item $E[F] = \sum_{n>0}nq^{n-1}p = p\sum_{n>0}nq^{n-1}$
    \bigskip

  \item $E[F] = p\cdot\frac{1}{(1-q)^2}$
    \bigskip

  \item $E[F] = p\cdot\frac{1}{p^2} = \frac{1}{p}$
  \end{itemize}

  So, in average, the server will fail after $\frac{1}{p}$ hours.
\end{frame}

% TODO: Linearity of Expectation

\section{Conclusion}

\begin{frame}
  \frametitle{Class Summary}
  \begin{itemize}
  \item \structure{Random Variable} is a total function that describes
    a numeric value for a set of probability events.
    \bigskip

  \item \structure{Probability Density Function} describes the
    probability of a Random Variable to have a certain value $a$
    \bigskip

  \item \structure{Expectation} is the weighted average of a PDF over
    its domain.
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Next Class}
  \begin{itemize}
  \item No Exercise Sheet for Next class!
    \bigskip

  \item Sample Exam;
    \bigskip

  \item Lecture Evaluation;
  \end{itemize}

  \vfill

  \begin{center}
    Please come!
  \end{center}
\end{frame}

\end{document}


\renewcommand*\chappic{img/satfeatures.pdf}
\renewcommand*\chapquote{To be usable effectively [\dots] these features must be related to instance hardness and relatively cheap to compute}
\renewcommand*\chapquotesrc{SATzilla}
\chapter{SAT features}
\label{ch:features}
%
At the very beginning, I was very intrigued by the question
\enquote{What is an \enquote{average} SAT problem?}. Answers to this
question can help to optimize SAT solver memory layouts and
find distinctive properties of CNFs.
Specifically for this thesis, I wanted to find out whether
our problems distinguish from \enquote{average} problems in any
way such that we can use this distinction for runtime optimization.

I came up with 8 questions related to basic properties of SAT problems
we will discuss in depth in this section. We will characterize an
average SAT problem in Section~\ref{sec:features-average}:
\begin{enumerate}
\item Given an arbitrary literal. What is the percentage it is positive?
\item What is the clauses / variables ratio?
\item How many literals occur only once either positive or negative?
\item What is the average and longest clause length among CNF benchmarks?
\item How many Horn clauses exist in a CNF?
\item Are there any tautological clauses?
\item Are there any CNF files with more than one connected variable component?
\item How many variables of a CNF are covered by unit clauses?
\end{enumerate}

We will now define the terms used in those questions.

\section{Definition}
\label{sec:features-definition}
%
\index{SAT feature}
\index{Feature value}
\begin{defi}[SAT feature]
  A \emph{SAT feature} is a statistical value (named \emph{feature value})
  retrievable from some given SAT problem.
\end{defi}

The most basic example of a SAT feature is the number of variables and clauses
of a given SAT problem. This SAT feature is stored in the CNF header of a SAT
problem encoded in the DIMACS format.

The general goal is to write a tool which evaluates several SAT features at the same
time and retrieves them for comparison with other problems. Therefore it should be
computationally easy to evaluate SAT features of a given SAT problem. A suggested
computational limit is given with polynomial complexity in terms of number of
variables and number of clauses for memory as well as runtime.
Sticking to this convention implies that evaluation of satisfiability must not be
necessary to evaluate a SAT feature as long as no polynomial algorithm to determine
satisfiability can be found. Hence the number of valid models cannot be a SAT feature
as far as satisfiability needs to be determined.
But no actual hard computational limit is defined.

\section{Related work}
\label{sec:features-related}
%
The most similar resource I found---looking at SAT features---was the
SATzilla project~\cite{satzilla2004,satzilla2008} in 2012. The authors used
91~SAT features categorized in 9 groups, originally described by Nudelman,
et al.~\cite{nudelman2004understanding}. Some features are only evaluated
if they can be evaluated within a given time frame (e.g. 20~seconds).

The following list provides an excerpt of the features:

\begin{description}
\item[\satfeature{nvarsOrig}] number of variables defined in the CNF header
\item[\satfeature{nvars}] number of active variables
\item[\satfeature{reducedVars}] \satfeature{nvarsOrig} reduced by \satfeature{nvars}, divided by \satfeature{nvars}
\item[\satfeature{vars-clauses-ratio}] nvars divided by the number of active clauses
\item[\satfeature{POSNEG-RATIO-CLAUSE-mean}] clause mean of $2 \cdot \left\| 0.5 - {\text{pos}}/{\text{length}}\right\|$ where $\text{pos}$ is the number of positive literals and $\text{length}$ its clause length
\item[\satfeature{POSNEG-RATIO-CLAUSE-entropy}] like \satfeature{POSNEG-RATIO-CLAUSE-mean} but its Shannon entropy
\item[\satfeature{TRINARY+}] number of clauses with clause length 1, 2 or 3 divided by number of active clauses
\item[\satfeature{HORNY-VAR-min}] minimum number of times a variable occurs in a Horn clause
\item[\satfeature{cluster-coeff-mean}]
   let neighbors of a clause be all clauses containing any literal negated
   and let clauses $c_1$ and $c_2$ be conflicting if $c_1$ contains literal $l$ and $c_2$ contains $-l$,
   then return the mean of two times the number of conflicting neighbors of a clause $c$
   divided by the number of unordered pairs of neighbors;
   returned iff computable within 20 seconds for all clauses
\end{description}

Please recognize that active clauses are the unsatisfied clauses after BCP has been applied.
Equivalently active variables are remaining variables after application of BCP.

The SAT solvers we use (Section~\ref{sec:sat-solvers}), also compute features they use
when computing a solution. For example, CryptoMiniSat 4.5.3 prints lines such as:

\begin{verbatim}
c [features] numVars 56118, numClauses 358991, var_cl_ratio 0.156,
binary 0.019, trinary 0.520, horn 0.387, horn_mean 0.000, horn_std
0.000, horn_min 0.000, horn_max 0.000, horn_spread 0.000,
vcg_var_mean 0.000, vcg_var_std 0.902, vcg_var_min 0.000,
vcg_var_max 0.000, vcg_var_spread -0.000, vcg_cls_mean 0.000,
...
\end{verbatim}

Even though we will partially use equivalent features (like Horn clauses),
many are actually related to the current state of evaluation like decisions
per conflicts. We consider this as a property of the evaluation and not the
SAT problem itself.

Many SAT solvers collect feature values to improve algorithm selection,
restart strategies and estimate problem sizes. Recent trends to apply Machine
Learning to SAT solving imply feature evaluation. SAT features and the resulting
satisfiability runtimes are used as training data for Machine Learning.
Another SAT solver using SAT features heavily for algorithm selection
besides SATzilla is ASlib~\cite{aslib}.

\section{Statistical properties}
\label{sec:features-stats}
%
For our SAT~features we need to define some basic statistical terminology.
Let $x_1, x_2, \ldots, x_n$ be a sequence of real numbers ($n \in \mathbb N$).
\begin{description}
  \item[Arithmetic mean] (or mean for short)
    is defined as
    \[ \overline{x} = \frac1n \sum_{i=1}^n x_i \]
  \item[Standard deviation] (or sd for short)
    with mean $\overline{x}$ is defined as
    \[ \sigma(x) = \sqrt{\frac1n \sum_{i=1}^n (x_i - \overline{x})} \]
  \item[Median]
    with $x_1 \leq x_2 \leq \ldots \leq x_n$
    (i.e. sorted ascendingly) is defined as
    \[
       m = \begin{cases}
         x_{|\text{mid}|+1} & \text{if } n \text{ odd} \\
         \frac{x_{\text{mid}} + x_{\text{mid} + 1}}{2} & \text{if } n \text{ even}
       \end{cases}
       \quad\text{ with } \text{mid} = \frac{n}{2}
    \]
    and often considered more \enquote{robust} than the arithmetic mean.
  \item[Entropy]
    is defined according to Claude Shannon's information theory:
    \[ H(x) = -\sum_{i=1}^n x_i \cdot \log_2(x_i) \]
    where $0 \cdot \log_2(0) \coloneqq 0$.
\end{description}

Furthermore \emph{count} refers to the number of elements $n$,
\emph{largest} refers to the maximum element $\max_{1 \leq i \leq n}(x_i)$
and \emph{smallest} refers to the minimum element $\min_{1 \leq i \leq n}(x_i)$.

\section{Suggested SAT features}
\label{sec:features-suggested}
%
\index{cnf-analysis}
We wrote a tool called cnf-analysis. The evaluated features are partially inspired
by SATzilla and lingeling. The latter prints basic statistics for every CNF it
evaluates.

A summary of our suggested SAT features is given:

\begin{description}
\item[\satfeature{clause\_variables\_sd\_mean}] \hfill{} \\
  mean of sd of variables in a clause
\item[\satfeature{clauses\_length\_(largest, smallest, mean, median, sd)}] \hfill{} \\
  statistics related to the clause length
\item[\satfeature{connected\_(literal, variable)\_components\_count}] \hfill{} \\
  two literals (variables) are connected if they occur in some clause together,
  count the number of connected components
\item[\satfeature{definite\_clauses\_count}] \hfill{} \\
  number of definite clauses in the CNF
\item[\satfeature{existential\_literals\_count}] \hfill{} \\
  number of existential literals in the CNF
\item[\satfeature{existential\_positive\_literals\_count}] \hfill{} \\
  number of positive, existential literals in the CNF
\item[\satfeature{(false, true)\_trivial}] \hfill{} \\
  is the CNF satisfied if all variables are claimed to be false (true)?
\item[\satfeature{goal\_clauses\_count}] \hfill{} \\
  number of goal clauses in the CNF
\item[\satfeature{literals\_count}] \hfill{} \\
  number of literals in the CNF (i.e. sum of clause lengths)
\item[\satfeature{literals\_frequency\_$k$\_to\_$k+5$}] \hfill{} \\
  let $n_l$ be the literal frequency of literal $l$,
  count the number of $n_l$ satisfying $\frac{k}{100} \leq n_l < \frac{k+5}{100}$
  where $k$ is a variable in $\{0,5,10,\ldots,90,95\}$ and $k=95$ counts
  $\frac{k}{100} \leq n_l$.
\item[\satfeature{literals\_frequency\_(largest, smallest, mean, median, sd)\_entropy}] \hfill{} \\
  statistics related to literal frequencies
\item[\satfeature{literals\_occurence\_one\_count}] \hfill{} \\
  number of literals with occurence $1$
\item[\satfeature{nbclauses}, \satfeature{nbvars}] \hfill{}
  number of clauses (variables) as defined in the CNF header
\item[\satfeature{negative\_literals\_in\_clause\_(smallest, largest, mean)}] \hfill{} \\
  statistics related to number of negative literals in clauses
\item[\satfeature{(positive, negative)\_unit\_clause\_count}] \hfill{} \\
  number of unit clauses with a positive (negative) literal
\item[\satfeature{positive\_literals\_count}] \hfill{} \\
  number of positive literals in CNF
\item[\satfeature{positive\_literals\_in\_clause\_(largest, smallest, mean, median, sd)}] \hfill{} \\
  statistics related to number of positive literals in clauses
\item[\satfeature{positive\_negative\_literals\_in\_clause\_ratio\_(mean, entropy)}] \hfill{} \\
  let $r_c$ be the number of positive literals divided by clause length of clause $c$,
  mean and related of all $r_c$
\item[\satfeature{positive\_negative\_literals\_in\_clause\_ratio\_mean}] \hfill{} \\
  mean of all $r_c$
\item[\satfeature{tautological\_literals\_count}] \hfill{} \\
  number of clauses which contain a tautological literal
\item[\satfeature{two\_literals\_clause\_count}] \hfill{} \\
  number of clauses with two literals
\item[\satfeature{variables\_frequency\_$k$\_to\_$k+5$}] \hfill{} \\
  same as \satfeature{literals\_frequency\_$k$\_to\_$k+5$} but for variables
\item[\satfeature{variables\_frequency\_(largest, smallest, mean, median, sd, entropy)}] \hfill{} \\
  same as \satfeature{literals\_frequency} but for variables
\item[\satfeature{variables\_used\_count}] \hfill{} \\
  number of variables with occurence greater 0
\end{description}

\section{Evaluation efficiency}
\label{sec:features-efficiency}
%
The resource requirements of those features have been classified:
\begin{description}
  \item[Type 1] read the files as bytestring, a DIMACS CNF parser is not necessary, constant memory is used
  \item[Type 2] features understand what a clause is, but still need constant memory
  \item[Type 3] subquadratic runtime and linear memory
  \item[Type 4] unrestricted
\end{description}
%
Memory and runtime is always considered in comparison with the filesize.

This classification should support computational considerations regarding feature evaluation tools.
The suggested SAT features above have been explicitly selected to avoid Type 4 implementations to limit the time to compute features.
Furthermore tools evaluating only a subset of features (like Type~2 features) can achieve better performance characteristics than general-purpose tools. For example, we wrote a dedicated tool to evaluate the maximum clause length of CNF files, which was much faster (175 GB, 1.5 hours).

The Python implementation triggered MemoryErrors on a computer with 4~GB RAM for a 770~MB CNF file.
Followingly a much more efficient Go implementation was implemented which requires much less memory and is much faster.
\texttt{bench\_573.smt2.cnf} (1.6~MB filesize) took 1 second in Go instead of 2 minutes in Python.
However, the data evaluated is less accurate compared to Python in terms of floating point precision, because Python unlike Go provides a nice implementation of statistical tools in the standard library. Go algorithms were written on our own.

In the following section, we define the dataset we consider.

\section{CNF dataset}
\label{sec:features-dataset}
%
To evaluate CNF features of a representative set of CNF files, it was necessary to identify equivalent CNF files in the best possible way.
Therefore we defined a hashing algorithm standardizing the CNF input provided to a SHA1 instance. Every CNF file is identifiable by its
\enquote{cnfhash 2.0.0} hash value.

In the next step a complete set of CNF files of previous SAT competitions was collected.
The following CNF file collections have been considered:

\begin{center}
  \begin{minipage}{0.48\linewidth}
    \begin{itemize}
      \itemsep2pt
      \item SAT Race 2008
      \item SAT09 Competition
      \item SAT-Race 2010
      \item SAT11 Competition
      \item SAT Challenge 2012
    \end{itemize}
  \end{minipage}
  \begin{minipage}{0.48\linewidth}
    \begin{itemize}
      \itemsep2pt
      \item SAT Competition 2013
      \item SAT Competition 2014
      \item SAT-Race 2015
      \item SAT Competition 2016
      \item SATlib~\cite{satlib}
    \end{itemize}
  \end{minipage}
\end{center}

The benchmarks are mostly contributed by the participants of the associated conferences.
Others are reused from previous years. Individual projects allow to generate CNF files
for specific problems in a selectable problem size; such as CNFgen~\cite{cnfgen} by
Massimo Lauria.

Some files turned out to be problematic. In SATlib, 3 gzipped files couldn't be decompressed and several files
contain empty clauses. Empty clauses are assumed to immediately falsify the CNF.
We removed empty clauses and evaluated the remaining CNF.
I removed trailing zeros in CNFs. Variants of the DIMACS standard also expect lines with a percent symbol to
terminate the CNF. Beside those minor issues documented as part of the cnf-analysis project,
175 gigabytes of CNF files have been evaluated with a total of 68,069 CNF files (62,251 unique CNF files).

\section{The average SAT problem}
\label{sec:features-average}
%
\begin{prop}
  The set of public benchmarks in SAT competitions between 2008 and 2015
  represent average SAT problems
\end{prop}

It is important to point out that public benchmark files are specifically chosen
to be evaluated before a conference is held. Hence they are expected to terminate
within a given time frame and are therefore not oversized. On the one hand this
ensures that the problems are feasible, but on the other hand they might be a
biased selection. At this point no better data set is available and therefore
we proceeded with this dataset.

According to my results, an average SAT problem consists of:
\begin{itemize}
  \item 83,542 clauses in average ranging from 21 to 53,616,734 (median = 430, $\sigma$ = 848,388)
  \item The longest clause we found had 61,473~literals, but the longest clause of an average CNF covers 20 literals.
  \item The total number of literals in a CNF ranges from 60 up to 150,609,758.
  \item The clause-variables ratio lies between 1.22 and 27,720 with mean = 9.54 and $\sigma$ = 139.
  \item The average length of a clause is expected to be 3. The largest clause length mean we found was 19.58 compared to 2.83 as the smallest clause length mean.
  \item Surprisingly, in average a CNF file has 205 connected literal components and 53 connected variable components. However both corresponding medians are $1$ meaning that at least have of the problems still have only one component. Component sizes have not been evaluated.
  \item In average, 32,787 clauses are definite and 35,094 clauses are goal clauses.
  \item In average, a literal occurs in 1.4~\% of the clauses of the CNF.
  \item 47~\% of literals in a clause are positive.
  \item The arithmetic mean tells 124 unit clauses per CNF file can be expected, but the median tells it is mostly 0.
  \item The largest variable found was 13,842,706 and 13,829,558~variables were used at most.
  \item Exactly one CNF file was true-trivial (namely \texttt{dubois/dubois100.cnf} of SATlib)
    whereas 13 CNF files were false-trivial (of SAT competition 2014 and SAT-Race 2015).
\end{itemize}


\documentclass{beamer}

\usepackage{kotex}
\usepackage{graphicx,psfrag,amsfonts,amsmath,amssymb}
\usepackage{multicol}
\usepackage{algorithm2e}

usetheme{metropolis}
\usefonttheme[onlymath]{serif}	% 수식 설정!!
\setbeamertemplate{section in toc}[sections numbered]

\title{Functional Logistic Regression with Sparse Functional PCA Method}
%\date{\today}
\date[Short Occasion]{August 21, 2019}
\author{Hyunsung Kim}
\institute{Department of Statistics\\ Chung-Ang University}

% A subtitle is optional and this may be deleted
\subtitle{}

%\author{F.~Author\inst{1} \and S.~Another\inst{2}}
% - Give the names in the same order as the appear in the paper.
% - Use the \inst{?} command only if the authors have different
%   affiliation.

%\institute[Universities of Somewhere and Elsewhere] % (optional, but mostly needed)
%{
%  \inst{1}%
%  Department of Computer Science\\
%  University of Somewhere
%  \and
%  \inst{2}%
%  Department of Theoretical Philosophy\\
%  University of Elsewhere}
% - Use the \inst command only if there are several affiliations.
% - Keep it simple, no one is interested in your street address.

%\date{Conference Name, 2013}
% - Either use conference name or its abbreviation.
% - Not really informative to the audience, more for people (including
%   yourself) who are reading the slides online

\subject{Functional Data Analysis}
% This is only inserted into the PDF information catalog. Can be left
% out.

% If you have a file called "university-logo-filename.xxx", where xxx
% is a graphic format that can be processed by latex or pdflatex,
% resp., then you can add a logo as follows:

% \pgfdeclareimage[height=0.5cm]{university-logo}{university-logo-filename}
% \logo{\pgfuseimage{university-logo}}

% Delete this, if you do not want the table of contents to pop up at
% the beginning of each subsection:
\AtBeginSubsection[]
{
  \begin{frame}<beamer>{Outline}
    \tableofcontents[currentsection,currentsubsection]
  \end{frame}
}

\def \bY {\mathbf{Y}}
\def \by {\mathbf{y}}
\def \bX {\mathbf{X}}
\def \bB {\mathbf{B}}
\def \bD {\mathbf{D}}
\def \bbeta {\boldsymbol{\beta}}
\def \btheta {\boldsymbol{\theta}}
\def \bTheta {\boldsymbol{\Theta}}
\def \bepsilon {\boldsymbol{\epsilon}}
\def \balpha {\boldsymbol{\alpha}}
\def \bgamma {\boldsymbol{\gamma}}
\def \bGamma {\boldsymbol{\Gamma}}
\def \bR {\boldsymbol{R}}
\def \bZ {\boldsymbol{Z}}
\def \bG {\boldsymbol{G}}
\def \bu {\boldsymbol{u}}
\def \bV {\boldsymbol{V}}

% Let's get started
\begin{document}

\begin{frame}
  \titlepage
\end{frame}

\begin{frame}{Outline}
  \tableofcontents
  % You might wish to add the option [pausesections]
\end{frame}

% Section and subsections will appear in the presentation overview
% and table of contents.
\section{Introduction}
\begin{frame}{Introduction}
	\begin{block}{Temporal gene expression data}
		\vspace{0.1cm}
		\begin{itemize}
			\item {
				The data was measured at $18$ equal time points($0, 7, ..., 119$).
			}
			\item {
				From this dense data, We generate 100 datasets based on the functional PCs.
			}
		\end{itemize}
	\end{block}
	\begin{block}{Classification for the simulated data}
		\vspace{0.1cm}
		\begin{itemize}
			\item {
				We compute FPC scores using sparse functional PCA method.
			}
			\item {
				Using the above FPC scores, we perform classification using the functional logistic regression.
			}
		\end{itemize}
	\end{block}
\end{frame}

\section{Methods}
\begin{frame}{Methods}
	\begin{block}{The Sparse Functional PCA}
		\vspace{0.1cm}
		\begin{itemize}
			\item {
				It can be applied the curves measured at irregular or sparse time points.
			}
			\item {
				 James \textit{et al.} (2001) used the reduced rank model to solve the functional PC problem.
			}
			\item {
				To fit above model, \texttt{EM} algorithm was used.
			}
		\end{itemize}
	\end{block}
\end{frame}

\begin{frame}{Methods}
	\begin{block}{Functional Logistic Regression}
		\vspace{0.1cm}
		$$ Y_i = \pi_i + \epsilon_i, \hspace{0.3cm} i=1,\dots,n $$
		where $Y_i=1$, if the curve $\in G_1$ and $Y_i=0$, if the curve $\in G_2$,
		%\vspace{-0.3cm}
		$$ \begin{aligned}
		   \pi_i &= P(Y=1|X=\boldsymbol{x}_i)\\
			 	  &= \frac{ \exp[\alpha + \int_T \beta(t)x_i(t) dt] }{1 + \exp[\alpha + \int_T \beta(t)x_i(t) dt]}
		\end{aligned}  $$
		with $X:T \rightarrow \mathbb{R}$ is the predictor, $\alpha$ is an intercept parameter, $\boldsymbol\beta : T \rightarrow \mathbb{R}$ is a coefficient function, and $\epsilon_i$ is the independent errors with zero mean.
	\end{block}
\end{frame}

\begin{frame}{Methods}
	\begin{block}{Functional Logistic Regression with functional PC approach}
		\vspace{0.1cm}
		$$ Y_i = \pi_i + \epsilon_i, \ i=1,\dots,n $$
		where $Y_i=1$, if the curve $\in G_1$ and $Y_i=0$, if the curve $\in G_2$,
		%\vspace{-0.3cm}
		$$ \pi_i = \frac{ \exp[\alpha + \sum_{k=1}^K \beta_k\xi_{ik}] }{1 + \exp[\alpha + \sum_{k=1}^K \beta_k\xi_{ik}]}, \ i=1,\dots,n $$
		with $\alpha$ is an intercept parameter and $\boldsymbol\beta$ is a coefficient function, $\xi_{ik}$ is $k$th FPC score for the $i$th individual, and $\epsilon_i$ is the independent errors with zero mean.
	\end{block}
\end{frame}

\section{Simulation}
\begin{frame}{Simulation}
	\begin{block}{The simulated datasets}
		\vspace{0.1cm}
		\begin{itemize}
			\item {
				The $100$ datasets are simulated from the first $5$ estimated FPCs from the temporal gene expression data.
			}
			\item {
				 Each dataset has $200$ curves with 2 groups($G_1, G_2$)
				 and is randomely divided to $100$ training and test sets for each.
			}
			\item {
				We perform the functional logistic regression for the training sets, and predict for the test sets.
			}
		\end{itemize}
	\end{block}
\end{frame}

\begin{frame}{Simulation}
	\begin{figure}[h] %%% t: top, b: bottom, h: here
		\begin{center}
			\includegraphics[width=1\linewidth]{img/1.png}
		\end{center}
		\label{fig:long}
		\label{fig:onecol}
		\caption{The mean curve and 5 FPC functions for $1$st training set}
	\end{figure}
\end{frame}

\begin{frame}{Simulation}
	\begin{figure}[h] %%% t: top, b: bottom, h: here
		\begin{center}
			\includegraphics[width=1\linewidth]{img/3.png}
		\end{center}
		\label{fig:long}
		\label{fig:onecol}
		\caption{Scatterplot of pairwise FPC scores for $1$st training dataset}
	\end{figure}
\end{frame}

\begin{frame}{Simulation}
	\begin{figure}[h] %%% t: top, b: bottom, h: here
		\begin{center}
			\includegraphics[width=1\linewidth]{img/2.png}
		\end{center}
		\label{fig:long}
		\label{fig:onecol}
		\caption{The curves classified by functional logistic regression for $1$st simulated dataset}
	\end{figure}
\end{frame}

\begin{frame}{Simulation}
	\begin{table}[ht]
		\caption{Classification error rates between Dense and Sparse method}
		\centering
		\tiny
		\begin{tabular}{lllllll}
			\hline
			No. of & \multicolumn{2}{l}{Group 1} & \multicolumn{2}{l}{Group 2} & \multicolumn{2}{l}{Overall} \\
			FPCs   & Dense & Sparse & Dense & Sparse & Dense & Sparse \\
			\hline
			$1$ & 32.72 (8.41) & 31.67 (0.08) & 32.70 (8.31) & 33.65 (0.08) & 32.71 (5.26) & 32.68 (0.06) \\
			$2$ & 22.16 (6.65) & 22.20 (0.08) & 22.06 (6.15) & 22.00 (0.07) & 22.11 (4.33) & 22.10 (0.05) \\
			$3$ &  7.58 (4.58) & 12.45 (0.11) &  8.26 (5.34) & 12.47 (0.09) &  7.92 (3.35) & 12.46 (0.09) \\
			$4$ &  7.14 (4.14) & 11.81 (0.09) &  7.62 (5.10) & 11.37 (0.08) &  7.38 (3.11) & 11.59 (0.08) \\
			$5$ &  7.40 (4.07) & 12.22 (0.11) &  7.86 (5.26) & 11.45 (0.08) &  7.63 (3.06) & 11.83 (0.09) \\
			\hline
		\end{tabular}
	\end{table}
\end{frame}

\begin{frame}{Simulation}
	\begin{block}{Comparison between Dense and Sparse FPCA method}
		\vspace{0.1cm}
		\begin{itemize}
			\item {
				The sparse method shows higher misclassification rate than the dense one.
			}
			\item {
				The Monte Carlo standard errors are much lower on the sparse method.
			}
			\item {
				For the data measured at all time points, the dense functional PCA method perform well than the sparse method.
			}
		\end{itemize}
	\end{block}
\end{frame}

\appendix
\section{Reference}
\begin{frame}
  \frametitle<presentation>{Reference}

  \begin{thebibliography}{10}
  	\beamertemplatearticlebibitems
	% Followed by interesting articles. Keep the list short.
	 \bibitem{Someone2000}
		%GARETH M. JAMES, TREVOR J. HASTIE, CATHERINE A. SUGAR
		James G.M., Hastie T.J., Sugar C.A.
		\newblock Principal component models for sparse functional data
		\newblock {\em Biometrika}, 87(3):587--602,
		2000.

   	\beamertemplatearticlebibitems
		\bibitem{Someone2008}
		 Zhou L. \textit{et al.}
		 \newblock Joint modeling of paired sparse functional data using principal components
		 \newblock {\em Biometrika}, 95(3):601--619,
		 2008.

    \beamertemplatearticlebibitems
    \bibitem{Someone2006}
    Leng. X. and Müller. HG.
    \newblock Classification using functional data analysis for temporal gene expression data
    \newblock {\em Bioinformatics}, 22(1):68--76,
    2006.

  \end{thebibliography}
\end{frame}

% All of the following is optional and typically not needed.
%\appendix
%\section<presentation>*{\appendixname}
%\subsection<presentation>*{For Further Reading}
%
%\begin{frame}[allowframebreaks]
%  \frametitle<presentation>{For Further Reading}
%
%  \begin{thebibliography}{10}
%
%  \beamertemplatebookbibitems
%  % Start with overview books.
%
%  \bibitem{Author1990}
%    A.~Author.
%    \newblock {\em Handbook of Everything}.
%    \newblock Some Press, 1990.
%
%
%  \beamertemplatearticlebibitems
%  % Followed by interesting articles. Keep the list short.
%
%  \bibitem{Someone2000}
%    S.~Someone.
%    \newblock On this and that.
%    \newblock {\em Journal of This and That}, 2(1):50--100,
%    2000.
%  \end{thebibliography}
%\end{frame}

\end{document}

Sparse FPCA/Application/logistic_reg/Functional logistic regression for sparse functional pca method.tex


\documentclass[Physics.tex]{subfiles}
\begin{document}
\chapter{Current of Electricity}
\section{Charge, current and potential difference}
\sldef{Electric charge} is the physical property of matter that causes it to experience a force when placed in an electromagnetic field. For charge to flow in a circuit, there must generally be a closed conducting path extending from the positive to negative terminals of an energy source.

The \sldef{potential difference} (p.d.) between two points in an electrical circuit is the electrical energy converted into other forms of energy per unit charge passing from one point to the other. One \sldef{volt} (\SI{1}{\volt}) is the p.d. between two points in an electrical circuit when one joule of electrical energy is converted to other forms of energy as one coulomb of charge passes from one point to the other.

\sldef{Current} is the rate of flow of charge through a particular cross-sectional area with respect to time. \begin{equation}I = \frac{\mathrm{d}Q}{\mathrm{d}t}\end{equation} The \sldef{conventional current} follows positive charges flowing from the positive to negative terminal.

The \sldef{charge} that passes through a given point is the product of the steady current flowing through that point and the time for which the current flows. \begin{equation}Q = It\end{equation} One \sldef{coulomb} (\SI{1}{\coulomb}) is the quantity of charge that flows through a point when a steady current of one ampere flows through that point for one second.
\section{Resistance and resistivity}
The \sldef{resistance} of a device is defined as the ratio of the p.d. across it to the current flowing through it. \begin{equation}R = \frac{V}{I}\end{equation} A resistor has a resistance of one \sldef{ohm} (\SI{1}{\ohm}) if a current of one ampere flows through it when a p.d. of one volt is applied across it.

The power expended by a resistor is the product of the current through it and the p.d. across it. \begin{equation}P = IV = I^2R = \frac{V^2}{R}\end{equation}

\sldef{Ohm's law} states that the ratio of p.d. across a conductor to the current flowing through it is a constant if physical conditions like temperature remain constant.

The resistance of a metal increases with temperature. As temperature increase, vibration of the lattice ions increases; free electrons collide more frequently with the ions, experiencing more obstructions and a lower drift velocity, so resistance increases.

The resistance of a semiconductor generally decreases with temperature, as more electrons are likely to have sufficient thermal energy to escape from the ions, so there are more electrons to act as charge carriers.

A \sldef{diode} is a device that allows current to pass only in one direction. A diode with p.d. in the direction with low resistance is forward biased, and vice versa. An excessive reverse biased p.d. can cause the diode to break down (at the breakdown voltage), leading to a sudden large current i.e. a short circuit.

The resistance of a material is generally proportional to its resistivity and length, and inversely proportional to its cross-sectional area. \begin{equation}R = \frac{\rho l}{A}\end{equation}
\section{Electromotive force and sources}
The \sldef{electromotive force} of a source is the energy converted from other forms to electrical energy per unit charge delivered round a complete circuit. \[\varepsilon = \frac{W}{Q}\]

The e.m.f. of a source is the ability of the source to convert other forms of energy to electrical energy, while p.d. across a resistor is the ability of the resistor to convert electrical energy to other forms of energy.

Sources of e.m.f. generally have an internal resistance. This resistance can simply be treated as a resistor in series with an ideal source.

For a battery with e.m.f. \(\varepsilon\), the terminal p.d. across it is \begin{equation}V = \varepsilon - Ir\end{equation}

The \sldef{output efficiency} of a circuit is generally given by \begin{equation}\frac{R}{R+r}\end{equation} where \(R\) is the resistance of all the devices in the circuit, and \(r\) is the internal resistance of the battery. This is at a maximum when \(R = r\).
\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}

\newcommand\ddfrac[2]{{\displaystyle\frac{\displaystyle #1}{\displaystyle #2}}}

\title{COP290-A1}
\author{Sayam Sethi 2019CS10399 \\ Mallika Prabhakar 2019CS50440 }
\date{March 2021}

\usepackage{natbib}
\usepackage{graphicx}

\begin{document}

\maketitle

\section{Metrics}
%define your utility and runtime metrics here
For any software, we need to consider a lot of aspects like accuracy, hardware resources, privacy, latency, etc. Improving one aspect might hamper the second aspect hence there is a trade-off. To analyse this trade-off, we have to use certain metrics to compare different approaches and optimisations. The utility function and runtime metric used for this assignment are described as follows-

\subsection{Penalty Function}
A penalty function is considered which takes into account both deviation in error and execution time into account.

\textbf{Specifications:} The penalty function was made keeping in account that higher reduction in execution time leads to a larger error and hence, if the execution time reduction is more, the contribution of the error to the function should be reduced. The final function considered was:
\begin{center}
    $\ddfrac{1 + \mathit{mean}(\mathit{square}(10000\cdot\mathit{diff}(\mathit{queueDensityBaseline}, \mathit{queueDensityCurr})))}{1 + \frac{\mathit{baseExecutionTime} - \mathit{currentExecutionTime}}{\mathit{baseExecutionTime}}}$
\end{center}
Initially many variations were explored in which the numerator and denominator were raised to different powers, however, they led to highly similar graphs and not much difference was observed in the relative trends. Hence, the simple ratio was finally considered.

\subsection{Runtime Metric}
Efficiency is one of the most important factors to consider while writing a program. Different programs have different order functions and hence there is a need to be able to compare different functions on the basis of the load they put on CPU and the time they take to run. Thus there arises a need to have a proper runtime metric.

\textbf{Specifications:}
We consider the time taken (in seconds) using $std::chrono$ for the program to run on the given video. The clock is started once the initial pre-processing is done and is stopped once the complete program is executed (after calculating and printing the values). To ensure uniformity across all runs of the code with different parameters, the system conditions were kept as constant as possible with the system on charging. Additionally, CPU usage was also monitored, where the percent usage of each core of the CPU was observed and the trends have been reported separately.

\section{Methods}
Following methods are the ones which we implemented and considered with respect to the sub-task 2 implementation:

\subsection{Sub-sampling frames}
Main idea behind sub-sampling is to process every $x\textsuperscript{th}$ frame i.e. if the last frame was $N$, the next frame to be processed will be the one at position $N+x$. In our implementation, we skip the processing of next $x-1$ frames by using a for loop to go through the frames in the designated video.

\subsection{Resolution reduction}
Resolution reduction is performed to decrease the processing time of various functions called on the frame matrix by reducing it size while trying to keep the error minimum. In our implementation, we reduce the resolution of each dimension of the video by a factor of $x$ and process it.

\subsection{Spatial threading}
Splitting work spatially by allotting different sections of every frame matrix to different threads is a lucrative method to spread the processing load. The reference frame is read and broken into $x$ segments which is stored in a struct. During the processing of each frame, the frame is directly passed to the thread functions, where based on the $thread\_id$, the corresponding segment is obtained and sum of the ``white pixels" obtained. In the same loop, the threads are joined and the values obtained are added and the result reported for that thread.

\subsection{Temporal threading}
This method involves reading multiple frames one after the other and then processing all of them at the same time. In a single iteration of the loop, $x$ frames are read and corresponding threads created and the frame passed to it. In the same iteration, the threads are combined so that they can be reused in the next iteration.

\section{Trade-off analysis}
While building a software, our purpose is to not only maximise the accuracy but also the other user and hardware specific features like latency in computation, temperature of the CPU unit and other hardware, security and privacy of data, etc. For this assignment, we have mainly considered the computation speed and accuracy for trade-off analysis.

\subsection{Sub-sampling frames}
Intuitively, the execution time is expected to decrease with a larger sub-sampling parameter. Indeed, this is what actually happens:
\begin{center}
    \makebox[\textwidth]{\includegraphics[width=1.25\textwidth]{Graphs/subsampling.png}}
    Sub-sampling graphs
\end{center}
From the graph, we observe that the execution time decreases by a large extent initially and then saturates at about $30s$. The penalty function also, as expected, increases with a larger sub-sampling factor. If one permits slight tolerance, then sub-sampling factor of $5$ is the most optimal, since there is a huge reduction in execution time, but a very small value of penalty function.

\subsection{Resolution reduction}
In this case again, we expect the execution time to reduce with the increase in the resolution parameter. The graph conveys the same results:
\begin{center}
    \makebox[\textwidth]{\includegraphics[width=1.25\textwidth]{Graphs/resolution.png}}
    Resolution graphs
\end{center}
Initially, there is a slight increase since resizing is not a constant time operation. Also, the reduction in the execution time is relatively lesser for smaller scaling factor and decreases suddenly. The execution time saturates in this too, at about $32s$. The penalty function increases considerably with higher scaling factor and shoots up beyond a factor of $3$. Thus, to get optimal results with a small tolerance, scaling factor under $1.5$ should be preferred, and to get considerable reduction in execution time but a relatively higher tolerance is acceptable, then a scaling factor between $1.5$ and $2$ is desirable.

\subsection{Spatial threading}
In this method, it is not easy to guess if execution time will improve with more threads or not since splitting each frame into segments has a large contribution to the execution time too. On running the code and plotting the graphs, we get:
\begin{center}
    \makebox[\textwidth]{\includegraphics[width=1.25\textwidth]{Graphs/spatialThreading.png}}
    Spatial threading graphs
\end{center}
Large variation is observed in the execution time with different number of threads. The execution time initially increases since creating more matrices takes more time than what is saved because of dividing the work into threads. On increasing the threads further, the execution time decreases since the time saved due to parallel processing is more than the time increase due to forming segments. The time taken increases slightly on increasing the number of threads since creating and joining larger number of threads is time consuming.\par
The penalty function is dependent only on the execution time since the error encountered is zero (and hence the numerator equals $1$). Thus, the most optimal number of threads in spatial threading is $4$, which gives the least execution time. However, it is to be noted that this is still more than the execution time that is obtained without using any threads.

\subsection{Temporal threading}
In this method, multiple frames are processed at once, hence, the execution time is expected to decrease with larger number of threads. This is what the graph shows:
\begin{center}
    \makebox[\textwidth]{\includegraphics[width=1.25\textwidth]{Graphs/temporalThreading.png}}
    Temporal threading graphs
\end{center}
Initially the execution time is larger compared to the baseline code since creating threads takes time. Also, the reduction in the execution time is clearly visible with a decrease in the number of threads. Additionally, the reduction in execution time gradually becomes lesser. Similar to spatial threading, the penalty function is dependent only on the execution time since the error is zero and hence the numerator equal to $1$. The most optimal parameter would be to use as many threads as possible that can be simultaneously processed by the computer ($12$ in the case of the device used to perform testing on).

\subsection{Note on CPU Usage}
The execution of only the main function is not directly a single thread operation and the processor uses multiple threads even for a single thread code:
\begin{center}
    \makebox[\textwidth]{\includegraphics[width=1.25\textwidth]{Graphs/nonThreadedCPU.png}}
    CPU usage on running a single thread code
\end{center}
In the previous image, the average CPU usage increased to over $60\%$ across all threads even though it was a single thread operation. Also, it can be noticed that at any point of time, there is one core which has close to $100\%$ usage and that thread keeps on changing.\par
For a multi-threaded code, the CPU usage looks as follows:
\begin{center}
    \makebox[\textwidth]{\includegraphics[width=1.25\textwidth]{Graphs/threadedCPU.png}}
    CPU usage on running a multi-thread code ($4$ threads)
\end{center}
Here, all the $12$ cores have a really high average CPU usage, over $80\%$, even though only $4+1$ (additional threads + main thread) threads were used. This can be possibly explained using the fact that the exact same set of cores is not used to create the threads, since breaking and joining of the threads happens multiple times throughout the code.

\end{document}