% Options for packages loaded elsewhere
\PassOptionsToPackage{unicode}{hyperref}
\PassOptionsToPackage{hyphens}{url}
\PassOptionsToPackage{dvipsnames,svgnames*,x11names*}{xcolor}
%
\documentclass[
]{article}
\usepackage{lmodern}
\usepackage{amssymb,amsmath}
\usepackage{ifxetex,ifluatex}
\ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex
  \usepackage[T1]{fontenc}
  \usepackage[utf8]{inputenc}
  \usepackage{textcomp} % provide euro and other symbols
\else % if luatex or xetex
  \usepackage{unicode-math}
  \defaultfontfeatures{Scale=MatchLowercase}
  \defaultfontfeatures[\rmfamily]{Ligatures=TeX,Scale=1}
\fi
% Use upquote if available, for straight quotes in verbatim environments
\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
\IfFileExists{microtype.sty}{% use microtype if available
  \usepackage[]{microtype}
  \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
}{}
\makeatletter
\@ifundefined{KOMAClassName}{% if non-KOMA class
  \IfFileExists{parskip.sty}{%
    \usepackage{parskip}
  }{% else
    \setlength{\parindent}{0pt}
    \setlength{\parskip}{6pt plus 2pt minus 1pt}}
}{% if KOMA class
  \KOMAoptions{parskip=half}}
\makeatother
\usepackage{xcolor}
\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available
\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}
\hypersetup{
  colorlinks=true,
  linkcolor=blue,
  filecolor=Maroon,
  citecolor=Blue,
  urlcolor=Blue,
  pdfcreator={LaTeX via pandoc}}
\urlstyle{same} % disable monospaced font for URLs
\usepackage{graphicx}
\makeatletter
\def\maxwidth{\ifdim\Gin@nat@width>\linewidth\linewidth\else\Gin@nat@width\fi}
\def\maxheight{\ifdim\Gin@nat@height>\textheight\textheight\else\Gin@nat@height\fi}
\makeatother
% Scale images if necessary, so that they will not overflow the page
% margins by default, and it is still possible to overwrite the defaults
% using explicit options in \includegraphics[width, height, ...]{}
\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio}
% Set default figure placement to htbp
\makeatletter
\def\fps@figure{htbp}
\makeatother
\setlength{\emergencystretch}{3em} % prevent overfull lines
\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
\setcounter{secnumdepth}{5}

%% pandoc-secnos: required package
\usepackage{cleveref}

\author{}
\date{}

\begin{document}

\newcommand{\Ber}{\operatorname{Ber}}
\newcommand{\E}{\operatorname{E}}
\newcommand{\V}{\operatorname{Var}}
\newcommand{\diag}{\operatorname{diag}}

\newcommand{\bi}{\mathbf{i}}
\newcommand{\bj}{\mathbf{j}}
\newcommand{\bt}{\mathbf{t}}

\newcommand{\bw}{{\boldsymbol{w}}}
\newcommand{\bu}{{\boldsymbol{u}}}
\newcommand{\bv}{{\boldsymbol{v}}}
\newcommand{\bx}{{\boldsymbol{x}}}
\newcommand{\by}{{\boldsymbol{y}}}
\newcommand{\bb}{{\boldsymbol{b}}}
\newcommand{\bz}{{\boldsymbol{z}}}
\newcommand{\bX}{{\boldsymbol{X}}}
\newcommand{\bY}{{\boldsymbol{Y}}}
\newcommand{\bZ}{{\boldsymbol{Z}}}

\hypertarget{gradient-descent}{%
\section{Gradient Descent}\label{gradient-descent}}

\hypertarget{motivation}{%
\subsection{Motivation}\label{motivation}}

In Linear Regression, we studied how to minimize the error function
\[ E = \sum_{j=1}^{N} (y_j-\sum_{s=1}^{k+1} x_{js}m_{s})^2 \] and
obtained an exact solution. In other cases we will encounter later, such
an exact solution is not feasible and we will have to use a method to
approximate an exact solution. One of the most common methods for this
purpose is \emph{gradient descent}.

\hypertarget{basic-algorithm}{%
\subsection{Basic Algorithm}\label{basic-algorithm}}

Consider a function \(E: \mathbb R^n \rightarrow \mathbb R\),
\({\boldsymbol{w}}=(w_1, w_2, \dots , w_n) \rightarrow E({\boldsymbol{w}})\).
The \emph{gradient} \(\nabla E\) of \(E\) is defined by
\[  \nabla E := \left ( \frac {\partial E}{\partial w_1}, \frac {\partial E}{\partial w_2}, \dots , \frac {\partial E}{\partial w_n} \right ). \]

\textbf{Proposition }: \emph{Assume that \(E({\boldsymbol{w}})\) is
differentiable in a neighborhood of \({\boldsymbol{w}}\). Then the
function \(E({\boldsymbol{w}})\) decreases fastest in the direction of
\(-\nabla E ({\boldsymbol{w}})\).}

\textbf{Proof:} For a unit vector \({\boldsymbol{u}}\), the directional
derivative \(D_{\boldsymbol{u}}E\) is given by
\[ D_{\boldsymbol{u}}E= \nabla E \cdot {\boldsymbol{u}}= | \nabla E | |{\boldsymbol{u}}| \cos \theta = | \nabla E | \cos \theta ,\]
where \(\theta\) is the angle between \(\nabla E\) and
\({\boldsymbol{u}}\). The minimum value of \(D_{\boldsymbol{u}}E\)
occurs when \(\cos \theta\) is \(-1\). ~ ~ \(\square\)

Set
\[ {\boldsymbol{w}}_{k+1}={\boldsymbol{w}}_k - \eta \nabla E({\boldsymbol{w}}_k) \]
where \(\eta >0\) is the step size or \emph{learning rate}. Then
\[ E({\boldsymbol{w}}_{k+1}) \le E({\boldsymbol{w}}_{k}).\] Under some
moderate conditions,
\[ E({\boldsymbol{w}}_k) \rightarrow \text{local minimum} \qquad \text{ as }\ k \rightarrow \infty .\]In
particular, this is true when \(E\) is convex or when \(\nabla E\) is
Lipschitz continuous.

\hypertarget{example}{%
\subsubsection{Example}\label{example}}

Consider \(E({\boldsymbol{w}})=E(w_1,w_2)= w_1^4+w_2^4-16w_1w_2\). Then
\(\nabla E({\boldsymbol{w}})= [ 4w_1^3-16w_2, 4w_2^3-16w_1]\). Choose
\({\boldsymbol{w}}_0=(1,1)\) and \(\eta =0.01\). Then
\[ {\boldsymbol{w}}_{30}=(1.99995558586289, 1.99995558586289)\] and we
get \[ E({\boldsymbol{w}}_{30})= -31.9999999368777. \]

We see that \({\boldsymbol{w}}_k \rightarrow (2,2)\) and \(E(2,2)=-32\).
Indeed, using multi-variable calculus, one can verify that when
\({\boldsymbol{w}}=(2,2)\), a local minimum of \(E({\boldsymbol{w}})\)
is \(-32\).

\textbf{Exercise}: Using multi-variable calculus, find all the local
minima of \(E({\boldsymbol{w}})\).

\begin{figure}
\hypertarget{fig:des}{%
\centering
\includegraphics[width=0.3\textwidth,height=\textheight]{descent.png}
\caption{Graph of \(E({\boldsymbol{w}})\)}\label{fig:des}
}
\end{figure}

\begin{figure}
\hypertarget{fig:tab}{%
\centering
\includegraphics[width=0.5\textwidth,height=\textheight]{table.png}
\caption{Values of \(E({\boldsymbol{w}}_k)\)}\label{fig:tab}
}
\end{figure}

\hypertarget{example-linear-regression-revisited}{%
\subsubsection{Example: Linear Regression
revisited}\label{example-linear-regression-revisited}}

We will apply gradient descent to linear regression in the lab session.

\textbf{Exercise}: Define
\(\sigma(x) = \dfrac {e^x}{e^x+1}= \dfrac 1 {1+e^{-x}}\). In Logistic
Regression we will minimize the following error function
\[ E ({\boldsymbol{w}}) = - \sum_{n=1}^N \{ t_n \ln y_n + (1-t_n) \ln (1-y_n)\},  \]
where we write \({\boldsymbol{w}}=(w_1, w_2, \dots , w_{k+1})\) and
\(y_n=\sigma(w_1 x_{n1}+ w_2 x_{n2} + \cdots + w_k x_{nk}+w_{k+1} )\).
Compute the gradient \(\nabla E({\boldsymbol{w}})\).

\hypertarget{newtons-method}{%
\subsection{Newton's Method}\label{newtons-method}}

Let us first consider the single-variable case.

\begin{itemize}
\item
  Let \(f: \mathbb R \longrightarrow \mathbb R\) be a single-variable
  (convex, differentiable) function.
\item
  To find a local minimum \(\Longleftrightarrow\) \qquad To find \(x^*\)
  such that \(f'(x^*)=0\)\\
  Make a guess \(x_0\) for \(x^*\) and set \(x=x_0+h\).
\item
  Using the Taylor expansion, we have
  \begin{align*} f(x) &=f(x_0+h) \approx f(x_0) + f'(x_0) h+ \tfrac 1 2 f''(x_0) h^2 \\ f'(x) = \frac{df}{dh}\frac {dh}{dx} & \approx \frac d {dh} \left ( f(x_0) + f'(x_0) h+ \tfrac 1 2 f''(x_0) h^2 \right ) \\
  &= f'(x_0) + f''(x_0)h \end{align*} If \(0 = f'(x_0) + f''(x_0)h\), we
  obtain \[ h = - f'(x_0)/f''(x_0). \]
\item
  We have shown that \[ x_1=x_0- f'(x_0)/f''(x_0) \] is an approximation
  of \(x^*\).
\item
  Repeat the process to obtain
  \[ \boxed{x_{k+1}=x_k- f'(x_k)/f''(x_k)},  \] and
  \(x_k \rightarrow x^*\) as \(k \rightarrow \infty\).
\end{itemize}

Now we consider the multi-variable case.

\begin{itemize}
\item
  Let \(E({\boldsymbol{w}})\) be a multi-variable function. The
  \emph{Hessian matrix} of \(E\) is defined by
  \[ \mathbf H E= \begin{bmatrix} \tfrac {\partial^2 E}{\partial w_1^2} &  \tfrac {\partial^2 E}{\partial w_1 \partial w_2} & \cdots &  \tfrac {\partial^2 E}{\partial w_1 \partial w_n} \\ \tfrac {\partial^2 E}{\partial w_2 \partial w_1} & \tfrac {\partial^2 E}{\partial w_2^2} & \cdots & \tfrac {\partial^2 E}{\partial w_2 \partial w_n} \\ \vdots & \vdots & \ddots & \vdots \\ \tfrac {\partial^2 E}{\partial w_n \partial w_1} & \tfrac {\partial^2 E}{\partial w_n \partial w_2} & \cdots & \tfrac {\partial^2 E}{\partial w_n^2}  \end{bmatrix}. \]
  That is,
  \(\mathbf H E= [ \tfrac {\partial^2 E}{\partial w_i \partial w_j}]\).
\item
  Generalizing the single-variable case, we obtain\\
  \[ \boxed{ {\boldsymbol{w}}_{k+1}= {\boldsymbol{w}}_k - \mathbf H E ({\boldsymbol{w}}_{k})^{-1} \nabla E({\boldsymbol{w}}_k)} . \]
\item
  Using a step size \(\eta\), the formula may be modified to be
  \[  \boxed{ {\boldsymbol{w}}_{k+1}= {\boldsymbol{w}}_k - \eta \mathbf H E ({\boldsymbol{w}}_{k})^{-1} \nabla E({\boldsymbol{w}}_k)} . \]
\item
  Newton's method is much faster than Gradient Descent. However, it may
  be expensive to compute the inverse of the Hessian matrix.
\end{itemize}

\hypertarget{example-1}{%
\subsubsection{Example}\label{example-1}}

\begin{itemize}
\tightlist
\item
  Consider \(E({\boldsymbol{w}})=E(w_1,w_2)= w_1^4+w_2^4-16w_1w_2\).
  Then
  \(\nabla E({\boldsymbol{w}})= [ 4w_1^3-16w_2, 4w_2^3-16w_1]^\top\).
\end{itemize}

\begin{align*} \mathbf H E({\boldsymbol{w}}) &= \begin{bmatrix} 12 w_1^2 & -16 \\ -16 & 12 w_2^2 \end{bmatrix} \\ \\ \mathbf H E({\boldsymbol{w}})^{-1} &= \frac 1 {9w_1^2 w_2^2 -16} \begin{bmatrix} \tfrac 3 4 w_2^2 & 1 \\ 1& \frac 3 4 w_1^2  \end{bmatrix} \\  \\
\mathbf H E^{-1} \nabla E & = \frac 1 {9w_1^2 w_2^2 -16} \begin{bmatrix} 3 w_1^3 w_2^2 -8 w_2^3 -16w_1 \\ 3 w_1^2 w_2^3 -8 w_1^3  -16w_2  \end{bmatrix} \end{align*}

\begin{itemize}
\tightlist
\item
  Choose \({\boldsymbol{w}}_0=(1.2,1.2)\) and \(\eta =1\). Then
  \({\boldsymbol{w}}_{9}=(2.00000004189571, 2.00000004189571)\),
  \(E({\boldsymbol{w}}_{9})= -31.9999999999999\).
\end{itemize}

\begin{figure}
\hypertarget{fig:tab}{%
\centering
\includegraphics[width=0.5\textwidth,height=\textheight]{table-1.png}
\caption{Values of \(E({\boldsymbol{w}}_k)\)}\label{fig:tab}
}
\end{figure}

\hypertarget{stochastic-gradient-descent-sgd}{%
\subsubsection{Stochastic Gradient Descent
(SGD)}\label{stochastic-gradient-descent-sgd}}

Typically in Machine Learning, the function \(E({\boldsymbol{w}})\) is
given by a sum of the form
\[ E({\boldsymbol{w}}) = \frac 1 N \sum_{n=1}^N E_n({\boldsymbol{w}}), \]
where \(N\) is the number of elements in the training set. When \(N\) is
large, computation of the gradient \(\nabla E\) may be expensive.

The SGD selects a sample from the training set in each iteration step
instead of using the whole batch of the training set, and use
\[ \frac 1 M \sum_{i=1}^M \nabla E_{n_i}({\boldsymbol{w}}) ,\] where
\(M\) is the size of the sample and
\(\{ n_1, n_2 , \dots , n_M \} \subset \{ 1,2, \dots , N \}\). The SGD
is commonly used in many Machine Learning algorithms.

\end{document}


% This is a LaTeX tutorial by Overleaf titled [Learn LaTeX in 30 minutes](https://www.overleaf.com/learn/latex/Learn_LaTeX_in_30_minutes).

\documentclass[12pt, letterpaper, oneside]{report}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage{amsmath}

\title{First document}
\author{Tran Phong Binh\thanks{phongbinh2511@gmail.com}}
\date{\today}

\begin{document}

\maketitle

\begin{abstract}
This is a simple paragraph at the beginning of the document. A brief introduction about the main subject.
\end{abstract}

\tableofcontents

\chapter{First Chapter}

\section{Introduction}
The first line after section is always fucked like this. Feels like an introduction to the section though.

First document. This is a simple example, with no extra parameters or packages included. Maybe if I put in more words, the difference between oneside and twoside will be revealed. I am not sure but this is what I would do. Unfortunately the experiment failed, I guess oneside and twoside implies a different meaning than what I have thought. Oh! I tried again, and it had nothing to do with the formatting of the content, but of the document itself! The margins are changed, they are shifted, together with the footnotes. I don't know if there are more things that change! So this is what working with \LaTeX{} feels like! Some of the \textbf{greatest} discoveries in \underline{science} were made by \textbf{\textit{accident}}. Below are some unordered lists stuff:
\begin{itemize}
    \item The individual entries are indicated with a black dot, a so-called bullet.
    \item The text in the entries may be of any length.
\end{itemize}
What if I want to continue the thing. Would I do it like this? Yes, yes it is. Now here it comes to the ordered lists:
\begin{enumerate}
    \item This is the first entry in our list
    \item The list numbers increase with each entry we add
\end{enumerate}
And then some texts follow that to be less awkward! In physics, the mass-energy equivalence is stated by the equation $E = mc^2$, discovered in 1905 by Albert Einstein. Again, the equation but in display mode will be written as
\[ E = mc^2 \]
for aesthetic. In natural units ($c = 1$), the formula expresses the identity
\begin{equation}
E = m
\end{equation}
Okay, that's cool. Now we are testing some texts after that.

\section{Second Section}
Well of-course this is the case too. Introduction stuff on the first line!

Some of the greatest \emph{discoveries}
in science were made by accident.

\begin{figure}[h]
    \centering
    \includegraphics[width=0.25\textwidth]{potato}
    \caption{potato world}
    \label{bar}
\end{figure}

As you can see in the figure \ref{bar}, this is the world of potato. Also for your curiosity, page \pageref{bar} is the page of that image.

\subsection{First Subsection}
I mean like why not? Introduction on the first line seems cool.

\textit{Some of the greatest \emph{discoveries}
in science were made by accident.}

\textbf{Some of the greatest \emph{discoveries}
in science were made by accident.}

\section*{Unnumbered Section}
\addcontentsline{toc}{section}{Unnumbered Section Name On ToC}
Well maybe it will become uncool somehow, but for the matter of writting, this first line introduction seems like a good convention.

Subscripts in math mode are written as $a_b$ and superscripts are written as $a^b$. These can be combined an nested to write expressions such as
\[ T^{i_1 i_2 \dots i_p}_{j_1 j_2 \dots j_q} =
T \left( x^{x_1}, \dots, x^{i_p}, e_{j_1}, \dots, e_{j_q} \right) \]

We write integrals using $\int$ and fractions using $\frac{a}{b}$. Limits are placed on integrals using superscripts and subscripts:
\[ \int_0^1 \frac{dx}{e^x} = \frac{e - 1}{e} \]

Lower case Greek letters are written as $\omega$ $\delta$ etc. while upper case Greek letters are written as $\Omega$ $\Delta$.

Mathematical operators are prefixed with a backslash as $\sin(\beta)$, $\cos(\alpha)$, $\log(x)$ etc.

\begin{center}
\begin{tabular}{ |c|c|c| }
    \hline
    cell1 & cell2 & cell3 \\
    cell4 & cell5 & cell6 \\
    cell7 & cell8 & cell9 \\
    \hline
\end{tabular}
\end{center}

Okay, I guess that's enough playing with toy table. It's time for some real shit. Table \ref{table:data} is an example of referenced \LaTeX{} elements.

\begin{table}[h!]
    \centering
    \begin{tabular}{ ||c|c c c|| }
        \hline
        No. & Col1 & Col2 & Col3 \\ [0.5ex]
        \hline \hline
        1 & 6 & 87837 & 787 \\
        2 & 7 & 78 & 5415 \\
        3 & 545 & 778 & 7507 \\
        4 & 545 & 18744 & 7560 \\
        5 & 88 & 788 & 6344 \\ [1ex]
        \hline
    \end{tabular}
    \caption{Table to test captions and labels}
    \label{table:data}
\end{table}

\end{document}


\chapter{Hashes}
\label{hashes}

This chapter presents another built-in type called a hash.
Hashes are one of Perl's best and most commonly used
features; they are the building blocks of many efficient
and elegant algorithms.

\section{A hash is a mapping}
\label{hash_descr}

\index{hash}
\index{type!hash}
\index{key}
\index{key-value pair}
\index{index}
\index{mapping}
A {\bf hash} is like an array, but more general.  In an
array, the indices or subscripts have to be integers; in a hash,
they can be (almost) anything.

A hash contains a collection of indices, which are called
{\bf keys}, and a collection of values.  Each key is associated with a single value.  A key and a value together form a pair
(an object of the Pair type), or a {\bf key-value pair}. A
hash can be viewed as a collection of key-value pairs, which
are sometimes called {\bf items} or elements.
\index{item}

In other programming languages, hashes are sometimes called
dictionaries, hash tables, maps or associative arrays.

In mathematical language, a hash represents a {\bf mapping}
from keys to values, so you can also say that each key
``maps to'' a value. As an example, we'll build a hash that
maps from English to Spanish words, so the keys and the
values are all strings.

In Perl, hash names start with the ``\verb"%"'' sigil. To create
a new hash, you just declare it this way:
\index{sigil}
\index{sigil, percent}

\begin{verbatim}
> my %eng2sp;
\end{verbatim}

This creates an empty hash. To add items to the hash,
you can use braces a.k.a. curly brackets (sometimes also
called ``curlies''):
\index{curly bracket}
\index{bracket!curly}
\index{brace}

\begin{verbatim}
> %eng2sp{'one'} = 'uno';
uno
\end{verbatim}
%
This line creates an item that maps from the key
\verb"'one'" to the value \verb"'uno'".

If the key is a string containing a single word (i.e.
without any space in the middle of it), there is a more
idiomatic shortcut to create the same hash entry:
\index{idiomatic}

\begin{verbatim}
> %eng2sp<one> = 'uno';
uno
\end{verbatim}
%

If we print the hash, we see a key-value pair with a
\verb'=>' pair constructor operator between the key
and value:
\index{pair constructor}

\begin{verbatim}
> say %eng2sp;
one => uno
\end{verbatim}
%
This output format is also an input format.  For example,
you can create a new hash with three items:

\begin{verbatim}
> my %eng2sp = ('one' => 'uno', 'two' => 'dos', 'three' => 'tres');
one => uno, three => tres, two => dos
\end{verbatim}
%

Using the \verb'=>' pair constructor operator between keys and
values is not required, you may use a comma as well:

\begin{verbatim}
my %eng2sp = ('one', 'uno', 'two', 'dos', 'three', 'tres');
\end{verbatim}
%

But the pair constructor has the advantage of showing more
graphically the key-value relations. The pair constructor
operator also makes the use of quotes non mandatory on its
left-hand side (if the key is a string with no space):

\begin{verbatim}
> my %eng2sp = (one => 'uno', two => 'dos', three => 'tres');
one => uno, three => tres, two => dos
\end{verbatim}
%

You might also use a more concise list syntax for the hash
assignment and Perl will happily convert the list into a
hash, provided the number of items in the input list is even:

\begin{verbatim}
> my %eng2sp = <one uno two dos three tres>;
one => uno, three => tres, two => dos
\end{verbatim}
%

You might be surprised by the output. The order of the
key-value pairs is usually not the order in which you
populated the hash. In general, the order
of items in a hash is unpredictable.

But that's not a problem because the elements of a hash
are never indexed with integer subscripts. Instead, you use
the keys to look up the corresponding values:

\begin{verbatim}
> say %eng2sp<two>;
dos
\end{verbatim}
%
The key \verb"two" always maps to the value \verb"'dos'"
so the order of the items doesn't matter.

If the key isn't in the hash, you get an undefined value:

\begin{verbatim}
> say %eng2sp<four>;
(Any)
\end{verbatim}
%
The {\tt elems} method or function works on hashes just
as on arrays; it returns the number of key-value pairs:
\index{elems function or method}
\index{function!elems}
\index{elems function or method}
\index{method!elems}

\begin{verbatim}
> say %eng2sp.elems;
3
> say elems %eng2sp
3
\end{verbatim}
%
The {\tt :exists} adverb also works on hashes as on
arrays; it tells you whether
something appears as a {\em key} in the hash (appearing
as a value is not good enough).
\index{membership!hash}
\index{exists adverb}
\index{adverb!:exists}

\begin{verbatim}
> %eng2sp<two> :exists;
True
> %eng2sp<four> :exists;
False
\end{verbatim}
%
To see whether something appears as a value in a hash, you
can use the {\tt values} method, which returns a collection of
values, and then use a loop (or possibly a {\tt grep}) to
look for the searched item.
\index{values function or method}
\index{method!values}
\index{grep}

\begin{verbatim}
my @vals = values %eng2sp;
for @vals -> $value {
    say "Found it!" if $value eq 'uno';           # -> Found it!
}
\end{verbatim}
%
Or more concisely:
\begin{verbatim}
say "Found it!" if grep {$_ eq 'uno'}, %eng2sp.values;
\end{verbatim}

Since {\tt grep} defaults to a smart match, this can be
made even more concise:

\begin{verbatim}
say "Found it!" if grep {'uno'}, %eng2sp.values;  # -> Found it!
\end{verbatim}

When looking for values, the program has to search the
elements of the list in order (or sequentially), as in
Section~\ref{find}.  As the list gets longer, the search
time gets longer in direct proportion.
\index{sequential search}
\index{search!sequential}

By contrast, when looking for keys, Perl uses a {\bf hashing}
algorithm that has a remarkable property: it takes about
the same amount of time no matter how many items are in
the hash. In other words, it performs really fast, compared
to the list size, when the
searched list is large. This is the reason why the solution to
the reverse pair exercise (Exercise~\ref{reverse_pair}) of the
previous chapter using a hash was almost three times
faster than the bisection search solution
(see Subsection~\ref{sol_reverse_pair}).

\label{ex_employees}
As an exercise, use the sample employee data of the
multidimensional array of Section~\ref{multidimensional array}
(p.~\pageref{multidimensional array}), load it into a hash and
lookup for some salaries. Hint: you don't need a
multidimensional structure for doing that with a hash.
Solution: \ref{sol_ex_employees}

\section{Common operations on hashes}

We've seen already that to populate a hash, you can just
assign an even list to it. The four syntactical forms
below are correct:

\begin{verbatim}
my %first_quarter = ("jan" => 1, "feb" => 2, "mar" => 3);
my %second_quarter = (apr => 4, may => 5, jun => 6);
my %third_quarter = jul => 7, aug => 8, sep => 9;
my %fourth_quarter = < oct 10 nov 11 dec 12 >;
\end{verbatim}

To add an element to a hash, just assign the hash with a key:

\begin{verbatim}
my %months = ("jan" => 1, "feb" => 2, "mar" => 3);
%months{'apr'} = 4;
say %months;         # -> apr => 4, feb => 2, jan => 1, mar => 3
\end{verbatim}

Remember that you can also do the same without having to
quote the keys if you use the angle brackets quote-word
operator (if the keys are strings):
\index{quote-word operator}
\index{angle bracket}

\begin{verbatim}
%months<apr> = 4;       # same as: %months{'apr'} = 4;
\end{verbatim}

or you can also use the {\tt push} function with a pair:
\index{push function}
\index{function!push}

\begin{verbatim}
> push %months, (may => 5);
apr => 4, feb => 2, jan => 1, mar => 3, may => 5
> my $new-pair = jun => 6
jun => 6
> push %months, $new-pair;
apr => 4, feb => 2, jan => 1, jun => 6, mar => 3, may => 5
\end{verbatim}
%

Using {\tt push} to add a pair to a hash is not exactly the same,
though, as making a hash assignment: if the key already
exists, the old value is not replaced by the new one---instead,
the old and the new ones are placed into an array (or,
if the old value is already an array, then the new value
is added to the array):

\begin{verbatim}
> push %months, (jan => '01');
{apr => 4, feb => 2, jan => [1 01], jun => 6, mar => 3, may => 5}
\end{verbatim}

To check whether an element exists, use the {\tt :exists}
subscript adverb:
\index{exists adverb}
\index{adverb!:exists}
\begin{verbatim}
> say True if  %months<jan>:exists;
True
\end{verbatim}

To check whether a value is defined for a given key,
use {\tt defined}:
\index{defined}

\begin{verbatim}
> say True if defined %months<apr>;
True
\end{verbatim}
%

To obtain the number of items in a hash, use the {\tt elems}
method:
\index{elems function or method}

\begin{verbatim}
say %months.elems;                # -> 6
\end{verbatim}

To remove a hash item, use the {\tt :delete} adverb:
\index{delete adverb}
\index{adverb!:delete}

\begin{verbatim}
> push %months, (jud => 7);      # Oops, a typo!
apr => 4, feb => 2, jan => 1, jud => 7, jun => 6, mar => 3, may => 5
> %months{'jud'}:delete;         # typo now removed
7
> say %months
apr => 4, feb => 2, jan => 1, jun => 6, mar => 3, may => 5
\end{verbatim}

To iterate over a hash, use:
\index{kv function or method}
\index{keys function or method}
\index{values function or method}
\index{pairs function or method}

\begin{itemize}
\item {\tt kv} to retrieve the interleaved keys and values;
\item {\tt keys} to retrieve the keys;
\item {\tt values} to retrieve the values;
\item {\tt pairs} to retrieve the key-value pairs;
\end{itemize}

For example:

\begin{verbatim}
> for %months.kv -> $key, $val { say "$key => $val" }
jan => 1
apr => 4
mar => 3
jun => 6
may => 5
feb => 2
> say keys %months;
(jan apr mar jun may feb)
> say values %months;
(1 4 3 6 5 2)
> say %months.pairs;
(jan => 1 apr => 4 mar => 3 jun => 6 may => 5 feb => 2)
\end{verbatim}
%

\section{Hash as a collection of counters}
\label{histogram}
\index{counter}

Suppose you are given a string and you want to count how many
times each letter appears.  There are several ways you could do it:

\begin{itemize}

\item You could create 26 variables, one for each letter of the
alphabet.  Then you could traverse the string and, for each
character, increment the corresponding counter, probably using
a chained conditional.

\item You could create an array with 26 elements.  Then you could
convert each character to a number (using the built-in function
{\tt ord}), use the number as an index into the array, and
increment the appropriate counter.

\item You could create a hash with characters as keys
and counters as the corresponding values.  The first time you
see a character, you would add an item to the hash.  After
that, you would increment the value of an existing item.

\end{itemize}

Each of these options performs the same computation, but each
of them implements that computation in a different way.
\index{implementation}

An {\bf implementation} is a way of performing a computation;
some implementations are better than others.  For example,
an advantage of the hash implementation is that we don't
have to know ahead of time which letters appear in the string
and we only have to make room for the letters that do appear.

Here is what the code might look like:

\begin{verbatim}
sub histogram (Str $string) {
    my %histo;
    for $string.comb -> $letter {
        %histo{$letter}++;
    }
    return %histo;
}
\end{verbatim}
%
The name of the function is {\tt histogram}, which is a statistical
term for a collection of counters (or frequencies).
\index{histogram}
\index{frequency}
\index{traversal}
\index{comb function and method}
\index{counter}

The first line of the
function creates an empty hash.  The {\tt for} loop traverses
the string.  Each time through the loop, if the character \verb'$letter' is
not in the hash, Perl creates a new item with key
\verb'$letter' and defaults the values to 0 when the ``++''
operator is called on it, so that the first value immediately
thereafter is 1.  If \verb'$letter'  is already in the hash,
the value is incremented.
\index{increment operator}

Here's how it works:

\begin{verbatim}
> say histogram("We all live in a yellow submarine")
W => 1, a => 3, b => 1, e => 4, i => 3, l => 5, (...) y => 1
\end{verbatim}
%
The histogram indicates that the letters \verb"'W'" and \verb"'b'" appear only once; \verb"'a'" and \verb"'i'" appear three times; \verb"'e'" appears four times, and so on.

\section{Looping and hashes}
\index{hash!looping with}
\index{looping!with hashes}
\index{traversal}

If you use a hash in a {\tt for} statement, it traverses
the pairs of the hash:

\begin{verbatim}
> for %eng2sp -> $pair { say $pair}
two => dos
three => tres
one => uno
\end{verbatim}
%

We have named the iteration variable \verb'$pair' to point
out more clearly that the program is iterating over key-value
pairs (actually Pair objects). You may use the {\tt key} and {\tt value} (notice
the singular) methods to access the key and value of a Pair.
For example, to reverse the order in which each line is printed:

\begin{verbatim}
> for %eng2sp -> $pair { say $pair.value ~ " <= " ~ $pair.key; }
dos <= two
tres <= three
uno <= one
\end{verbatim}

Again, the keys are in no particular order.  To traverse
the keys in sorted order, you can use the {\tt keys}
(plural) and {\tt sort} functions or methods:
\index{keys function or method}
\index{method!keys}
\index{sort!function or method}
\index{method!sort}

\begin{verbatim}
my %histo = histogram("We all live in a yellow submarine");
for %histo.keys.sort -> $key {
    say "$key\t%histo{$key}";
}
\end{verbatim}

\section{Reverse lookup}
\label{raise}
\index{hash!lookup}
\index{hash!reverse lookup}
\index{lookup, hash}
\index{reverse lookup, hash}

Given a hash \verb'%hash' and a key \verb'$k', it is easy to
find the corresponding value \verb'$val = %hash{$k}'.
This operation is called a {\bf lookup} and, as already
mentioned, this is fast even when the hash is very large.
\index{lookup}
\index{hash lookup}

But what if you have \verb'$val' and you want to find
\verb'$k'? You have three problems: first, there might be
more than one key that maps to the value \verb'$val'; depending
on the application, you might be able to pick one, or you
might have to make an array that contains all of them.
Second, there is no simple syntax to do a {\bf reverse lookup};
you have to search. Third, it might be time-consuming if
the hash is large.

Here is a function that takes a value and returns the first
key that maps to that value:
\index{reverse lookup}

\begin{verbatim}
sub reverse-lookup (%hash, $val) {
    for %hash -> $pair {
        return $pair.key if $pair.value eq $val;
    }
    return;
}
\end{verbatim}
%
This subroutine is yet another example of the search pattern.
If we get to the end of the loop, that means \verb'$val'
doesn't appear in the hash as a value, so we return an
undefined value (Nil). Here, the responsibility to react
to that situation is left to the caller. An alternative
might be to raise an exception, which would still have
to be dealt with by the caller. However, since direct
lookup with the key is not raising an exception but simply
returning an undefined value when the key does not exist,
it makes sense for {\tt reverse-lookup} to have the same
behavior when the value is not found.

Here is an example of a successful reverse lookup:

\begin{verbatim}
> my %histo = histogram('parrot');
a => 1, o => 1, p => 1, r => 2, t => 1
> my $key =  reverse-lookup %histo, "2";
r
\end{verbatim}
%
And an unsuccessful one:

\begin{verbatim}
> say reverse_lookup %histo, "3";
Nil
\end{verbatim}
%

Another more concise way to do reverse lookup would be to
use {\tt grep} to retrieve a list of values satisfying our
condition:
\begin{verbatim}
 say grep { .value == 2 }, %histo.pairs;   # (r => 2)
\end{verbatim}

or to an expression with the {\tt first} built-in function
to retrieve only the first one:
\begin{verbatim}
my %histo = histogram('parrot');
say %histo.pairs.first: *.value == 1;      # ->  p => 1
\end{verbatim}

\index{whatever}
but this latter example uses the ``*'' placeholder parameter
which we haven't covered yet in this book. Let's just say that,
here, the ``*'' stands successively for every pair of the hash,
and the {\tt first}function returns the first pair that matches
the condition on the value (see
Section~\ref{whatever star parameter} for details on the
``*'' parameter).

A reverse lookup is much slower than a forward lookup; if you
have to do it often, or if the hash gets big, the performance
of your program will suffer.
\index{reverse lookup}

\section{Testing for existence}
\index{existence!testing for}

A quite common task is to determine whether something exists or
if a given value has already been seen in a program. Using a
hash is usually the best solution because finding out whether
there is an entry for a given key is very simple and also
very efficient: you just need to store the values that you
want to watch as a key entry in a hash, and then check for
its existence when needed.

In such a case, you often don't really
care about the value and you might put basically anything.
It is quite common in that case to use ``1'' as a value, but
you might as well store {\tt True} or any other value you
like.

Suppose we want to generate 10 random integers between 0 and
49, but want to make sure that the integers are unique. We
can use ten times the {\tt rand} method on the desired range. But
the likelihood to hit twice the same number is far from
negligible (see Exercise~\ref{birthdays} on the so-called
birthday paradox and its solution (Subsection~\ref{sol_birthdays})
for a similar situation). For example, trying this:
\index{duplicate!checking}
\index{rand function}

\begin{verbatim}
> my @list;
[]
> push @list, 50.rand.Int for 1..10;
> say @list;
[12 25 47 10 19 20 25 42 33 20]
\end{verbatim}

produced a duplicate value in the list (25) on the first try.
And the second try produced three pairs of duplicates:

\begin{verbatim}
> say @list;
[45 29 29 27 12 27 20 5 28 45]
\end{verbatim}

We can use a hash to reject any generated random integer
that has already been seen. The following is a possible way to
code this:
\index{rand function}

\begin{verbatim}
my @list;
my %seen;
while @list.elems < 10 {
    my $random = 50.rand.Int;
    next if %seen{$random}:exists;
    %seen{$random} = 1;
    push @list, $random;
}
say @list;
\end{verbatim}

Every valid integer generated is added to both the \verb'%seen'
hash and the output list. But before doing
that, the generated integer is checked against the
\verb'%seen' hash to verify that it has not been seen yet.
When this program is finished running, the list has ten unique (pseudo-)random integers.

We have made it step by step and kept two separate data
structures, the \verb'@list' output array and the
\verb'%seen' hash, to make the process as clear as possible.
If you think about it, however, \verb'@list' and \verb'%seen'
have essentially the same content at any step through the
loop. We don't really need to keep track of the same data in
two places. Since having a hash is important for checking that the
output values are unique, we can get rid of \verb'@list'
and write a more concise and probably more idiomatic version
of the same program:
\index{idiomatic}
\index{push function}

\begin{verbatim}
my %seen;
while %seen.elems < 10 {
	my $random = 50.rand.Int;
	push %seen, ($random => 1) unless %seen{$random}:exists;
}
say keys %seen;      # -> (39 12 46 27 14 21 4 38 25 47)
\end{verbatim}

This can be further simplified. It is not really necessary
here to check whether the generated integer exists in the
hash: if it does exist, the old hash element will be replaced
by the new one, and the hash will be essentially unchanged. In
addition, when evaluated in a scalar numeric context, a
hash returns the number of its elements, so that the call
to the {\tt .elems} is not necessary. This is the new version:
\index{scalar context}
\index{elems function or method}

\begin{verbatim}
my %seen;
%seen{50.rand.Int} = 1 while %seen < 10;
say keys %seen;      # -> (46 19 5 36 33 1 20 45 47 30)
\end{verbatim}

This last version is probably more concise and more idiomatic,
but that's not meant to say that it is better. It is
perfectly fine if you prefer the second or the first version,
maybe because you find it clearer. Use whichever version you
like better, or your own modified version provided it does
the expected work. This is Perl, \emph{there
is more than one way to do it} (TIMTOWTDI).
\index{TIMTOWTDI}

Note however that the pure hash version doesn't keep the order
in which the numbers were generated, so (pseudo-)randomness
might not be as good.

Also note, by the way, that Perl has a {\tt pick} function or
method to choose elements at random from a list without
repetition.
\index{pick function or method}

\section{Hash keys are unique}

It is not possible to have the same key in a hash more than once.
Trying to map a new value to a key will replace the old
value with the new one. Here is an example of hash creation
with duplicates keys:
\index{duplicate}

\begin{verbatim}
> my %friends = (Tom => 5, Bill => 6, Liz => 5, Tom => 7, Jack => 3)
Bill => 6, Jack => 3, Liz => 5, Tom => 7
\end{verbatim}

Because two of our friends are named Tom, we lose the data
associated with the first of them. This is something you should
be careful about: hash keys are unique, so you'll lose some items
if the data associated with your keys has duplicates. The
next section will show some ways of dealing with this
possible problem.

But this key uniqueness property also has very interesting upsides.
For example, a typical way of removing duplicates from a list of
items is to assign the list items to the keys of a hash (the
value does not matter); at the end of the process, the list of
keys has no duplicates.

\begin{verbatim}
> my @array = < a b c d s a z a r e s d z r a >
[a b c d s a z a r e s d z r a]
> my %unique = map { $_ => 1 }, @array;
a => 1, b => 1, c => 1, d => 1, e => 1, r => 1, s => 1, z => 1
> my @unique_array = keys %unique;
[z a e s d c b r]
\end{verbatim}

As you can see, duplicates have been removed from the output
array. In such a simple case, the {\tt unique} built-in function
would have been sufficient to remove duplicates from
\verb'@array', but within a more complex program,
it is quite common to use a hash (often called \verb'%seen')
to check whether a value has already been seen.
\index{unique function}
\index{map}

\section{Hashes and arrays}
\label{invert}

\index{inverting a hash}
\index{hash!invert}
Inverting a hash can be very easy if it is known that the
values can happen only once (that they are unique). Consider
for example a hash mapping months to their number in the year
(we limit the example to five months for brevity):

\begin{verbatim}
> my %months = jan => 1, feb => 2, mar => 3, apr => 4, may => 5;
apr => 4, feb => 2, jan => 1, mar => 3, may => 5
\end{verbatim}
%

We can transform the key-value pairs into a flat list,
reverse the list and assign the reversed list to a new
hash:

\begin{verbatim}
> my %rev_months = %months.kv.reverse;
1 => jan, 2 => feb, 3 => mar, 4 => apr, 5 => may
\end{verbatim}
\index{kv function or method}
\index{reverse function or method}
%

We now have a new hash mapping month numbers to their names.
This can be very handy if a hash is known to be bijective, but this approach does not work correctly if a value can
happen more than once: in such a case, some pairs will be
lost:

\begin{verbatim}
> my %months = jan => 1, january => 1, feb => 2, february => 2;
feb => 2, february => 2, jan => 1, january => 1
> my %rev_months = %months.kv.reverse;
1 => january, 2 => february
\end{verbatim}

Arrays can appear as values in a hash.  For example, if you
are given a hash that maps from letters to frequencies, you
might want to invert it; that is, create a hash that maps
from frequencies to letters.  Since there might be several
letters with the same frequency, each value in the inverted hash
should probably be an array of letters.
\index{inverting a hash}
\index{hash!invert}

Here is a function that inverts such a hash:

\begin{verbatim}
sub invert-hash (%in-hash) {
    my %out-hash;
    for %in-hash.kv -> $key, $val {
        push %out-hash{$val}, $key;
    }
    return %out-hash;
}
\end{verbatim}
%
Each time through the loop, a hash item is assigned to the  \verb'$key' and \verb'$val' variables, and \verb'$key' is
appended to the value \verb'%output-hash' for the \verb'$val'
key; if that value does not exist yet, it is created. At
the end of the process, the values of \verb'%output-hash' are
all anonymous arrays.

Here is an example:

\begin{verbatim}
my %rev-hist = invert-hash histogram 'parrot';
say %rev-hist;
dd %rev-hist;
\end{verbatim}

This will display:

\begin{verbatim}
1 => [p a o t], 2 => [r]
Hash %rev-hist = {"1" => $["p", "a", "o", "t"], "2" => $["r"]}
\end{verbatim}

Notice that the {\tt say} function gives a simple representation
of the hash data, and that the new {\tt dd} (short for ``data
dumper'') function used here gives more detailed information.
{\tt dd} is not very commonly used in normal programs, but
can be quite useful while debugging a program to display a
detailed description of a complex data structure
\footnote{To tell the full truth, {\tt dd} is not standard
Perl~6, it is a Rakudo-specific debugging feature. A future
implementation of Perl~6 not based on Rakudo might not have
it.}.
\index{say function or method}
\index{dd function}

\verb'%output-hash' contains two items (two pairs) whose
values are anonymous arrays. You can access the second
element of the first array using the hash value
\verb'%rev-hist{"1"}' as if it was any ordinary array name,
with this simple syntax:

\begin{verbatim}
say %rev-hist{"1"}[1];  # -> a
\end{verbatim}

\begin{figure}
\centerline
{\includegraphics[scale=0.8]{figs/hash1.png}}
\caption{State diagram.}
\label{fig.hash1}
\end{figure}

Figure~\ref{fig.hash1} is a state diagram showing \verb'%hist'  and \verb'%rev-hist' .
A hash is represented as a box with the type {\tt hash} above it
and the key-value pairs inside.
\index{state diagram}
\index{diagram!state}

Arrays can be values in a hash, as this example shows, but they
cannot be keys.  If you try, you're likely to end up with a
key that contains only one item of the array, but most probably
not what you intended:

\begin{verbatim}
my @a = 'a' .. 'c';
my %h;
%h{@a} = 5;
say %h;  # -> a => 5, b => (Any), c => (Any)
\end{verbatim}

Here, Perl interpreted the \verb'%h{@a} = 5;' assignment
as a a slice assignment, i.e. assumed that we were
trying to populate three items in one go, one for each
element of the array.

\index{hash!function}
\index{hashable}
As mentioned earlier, a hash is implemented using
a hashing function and that means that the keys have to
be \emph{hashable} \footnote{This is not entirely true. The
keys of a ``normal'' hash must be hashable and therefore
immutable. There is another type of hash, object hashes,
for which the need to have immutable keys does not apply.}.
A {\bf hash} is a function that takes
a value (of any kind) and returns an integer.  Hashes use
these integers, called hash values, to store and look up
key-value pairs.
\index{immutability}

This system works fine if the keys are immutable.  But if the
keys were mutable, like arrays, bad things would happen. For example,
when you create a key-value pair, Perl would hash the key and
store it in the corresponding location.  If you modify the
key and then hash it again, it would go to a different location.
In that case, you might have two entries for the same key,
or you might not be able to find a key.  Either way, the
hash wouldn't work correctly.

That's why keys have to be hashable, and why mutable types like
arrays aren't. So Perl will do something else that can be
useful (such as creating three distinct hash items in the
example above), but will not hash the array itself.

Since hashes are mutable, they can't be used as keys,
but they {\em can} be used as values, so that you can
have nested hashes.

\section{Memos}
\label{memoize}
\index{memoize}
\index{cache}
\index{memo}
\index{Fibonacci}

If you played with the {\tt fibonacci} subroutine from
Section~\ref{one.more.example}, you might have noticed that
the bigger the argument you provide, the longer the
subroutine takes to run. Furthermore, the run time
increases extremely quickly.
\index{Fibonacci!function}
\index{function!Fibonacci}

To understand why, consider Figure~\ref{fig.fibonacci}, which shows
the {\bf call graph} for {\tt fibonacci} with {\tt n=4}:

\begin{figure}
\centerline
{\includegraphics[scale=0.7]{figs/fibonacci.png}}
\caption{Call graph.}
\label{fig.fibonacci}
\end{figure}

A call graph shows a set of subroutine frames, with lines
connecting each frame to the frames of the functions it
calls.  At the top of the graph, {\tt fibonacci} with
\verb'$n=4' calls {\tt fibonacci} with \verb'$n=3' and
\verb'$n=2'.  In turn, {\tt fibonacci} with \verb'$n=3' calls
{\tt fibonacci} with \verb'$n=2' and \verb'$n=1'.  And so on.
\index{function frame}
\index{frame}
\index{call graph}

Count how many times {\tt fibonacci(0)} and {\tt fibonacci(1)}
are called.  This is an inefficient solution to the problem,
and it gets much worse as the argument gets bigger.
\index{memo}

One solution is to keep track of values that have already been
computed by storing them in a hash.  A previously computed value
that is stored for later use is called a {\bf memo}.  Here is a
``memoized'' version of {\tt fibonacci}:

\begin{verbatim}
my %known = 0 => 1, 1 => 1;
say fibonacci(10);
sub fibonacci ($n) {
    return %known{$n} if %known{$n}:exists;
    %known{$n} = fibonacci($n-1) + fibonacci($n-2);
    return %known{$n};
}
\end{verbatim}
%

\verb'%known' is a hash that keeps track of the Fibonacci
numbers we already know.  It starts with
two items: 0 and 1, which both map to 1.

Whenever {\tt fibonacci} is called, it checks \verb'%known'.
If the result is already there, it can return
immediately.  Otherwise, it has to
compute the new value, add it to the hash, and return it.

If you run this version of {\tt fibonacci} and compare it with
the original, you will find that it is much faster, especially
for a large argument (say more than 30).

\index{cache}
\index{memoize}
Memoizing is a form of \emph{caching}, i.e. storing in memory
the result of a (presumably costly) computing operation in
order to avoid computing it again. This process is
sometimes called ``trading memory against CPU cycles''.  In
some cases, such as our Fibonacci recursive example here, the gain
can be absolutely huge: calculating the 100th Fibonacci
number would take billions of years with the original recursive
subroutine and it takes only a split second with the memoized
version.

Please note that in the specific case of the Fibonacci function,
we are storing values for each successive integer; we could
have memoized the Fibonacci numbers in an array rather than
in a hash (and it might even be slightly more efficient), but
using a hash for such purpose is a more general solution,
working even when the memo keys are not consecutive integers.

As an exercise, try to rewrite the {\tt fibonacci} subroutine
using an array instead of a hash to memoize the calculated
Fibonacci numbers.

\section{Hashes as dispatch tables}
\label{dispatch}

You may
need a procedure to launch some action depending on the
value of a parameter received by the program. To do that,
you could use a series of
\verb'if {...} elsif {...} else {...}'
statements like this:

\begin{verbatim}
sub run-stop  { ... };
sub run-start { ... };
my $param = get-param;
if $param eq "stop" {
    run_stop;
} elsif $param eq "start" {
    run-start;
} elsif $param = "h" {
    say $help;
} elsif $param = "help" {
    say $help;
} elsif $param = "v" {
    $verbose = True;
} else {
    die "Unknown option $param";
}
\end{verbatim}

This approach is boring and error prone. Using a dispatch table
is often a simpler solution.

A dispatch table is a data structure mapping identifiers to
code references or subroutine objects. Applied to the
above scenario, it could look like this:

\begin{verbatim}
sub run-stop  { ... };
sub run-start { ... };
my %dispatch = {
    stop  => &run_stop,
    start => &run_start,
    h     => { say $help; },
    help  => { say $help; },
    v     => { $verbose = True;},
}
my $param = get-param;
die "Unknown option $param" unless %dispatch{$param}:exists;
%dispatch{$param}(); # execute the action specified in %dispatch
\end{verbatim}

The \verb'%dispatch' hash defines the action depending on
the parameter used as a key. The \verb'%dispatch{$param}()'
statement calls the required action.

This approach is a bit more concise and slightly cleaner, but there
are some other advantages. It is more maintainable: if you need
to add one option, you just need to add one entry to the hash
and don't have to add code in the middle of a complicated
chain of nested \verb'if {...} elsif {...} else {...}'
statements at the risk of breaking up something.

Another upside is that the dispatch table can be dynamically
modified at run time, for example depending on certain
external circumstances (for example the day in the month when
the program is running) or in accordance with a configuration
file. This means that it is possible to dynamically modify
the behavior of a program after compile time, while it is
already running. This paves the way to some very interesting
advanced programming techniques which are beyond the scope
of this book.

Note that we have been using hashes for our dispatch tables,
and this is the most common way to implement them. If it
makes sense to have small integers as keys, you could also
implement a dispatch table as an array. This is the case,
for example, with numbered menu items where the user is
prompted to type a number to indicate which menu option
to activate.

\section{Global variables}
\index{global variable}
\index{variable!global}
\index{lexical variable}
\index{variable!lexical}

In the previous example, \verb'%known' is created outside the subroutine,
so it belongs to the whole main package.
Such variables are sometimes called {\bf global}
because they can be accessed from any function.  Unlike ``local''
lexical variables, which usually disappear when their scope
ends, global variables persist from one subroutine call to
the next.
\index{flag}

It is common to use global variables for {\bf flags}; that is,
boolean variables that indicate (``flag'') whether a condition
is true.  For example, some programs use a flag named
\verb'$verbose' to control the level of detail in the
output:

\begin{verbatim}
my $verbose = True;
sub example1 {
    say 'Running example1' if $verbose;
    # ...
}
\end{verbatim}
%

Global variables are also sometimes used for environment
variables and parameters passed to the program, as well
as for storing a large
data structure that is the centerpiece of a program, in order
to avoid copying it when passing it around as an argument to
subroutines.

But, asides from those specific cases, it is usually
considered poor practice to use global variable, because
it creates dependencies and unexpected ``action-at-a-distance''
behaviors between various parts of a program and may lead to
difficult-to-track bugs.

\section{Debugging}
\index{debugging}

As you work with bigger datasets it can become unwieldy to
debug by printing and checking the output by hand.  Here are some
suggestions for debugging large data sets:

\begin{description}

\item[Scale down the input:] If possible, reduce the size of the
dataset.  For example if the program reads a text file, start with
just the first 10 lines, or with the smallest example you can find.
You can either edit the files themselves, or (better) modify the
program so it reads only the first {\tt n} lines.

If there is an error, you can reduce {\tt n} to the smallest
value that manifests the error, and then increase it gradually
as you find and correct errors.

\item[Check summaries and types:] Instead of printing and checking the
entire dataset, consider printing summaries of the data: for example,
the number of items in a hash or the total of a list of numbers.

A common cause of runtime errors is a value that is not the right
type.  For debugging this kind of error, it is often enough to print
the type of a value (think about the {\tt .WHAT} method).
\index{WHAT}

It is often useful to add typing to your variables. Where you
expect a string, make sure you type the variable or subroutine
parameter with {\tt Str}. If you expect an integer, type it with
{\tt Int}. If you expect an {\tt Int} of a certain range, create
a subset for it as in Section~\ref{guardian} (p.~\pageref{guardian})
and type the variable with that.
\index{subset!type}
\index{type subset}

\item[Write self-checks:]  Sometimes you can write code to check
for errors automatically.  For example, if you are computing the
average of a list of numbers, you could check that the result is
not greater than the largest element in the list or less than
the smallest.  This is called a ``sanity check'' because it detects
results that are ``insane''.
\index{sanity check}
\index{consistency check}

Another kind of check compares the results of two different
computations to see if they are consistent.  This is called a
``consistency check''.

\item[Format the output:] Formatting debugging output
can make it easier to spot an error.  We saw an example in
Section~\ref{factdebug}.  The {\tt dd} function displays
helpful details on a composite or complex data structure.

\index{dd function}
\index{function!dd}

\end{description}

Again, time you spend building scaffolding can reduce
the time you spend debugging.
\index{scaffolding}

\section{Glossary}

\begin{description}

\item[mapping:] A relationship in which each element of one set
corresponds to an element of another set.
\index{mapping}

\item[hash:] A mapping from keys to their
corresponding values.
\index{hash}

\item[key-value pair:] The representation of the mapping from
a key to a value.
\index{key-value pair}

\item[item:] In a hash, another name for a key-value
  pair.
\index{item!hash}

\item[key:] An object that appears in a hash as the
first part of a key-value pair.
\index{key}

\item[value:] An object that appears in a hash as the
second part of a key-value pair.  This is more specific than
our previous use of the word ``value''.
\index{value}

\item[implementation:] A way of performing a computation.
\index{implementation}

\item[hashtable:] The algorithm used to implement hashes.
\index{hashtable}

\item[hash function:] A function used by a hashtable to
compute the location of a key.
\index{hash!function}

\item[hashable:] A type that has a hash function.  Immutable
types like numbers and strings are hashable; mutable types
like arrays and hashes are not.
\index{hashable}

\item[lookup:] A hash operation that takes a key and finds
the corresponding value.
\index{lookup}

\item[reverse lookup:] A hash operation that takes a value and finds
one or more keys that map to it.
\index{reverse lookup}

\item[call graph:] A diagram that shows every frame created during
the execution of a program, with an arrow from each caller to
each callee.
\index{call graph}
\index{diagram!call graph}

\item[memo:] A computed value stored to avoid unnecessary future
computation.
\index{memo}

\item[global variable:]  A variable defined outside any
subroutine or other block.  Global variables can be
accessed from any subroutine.
\index{global variable}

\item[flag:] A boolean variable used to indicate whether a condition
is true.
\index{flag}

\end{description}

\section{Exercises}

\begin{exercise}
\label{wordlist2}
\index{set!membership}
\index{membership!set}

Write a subroutine that reads the words in {\tt words.txt} and
stores them as keys in a hash.  (It doesn't matter what the
values are.)  Then you can use the {\tt exists} adverb
as a fast way to check whether a string is in
the hash.

If you did Exercise~\ref{bisection}, you can compare the speed
of this implementation with a hash and the bisection search.

Solution: \ref{sol_wordlist2}

\end{exercise}

\begin{exercise}
\label{mem_ackerman}
Memoize the Ackermann function from Exercise~\ref{ackermann}
and see if memoization makes it possible to evaluate the
subroutine with bigger arguments.  Hint: no.
Solution: \ref{sol_mem_ackerman}.
\index{Ackermann function}
\index{function!ack}

\end{exercise}

\begin{exercise}
\index{duplicate}
\label{has_duplicates_hash}

If you did Exercise~\ref{has_duplicates}, you already have
a function named \verb"has-duplicates" that takes a list
as a parameter and returns {\tt True} if any object
appears more than once in the list.

Use a hash to write a faster, simpler version of
\verb"has-duplicates".
Solution: \ref{sol_has_duplicates_hash}.

\end{exercise}

\begin{exercise}
\label{exrotatepairs}
\index{letter rotation}
\index{rotation, letters}

Two words are ``rotate pairs'' if you can rotate one of them
and get the other (see \verb"rotate_word" in
Exercise~\ref{rotate}) using the Caesar Cipher.
\index{Caesar cipher}

Write a program that reads a wordlist (e.g. {\tt words.txt}
and finds all the rotate pairs.
Solution: \ref{sol_exrotatepairs}.

\end{exercise}

\begin{exercise}
\label{homophones}
\index{Car Talk}
\index{Puzzler}

Here's another Puzzler from {\em Car Talk}
(\url{http://www.cartalk.com/content/puzzlers}):

\begin{quote}
This was sent in by a fellow named Dan O'Leary. He came upon a common
one-syllable, five-letter word recently that has the following unique
property. When you remove the first letter, the remaining letters form
a homophone of the original word, that is a word that sounds exactly
the same. Replace the first letter, that is, put it back and remove
the second letter and the result is yet another homophone of the
original word. And the question is, what's the word?

Now I'm going to give you an example that doesn't work. Let's look at
the five-letter word, `wrack.' W-R-A-C-K, you know like to `wrack with
pain.' If I remove the first letter, I am left with a four-letter
word, 'R-A-C-K.' As in, `Holy cow, did you see the rack on that buck!
It must have been a nine-pointer!' It's a perfect homophone. If you
put the `w' back, and remove the `r,' instead, you're left with the
word, `wack,' which is a real word, it's just not a homophone of the
other two words.

But there is, however, at least one word that Dan and we know of,
which will yield two homophones if you remove either of the first two
letters to make two, new four-letter words. The question is, what's
the word?
\end{quote}
\index{homophone}
\index{reducible word}
\index{word, reducible}

You can use the hash from Exercise~\ref{wordlist2} above to check
whether a string is in {\tt words.txt}.

To check whether two words are homophones, you can use the CMU
Pronouncing Dictionary.  You can download it from
\url{http://www.speech.cs.cmu.edu/cgi-bin/cmudict}.
\index{CMU Pronouncing Dictionary}

Write a program that lists all the words in {\tt words.txt}
(or in the CMU dictionary) that solve the Puzzler.
Solution: \ref{sol_homophones}.

\end{exercise}


\documentclass[a4paper,11pt]{article}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{float}
\usepackage[utf8]{inputenc}

\title{MaS Assignment II\\
\large Percolation}
\author{Klaas Kliffen s2369494}
\date{\today}

\begin{document}

\maketitle

\section{Introduction}
The percolation model can be seen as a extremely simple randomised growth model without particle diffusion.
At each step, for all available unmarked neighbour sites to an existing cluster, each one is marked occupied with a fixed probability $p$ or empty with probability $1-p$. Only unmarked sites may be occupied. The probability is the only parameter of this model.

\subsection{Initialisation}
This assignment considers a percolation growth on a square lattice with a fixed size of ($2N+1\times(2N+1)$. The clusters starts at the center of the lattice at $(N,N)$, the rest of the lattice remains unmarked.

\subsection{Stopping conditions}
The model stops when either (a) all neighbour sites of a cluster are marked empty or (b) the cluster ahas reached the border of the lattice, i.e. any site in row or column number 1 or $2N+1$ is occupied.\\
Case (a) refers to a non-percolating or finite cluster. Case (b) is called a percolating or infinite cluster.

\subsection{Cluster size statistics}
The first core problem will focus on statistics on the size of generated clusters and whether they are finite or infinite.\\

\subsection{System size dependence}
The size of the lattice may influence the growth of clusters and the number of clusters becoming percolating.

\section{Methods}
This section globally describes the parameters used for this assignment.
\subsection{Cluster size statistics}
For each probability 100 runs will be used to calculate the mean and standard deviation on the size of a cluster. Only non percolating clusters will be used in the statistics, since the size of the cluster for percolating clusters is not defined.
The clusters will be generated on a lattice of size 99 by 99 sites. Values of $p$ in the range of [0.3,0.7] will be used. Values for either smaller than 0.3 or larger than 0.7 will be neglected. Either the value of $p$ will be too small and not generate a cluster or if $p$ is too large, all clusters will reach the boundary. A step size for $p$ of 0.05 is used.\\

\subsection{System size dependence}
The system size dependence uses the same method as with the cluster size statistics, only using different size for the lattice. The sizes used for the lattices are: $11\times11$,$21\times21$ and $41\times41$

\section{Results}
\subsection{Cluster size statistics}
\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{img/s99r100.png}
\caption{100 runs on a 99 by 99 lattice with $p$ in [0.3,0.7]}
\label{s99}
\end{figure}
In figure \ref{s99} the blue line represent the mean cluster size for a given $p$ with error bars showing the standard deviation. The green bars represent the number of runs which yielded a percolating cluster.\\
The mean cluster size increases until $p = 0.55$. Above that probability the mean cluster size will decrease, due to there being more percolating clusters.
An interesting fact can be seen that until $p = 0.55$ the standard deviation increases. The standard deviation at $p = 0.5$ and $p = 0.55$ is larger than the mean cluster size. This indicates a large spread of cluster sizes, either being very large or very small.\\
The number of infinite clusters slowly increases after $p = 0.55$ reaching 100 for the higher values for $p$. Since the number of runs was 100, this fraction will be 1.
This means that after $p = 0.55$ it becomes more probably for a cluster to be percolating than being a finite cluster.

\subsection{System size dependence}
\begin{figure}[H]
\centering
\includegraphics[width=.8\textwidth]{img/s11r100.png}
\caption{100 runs on a 11 by 11 lattice with $p$ in [0.3,0.7]}
\label{s11}
\end{figure}
\begin{figure}[H]
\centering
\includegraphics[width=.8\textwidth]{img/s21r100.png}
\caption{100 runs on a 21 by 21 lattice with $p$ in [0.3,0.7]}
\label{s21}
\end{figure}
\begin{figure}[H]
\centering
\includegraphics[width=.8\textwidth]{img/s41r100.png}
\caption{100 runs on a 41 by 41 lattice with $p$ in [0.3,0.7]}
\label{s41}
\end{figure}
Comparing the cluster size and fraction of percolating cluster for different lattice sizes for the values of $p = 0.3$ and $p = 0.7$ yields similar values for cluster size and fraction of percolating clusters for all lattice sizes.\\
A difference between the size can be seen at which value for $p$ the fraction of percolating cluster starts to emerge. In figure \ref{s11} this start even at the lowest value of $p = 0.3$. Also, the increase seems to be almost linear for increasing the value of $p$. While in figures \ref{s21},\ref{s41} and also in figure \ref{s99} seems to be more like a inverse tangent function with a translation to the right. The probability $p$ at which the percolating start to appear, increases with the lattice size.
For an infinite lattice, the fraction would only start to appear at $p=1$. As there are an infinite amount of neighbour sites with a probability for each one to be empty, eventually stopping the cluster to grow.\\
Another difference can be seen at the probability $p=0.5$. Increasing the size of the lattice also increases the mean cluster size and the standard deviation of cluster sizes.

\section{Conclusion and discussion}
\subsection{Cluster size statistics}
The mean cluster size grows exponentially with $p$. The cluster sizes however are very skewed for very large and very small clusters, with a large standard deviation.
The fraction of percolating clusters also increases with $p$. Only starting at $p=0.55$ for a lattice with size $99\times99$.
\subsection{System size dependence}
The model seems to be dependent on the size of the lattice. For $p = 0.3$  and $p = 0.7$ the values for the mean cluster size and number of percolating clusters might be similar. But for values in between these statistics do differ.\\
Another observation is at which value for $p$ the percolating clusters start to appear, increasing with the cluster size. Eventually becoming $p = 1$ for an infinite lattice.
\subsection{Discussion}
In this assignment the cluster always started at the center point of the lattice. A cluster started at a random point in the lattice might yield different results. Especially if the point is chosen near one of the borders, the number of percolating clusters might increase.

\end{document}


\documentclass[10pt]{article}

\usepackage{fancyvrb}
%% \usepackage{hyperref}

\title{Parallel and Distributed Programs\\
Lecture F12\\
24 February, 2014
}
\author{Joe Armstrong}

\fvset{frame=single,numbers=left,numbersep=3pt}

\begin{document}

\maketitle

\tableofcontents

\section{Introduction}
In this lecture we'll look at some ways of writing {\sl
  potentially parallel} programs.

In Erlang we write concurrent programs. The algorithms in the lecture
will run sequentially if run on a single core machine, but will run
in parallel if run on a multi-core computer. That's why they are
called ``potentially parallel.''

To make a parallel program either the programmer must say what is to be
done in parallel, or the system must discover this automatically.

Automatic parallelisation of programs has been studied for many years.
It is very good for certain classes of problem
and very bad for other classes of problem.

\subsection{GPU Parallism}

GPU = Graphics Programming Unit. All the computations are the same.

For example: To compute the temperature distribution over a conducting
surface we divide the surface into a grid and repeatedly average the
temperatures at the four corners of the squares surrounding each of
the points on the grid.

\begin{verbatim}

             |
             X (i,j+1)
             |
             |
----X--------X-------X----
  (i-1,j)  (i,j)   (i+1,j)
             |
             |
             X (i,j-1)
             |
             |

  T[k+1] = (T[k][i,j+1] + T[k][i-1,j] +
            T[k][i+1,j] + T[k][i,j-1])/4
\end{verbatim}

Erlang is not suited for this.

\subsection{Embarrassing Parallelism}

The nature of the problem is naturally parallel and there are a lot of
things to be done.  There is little or no communication between the
tasks. The parallelism is obvious and inherent in the problem.

Example: A web server with millions of simultaneous connected users.
Erlang is very good at this.

\subsection{Declarative Parallelism}

Sometimes called ``functional parallelism''. In a pure functional
programming language \verb+f(g(...), h(...))+ \verb+g(...)+ and
\verb+h(...)+ can be evaluated in any order and in parallel.

Unfortunately, while this is true, it is still a research problem how
to map the computation onto the hardware we have {\sl today}, and {\sl
  hardware changes in time} so the solution we find today will not
work tomorrow.

There have been a large number of attempts to build parallel hardware to
do this. None have provided lasting solutions.

\section{Expressing Parallel Behaviour}

Different languages have syntactic constructs for expressing parallel
behavior, or can do this algorithmically.  This section reviews some
commonly used methods for expressing parallel behavior.

\subsection{ParBegin .. ParEnd}

\begin{verbatim}
ParBegin
   x;
   y;
   z;
ParEnd
\end{verbatim}

The above expression means evaluate \verb+x+, \verb+y+ and \verb+z+ in parallel.

\verb+Fork+ ... \verb+Join+ is also used.

If shared memory is involved, simultaneous access to the shared memory must be
avoided. If no shared memory is involved the computations can proceed in parallel.

Both \verb+ParBegin+ .. \verb+ParEnd+ and \verb+Fork+ ... \verb+Join+ can be
implemented using the lower level primitives \verb+promise+ and \verb+yield+.

\subsection{Promises}

A ``{\sl Promise}'' is a commitment to deliver a value in the future.

\begin{verbatim}
    P
    |
    |  promise(F)
T1  +------>-------+
    |              |
    |              |
    |              |
    +------<-------+
    |
    |
T2  +
    |
\end{verbatim}

At some time \verb+T1+ a process \verb+P+ wants to compute \verb+F()+ so it calls
\verb+promise(F)+ which returns a tag \verb+Tag+. \verb+P+ continues,
and \verb+F()+ is computed in parallel. At some time later (\verb+T2+) when
\verb+P+ wants the return value it calls \verb+yield(Tag)+. \verb+yield+ blocks until
the result of the computation is available.

Promises can be implmeneted in Erlang as follows:

\VerbatimInput{promises.erl}

\verb+promises:test1()+ blocks until the result is available:

\begin{Verbatim}
> promises:test1().
.... yawn ..
...  shell is blocked ...
... I can't do anything ...
165580141
\end{Verbatim}

Using a promise we can do {\sl useful things} \textregistered{} while
the promised computation is being performed:

\begin{Verbatim}
2> Tag = promises:test2().
Ref<0.0.0.104>
3> promises:query(Tag).
not_finished
4> 213813813861736*128763816381638163.
27531482667950334890792863030968
5> promises:query(Tag).
not_finished
6> 19379127391273971239719379173979372193137*18763861238183638.
363627257286948472579176034410575587459129627874409292406
7> promises:query(Tag).
{finished,165580141}
\end{Verbatim}

{\sl Note: If a command fails in the shell, the promise will not
be honoured, the results will be send back to the old shell which has died!}

\subsection{Broken Promises}

What happens if \verb+P+ dies {\sl Before} the worker has computed
its response? Should the worker be killed?

What happens if the worker dies while computing F. Who should be told?
Who should be killed?

Distributed Computing is the art of repairing broken promises.

\subsection{Maps}

Here is a simple implement of \verb+smap+ (Sequential map) and
\verb+pmap+ (Parallel map).

\VerbatimInput{maps.erl}

In the non-failure case \verb+smap+ and \verb+pmap1+ behave
identically.

\begin{Verbatim}
1> Double = fun(X) -> 2*X end.
#Fun<erl_eval.6.71889879>
2> maps:smap(Double,[1,2,3,4]).
[2,4,6,8]
3> maps:pmap(Double,[1,2,3,4]).
[2,4,6,8]
\end{Verbatim}

Now let's try and double the list \verb+[1,2,3,a,3,4,b]+. In the
failure case \verb+smap+ will fail on the first error (ie when
computing \verb+2*a+) but \verb+pmap+ will fail for every bad element in
the list.

\begin{Verbatim}
4> maps:smap(Double,[1,2,a,3,4,b]).
** exception error: an error occurred when evaluating an
   arithmetic expression
     in operator  */2
        called as 2 * a
     in call from maps:'-smap/2-lc$^0/1-0-'/2 (maps.erl, line 5)
     in call from maps:'-smap/2-lc$^0/1-0-'/2 (maps.erl, line 5)
5> maps:pmap(Double,[1,2,a,3,4,b]).
[2,4,
 {'EXIT',{badarith,[{erlang,'*',[2,a],[]},
                    {maps,'-pmap/2-fun-0-',3,
                          [{file,"maps.erl"},{line,10}]}]}},
 6,8,
 {'EXIT',{badarith,[{erlang,'*',[2,b],[]},
                    {maps,'-pmap/2-fun-0-',3,
                          [{file,"maps.erl"},{line,10}]}]}}]

\end{Verbatim}

\subsection{Broken Maps}

The idea of \verb+pmap+ seems simple {\sl but it is not}.
Here are some of the problems that might occur.

\begin{description}

\item A computation in the map fails\\
Assume one of the computations of
\verb+F(I)+ fails. We could kill all the other computations. But
what happens if we don't really care if the computation failed? Many
of the earlier computations might have succeeded so why drop the
results? The computations might have taken a long time, it would be a  shame to
loose the results.

\item We have too much concurrency\\
\verb+pmap+ will be {\sl too parallel} if we have a list of 1M processes
and the function is {\sl Double} then pmapping the list would be wrong.
We'd create too many processes.

\item The overheads are too great\\
It might turn out that creating a parallel computation
costs more than just doing the sequential computation.

\end{description}

{\sl Discussion:} \verb+map+ and \verb+pmap+ have identical behavior
when there are no errors. In \verb+pmap+ we fail immediately on the
first error. In \verb+pmap+ we cannot easily fail on the first error.
(We could link all the parallel processes together, and let them all
die if any one dies)

\subsection{Map-Reduce}

Map-reduce is the name given to a method of parallelising
computations.  Note: The word ``map'' does not mean the same as the
word map used in functional programming.

A full map-reduce framework provides the following:

\begin{itemize}

\item A way of starting a number of parallel computations.

\item A number of parallel proceses which perform computations. The map processes
  performs some computation and sends a stream of \verb+{Key,Value}+
  results to a reduce process.

\item A reduce process which merges values with the same Key.
\end{itemize}

For example: Calculating the frequency of words in a large set of
documents.  Assume the number of documents is \verb+K+.

Here's how we might do this in Erlang:

\begin{itemize}
\item We spawn a reducer process, Call this process \verb+Reducer+.
\item We spawn \verb+K+ parallel processes (workers) , and give each
  process the name of a file to work on, and the process identifier
  \verb+Reducer+.

\item Each worker analyses the data in the file it was given and sends
  a sequence of \verb+{Word,Count}+ pairs to the reducer process.
  For example, if the word ``tuple'' occurred ten times in a
  document it would send a \verb+{<<"tuple">>,10}+ tuple to the
  reduce process.

\item The reducer process merges the word count streams. So for
  example if it has received the tuples \verb+{<<"tuple">>,10}+ and
  \verb+{<<"tuple">>,15}+ it will know that the number of times the
  word has been seen is 25.

\end{itemize}

Several practical issues have to be dealt with when implementing
map-reduce:

\begin{itemize}

\item Termination. How do we know when the map phase has terminated.
  Should the reducer wait for all map processes to terminate before
  it terminates.

\item Bottlenecks. The reduce process could become a bottleneck. All
  the results eventually go to a single process. We might want to
  avoid this.
\end{itemize}

In practice we would probably limit the time for a map-reduce
operation to a fixed time, and build a tree of reducers. In the
word-count example we could build two reducers (one for words
beginning \verb+a..m+ and \verb+n..z+ or 26 reducer processes for
letters starting \verb+a+, \verb+b+ and so on.

\section{Components}

Parallel systems are often build from compents. Here are afew examples:

\subsection{Server}

A server has state. When it receives requests from a client, it
performs a computation which might update the state and then sends
a message back to the client.

(This is the server in the so call Client-Server Model)

\subsection{Publisher}

A publisher has a list of channels. \verb+[{Channel, [Pid1, Pid2, ...]}]+
The publisher accepts four input messages.

\begin{description}

\item \verb+{subscribe,Channel,Pid}+\\
  Adds \verb+Pid+ to the list
  of pids associated with \verb+ChannelName+. Creates a new channel if
  a channel does not exist.

\item \verb+{unsubscript_all,Pid}+\\
removes all subscriptions that Pid has.

\item \verb+{unsubscript_all,Channel, Pid}+\\
Remove the \verb+Pid+ to \verb+Channel+.

\item \verb+{publish,Channel,Msg}+\\
Sends Msg to all the subscribers associated with
\verb+Channel+. It is not an error to send a message to a channel with
zero subscribers.

\end{description}

Publisher has no errors.

A publisher is the PUB in the so called PUB-SUB model.

Note: PUB-SUB is an enormously important model: Twitter, Facebook,
Mailing Lists, IRC Newsletters are all variants on PUB-SUB

Actually PUB-SUB is a confusing term. It could mean the names of two
different processes. A Publisher Process that talks to a Subscriber
Process. Or, it could mean the name of two different message sent to a
publisher process.

\subsection{Dealer}

A dealer has \verb+N+ output ports.
Messages sent to the input ports are round-robbined to
the output ports.

\begin{description}

\item \verb+{add_output,Pid}+\\
add Pid to the list of output ports.

\item \verb+{send,Message}+\\
  Send \verb+Message+ to one of the output ports.

  {\sl errors: There are no output ports}
\end{description}

\subsection{Router}

A router has an internal routing table which is a list of pairs of the form.
\verb+[{Tag,OutPort}]+. Message sent to the input port must have
a tag, these messages are sent to the appropriate output port.

\begin{description}
\item \verb+{add_route,Tag,Pid}+\\
  Adds a routing table entry \verb+{Tag, Pid}+ to the
  routing table.

  {\sl errors: Tag already used}.

\item \verb+{unroute,Tag}+\\
  Removes a routing table entry with \verb+Tag+ from the routing table.

\item \verb+{send,Tag,Msg}+\\
  Sends \verb+Msg+ to the process assocated with \verb+Tag+.

  {\sl errors: No process associated with Tag}

\end{description}

\subsection{Middle Man}

A middle man translates between two different messaging formats.  It has
two ports, call these \verb+a+ and \verb+b+.  Messages to port
\verb+a+ are in format \verb+one+ and messages to port \verb+b+ are in
format \verb+two+.

Messages arriving at port \verb+a+ should be converted from format
\verb+one+ to format \verb+two+ and sent to port \verb+b+ and {\sl
  vice versa.}

Here's a middle man that converts bwteen Metric and US meaasurements:

\begin{verbatim}

                 +-----+
                 |     |
      {km,3}   A |     |B  {miles,1.864}
  ------->-------+     +------->---------
                 |     |
                 +-----+
\end{verbatim}

And in the opposite direction:

\begin{verbatim}

                 +-----+
                 |     |
    {km,3.218} A |     |B  {miles,2}
  -------<-------+     +-------<---------
                 |     |
                 +-----+
\end{verbatim}

Examples: Convert JSON messages to XML messages, or Erlang terms.
Convert SAX events to XML terms.

\section{Router Code}

\VerbatimInput{router.erl}

The router assumes all messages are
are of the form: \verb+{do,Cmd,Epid}+
If the command works it does what it is supposed to do. If it fails
it sends a message to \verb+Epid+

\section{Publisher Code}

Here's a simple PUB-SUB publisher process. There is no error
handling.

\VerbatimInput{pubsub0.erl}

\end{document}


\documentclass{beamer}

\usepackage{beamerthemevictor,comment,verbatim}

\input{tutmacs}
\input{slidemacs}
\input idxmacs

\begin{document}

\title{Python}
\author{Victor Eijkhout}
\date{Notes for CS 594 -- Fall 2004}

\frame{\titlepage}

\section{Introduction}

\subsectionframe{The very basics}

\frame[containsverbatim]{
\frametitle{Running}
\begin{itemize}
\item Interactively: \n{python}
\item Call python: \n{python myfile.py}
\item Make executable: \n{chmod +x myfile.py}
\begin{verbatim}
file:
#!/usr/bin/python
...
\end{verbatim}
also \verb+#!/usr/bin/env python+
\end{itemize}

Use emacs: python mode
}

\frame{
\frametitle{Code layout}

Every statement on a line by itself: no semicolon needed.

With semicolon: more than one statement on a line.

Continuation by escaping line end.

Comments start with~\n{\#}.
}

\sectionframe{Data}
\subsection{Simple stuff}

\frame{
\frametitle{Numbers}
Booooring

\n{2/3} is zero; \n{1.*2/3} float
}

\frame{
\frametitle{Strings}

Strings in single or double quotes\\
otherwise largely like lists.

Concatenate: \n{'a'+"b"}

multiply \n{5*'word'}
}

\subsectionframe{Lists}

\frame{
\frametitle{The basics}
List: \n{a=[1,2,'a','bcd']}

List run from~0: \n{a[1:3]} is \n{[2,'a']}

Slices: \n{a[2:]}, \n{a[:3]}, \n{a[:]}\\
(that last one is a copy, as opposed to~\n{a}: copy of pointer)

Assign to slices: \n{a[2]=[3,4]}, \n{a[3:4]=[]}

Concatenate: \n{a+b[:3]}

Length: \n{len(a)}

Multidimensional: list of lists \n{a[3]=[4,5]}
}

\frame{
\frametitle{Other list manipulation}

\n{a.insert(i,x)} insert $x$ before element~$i$

\n{a.remove(x)} remove $x$; has to be present

\n{a.count(x)}, \n{a.index(x)} how many times present, where?

\n{a.append(x)}, \n{a.extend(L)} add at end of list
}

\frame{
\frametitle{Tips and tricks}

\begin{itemize}
\item A simple assignment \n{list1 = list2} does not create a second list:
it only copies a pointer. To create~\n{list1} as copy of~\n{list2},
do~\n{list1 = list2[:]}.
\item Create empty list: \n{a= n*[0]}
\end{itemize}
}

\sectionframe{Statements}

\frame[containsverbatim]{
\frametitle{White space}
White space is significant: in conditionals and loops and such, the
clauses are indented.
\begin{verbatim}
if a>0:
    do_something
    and more
else:
    yet more
\end{verbatim}
}

\frame[containsverbatim]{
\frametitle{Python support in emacs}
Indentation is done automatically
\begin{description}
\item[return] next line with proper indentation
\item[delete] to the previous indentation level
\item[tab] the the next level, but only if that is possible:
\begin{verbatim}
if a>0:
    do_something
    and more
after
\end{verbatim}
the \n{after} can be tabbed to belong in the conditional.
\end{description}
}

\subsectionframe{Control structures}

\frame[containsverbatim]{
\frametitle{Conditionals}
\begin{verbatim}
if a>0:
    print "yes"
elif b<0:
    print "no"
else:
    print "hm"
\end{verbatim}
Also
\begin{verbatim}
if a>0: print a
\end{verbatim}
}

\frame[containsverbatim]{
\frametitle{Loops}
\begin{verbatim}
for w in words:
    print w
\end{verbatim}
range over array elements. Numerical index:
\begin{verbatim}
for i in range(4):
    f(i)
\end{verbatim}
ranges over index array \n{[0,1,2,3]}.
Also \n{range(1,4)} is \n{[1,2,3]}; \n{range(4,1,-1)} is~\n{[4,3,2]}.
}

\frame[containsverbatim]{
\frametitle{While loop}
\begin{verbatim}
while p>0:
    p = prev[p]
\end{verbatim}

\n{break} and \n{continue} statements
}

\frame[containsverbatim]{
\frametitle{Functions}
Define
\begin{verbatim}
def fact(n):
    if n==0: return 1
    else:
        return n*fact(n-1)
\end{verbatim}
All variables local unless
\begin{verbatim}
    global x
\end{verbatim}
included
}

\subsection{Input/output}

\frame[containsverbatim]{
\frametitle{I/O}
Use \n{raw_input}; the \n{input} command evaluates its
input (which can be dangerous, since you can sneak system commands
in there).

Output with \n{print}. This automatically inserts a newline; prevent
that with \n{print,}. For precise control over spacing and newlines:
\begin{verbatim}
import sys
sys.stdout.write(<something>)
\end{verbatim}
This only works with strings: \n{str(23)} et cetera for conversion
}

\frame[containsverbatim]{
\frametitle{EOF}
\begin{verbatim}
try:
    a = raw_input()
    print "line:",a
except (EOFError):
    break
\end{verbatim}
}

\end{document}


\hypertarget{structIir_1_1Butterworth_1_1BandStop}{}\doxysubsection{Iir\+::Butterworth\+::Band\+Stop$<$ Filter\+Order, State\+Type $>$ Struct Template Reference}
\label{structIir_1_1Butterworth_1_1BandStop}\index{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$@{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$}}

{\ttfamily \#include $<$Butterworth.\+h$>$}

Inheritance diagram for Iir\+::Butterworth\+::Band\+Stop$<$ Filter\+Order, State\+Type $>$\+:\begin{figure}[H]
\begin{center}
\leavevmode
\includegraphics[height=2.434783cm]{structIir_1_1Butterworth_1_1BandStop}
\end{center}
\end{figure}
\doxysubsubsection*{Public Member Functions}
\begin{DoxyCompactItemize}
\item
void \mbox{\hyperlink{structIir_1_1Butterworth_1_1BandStop_a88300cc9eef7b5eda35f0381f7238c16}{setup}} (double sample\+Rate, double center\+Frequency, double width\+Frequency)
\item
void \mbox{\hyperlink{structIir_1_1Butterworth_1_1BandStop_a7d72486d044fef3dce19c9756de995ba}{setupN}} (int req\+Order, double sample\+Rate, double center\+Frequency, double width\+Frequency)
\item
void \mbox{\hyperlink{structIir_1_1Butterworth_1_1BandStop_aff496377a74e44a7c5e88e5d9b5abe5c}{setupN}} (double center\+Frequency, double width\+Frequency)
\item
void \mbox{\hyperlink{structIir_1_1Butterworth_1_1BandStop_ab88fa7f17faa3de891c9db4766b3cba6}{setupN}} (int req\+Order, double center\+Frequency, double width\+Frequency)
\end{DoxyCompactItemize}

\doxysubsubsection{Detailed Description}
\subsubsection*{template$<$int Filter\+Order = 4, class State\+Type = Direct\+Form\+II$>$\newline
struct Iir\+::\+Butterworth\+::\+Band\+Stop$<$ Filter\+Order, State\+Type $>$}

\mbox{\hyperlink{namespaceIir_1_1Butterworth}{Butterworth}} Bandstop filter.
\begin{DoxyParams}{Parameters}
{\em Filter\+Order} & Reserves memory for a filter of the order Filter\+Order \\
\hline
{\em State\+Type} & The filter topology\+: \mbox{\hyperlink{classIir_1_1DirectFormI}{Direct\+FormI}}, \mbox{\hyperlink{classIir_1_1DirectFormII}{Direct\+Form\+II}}, ... \\
\hline
\end{DoxyParams}

\doxysubsubsection{Member Function Documentation}
\mbox{\Hypertarget{structIir_1_1Butterworth_1_1BandStop_a88300cc9eef7b5eda35f0381f7238c16}\label{structIir_1_1Butterworth_1_1BandStop_a88300cc9eef7b5eda35f0381f7238c16}}
\index{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$@{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$}!setup@{setup}}
\index{setup@{setup}!Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$@{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$}}
\doxyparagraph{\texorpdfstring{setup()}{setup()}}
{\footnotesize\ttfamily template$<$int Filter\+Order = 4, class State\+Type  = Direct\+Form\+II$>$ \\
void \mbox{\hyperlink{structIir_1_1Butterworth_1_1BandStop}{Iir\+::\+Butterworth\+::\+Band\+Stop}}$<$ Filter\+Order, State\+Type $>$\+::setup (\begin{DoxyParamCaption}\item[{double}]{sample\+Rate,  }\item[{double}]{center\+Frequency,  }\item[{double}]{width\+Frequency }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}

Calculates the coefficients with the filter order provided by the instantiation
\begin{DoxyParams}{Parameters}
{\em sample\+Rate} & Sampling rate \\
\hline
{\em center\+Frequency} & Centre frequency of the bandstop \\
\hline
{\em width\+Frequency} & Width of the bandstop \\
\hline
\end{DoxyParams}
\mbox{\Hypertarget{structIir_1_1Butterworth_1_1BandStop_aff496377a74e44a7c5e88e5d9b5abe5c}\label{structIir_1_1Butterworth_1_1BandStop_aff496377a74e44a7c5e88e5d9b5abe5c}}
\index{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$@{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$}!setupN@{setupN}}
\index{setupN@{setupN}!Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$@{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$}}
\doxyparagraph{\texorpdfstring{setupN()}{setupN()}\hspace{0.1cm}{\footnotesize\ttfamily [1/3]}}
{\footnotesize\ttfamily template$<$int Filter\+Order = 4, class State\+Type  = Direct\+Form\+II$>$ \\
void \mbox{\hyperlink{structIir_1_1Butterworth_1_1BandStop}{Iir\+::\+Butterworth\+::\+Band\+Stop}}$<$ Filter\+Order, State\+Type $>$\+::setupN (\begin{DoxyParamCaption}\item[{double}]{center\+Frequency,  }\item[{double}]{width\+Frequency }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}

Calculates the coefficients with the filter order provided by the instantiation
\begin{DoxyParams}{Parameters}
{\em center\+Frequency} & Normalised centre frequency (0..1/2) of the bandstop \\
\hline
{\em width\+Frequency} & Normalised width of the bandstop \\
\hline
\end{DoxyParams}
\mbox{\Hypertarget{structIir_1_1Butterworth_1_1BandStop_ab88fa7f17faa3de891c9db4766b3cba6}\label{structIir_1_1Butterworth_1_1BandStop_ab88fa7f17faa3de891c9db4766b3cba6}}
\index{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$@{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$}!setupN@{setupN}}
\index{setupN@{setupN}!Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$@{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$}}
\doxyparagraph{\texorpdfstring{setupN()}{setupN()}\hspace{0.1cm}{\footnotesize\ttfamily [2/3]}}
{\footnotesize\ttfamily template$<$int Filter\+Order = 4, class State\+Type  = Direct\+Form\+II$>$ \\
void \mbox{\hyperlink{structIir_1_1Butterworth_1_1BandStop}{Iir\+::\+Butterworth\+::\+Band\+Stop}}$<$ Filter\+Order, State\+Type $>$\+::setupN (\begin{DoxyParamCaption}\item[{int}]{req\+Order,  }\item[{double}]{center\+Frequency,  }\item[{double}]{width\+Frequency }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}

Calculates the coefficients
\begin{DoxyParams}{Parameters}
{\em req\+Order} & The actual order which can be less than the instantiated one \\
\hline
{\em center\+Frequency} & Normalised centre frequency (0..1/2) of the bandstop \\
\hline
{\em width\+Frequency} & Normalised width of the bandstop \\
\hline
\end{DoxyParams}
\mbox{\Hypertarget{structIir_1_1Butterworth_1_1BandStop_a7d72486d044fef3dce19c9756de995ba}\label{structIir_1_1Butterworth_1_1BandStop_a7d72486d044fef3dce19c9756de995ba}}
\index{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$@{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$}!setupN@{setupN}}
\index{setupN@{setupN}!Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$@{Iir::Butterworth::BandStop$<$ FilterOrder, StateType $>$}}
\doxyparagraph{\texorpdfstring{setupN()}{setupN()}\hspace{0.1cm}{\footnotesize\ttfamily [3/3]}}
{\footnotesize\ttfamily template$<$int Filter\+Order = 4, class State\+Type  = Direct\+Form\+II$>$ \\
void \mbox{\hyperlink{structIir_1_1Butterworth_1_1BandStop}{Iir\+::\+Butterworth\+::\+Band\+Stop}}$<$ Filter\+Order, State\+Type $>$\+::setupN (\begin{DoxyParamCaption}\item[{int}]{req\+Order,  }\item[{double}]{sample\+Rate,  }\item[{double}]{center\+Frequency,  }\item[{double}]{width\+Frequency }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}

Calculates the coefficients
\begin{DoxyParams}{Parameters}
{\em req\+Order} & The actual order which can be less than the instantiated one \\
\hline
{\em sample\+Rate} & Sampling rate \\
\hline
{\em center\+Frequency} & Centre frequency of the bandstop \\
\hline
{\em width\+Frequency} & Width of the bandstop \\
\hline
\end{DoxyParams}

The documentation for this struct was generated from the following file\+:\begin{DoxyCompactItemize}
\item
iir/Butterworth.\+h\end{DoxyCompactItemize}