\documentclass[a4paper]{article}
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
% package for including graphics with figure-environment
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{tikz}
\usepackage{amsmath}
\usepackage{multirow}
\usepackage{qtree}
% colors for hyperlinks
% colored borders (false) colored text (true)
\hypersetup{colorlinks=true,citecolor=black,filecolor=black,linkcolor=black,urlcolor=black}

% package for bibliography
\usepackage[authoryear,round]{natbib}
% package for header
\usepackage[automark,headsepline]{scrlayer-scrpage}
\pagestyle{scrheadings}
\ihead[]{BB84 Protocol}
\cfoot[]{\pagemark}

\begin{document}
	\title{

	\vspace{1cm}
	\Huge BB84 Protocol
	}

	\vspace{1cm}

	% if you are the only author, you might use the following
	% \author{Name of student}

	% Insert here your name and correct mail address
	\author{\Large {Sriram Devata} \Large \\ 2019113007
	\vspace{0.5cm}}

	% name of the course and module
	\date{
	\large Algorithm Analysis And Design \\
	\vspace{1cm}
	}

	\maketitle
	\setlength{\parindent}{0pt}

\vspace{8cm}
\begin{abstract}
\begin{center}
This document gives an overview of the BB84 protocol for the implementation of Quantum Key Distribution(QKD) and the relevant principles needed to understand it.
\end{center}
\end{abstract}
	\newpage
	\tableofcontents
	\newpage

\section{Introduction} % (fold)
\label{sec:introduction}
All cryptography algorithms depend on secure sharing of keys. Though symmetrical cryptography algorithms are faster over asymmetrical cryptography algorithms, the risk of the shared key being intercepted makes them an inferior choice to an asymmetrical algorithm.
\\ \\
When two parties (for simplicity, assume these parties to be Alice and Bob) are trying to establish communication by sharing keys to encrypt future information exchange, there's a risk of an eavesdropper (for simplicity, assume the eavesdropper is called Eve) intercepting the communication and figuring out the key that Alice and Bob decide to use. Besides having no point of encoding and decoding the information, Alice and Bob are not aware if their key has been compromised and would have the false belief that they have established a secure line of communication.
\\ \\
Using the BB84 protocol, Alice and Bob will know if Eve is eavesdropping and will not use the compromised key for further communication.
% section introduction (end)

\section{Background} % (fold)
\label{sec:background}

\subsection{Qubits} % (fold)
\label{sub:qubits}

In any computational devices, a building block is a two-state system (0, 1). In quantum mechanics, any system can exist in a \textit{superposition} of possible states. A qubit is a 2-state quantum system. It can exist in either of the possible states, or in a superposition of the two states. In general, the state of a qubit can be described as:
\begin{gather*}
    |\psi\rangle = \alpha|0\rangle + \beta|1\rangle\text{, with }\alpha, \beta \in \mathbb{C} \text{ and }|\alpha|^2 + |\beta|^2 = 1
\end{gather*}

\subsection{Measurement of a Qubit} % (fold)
\label{sub:measurement of a qubit}

The superposition $|\psi\rangle = \alpha|0\rangle + \beta|1\rangle$ is the \textit{private} information of a qubit. In order for us to know the state of the qubit, we have to make a measurement. When we make a measurement, we end up with one bit of information - if the qubit collapsed to the state $|0\rangle$ or $|1\rangle$.  It collapses to the state $|0\rangle$ with probability $|\alpha|^2$ and collapses to the state $|1\rangle$ with probability $|\beta|^2$.
\\ \\
Once the qubit collapses to a state after we make a measurement, the superposition of the qubit is lost and it now exists in the collapsed state. This means that it is not possible to find the values $|\alpha|^2$ and $|\beta|^2$ by making multiple measurements on a qubit to get a probability distribution.
\\ \\
A qubit can be visualized as a vector on a plane. The state of any qubit can be expressed as a linear combination of two orthogonal states. Two standard basis sets are the Z-basis states \{$|0\rangle$, $|1\rangle$\} and the X-basis states\{$|-\rangle$, $|+\rangle$\}.
\\ \\
\begin{figure}[h]
\centering
\begin{tikzpicture}

\draw[thick,<->] (-3,0) -- node[right=85pt] {$|0\rangle$} (3,0);
\draw[thick,<->] (0,-3) -- node[above=85pt] {$|1\rangle$} (0,3);
\draw[thick,blue,<->] (-2.12,-2.12) -- node[right=80pt, above=62pt] {$|+\rangle = \frac{|1\rangle+|0\rangle}{\sqrt{2}}$} (2.12,2.12);
\draw[thick,blue,<->] (-2.12,2.12) -- node[left=80pt, above=62pt] {$|-\rangle = \frac{|1\rangle-|0\rangle}{\sqrt{2}}$} (2.12,-2.12);

\end{tikzpicture}
\caption{The Standard(Z) basis and the \textcolor{blue}{Signed(X) basis}}
\end{figure}

In general, we can choose any two orthogonal states and express the state of a qubit as a linear combination of these two states.

\subsection{Measurements In Different Bases} % (fold)
\label{sub:measurement in different bases}

A central component of the BB84 protocol is the collapse of the qubit when it is measured in a basis different from the basis it was encoded in. For example, if the bit 0 was encoded onto a qubit in the Z basis and is measured in the X basis, the result of the measurement would be 0 or 1 with equal probability.

\begin{figure}[h]
\centering
\begin{tikzpicture}

\draw[thick,<->] (0,0) -- node[right=45pt] {$|0\rangle$ (0 in Z-basis)} (3,0);
\draw[thick,blue,<->] (-2.12,-2.12) -- node[right=90pt, above=62pt] {$|+\rangle$ (0 in X-basis)} (2.12,2.12);
\draw[thick,blue,<->] (-2.12,2.12) -- node[left=90pt, above=62pt] {$|-\rangle$ (1 in X-basis)} (2.12,-2.12);
\draw[thick, densely dotted,red] (2.5,0) -- node[] {50\%} (1.3,1.3);
\draw[thick, densely dotted,red] (2.5,0) -- node[] {50\%}(1.3,-1.3);

\end{tikzpicture}
\caption{Measuring Z-basis 0 in the X-basis}
\end{figure}

\subsection{Quantum Communication Channel} % (fold)
\label{sub:quantum communication channel}
Classical communication channels can transmit classical information. Bits can be encoded as high or low voltages. This cannot be used sending and receiving qubits. To transmit quantum information, BB84 utilizes a separate Quantum communication channel to send and receive the states of qubits. The practical implementation of such a communication channel can be a fibre-optic cable that conserves the polarisation of photons. The state of a qubit can be encoded in the polarization of a photon. We can map the possible polarization states of a photon to the vector representation of a qubit.

\begin{figure}[h]
\centering
\begin{tikzpicture}

\draw[thick,->] (0,0) -- node[right=50pt] {\shortstack{$\text{0 in Z-basis}\\0^\circ$}} (3,0);
\draw[thick,->] (0,0) -- node[above=50pt] {\shortstack{$\text{1 in Z-basis}\\90^\circ$}} (0,3);
\draw[thick,blue,->] (0,0) -- node[right=60pt, above=30pt] {\shortstack{$\text{0 in X-basis}\\45^\circ$}} (2.12,2.12);
\draw[thick,blue,->] (0,0) -- node[left=55pt, above=30pt] {\shortstack{$\text{1 in X-basis}\\135^\circ$}} (-2.12,2.12);

\end{tikzpicture}
\caption{Mapping the Polarization of a Photon to Qubit states}
\end{figure}

In this particular quantum communication channel, the bits encoded in different bases and the resulting qubits can be shown as $\nwarrow, \uparrow, \nearrow \text{and} \rightarrow$.

\begin{table}[h]
    \centering
    \begin{tabular}{| c | c | c | c | c | c | c | c | c | c | c | c | c | c | c | c | c | c |}
    \hline
     Bit to encode & \multicolumn{8}{c|}{0} & \multicolumn{8}{c|}{1} \\
     \hline
     Encoding Basis & \multicolumn{4}{c|}{Z} & \multicolumn{4}{c|}{X} & \multicolumn{4}{c|}{Z} & \multicolumn{4}{c|}{X} \\
     \hline
     Photon Polarization & \multicolumn{4}{c|}{$\rightarrow$} & \multicolumn{4}{c|}{$\nearrow$} & \multicolumn{4}{c|}{$\uparrow$} & \multicolumn{4}{c|}{$\nwarrow$} \\
     \hline
     Measurement Basis & \multicolumn{2}{c|}{Z} & \multicolumn{2}{c|}{X} & \multicolumn{2}{c|}{Z} & \multicolumn{2}{c|}{X} & \multicolumn{2}{c|}{Z} & \multicolumn{2}{c|}{X} & \multicolumn{2}{c|}{Z} & \multicolumn{2}{c|}{X} \\
     \hline
     Measured Bit & \multicolumn{2}{c|}{0} & 0 & 1 & 0 & 1 & \multicolumn{2}{c|}{0} & \multicolumn{2}{c|}{1} & 0 & 1 & 0 & 1 & \multicolumn{2}{c|}{1} \\
     \hline

     \hline
    \end{tabular}
    \caption{Qubits when encoded and measured in different bases}
    \label{tab:qubit measurements}
\end{table}

\section{Overview of the Procotol} % (fold)

The protocol initially assumes that there are two channels(quantum and classical) of communication available to Alice and Bob, and Eve is able to intercept information in both the channels. Alice chooses a random string of bits, and decides to encode each of these bits as a qubit, in either the Z-basis or X-basis at random. She keeps track of the initial bit string and the set of bases she chose to encode each bit. She sends the resulting qubits over to Bob through the quantum channel. \\

As Bob receives the qubits, he measures each of the qubit in either the Z-basis or X-basis at random. He keeps track of the bits he measures and the bases he chose to measure each qubit. Bob now sends the set of bases he chose to measure each qubit and sends it to Alice. Alice responds to this by telling Bob the qubits for which Bob chose the same bases to measure in. Alice and Bob discard the bits where they both chose different bases to encode and measure. \\

Alice and Bob take a small subset of the bits and compare them. Assuming that there is proper noise correction and there is no measurement errors, if Eve didn't eavesdrop, Alice and Bob would end up with the same bits. They discard this subset of bits, and use the rest of the bits to form a secret key for further communication. \\

If the small subset of bits do not agree with each other, it would indicate that an eavesdropper, Eve, has intercepted the message, and the communication isn't secure. This is because if Eve wants to figure out the key, she would have to make measurements on the qubits sent by Alice in either of the bases chosen at random. According to the no cloning theorem, she can't make a replica of Alice's qubit to forward to Bob and make a measurement on her copy of the qubit. She either has to send the qubit she has measured, or guess the basis and send a new encoded qubit to Bob.

\section{Simulation Code} % (fold)
An implementation of the BB84 protocol in Qiskit can be found as an \href{https://github.com/ishaanshah/Cuantum-Qomputing/tree/master/Code/src/BB84}{interactive example here}.

\section{Analysis}

For each bit that Alice encodes, the following flowchart illustrates the bit that Bob will measure if Eve doesn't intercept the qubit, and the bit that Eve measures if Eve intercepts the qubit.

\vspace{1\baselineskip}

\Tree [.{Alice encodes $bit \in \{0,1\}$ in basis $b \in \{X, Z\}$}
        [.{Eve intercepts qubit} [.{Eve measures \\ qubit using basis $b$} {Eve reads $bit$} ]
        [.{Eve measures \\ qubit using basis $b'$} {Eve reads \\ $bit$} {Eve reads \\ $bit'$} ] ]
        [.{Bob receives qubit} [.{Bob measures \\ qubit using basis $b$} {Bob reads \\ $bit$} ] [.{Bob measures \\ qubit using basis $b'$} {Bob reads \\ $bit$} {Bob reads \\ $bit'$}  ] ] ]

\vspace{1\baselineskip}

Since both Eve and Bob choose the basis to measure in randomly, there is a $50\%$ chance that they choose the right basis. If they choose the wrong basis, there is a $50\%$ chance that they measure the correct bit. In the BB84 protocol implementation attached with this document, Eve forwards the qubit that she already made the measurement on to Bob. This means that in the case when Eve chose the wrong basis and the qubit collapsed to $bit'$, Bob measures $bit$ and $bit'$ with different probabilities.

\Tree [.{Alice encodes $bit \in \{0,1\}$ in basis $b \in \{X, Z\}$}
        [.{Eve intercepts qubit} [.{$50\%$\\Eve measures \\ qubit using basis $b$}
        [.{Eve reads $bit$} {$75\%$\\Bob reads\\$bit$} {$25\%$\\Bob reads\\$bit'$} ] ]
        [.{$50\%$\\Eve measures \\ qubit using basis $b'$} {$50\%$\\Eve reads\\$bit$ \\ \vdots}
        {$50\%$ \\ Eve reads \\ $bit'$ \\ \vdots} ] ]
        [.{Bob receives qubit} [.{$50\%$\\Bob measures \\ qubit using basis $b$} {Bob reads \\ $bit$} ] [.{$50\%$\\Bob measures \\ qubit using basis $b'$} {$50\%$ \\ Bob reads \\ $bit$} {$50\%$ \\ Bob reads \\ $bit'$}  ] ] ]

\vspace{1\baselineskip}

In the case where Eve intercepts the qubit and measures using the basis $b'$,

\vspace{1\baselineskip}

\Tree [.{\vdots\\$50\%$\\Eve measures\\qubit using basis $b'$}
        [.{$50\%$\\Eve reads\\$bit$}
        {$75\%$ \\ Bob reads \\ $bit$} {$25\%$ \\ Bob reads \\ $bit'$} ]
        [.{$50\%$ \\ Eve reads \\ $bit'$}
        {$25\%$ \\ Bob reads \\ $bit$} {$75\%$ \\ Bob reads \\ $bit'$} ] ]

\vspace{1\baselineskip}

Such an analysis will help us to choose optimal initial bit string length and the subset of the bit string to choose, to bring the number false negatives of an eavesdropper within the error threshold.

\end{document}


\documentclass[10pt, conference, compsocconf]{IEEEtran}
\IEEEoverridecommandlockouts

\usepackage{cite}
\usepackage{pgf}
\usepackage{amsmath,amssymb}
\usepackage{amsthm}
\usepackage{url}
\usepackage{tikz}
\usepackage{common}

% correct bad hyphenation here
\hyphenation{op-tical net-works semi-conduc-tor}

\begin{document}

\title{Straight-line Programs: A Practical Test\thanks{The authors acknowledge support
from the Ministry for Education and Science of Russia, grant 2.1.1/13995, and
from the Russian Foundation for Basic Research, grant 10-01-00524.}}

\author{\IEEEauthorblockN{Ivan Burmistrov}
\IEEEauthorblockA{Faculty of Mathematics and Mechanics \\
Ural State University\\
Ekaterinburg, Russia\\
burmistrov.ivan@gmail.com}
\and
\IEEEauthorblockN{Lesha Khvorost}
\IEEEauthorblockA{Faculty of Mathematics and Mechanics \\
Ural State University\\
Ekaterinburg, Russia\\
jaamal@mail.ru}
}

\maketitle

\begin{abstract}
We present an improvement of Rytter's algorithm that constructs a straight-line program for a given text and show that
the improved algorithm is optimal in the worst case with respect to the number of AVL-tree rotations. Also we compare
Rytter's and ours algorithms on various data sets and provide a comparative analysis of compression ratio achieved by
these algorithms, by LZ77 and by LZW.
\end{abstract}

\begin{IEEEkeywords}
straight-line program; LZ-compression; AVL-tree;
\end{IEEEkeywords}

\IEEEpeerreviewmaketitle

\section{Introduction}

Nowadays searching algorithms on huge data sets attract much attention. To reduce the input size one needs algorithms
that can work directly with a compressed representation of input data.

Various compressed representations of strings are known: straight-line programs (SLPs)~\cite{SLPConstruction},
collage-systems \cite{collages}, string representations using antidictionaries \cite{antidictionaries}, etc. Nowadays
text compression based on context-free grammars such as SLPs has become a popular research direction. The reason for
this is not only that grammars provide well-structured compression but also that the SLP-based compression is,
in a sense, polynomially equivalent to the compression achieved by the Lempel-Ziv algorithm that is widely used in
practice. It means that, given a text $S$, there is a polynomial relation between the size of an SLP that
derives $S$ and the size of the dictionary stored by the Lempel-Ziv algorithm, see \cite{SLPConstruction}. It should
also be noted that classical LZ78 \cite{LZ78} and LZW \cite{LZW} algorithms can be considered as special cases of grammar
compression. (At the same time other compression algorithms from the Lempel-Ziv family---such as LZ77 \cite{LZ77} and
run-length compression---do not fit directly into grammar compression model.)

Using the fact that SLPs are nicely structured, several researchers keep developing analogues of classical string
algorithms that (at least theoretically) perform quite well on SLP-compressed representations: \textbf{Pattern matching}
\cite{PM_and_HD}, \textbf{Longest common substring} \cite{LCSubstring}, \textbf{Computing all palindromes} \cite{LCSubstring}, some
versions of \textbf{Longest common subsequence} \cite{LCS_P}. At the same time, constants hidden in big-O notation for
algorithms on SLPs are often very big. Also the aforementioned polynomial relation between the size of an SLP for a
given text and the size of the LZ77-dictionary for the same text does not yet guarantee that SLPs provide good
compression ratio in practice. Thus, a major questions is whether or not there exist SLP-based compression models suitable
to practical usage? This question splits into two sub-questions addressed in the present paper: How difficult is it to
compress data to an SLP-representation? How large compression ratio do SLPs provide as compared to
classic algorithms used in practice?

Let us describe in more detail the content of the paper and its structure. Section 2 gathers some preliminaries about
SLPs. In Section 3 we present an improved version of Rytter's algorithm~\cite{SLPConstruction} for constructing
an SLP-presentation of a given text. In Section 4 we compare the improved algorithm vs. the original algorithm
from~\cite{SLPConstruction} and also present results of a comparison of compression ratio between the two SLP-based
algorithms and LZ77. In Section 5 we summarize our results.

\section{Preliminaries}

We consider strings of characters from a fixed finite alphabet $\Sigma$. The \emph{length} of a string
$S$ is the number of its characters and is denoted by $|S|$. The \emph{concatenation} of strings $S_1$ and $S_2$
is denoted by $S_1 \cdot S_2$. A \emph{position} in a string $S$ is a point between consecutive characters. We number
positions from left to right by $1,2,\dots,|S|-1$. It is convenient to consider also the position 0 preceding the text
and the position $|S|$ following it. For a string $S$ and an integer $0 \leq i \leq |S|$ we define $S[i]$ as character
between positions $i$ and $i+1$ of $S$. For example $S[0]$ is a first character of $S$. A \emph{substring} of $S$
starting at a position $\ell$ and ending at a position $r$, $0\leq \ell < r \leq |S|$, is denoted by $S[\ell \dots r]$ (in other
words $S[\ell \dots r] = S[\ell] \cdot S[\ell + 1] \cdot \mbox{\dots} \cdot S[r-1]$).

A \emph{straight-line program} (SLP) $\slp{S}$ is a sequence of assignments of
the form: $$\slp{S}_1 = expr_1,\ \slp{S}_2 = expr_2, \dots, \slp{S}_n =
expr_n,$$ where $\slp{S}_i$ are \emph{rules} and $expr_i$ are expressions of
the form:
\begin{itemize}
\item $expr_i$ is a character of $\Sigma$ (we call such rules \emph{terminal}), or
\item $expr_i = \slp{S}_\ell\cdot \slp{S}_r \ (\ell, r < i)$ (we call such rules \emph{nonterminal}).
\end{itemize}

Thus, an SLP is a context-free grammar in Chomsky normal form. Obviously every SLP generates exactly one string over
$\Sigma^+$. This string is referred to as the \emph{text} generated by an SLP. For a grammar $\slp{S}$ generating a text
$S$, we define \emph{parse-tree} of $S$ as a derivation tree of $S$ in $\slp{S}$. We identify terminal symbols with
their parents in this tree; after this identification every internal node has exactly two sons. We accept the
following conventions in the paper: every SLP is denoted by a capital blackboard bold letter, for example, $\slp{S}$.
Every rule of this SLP (and every internal node in its parse-tree) is denoted by the same letter with indices, for
example, $\slp{S}_1,\slp{S}_2,\dots$. The \emph{size} of an SLP $\slp{S}$ is the number of its rules and is denoted by
$|\slp{S}|$. The \emph{concatenation} of SLPs $\slp{S}_1$ and $\slp{S}_2$ is an SLP that derives
$S_1 \cdot S_2$ and denoted by $\slp{S}_1 \cdot \slp{S}_2$.

The \emph{height} of a node in a binary tree is defined as follows. The height
of terminal node (leaf) is equal to 0 by definition. The height of a
nonterminal node is equal to 1 + the maximum of the heights of its children. An
\emph{AVL-tree} $\slp{T}$ is a binary tree such as for every non-terminal node
the heights of its children can differ at most by 1. In the paper we consider
only SLPs whose parse-trees are AVL-trees. We denote the height of the rule
$\slp{S}_i$ by $h(\slp{S}_i)$.

\section{Algorithms for constructing SLPs}

\subsection{Rytter's algorithm and its bottleneck}

The problem of constructing a minimal size grammar generating a given text is
known to be NP-hard. Hence we should look for polynomial-time approximation
algorithms. One of the key approaches of such algorithms is to construct a
factorization of a given text and to build some binary search tree using the
factorization. It is obvious that the size of the resulting grammar depends on
factorization. In \cite{SLPConstruction} Rytter considered factorizations
defined by the LZ77 encoding algorithm.

\begin{definition}
The LZ-factorization of a text $S$ is given by a decomposition: $S = F_1 \cdot F_2 \cdot \mbox{\dots} \cdot F_k$, where
$F_1 = S[0]$ and $F_i$ is the longest prefix of $S\substr{|F_1\cdot \mbox{\dots} \cdot F_{i-1}|}{|S|}$ which occurs as
substring in $F_1 \cdot \mbox{\dots} \cdot F_{i-1}$ or $S[|F_1\cdot \mbox{\dots} \cdot F_{i-1}|]$ in case this prefix is
empty.
\end{definition}

The following theorem gives an estimation of the size of an SLP constructed on
the basis on a LZ-factorization.

\begin{thm}[\rm{\cite{SLPConstruction}}]
Given a string $S$ of length $n$ and its LZ-factorization of length $k$, one
can construct an SLP for $S$ of size $O(k \log n)$ in time $O(k \log n)$.
\end{thm}

The proof of the above theorem contains an algorithm for constructing SLPs. We
remind here some key ideas of the algorithm as they are important for the
further discussion. For efficient concatenation of grammars the algorithm uses
the following lemma:

\begin{lem}[{\rm\cite{SLPConstruction}}]
Assume $\slp{S}_1, \slp{S}_2$ are two nonterminals of AVL-balanced grammars.
Then we can construct in $O(|h(\slp{S}_1) - h(\slp{S}_2)|)$ time an
AVL-balanced grammar $\slp{S} = \slp{S}_1 \cdot \slp{S}_2$ that derives the
text $S_1\cdot S_2$ by adding only $O(|h(\slp{S}_1) - h(\slp{S}_2)|)$
nonterminals.
\end{lem}

\noindent \textsc{Problem:} \textbf{SLP Construction}

\noindent \textsc{Input:} a string $S$ and its LZ-factorization $F_1, F_2,
\dots, F_k$.

\noindent \textsc{Output:} an SLP $\slp{S}$ that derives $S$.

\noindent \textsc{Algorithm:} The algorithm constructs an SLP by induction on
$k$.

\textbf{Base:} Initially $\slp{S}$ is equal to the terminal rule that derives
$S[0]$.

\textbf{Step:} Let $i > 1$ be an integer and an SLP $\slp{S}$ that derives
$F_1\cdot F_2 \cdots F_i$ has already been constructed. Since a
LZ-factorization of $S$ is fixed, an occurrence of $F_{i+1}$ in $F_1\cdot F_2
\cdots F_i$ is known. From this, the algorithm extracts rules $\slp{S}_1,
\dots, \slp{S}_\ell$ such that $F_{i+1} = S_1 \cdot S_2 \cdots S_\ell$. Since
$\slp{S}$ is balanced, we have $\ell = O(\log |S|)$. Then, using Lemma III.2,
the algorithm concatenates rules in some specific order (see
\cite{SLPConstruction} for details) and sets the next value of $\slp{S}$ to be
equal to the result of concatenating the previous value of $\slp{S}$ with
$\slp{S}_1 \cdots \slp{S}_\ell$.

\begin{figure}[th]
\AVLrotations
\end{figure}

It is well-known that adding a new rule to an AVL-tree is quite a complex
operation. Every adding operation generates a sequence of rotations of the
tree. There are two types of rotations symbolically presented in Figure 1, see
\cite{SLPConstruction} for more details. Every rotation may generate at most
three new rules. Also every rotation may generate three unused rules. There are
two possible directions for an optimization of the algorithm: to construct more
compact grammar and to minimize the number of queries to AVL-trees. Minimizing
of the number of queries to AVL-trees becomes important when the size of input
text becomes huge and we cannot store an AVL-tree in the memory. Formally it
means that costs of a query to an AVL-tree are greater than costs of
calculations in memory.

The following example illustrates a bottleneck of the algorithm
from~\cite{SLPConstruction}:

\noindent \textbf{Example 1:} Let $n$ be an integer and $S$ be a text of length
$2^n$. Suppose the algorithm has already built an SLP $\slp{S}$ that derives
$S\substr{0}{2^{n-1}}$. Let $F_1 \cdot F_2 \cdots F_{n-1}$ be LZ-factorization
of $S\substr{2^{n-1}}{2^n}$ such that $|F_i| = 2^{n-i-1}$, where $i \in
\overline{1\dots n-1}$.

Let us estimate the number of rotations that may be generated in the course of
the sequence of concatenations $(((\slp{S} \cdot \slp{F}_1)\cdot
\slp{F}_2)\dots) \cdot \slp{F}_{n-1}$ in the worst case. In order to get an
upper bound, we suppose that the resulting tree grows after every
concatenation. So we get the following estimation: \begin{multline*}
\sum_{i=1}^{n-1} (\log{2^{n-1}} + (i-1) - \log{2^{n-i-1}}) = \\
\frac{(n-1)(n-2)}{2}(1 + \log{2}) - (n-2) = \Theta(n^2).
\end{multline*}

Notice that if the algorithm could choose another order of concatenations, then
the number of rotations generated by the algorithm could be essentially
smaller, namely,close to $O(n)$. For instance, in the case of concatenating in
the reverse order $\slp{S} \cdot (\slp{F}_1 \dots (\slp{F}_{n-2} \cdot
\slp{F}_{n-1}))$, we get the following estimation:
$$\sum_{i=1}^{n-1} (\log{2^{n-i}} - \log{2^{n-i-1}}) = (n-1)\log{2} =
\Theta(n).$$

Our next example shows that several factors can be processed together if they
occur in a single SLP.

\noindent \textbf{Example 2:} Let $n > 0$ be an integer and $S = b\cdot
a^{2^{n-1}}\cdot b \cdot a^{2^{n-2}} \cdots b \cdot a$. So the length of $S$ is
equal to $2^n + n - 2$. Consider the LZ-factorization of $S$: $$ b \cdot a
\cdot a \cdot a^2 \cdot a^4 \cdots a^{2^{n-2} - 1} \cdot ba^{2^{n-2}} \cdot
ba^{2^{n-3}} \cdots ba.$$ Let $\slp{S}_1$ be an SLP that derives $b \cdot
a^{2^{n-1}}$. It is obvious that all other factors starting with $b \cdot
a^{2^{n-2}}$ occur in $S_1$, therefore the algorithm may process them together.
So the algorithm may construct SLPs $\slp{S}_2$ that derives $b \cdot
a^{2^{n-2}}$, $\slp{S}_3$ that derives $b \cdot a^{2^{n-3}}$, etc. Finally the
algorithm concatenates the SLPs in the following order:
$\slp{S}_1\cdot(\dots(\slp{S}_{n-3}\cdot(\slp{S}_{n-2} \cdot \slp{S}_{n-1})))$.

The main ideas of our improved  algorithm are to process several factors
together and to concatenate each group of factors choosing an optimal order.
The intuition behind the algorithm is very simple: if the algorithm has already
constructed a huge SLP, then most factors occur in the text generated by this
SLP and may be processed together.

\subsection{An improved algorithm}

\noindent \textsc{Input:} a text $S$ and its LZ-factorization $F_1, F_2, \dots,
F_k$.

\noindent \textsc{Output:} an SLP $\slp{S}$ that derives $S$.

\noindent \textsc{Algorithm:} The algorithm constructs an SLP by induction on
$k$.

\textbf{Base:} Initially $\slp{S}$ is equal to the terminal rule that derives
$S[0]$.

\textbf{Step:} Let $i$ be an integer and $\slp{S}$ be an SLP that derives
$F_1\cdot F_2 \cdots F_i$. Let $\ell \geq 1$ be a maximal integer such that
$F_{i+\ell}$ occurs in $F_1 \cdots F_i$. Since the LZ-factorization of $S$ is
fixed, the value of $\ell$ can be found by using a linear time search on
factors. Let $\slp{F}_1$ be an SLP that derives $F_{i+1}$, let $\slp{F}_2$ be
an SLP that derives $F_{i+2}$, etc.

Initially the algorithm concatenates $\slp{F}_{i+1}, \dots, \slp{F}_{i+k}$
using dynamic programming. Let $\varphi(p, q)$ be the function that calculates
an upper bound of the number of rotations of a grammar tree that are performed
during the process of concatenation of $\slp{F}_p, \slp{F}_{p+1}, \dots,
\slp{F}_q$. The algorithm uses the following recurrent formula to calculate
$\varphi(p,q)$:
$$\varphi(p, q) = \begin{cases}
0 &\text{if } p=q, \\
\min_{r = p}^q(\varphi(p, r) + \varphi(r+1, q) +&\\
|\log(|f_{i+p}|+\dots+|f_{i+r}|) -& \\
\log(|f_{i+r+1}|+\dots+|f_{i+q}|)|) &\mbox{otherwise}.
\end{cases}$$
The formula is correct since the concatenation of $\slp{F}_p$ and $\slp{F}_q$
generates at most. $2\cdot|h(\slp{F}_p) - h(\slp{F}_q)|$ rotations  In the
formula we omit the constant factor 2 since, otherwise, all values of $\varphi$
will be proportional to the factor.

The algorithm calculates the values of $\varphi(p, q)$ using a $\varphi$-table.
The $\varphi$-table is a $((k-1) \times (k-1))$-table that stores information
about the value of $\varphi(p,q)$ and about the optimal value of index $r$. The
cell in the $p$-th row and the $q$-th column where $p \leq q$ always contains
zero values. Initially the algorithm fills out cells such that $q - p = 1$,
next it fills out cells such that $q - p = 2$, etc. Thus, the algorithm does
not recompute recursively the values of $\varphi(p, r)$ and $\varphi(r+1, q)$
since they already exist in the $\varphi$-table.

Every single value of $\varphi(p, q)$ can be calculated using $O(k)$ time (the
pseudo-code of the corresponding procedure is presented in Figure 2). Thus, the
algorithm fills out the $\varphi$-table using $O(k^3)$ time and $O(k^2)$ space.

\begin{figure}[th]
\ComputingFiValue
\end{figure}

Notice that $\varphi(1, k)$ is equal to the optimal number of rotations of the
grammar tree in the worst case (i.e.\ in the case when each concatenation of
$\slp{F}_p$ and $\slp{F}_q$ generates $O(|h(\slp{F}_p) - h(\slp{F}_q)|)$
rotations). Since the $\varphi$-table contains information about the optimal
value of index $r$ in every cell, the algorithm can determine an optimal order
of concatenations for $\slp{F}_{i+1}, \dots, \slp{F}_{i+k}$. Using the optimal
order, the algorithm constructs an SLP $\slp{F}$ that derives $F_{i+1},
F_{i+2}, \dots, F_{i+l}$. Finally the algorithm concatenates $\slp{S}$ and
$\slp{F}$ and sets $\slp{S}$ to be equal to $\slp{S} \cdot \slp{F}$.

\begin{thm}
Let $f_1, f_2, \cdots, f_k$ be the LZ-factorization of a text $w$. The above
algorithm constructs an SLP for $w$ of size $O(k\log n)$.
\end{thm}
\proof{} The LZ-factorization can be divided into independent groups of
factors. Let $\ell$ be the number of the groups and let $k_1, k_2, \dots,
k_\ell$ be the sizes of the corresponding groups. The number of new rules
generated during concatenation of all factors in the $i$-th group is upper
bounded by $2\cdot(\log (|f_{k_i}| + |f_{k_i + 1}| + \dots +
|f_{k_{i+1}-1}|)\cdot(k_{i+1} - k_i))$. So the number of new rules generated
during a single step of the algorithm is upper bounded by
$2\cdot(\log(|f_{k_1}| + |f_{k_1 + 1}| + \dots + |f_{k_i -1}|) + (k_{i+1} - k_i
- 1)\cdot(\log (|f_{k_i}| + \dots + |f_{k_{i+1}-1}|))$. Finally the number of
rotations is upper bounded by $2\cdot(k_{i+1} - k_i)\cdot \log n$.

Totally the algorithm generates at most $O(k\log n)$ new rules.

Since concatenation of groups of factors is optimal in the worst case, it is
difficult to determine the precise time complexity of the improved algorithm as
a function of input size. Notice that if the size of every group of factors is
equal to 1, then we have the algorithm from \cite{SLPConstruction}. Otherwise
the new algorithm generates less rotations than the standard algorithm but the
new algorithm spends some extra time for calculating order of concatenation. So
in the next section we propose a practical comparison between the two
algorithms.

\section{Practical results}

\subsection{Setup of experiments}
Obviously, the nature of input strings highly affects compression
time and compression ratio. In this paper we consider three types of strings:

\begin{itemize}
  \item DNA sequences (downloaded from DNA Data Bank of Japan, \url{http://www.ddbj.nig.ac.jp/});
  \item Fibonacci strings;
  \item random strings over four letters alphabet.
\end{itemize}

These types of strings have been chosen for the following reasons. Fibonacci
strings are one of the best inputs to the SLP Construction problem. So we can
estimate potential of SLPs as compression model. Random strings are considered
to be incompressible and, potentially, they are the worst input for the SLP
Construction problem. DNA sequences are real data used in practice.

Let us introduce algorithms that have been tested. Our test suite contains a
classic version of the Lempel-Ziv algorithm proposed in \cite{LZ77}. We use the
following brief notation for it: \textbf{lz77}. The size of scanning window is
equal to 32Kb. The reader may think that 32Kb is too short but we choose this
value to study how the window size influences compression ratio.

Notice that the size of scanning window affects the length of factors and the
total size of a LZ-factorization. So if we want to compress longer factors,
then we need to spend more space. In other words, the size of scanning window
directly affects the size of compressed presentation. In the same time there is
no direct relation between the structure of an SLP and the size of the text
derived from it. Therefore using shorter LZ-factorizations leads to
constructing more compact SLPs. From this argument it follows that there is a
reason to consider a version of the Lempel-Ziv algorithm with an infinite
scanning window. We use the following brief notation for it: \textbf{lz77inf}.

Additionally we test yet another classic algorithm from the Lempel-Ziv family:
the Lempel-Ziv-Welch algorithm (briefly \textbf{lzw}). Finally our test suite
contains the classic algorithm for SLP Construction problem (briefly
\textbf{SLPclassic}) proposed in \cite{SLPConstruction} and the algorithm
presented in Section~3 (briefly \textbf{SLPnew}).

For convenience, we accept the following common notation for the tested
algorithms in all following graphs:

\algorithmNotations

We estimate the performance of a compression algorithm in terms of compression
ratio and execution time. For SLP compression algorithms we additionally
calculate the number of rotations. We calculate compression ratio as the ratio
between the size of compressed presentation and the size of input string. We
measure compression ratio in percents. For example, the formula for SLP
compression ratio looks like $\frac{|\slp{S}|}{|S|} \cdot 100$. Since ratio
between serialized SLP rule representation and serialized LZ-factor
representation is greater than 1 (but constant) we may encounter the following
situation: SLP compression ratio is less than LZ compression ratio but the file
containing an SLP is larger than the file containing the corresponding
LZ-dictionary. We should note that the situation changes with input growing.

All algorithms have been implemented using Java SE and tested on Intel Core
i-5(661), 4Gb RAM (Microsoft Windows 7 x64 OS).

\DNARotations

\subsection{SLP construction results}

In this subsection we discuss performance of both SLP construction algorithms.

As expected, both algorithms work infinitely fast on Fibonacci strings and
construct extremely compact representations. For example, on the 36-th
Fibonacci word of size 36.9Mb both algorithms work within 1ms and build SLPs of
size 100. This fact shows that there is a class of strings whereon the SLP
compression model is very efficient.

\DNASpeedTestInMemory

A diagram at Figure 3 shows how the suggested optimization of concatenation
affects the number of rotations of AVL trees on DNA sequences.

\DNASpeedTestInFile

To illustrate how the optimization affects  construction time we present two
tests. In the first one, the algorithms have stored all SLPs being constructed
in operational memory, while in the second one, the SLPs have been stored in an
external file so that every rotation of an AVL-tree forces I/O operations with
the file. We expect that there is no big difference between the algorithms in
the first setting since the new algorithm spends extra time when computing the
most efficient order of concatenation. But what happens when costs of accessing
AVL-trees are greater than costs of computing the order? This situation
naturally appear when the size of input is huge and we cannot store all the
trees in operational memory. So at figures 4 and 5 presented the execution time statistic for both ways of SLPs
storing.

\RandomRotations

\RandomSpeedTestInMemory

\RandomSpeedTestInFile

At Figures 6, 7 and 8 presented the results of testing both algorithms on random strings.

The Figures 3-8 shows that the new algorithm is more stable since it is on
average as fast as the classic algorithm in operational memory and on average
two times faster when a file system is used.

\subsection{Compression ratio results}

Here we present a comparative analysis of compression ratios for DNA sequences
and random strings:

Figures 9 and 10 shows that compression ratio provided by the new algorithm is
close to compression ratio provided by the classic algorithm. In case of DNA
sequences the new algorithm generates many rotations of small AVL trees and as
a result it constructs a wider SLP than the classic algorithm. Also tests shows
that SLP compression ratio directly depends on the size of LZ-factorization
(fluctuations in the results obtained by both classic and new algorithms are
similar to fluctuations in the results obtained by \textbf{lz77inf} and the
size of compressed presentation obtained by \textbf{lz77inf} is equal to the
LZ-factorization size).

\DNACompression

\section{Conclusion}

In this paper we present an improved algorithm for SLP construction. The
algorithm is similar to Rytter's algorithm proposed in \cite{SLPConstruction}
but it seems to be more efficient on large inputs. However, there is an open
issue related to the new algorithm: how to manage groups of factors optimally?
For example, suppose that we already have built an SLP $\slp{L}$ of height $h$
and suppose that the next group of factors generates an SLP $\slp{R}$ of height
$2h$. It looks inefficient to construct the whole $\slp{R}$ and then
concatenate it with $\slp{L}$. A possible solution is to let the allowed size
of the group gradually grow as the algorithm progresses on the input string.

In the paper we also present practical tests of SLPs as compression model. As a
result we should conclude that the existing ways of SLP construction are quite
slow. On the other hand, the tests confirm that compression ratio provided by
SLPs is close to compression ratio provided by the family of Lempel-Ziv
algorithms (and may be even closer than we initially expected).

\RandomCompression

\section*{Acknowledgment}

The authors would like to thank \emph{Mikhail V. Volkov} for his encouragement
and critical notes and \emph{Anna Kozlova} for her help in gathering
statistics. The authors would like to thank anonymous referees for their reviews.

\begin{thebibliography}{1}
\bibitem{LCSubstring}
A.~Ishino, A. Shinohara, T.~Nakamura, W.~Matsubara, S.~Inenaga, and
K.~Hashimoto.
\newblock Computing longest common substring and all palindromes from
  compressed strings.
\newblock In {\em SOFSEM}, volume 4910 of {\em Lecture Notes in Computer
  Science}, pages 364--375. Springer, 2008.

\bibitem{PM_and_HD}
Y.~Lifshits.
\newblock Processing compressed texts: A tractability border.
\newblock In {\em CPM}, volume 4580 of {\em Lecture Notes in Computer Science},
  pages 228--240. Springer, 2007.

\bibitem{SLPConstruction}
W.~Rytter.
\newblock Application of Lempel-Ziv factorization to the approximation of
  grammar-based compression.
\newblock {\em Theor. Comput. Sci.}, 302(1-3):211--222, 2003.

\bibitem{collages}
Y.~Shibata, M. Takeda, A.~Shinohara, T.~Kida, T.~Matsumoto, and S.~Arikawa.
\newblock Collage system: a unifying framework for compressed pattern matching.
\newblock {\em Theor. Comput. Sci.}, 1(298):253--272, 2003.

\bibitem{antidictionaries}
A.~Shinohara, Y.~Shibata, M.~Takeda, and S.~Arikawa.
\newblock Pattern matching in text compressed by using antidictionaries.
\newblock In {\em CPM}, volume 1645 of {\em Lecture Notes in Computer Science},
  pages 37--49. Springer, 1999.

\bibitem{LCS_P}
A. Tiskin.
\newblock Faster subsequence recognition in compressed strings.
\newblock {\em CoRR}, abs/0707.3407, 2007.

\bibitem{LZW}
T.~A. Welch.
\newblock A technique for high-performance data compression.
\newblock {\em IEEE Computer}, 17(6):8--19, 1984.

\bibitem{LZ77}
J.~Ziv and A.~Lempel.
\newblock A universal algorithm for sequential data compression.
\newblock {\em IEEE Transactions on Information Theory}, 23(3):337--343, 1977.

\bibitem{LZ78}
J.~Ziv and A.~Lempel.
\newblock Compression of individual sequences via variable-rate coding.
\newblock {\em IEEE Transactions on Information Theory}, 24(5):530--536, 1978.
\end{thebibliography}
\end{document}


\section{Introduction}
In this report we explain the ideas behind parallelizing the given system. This was done by first flattening the vectors/arrays of the system. To increase the degree of parallelism, array expansion and privatization of simple non-array variables, which are later transformed into inlined scalar variables, was applied. To further increase the degree of parallelism over the whole system, loop distribution was applied throughout the system from the main loop. To keep memory access coalesced, loops where interchanged where possible and/or matrices where transposed.

The system has one inherently sequential loop, in between parallel loops, which was distributed to be the outer loop via array expansions (making the inner arrays a dimension higher) and loop distribution.

Unfortunately we were unable to complete the programming part. We misused a lot of time on the convertion from the flat sequential program to CUDA kernels, whereby we created a PrivGlob structure completely on CUDA (to omit having to copy arrays to and from the kernels). This however made debugging hell, and after tens of hours debugging without valid results we had to revert back to the flat sequential solution and create kernels one at a time, with the PrivGlobs placed completely in hos memory. We should have had realized this sooner, but alas we where stubborn thinking a valid solutions was right around the next logical error.

Due to this, we ran out of time before parallelizing the rollback part, which performance wise is also the most essential part. This also means that we have not optimized the given tridag kernel. However, this report should still give the ideas behind parallelizing the system.


\documentclass[12pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{graphicx}
\author{Madhur Singhal\\2015CS10235}

\title{Parallel Convex Hull}
\date{26 February, 2018}
\begin{document}
\maketitle
\section{Introduction}
The convex hull or convex envelope of a set X of points in the Euclidean plane or in a Euclidean space (or, more generally, in an affine space over the reals) is the smallest convex set that contains X. Intuitively it is the set of boundary points in a given set of points, that is a surface made from joining In this assignment we implement the quick hull algorithm to solve the problem of finding the convex hull of a set of points using C++ and OpenMP.
\section{Algorithm}
The algorithm used by me is presented below.
\begin{enumerate}

   \item We find the point with minimum x-coordinate lets say, min\_x and similarly the point with maximum x-coordinate, max\_x. These are guaranteed to be in the convex hull.
 \item   Make a line joining these two points, say L. This line will divide the the whole set into two parts. We enable nested parallelism in openmp and use sections to assign the two parts to separate threads.
    \item Each thread finds the point with maximum distance from the line L. P forms a triangle with the points min\_x, max\_x. It is clear that the points residing inside this triangle can never be the part of convex hull. Wedistinguish the direction by comparing the sign of the distance from the other point.
    \item The above step divides the problem into two sub-problems. The line joining the points P and min\_x and the line joining the points P and max\_x are new lines and the points residing outside the triangle is a new set of points.  We solve the further suproblems recursively using new threads as needed. We repeat these steps until there is no point on the desired side of the line after which we add the end points of this line to the convex hull.

\end{enumerate}
\section{Design Decisions}
I opted to use task level parallelism (albeit without the OpenMP task primitive) for this task. It was clear that the recursive calls to quickhull were independent of each other and thus could be executed simultaneously. I used a std::set to store the result so that duplicate points are not added and utilized an OpenMP critical section at the portion where points are added to the set since standard library structures are not thread safe. Nested Parallelism using the omp\_set\_nested function was helpful in distributing the load across the threads. OpenMP sections were used inside a parallel block to enable different threads to execute different recursive calls.
\section{Analysis}
\includegraphics[scale=0.52]{spd2}\\
\includegraphics[scale=0.52]{eff2}
I ran the program on various sizes of input files and plotted the results. As can be seen speedup saturates at 2-4 processing elements because of the computer I am running this on. The smallest example actually results in less than one speedup since thread creation overhead dominates.\section{Conclusion}
Thus our parallel implementation performs better than the serial one in this case too. A speedup of 1.35 seems to be the best value we get.
\end{document}


\chapter{Partially Chosen Plaintext Attack}\label{ch:pcpa}

Traditionally, cryptographers have used games for security analysis. Such games
include the indistinguishability under chosen plaintext attack (IND-CPA), the
indistinguishability under chosen ciphertext attack/adaptive chosen ciphertext
attack (IND-CCA1, IND-CCA2)
etc\footnote{\url{https://en.wikipedia.org/wiki/Ciphertext_indistinguishability}}.

In this chapter, we introduce a definition for a new property of encryption
schemes, called indistinguishability under partially-chosen-plaintext-attack
(IND-PCPA). We also provide comparison between IND-PCPA and other known forms of
cryptosystem properties.

\section{Partially Chosen Plaintext Indistinguishability}\label{sec:indpcpa}

\subsection{Definition}

IND-PCPA uses a definition similar to that of IND-CPA.

For a probabilistic asymmetric key encryption algorithm, indistinguishability
under partially chosen plaintext attack (IND-PCPA) is defined by the following
game between an adversary and a challenger.

\begin{itemize} \item The challenger generates a key pair \begin{math}P_k,
S_k\end{math} and publishes \begin{math}P_k\end{math} to the adversary.  \item
The adversary may then perform a polynomially bounded number of encryptions or
other operations.  \item Eventually, the adversary submits two distinct chosen
plaintexts \begin{math}M_0, M_1\end{math} to the challenger.  \item The
challenger selects a bit \begin{math}b\in{0, 1}\end{math} uniformly at random.
\item The adversary can then submit any number of selected plaintexts
\begin{math}R_i, i\in N, |R| \geq 0\end{math}, for which the challenger sends
the ciphertext \begin{math}C_i = E(P_k, M_b||R_i)\end{math} back to the
adversary.  \item The adversary is free to perform any number of additional
computations or encryptions, before finally guessing the value of
\begin{math}b\end{math}.  \end{itemize}

A cryptosystem is indistinguishable under partially chosen plaintext attack, if
every probabilistic polynomial time adversary has only a negligible advantage on
finding \begin{math}b\end{math} over random guessing. An adversary is said to
have a negligible advantage if a win in the above game can be achieved with
probability \begin{math}\frac{1}{2} + \epsilon(k)\end{math}, where
\begin{math}\epsilon(k)\end{math} is a negligible function in a security
parameter \begin{math}k\end{math}.

Intuitively, we can think of the adversary as having the ability to modify the
plaintext of a message, by appending a chosen portion of data to it, without
prior knowledge of the plaintext itself. He can then acquire the ciphertext of
the modified text and perform any kinds of computations on it. A system would
then be described as IND-PCPA, if the adversary is unable to gain more
information about the plaintext, than he would by guessing at random.

\subsection{IND-PCPA vs IND-CPA}

Suppose the adversary submits the empty string as the chosen plaintext, a choice
which is allowed by the definition of the game. The ciphertext that the
challenger would then send back would be \begin{math}C_i = E(P_k, M_b||"\ ") =
E(P_k, M_b)\end{math}, which is the ciphertext returned from the challenger in
the context of the IND-CPA game.

Therefore, if the adversary has the ability to beat the game of IND-PCPA, i.e.
if the system is not indistinguishable under partially chosen plaintext attacks,
he also has the ability to beat the game of IND-CPA. This assumption
provides an informal proof that IND-PCPA is at least as strong as IND-CPA.

\section{PCPA on compressed encrypted protocols}\label{sec:cepcpa}

In this section we will investigate the relationship between compression and
encryption, regarding how partially chosen plaintext attacks can exploit either
method in protocols that allow such functionality schemes.

\subsection{Compression-before-encryption and vice versa}

When having a system that applies both compression and encryption on a given
plaintext, it would be interesting to investigate the order those
transformations should be executed.

Lossless compression algorithms rely on statistical patterns to reduce the size
of the plaintext data without losing information. Such method is possible, since
most real-world data present statistical redundancy. However, such compression
algorithms will fail to effectively compress certain kinds of data sets that
display no statistical patterns.

Encryption algorithms, on the other hand, rely on adding randomness to the
produced ciphertext. If the ciphertext presented high entropy, these statistical
patterns could be exploited in order to deduce information about the plaintext.

In the scheme where we apply compression after encryption, the ciphertext to be
compressed would demonstrate no statistical analysis exploits resulting in
poor compression performance. In addition, compression after encryption would
not increase the security of the protocol.

On the contrary, applying encryption after compression seems a more
preferable solution. The compression algorithm can use the statistical
redundancies of the plaintext to perform well, while the encryption algorithm
should produce a seemingly random stream of data. Also, since compression
increases entropy per symbol, this scheme should make it harder for attackers,
who rely on differential cryptanalysis, to break the system.

\subsection{PCPA scenario on compression-before-encryption protocol}

Let's assume a system that composes encryption and compression in the following
manner:

\begin{math}c = Encrypt(Compress(m))\end{math}

where \begin{math}c\end{math} is the ciphertext and \begin{math}m\end{math} is
the plaintext.

Suppose the plaintext contains a specific secret among random strings of data
and the attacker can issue a PCPA with a chosen plaintext, which we will call
reflection. The plaintext then takes the form:

\begin{math}m = n_1 || secret || n_2 || reflection || n_3\end{math}

where \begin{math}n_1, n_2, n_3\end{math} are random nonces.

If the reflection is the same as the secret, the compression mechanism will
recognize this pattern and compress the two data portions. Otherwise, the
two strings will not demonstrate any statistical redundancy and compression will
perform worse. As a result, in the first case the compressed unencrypted data
will be smaller than in the second case.

Usually encryption is done by a stream or a block cipher. In the first case, the
lengths of a plaintext and the corresponding ciphertext are identical, whereas
in the second case they differ by the number of padding bits, which is
relatively small. That way, for a system as the one mentioned, an adversary
could identify a pattern and extract information about the plaintext, based on
the lengths of the two ciphertexts.

\section{Known PCPA exploits}\label{sec:known_pcpa}

In this section, we present known attacks that use the partially chosen
plaintext attack vector.

\subsection{CRIME}

\texttt{Compression Ratio Info-leak Made Easy} (CRIME) \cite{crime} is a
security exploit that was revealed at the 2012
ekoparty\footnote{\url{https://www.ekoparty.org}}. As stated, "it decrypts
HTTPS traffic to steal cookies and hijack sessions".

In order for the attack to succeed there are two requirements. Firstly, the
attacker should be able to sniff the victim's network traffic to see the
request/response packet lengths. Secondly, the victim should visit a website
controlled by the attacker or surf on non-HTTPS sites, in order for the CRIME
script to be executed.

If the above requirements are met, the attacker makes a guess for the secret to
be stolen and asks the browser to send a request with this guess included in the
path. The attacker can then observe the length of the request and, if the
length is less than usual, it is assumed that the guessed string was compressed
with the secret, so it was correct.

CRIME has been mitigated by disabling TLS and SPDY compression on both Google
Chrome and Mozilla Firefox browsers, as well as various server software
packages. However, HTTP compression is still supported, while some web servers
that support TLS compression are also vulnerable.

\subsection{BREACH}

\texttt{Browser Reconnaissance and Exfiltration via Adaptive Compression of
Hypertext (BREACH)} \cite{breach} is a security exploit that is based on CRIME.
Presented on the 2013 Black Hat USA
conference\footnote{\url{https://www.blackhat.com}}, it targets the size of
compressed HTTP responses and extracts secrets hidden in the response body.

Like the CRIME attack, the attacker needs to sniff the victim's network traffic,
as well as force the victim's browser to issue requests to the chosen endpoint.
Additionally, it works against stream ciphers only and assumes zero noise in the
response. Moreover, it demands a known prefix for the secret, although a
proposed solution for this condition would be to guess the first two characters
of the secret in order to bootstrap the attack.

From then on, the methodology is in general the same as CRIME's. The attacker
guesses a value, which is then included in the response body along with the
secret and, if correct, it is compressed well with it resulting in smaller
response length.

BREACH has not yet been fully mitigated, although Gluck, Harris and Prado
proposed various counter measures for the attack. We will investigate these
mitigation techniques in depth in Chapter \ref{ch:mitigation}.


\documentclass[12pt]{article}
%\linespread{1}
\usepackage[margin=1.0in]{geometry}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{multicol}
\usepackage{graphicx}
\usepackage{url}
\usepackage{fancyhdr}
\usepackage{color}

\newcommand\blfootnote[1]{%
  \begingroup
  \renewcommand\thefootnote{}\footnote{#1}%
  \addtocounter{footnote}{-1}%
  \endgroup
}

\title{Distributed Sampled Convex Problems}
\author{David Eckman (dje88) and Calvin Wylie (cjw278)}
\date{}

\begin{document}

%\noindent
\setlength{\parindent}{24pt}

\maketitle

\begin{abstract}
It is well-known that chance constrained programs are in general intractable because of the non-convexity of the feasible region and the requirement of prior information about the distribution of the uncertain parameters.
By solving a sampled convex program with sufficiently many samples of the uncertain parameters, one can find solutions that are feasible for the chance constrained program with high probability.
Sampled convex problems are more easily solved because they are characterized by a finite number of constraints and require only that one can obtain i.i.d. samples of the uncertain parameters.
However, the required number of samples may still result in a sampled convex program that is expensive to solve exactly.
We therefore consider a procedure for obtaining approximate solutions to these sampled convex programs by distributing the constraints across processors and solving smaller subproblems, followed by a consensus on the active constraints.
We experiment with a classical portfolio optimization problem and study the trade-offs that arise from this method: including wall clock time, expected violation probability, and feasibility probability for the chance constrained program.
\end{abstract}

\blfootnote{Full source code and data for this paper is available at \url{https://github.com/calvinwylie/pscp}.}

\section*{Introduction}
Many real-world problems in areas as diverse as finance, energy, and emergency services can be modeled by chance constrained optimization problems \cite{bental09}.
For example, chance constraints can represent an acceptable level of risk in an investment or a fixed level of service that is guaranteed.
A typical chance constrained program, \ref{ccp}, is given by
\begin{align}\label{ccp}
\begin{split}
\begin{aligned}
    & \underset{x \in \mathcal{X}}{\text{minimize}}
    & & c^T x \\
    & \text{subject to}
    & & \mathbb{P}\{f(x,\delta) \leq 0\} \geq 1-\epsilon.
\end{aligned}
\end{split} \tag{CCP$_\epsilon$}
\end{align}
where $\mathcal{X} \subseteq \mathbb{R}^n$ is compact, $\delta$ is an uncertain parameter distributed with probability measure $\mathbb{P}$ over an uncertainty set $\Delta \subseteq \mathbb{R}^d$, and $f:\mathcal{X} \times \Delta \mapsto \mathbb{R}$ is convex in $x$ for any fixed $\delta$.
The parameter $0 \leq \epsilon \leq 1$ represents the acceptable probability with which the constraint $f(x,\delta) \leq 0$ can be violated.
\ref{ccp} is notoriously intractable because the feasible region is non-convex for general probability measures.
Moreover, obtaining a convex approximation relies on knowing the distribution $\mathbb{P}$ (or at least the moments) a priori.

One method for finding ``good'' solutions which are also feasible for \ref{ccp} with high probability is to take $N$ i.i.d. samples $(\delta_1, \ldots, \delta_N)$ and solve the corresponding sampled convex program, \ref{scp}:
\begin{align}\label{scp}
\begin{split}
\begin{aligned}
    & \underset{x \in \mathcal{X}}{\text{minimize}}
    & & c^T x \\
    & \text{subject to}
    & & f(x,\delta_i) \leq 0 \quad i = 1, \ldots, N.
\end{aligned}
\end{split} \tag{SCP$_N$}
\end{align}
Notice that in contrast to \ref{ccp}, \ref{scp} is more tractable since it has a finite number of constraints and a convex feasible region. Note that we also need not explicitly know $\mathbb{P}$, but instead only need to be able to obtain i.i.d. samples from it.

Let $x_N^*$ denote the (random) solution to \ref{scp} obtained by taking $N$ samples.
For a fixed $x_N^*$, let $V(x_N^*) := \mathbb{P}\{\delta \in \Delta: f(x_N^*, \delta) > 0\}$ be the corresponding violation probability.
If $V(x_N^*) \leq \epsilon$, then $x_N^*$ is feasible for \ref{ccp}.

Sampled convex programs do not provide a guarantee that $V(x_N^*) \leq \epsilon$ for a given sample size $N$.
Instead, Calafiore and Campi \cite{campi05} proved that one can bound the number of samples needed to ensure that $\mathbb{P}^N\{V(x_N^*) > \epsilon\} \leq \beta$ for $0 < \beta \leq 1$,  where $\mathbb{P}^N$ is the product probability measure.
That is, the optimal solution to \ref{scp} is feasible for \ref{ccp} with probability greater than $1 - \beta$.
Their bound of $N \geq 1/(\epsilon\beta) - 1$ is simple to calculate, but is quite loose in terms of asymptotics.
The bound was further tightened in \cite{campi06} to choosing $N$ to satisfy
\[ \mathbb{P}^N\{V(x_N^*) > \epsilon\} \leq \binom{N}{d}(1-\epsilon)^{N-d}. \]
and then later by Campi and Garatti in \cite{campi08} to the solution of a binomial equation:
\[ \mathbb{P}^N\{V(x_N^*) > \epsilon\} = \sum_{i=0}^{d-1} \binom{N}{i} \epsilon^i (1-\epsilon)^{N-i}. \]
Note that all of these bounds are independent of the distribution $\mathbb{P}$, which means that $N$ can be calculated without prior knowledge of $\mathbb{P}$.
Even for modest $\epsilon$ and $\beta$, the required $N$ may be large enough that the resulting \ref{scp} instance is very computationally expensive to solve exactly, especially if the number of decision variables, $n$, is also large.

One idea for addressing this challenge is to decompose \ref{scp} into smaller sampled convex programs to be solved in parallel before forming some consensus across the subproblems.
Because the subproblems have only a fraction of the constraints, they can potentially be solved much quicker than the full \ref{scp} problem.
However, because each subproblem is only considering a subset of the constraints, there is a loss associated with this simplification: the solution returned will likely violate some of the constraints of \ref{scp} and therefore is less likely to be a feasible solution for \ref{ccp}.
But for some applications, the speedup in terms of time to solution due to parallelism may be worth the decrease in the probability that the solution is \ref{ccp}-feasible.
%In particular, we would like to study how much is lost with respect to \ref{ccp} feasibility by taking the active constraints from the subproblems and solving a final sampled convex program with these constraints.

\section*{Parallel Procedure for \ref{scp}}
To motivate our proposed procedure, recall that a constraint $\{ x: f(x) \leq 0 \}$ is \emph{active} at $\bar{x}$ if $f(\bar{x}) = 0$. Assuming some regularity conditions, a feasible convex program in $n$ dimensional space will have at most $n$ active constraints at the optimal solution.
Therefore the problem of finding an optimal solution to \ref{scp} can be viewed as the problem of identifying the $n$ active constraints at the optimal solution.

A recent paper by Carlone et al. \cite{carlone2014} studies solving large-scale convex programs in a distributed fashion.
It assumes a directed graph topology on the communication network connecting a set of processors.
Initially, each processor is given a subset of the constraints of the full problem with which it solves a convex subproblem.
It then identifies the active constraints at that solution and passes them to other processors according to the network topology.
Each processor receives the incoming constraints and resolves a convex subproblem with their original set of constraints plus the new constraints.
It was proven that after an almost surely finite number of iterations, all processors will reach a consensus about which constraints are active at the optimal solution to the master problem.
Note that the objective in \cite{carlone2014} is to recover the exact solution to the convex program, whereas we are interested in finding an approximate solution in far less time.

In practice, the procedure in \cite{carlone2014} would take far too long to run to completion, especially since there is no bound on the number of iterations needed for consensus.
%The only instance you would fully run a procedure like this is if the master problem is too large to solve with existing software.
Instead, we will consider a variation of this procedure which only runs for two iterations and in which the topology of the communication network is a star graph in which all ``worker'' processors pass active constraints to a ``master'' processor for a second-stage re-solve.
Because there are only two iterations, the solution returned will likely not be the exact solution to \ref{scp}.
However this approximate solution can be calculated in less time, and thus it may be worth a loss in the probability of it being a \ref{ccp}-feasible solution.

Suppose that we have available one master processor and $p$ worker processors.
For simplicity, suppose that $N = mp$ for some positive integer $m > n$.
Our proposed procedure is as follows:

\bigskip
\textbf{Parallel \ref{scp} Procedure}
\begin{enumerate}
\item Acquire $N$ i.i.d. samples $\delta_1, \ldots, \delta_N$.

\item For each worker processor $1 \leq j \leq p$, solve the subproblem
\begin{equation*}
\begin{aligned}
    & \underset{x \in \mathcal{X}}{\text{minimize}}
    & & c^T x \\
    & \text{subject to}
    & & f(x,\delta_i) \leq 0 \quad i = m(j-1) + 1, \ldots, mj.
\end{aligned}
\end{equation*}
Let $\tilde{x}_j$ denote the corresponding optimal solution.
Determine the set of active constraints at the optimal solution $\tilde{x}_j$.
Assuming that each sampled convex program is fully supported with probability one (i.e. that there are $n$ active constraints),
label the samples corresponding to these constraints $\delta_{(1)}^j, \ldots, \delta_{(n)}^j$.
Pass the set of active constraints to the master processor.

\item On the master processor, solve the sampled convex program
\begin{equation*}
\begin{aligned}
    & \underset{x \in \mathcal{X}}{\text{minimize}}
    & & c^T x \\
    & \text{subject to}
    & & f(x,\delta_{(i)}^j) \leq 0 \quad i = 1,\ldots,n \text{ and } j = 1,\ldots,p.
\end{aligned}
\end{equation*}
Let $\tilde{x}$ denote the corresponding optimal solution.
\end{enumerate}

Changing the number of processors $p$ will impact the time to solution (wall clock time) of the procedure in two conflicting ways, as the sizes of both the first-stage subproblems and the second stage master problem are changed.
With more processors, the subproblems are smaller, but the total number of constraints passed to the master program ($np$) is larger.
Also, as the number of constraints in the master problem increases, we would expect $\mathbb{P}^N\{V(\tilde{x}) > \epsilon\}$ to decrease.
The main research question of this study will be: \textit{given these trade-offs, what is the optimal number of processors to use?}

We will also consider another version of this problem.
Assume that we are willing to commit to the wall clock time needed to solve \ref{scp} exactly.
But instead of solving \ref{scp} we want to use the time to take more than $N$ samples and apply the distributed procedure in hopes of increasing the probability that the returned solution is feasible for \ref{ccp}.
\textit{What then is the best number of processors and number of constraints per processor?}

\section*{Example Problem}

We consider the classical portfolio optimization problem of selecting an allocation $y$ for investing one dollar over $n$ assets with uncertain returns $r_i$ for $i = 1, \ldots, n$.
Of these assets, we assume that the $n$th asset has a fixed payout; it might, for example, represent a bond with a certain return.
Each allocation of assets defines a portfolio.
Our objective is to choose a portfolio that maximizes the value-at-risk (VaR), denoted by $t$, for an acceptable risk level $\epsilon$.
Here VaR refers to the $\epsilon$-percentile of the distribution of returns for a particular portfolio as opposed to the (more traditional) $\epsilon$-percentile of the distribution of losses.

This setting gives rise to the following chance constrained program:
\begin{align}\label{Portfolioccp}
\begin{split}
\begin{aligned}
    & \underset{y, \,t}{\text{maximize}}
    & & t \\
    & \text{subject to}
    & & \mathbb{P}\left\{ \sum_{i=1}^{n-1} r_i y_i + r_n y_n \geq t \right\} \geq 1-\epsilon \\
    & & & \sum_{i=1}^n y_i = 1 \\
    & & & y_i \geq 0 \quad i = 1, \ldots, n.
\end{aligned}
\end{split} \tag{Portfolio CCP}
\end{align}

We will assume that the vector of uncertain returns $(r_1, \ldots, r_{n-1})$ is distributed according to a multivariate log-normal distribution with mean vector $\mu$ and covariance matrix $\Sigma$.
A log-normal distribution is a typical model for asset return because it is unimodal with support on the positive real line.
A more sophisticated approach might sample the returns from an empirical distribution function built from past data.
In the experiments we perform, we take $n=200$, with one asset fixed at a return of 1.05.
The rest of the assets are uncorrelated, with means increasing on the interval [1.06,1.23] and variances increasing on the interval [0.05, 1.045].

By taking samples $r^{(j)}$ for $j = 1, \ldots, N$, the corresponding sampled convex program is
\begin{align}\label{Portfolioscp}
\begin{split}
\begin{aligned}
    & \underset{y, \,t}{\text{maximize}}
    & & t \\
    & \text{subject to}
    & & \sum_{i=1}^{n-1} r_i^{(j)} y_i + r_n^{(j)} y_n \geq t \quad j = 1, \ldots, N \\
    & & & \sum_{i=1}^n y_i = 1 \\
    & & & y_i \geq 0 \quad i = 1, \ldots, n.
\end{aligned}
\end{split} \tag{Portfolio SCP$_N$}
\end{align}

\ref{Portfolioscp} has several nice properties:
\begin{itemize}
\item The objective function and constraints of \ref{Portfolioscp} are linear. Therefore we can use standard linear programming techniques to solve the sampled convex program and subproblems.
\item Because of the fixed asset $n$, it is possible to easily find a basic feasible solution by taking $y_n = 1$, $y_i = 0$ for $i = 1, \ldots, n-1$ and $t = r_n$.
\item The feasible allocations $y$ sit within a unit simplex.
\end{itemize}

In our experiments, we took $\epsilon = 0.05$ and $\beta = 1 \times 10^{-5}$ and calculated a required sample size of $N = 5312$.

\section*{Implementation}

To solve our example portfolio optimization problem, we implemented a C interface to solve linear programs in parallel using the GNU Linear Programming Kit (GLPK) \footnote{\url{https://www.gnu.org/software/glpk/}}.
We chose to use the Simplex algorithm to solve our linear programs.
As our constraints are very dense, interior point methods would likely be very slow.
To provide parallelism, we coded to the Message Passing Interface (MPI) standard.
The code was compiled with the Intel C compiler (icc), and experiments performed on a single 12-core Intel Xeon E5-2620 v3 processor.

Given a linear program with $m$ constraints and $n$ variables, the folklore of the simplex method is that it takes $O(m)$ pivots to reach the optimal solution for a ``typical'' linear program (the worst case is always $2^n - 1$).
Thus we should observe a roughly linear speedup when decomposing our sampled program into smaller subproblems.
Also given a problem where $m >> n$, it may make sense to solve the dual program instead (which can be done through the so-called Dual Simplex Algorithm), which has only $n$ constraints.

Figure \ref{fig:fig_simplex_time} shows some experimental timings for the simplex and dual simplex methods applied to our problem with $n = 200$ assets on a single core.
It shows a superlinear increase in wall clock time with respect to the number of constraints.

However, we do not realize any speedup through solving the dual program.
We guess this is because our sampled constraints are on average very similar to each other, and thus not as many pivots are needed to reach an optimal extreme point as would be required in a more ``standard'' linear program.

\begin{figure}[ht]
	\centering
		\includegraphics[scale=0.9]{../plot/figs/fig_simplex_time.png}
	\caption{Wall Clock Times for Simplex and Dual Simplex Methods.}
	\label{fig:fig_simplex_time}
\end{figure}

\section*{Performance Analysis}

The program spends time in two main areas: communicating the constraint data through MPI and solving the linear programs in GLPK.
As these are two mature and heavily optimized libraries, manual performance tuning of the routines called would likely not lead to significant speedups.
We can however, reason about the overall performance of our program at a high level.

Assuming that the number of constraints is held constant, as Figure~\ref{fig:fig_simplex_time} suggests, we should see a linear decrease in the wall clock time of the simplex algorithm when distributing the constraints across different processors.
The time required for the final solve on the master processor (using active constraints from the subproblems) should be small, as we can warm-start this optimization with an optimal basis from a subproblem.

However, as we increase the number of processors, the time to communicate the initial constraints (through an \texttt{MPI\_Scatter}) and the active constraints (through \texttt{MPI\_Gather}) will increase.
Through profiling, we verified that there is a roughly $O(p^2)$ relationship between communication time and the number of processors $p$.

This suggests that we should initially see decreases in wall clock time as we add processors, but at some point the communication time will start to dominate the savings gained by splitting our problem.
Figure~\ref{fig:wct_numproc} shows this experimentally.

\begin{figure}[ht]
    \centering
        \includegraphics[scale=0.9]{../plot/figs/wct_numproc.png}
    \caption{Wall Clock Times by Number of Processors.}
    \label{fig:wct_numproc}
\end{figure}

\section*{Experiments}
Let $\tilde{x} := (\tilde{y}, \tilde{t})$ denote the solution returned by running the parallel \ref{scp} procedure.
Even though the distribution $\mathbb{P}$ is known, calculating $V(\tilde{x})$ exactly is intractable.
Instead we estimated $V(\tilde{x})$ via Monte Carlo simulation, generating $R = 5000$ i.i.d. realizations of the returns $r$ and calculating
\[ \hat{V}(\tilde{x}) = \frac{1}{R} \sum_{j = 1}^R \mathbf{1}\left\{ \sum_{i=1}^{n-1} r_i^{(j)} \tilde{y}_i + r_n^{(j)} \tilde{y}_n < \tilde{t}\right\}. \]

We performed $M = 400$ macro-replications of this procedure, with corresponding solutions $\tilde{x}^{(k)}$ for $k = 1, \ldots, M$, and calculated the following point estimates of $\mathbb{E}[V(\tilde{x})]$ and $\mathbb{P}^N\{V(\tilde{x}) > \epsilon\}$:
\[ \widehat{\mathbb{E}[V(\tilde{x})]} := \frac{1}{M} \sum_{k=1}^M \hat{V}(\tilde{x}^{(k)}), \]
and
\[ \widehat{\mathbb{P}^N\{V(\tilde{x}) > \epsilon\}} := \frac{1}{M} \sum_{k=1}^M \mathbf{1}\left\{\hat{V}(\tilde{x}^{(k)}) > \epsilon\right\}. \]
We used the Clopper-Pearson exact binomial test to form approximate confidence intervals for these two performance measures.

When we divided the 5312 constraints evenly among $p$ processors, our estimates for $\mathbb{E}[V(\tilde{x})]$ and $\mathbb{P}^N\{V(\tilde{x}) > \epsilon\}$ are shown in Figures~\ref{fig:expviolprob_numproc} and \ref{fig:probviolprobgreateps_numproc}.

%\begin{figure}[ht]
	%\centering
		%\includegraphics{../plot/figs/objfnval_num_proc.png}
	%\caption{VaR $\tilde{t}$ by Number of Processors}
	%\label{fig:objfnval_num_proc}
%\end{figure}
%
%\begin{figure}[ht]
	%\centering
		%\includegraphics{../plot/figs/wct_objfnval_frontier.png}
	%\caption{Trade-off between Time and VaR $\tilde{t}$}
	%\label{fig:wct_objfnval_frontier}
%\end{figure}

\begin{figure}[ht]
	\centering
		\includegraphics[scale=0.9]{../plot/figs/expviolprob_numproc.png}
	\caption{$\mathbb{E}[V(\tilde{x})]$ by Number of Processors.}
	\label{fig:expviolprob_numproc}
\end{figure}

\begin{figure}[ht]
	\centering
		\includegraphics[scale=0.9]{../plot/figs/probviolprobgreateps_numproc.png}
	\caption{$\mathbb{P}^N\{V(\tilde{x}) > \epsilon \}$ by Number of Processors.}
	\label{fig:probviolprobgreateps_numproc}
\end{figure}

Though at four processors and above the violation probability $V(\tilde{x})$ is less than $\epsilon$ in expectation, the probability of \ref{ccp} constraint violation $\mathbb{P}^N\{V(\tilde{x}) > \epsilon \}$ is still very high (recall that we are aiming for $\mathbb{P}^N\{V(\tilde{x}) > \epsilon \} \leq \beta = 10^{-5}$).

Figure~\ref{fig:wct_probviolprobgreateps_frontier} shows the trade-off between wall clock time and $\mathbb{P}^N\{V(\tilde{x}) > \epsilon \}$.
This Figure shows that despite returning solutions with high probabilities of infeasibility, the parallel \ref{scp} procedure could run in a fraction of the time of the serial procedure.
This suggests that we could use some of the time savings to run the parallel procedure on more than $N$ samples, recovering solutions with feasibility probabilities competitive with the serial procedure, with still significant speedup.

\begin{figure}[ht]
	\centering
		\includegraphics[scale=0.9]{../plot/figs/wct_probviolprobgreateps_frontier.png}
	\caption{Trade-off between Time and $\mathbb{P}^N\{V(\tilde{x}) > \epsilon \}$.}
	\label{fig:wct_probviolprobgreateps_frontier}
\end{figure}

In another set of experiments, we increased the number of samples given to each worker processor, typically doubling or quadrupling.
For example, with $N = 5312$, running the parallel procedure with eight processors means that each processor receives $N/8 = 664$ samples.
Here we ran the same procedure on eight processors, except with 1328 and 2656 constraints for each processor (so now the total number of samples generated was 10624 and 21248).

Figure \ref{fig:wct_numconstraint} shows the wall clock time results for the instances we tested.
Observe that the lowest point on each colored curve corresponds to the case of running the original problem of $N = 5312$ constraints.
From Figure \ref{fig:wct_numconstraint}, we can see the increases in wall clock time for a fixed number of processors is again superlinear.
In addition, doubling the number of constraints per processor (for $p =4$, 8, and 16) appears to outperform the serial procedure in terms of wall clock time and quadrupling the number of constraints per processor (for $p = 4$ and 8) has comparable times to the serial procedure.

\begin{figure}[ht]
	\centering
		\includegraphics[scale=0.9]{../plot/figs/wct_numconstraint.png}
	\caption{Wall Clock Times by Number of Constraints/Processor.}
	\label{fig:wct_numconstraint}
\end{figure}

Figures \ref{fig:expviolprob_numconstraint} and \ref{fig:probviolprobgreateps_numconstraint} show $\mathbb{E}[V(\tilde{x})]$ and $\mathbb{P}^N\{V(\tilde{x}) > \epsilon \}$, respectively, for these same instances.
On these plots, the highest point on each colored curve corresponds to the case of running the original problem.
As expected, both performance measures improve as the number of constraints per processor are increased.
It appears that doubling the number of constraints per processor is enough for the parallel procedure to match or surpass the performance of the serial procedure.

\begin{figure}[ht]
	\centering
		\includegraphics[scale=0.9]{../plot/figs/expviolprob_numconstraint.png}
	\caption{$\mathbb{E}[V(\tilde{x})]$ by Number of Constraints/Processor.}
	\label{fig:expviolprob_numconstraint}
\end{figure}

\begin{figure}[ht]
	\centering
		\includegraphics[scale=0.9]{../plot/figs/probviolprobgreateps_numconstraint.png}
	\caption{$\mathbb{P}^N\{V(\tilde{x}) > \epsilon \}$ by Number of Constraints/Processor.}
	\label{fig:probviolprobgreateps_numconstraint}
\end{figure}

Finally, the wall clock time and $\mathbb{P}^N\{V(\tilde{x}) > \epsilon \}$ are shown together in Figure \ref{fig:wct_probviolprobgreateps_frontier_constraint}.
Each colored line represents a frontier for a fixed number of processors where the left-point on each curve is the original problem.
The number of constraints per processor increases as one moves to the right along each curve.
Based on this plot, an ``optimal'' number of configuration appears to be eight processors with 1328 constraints per processor (the middle red point) because it attains the $\mathbb{P}^N\{V(\tilde{x}) > \epsilon \}$ of the serial solution in less than half the time.

\begin{figure}[ht]
	\centering
		\includegraphics[scale=0.9]{../plot/figs/wct_probviolprobgreateps_frontier_constraint.png}
	\caption{Trade-off between Time and $\mathbb{P}^N\{V(\tilde{x}) > \epsilon \}$}
	\label{fig:wct_probviolprobgreateps_frontier_constraint}
\end{figure}

\section*{Conclusions}
We presented a parallel procedure for obtaining approximate solutions to sampled convex programs.
For a modest-sized portfolio optimization problem, we observed that this parallel procedure can dramatically reduce the run time with some loss in \ref{ccp} feasibility.
In our MPI implementation we observed marginally decreasing gains in run-time performance as the number of processors increased, due to increased communication and the need to solve a larger second-stage problem.
We believe that for more complex sampled convex programs---those with more decision variables and constraints or those of harder complexity than linear programs---the benefits of a distributed approach would be even more pronounced.
When we experimented with adjusting the number of constraints per processor, our main conclusion was that one can typically sample twice as much while achieving faster run-times and \ref{ccp}-feasibility competitive with the serial procedure.

\section*{Open Extensions}
If we had more time, there are a number of open areas we would have wanted to explore:
\begin{itemize}
\item Experiment on an actual financial data set with forecasts for asset returns.
\item Compare the quality of the solution returned by our procedure with other solutions found by solving over convex inner approximations of the \ref{ccp} feasible region.
\item Extend our procedure to run multiple iterations of the worker/master sequence in hopes of better recovering the optimal active constraints.
\item Test for large problem sizes and other problems whose solution methods are of harder complexity than those of linear programming.
\end{itemize}

%We intend to conduct experiments on the performance of the parallel procedure versus the traditional serial \ref{scp} procedure.
%For measures of performance of the solutions $x^*$ and $x^{**}$, we will look at the objective function values, the expected violation probabilities, and the probabilities that the violations probabilities are greater than $\epsilon$.
%We will make comparisons based on both the total number of samples and the total wall clock times of the two procedures.
%
%We will also experiment with variations of the basic procedure.
%One variation would involve communicating the support constraints of the master's problem back to the slaves and doing new sampling before solving new subproblems for each slaves.
%This could be done over a number of iterations and we could study the performance of the returned solutions at each iteration.
%
%Another variation would again involve communicating support constraints of the master's problem back to the slaves, but instead of doing new sampling, we re-solve the subproblems with the additional shared support constraints.
%After sending the support constraints of these new subproblems to the master, we could perform one final solve and get a solution that is no worse than the previous.
%This is because passing support constraints from the master to the slaves would allow for constraints that were not previously support constraints to become support constraints.

\bibliographystyle{siam}
\bibliography{references}

\end{document}