\chapter[An Algorithm for Incrementally Enumerating Bitriangles]{An algorithm for incrementally enumerating Bitriangles using DP}\label{incr-algo-bt-dp}
In this chapter, we first give the foundation to define and implement an algorithm for incrementally enumerating bitriangles in a large bipartite graph.
Then, we introduce an algorithm based on the \acrshort{dp}. In particular,  we present the pseudo-code of the stages of the proposed $\dpbt$.
Finally, we provide a proof of correctness of our proposal and the main details of the implementation of the $\dpbt$ using the \acrshort{dpbt}.

\section{Preliminaries Definitions}\label{sec:prem:def}
In order to understand how the algorithm works, we need to provide some basic definitions, and define other small structural units that we use for giving a solution to the problem.
Let's enumerate all those definitions in the following paragraphs.

\begin{definition}[\acrlong{bg}]
A \acrfull{bg} is an undirected graph $G=(V,E)$  such that $V=(U\cup L)$, $U\cap L=\emptyset$ and $E\subseteq U\times L$.
\end{definition}

Additionally, without loss of generality, we assume that  $U\subseteq \mathbb{N}$ and $L\subseteq \mathbb{N}$. Consequently, $(U,<)$  and $(L,<)$ are strict total orders. We can see an example of a \acrshort{bg}
in \autoref{fig:bipartite-graph-example}.

\begin{figure}[ht]
\centering
\inputtikz{bipartite}
\caption[{[\acrshort{iebt}] Example of \acrlong{bg}}]{A bipartite graph in which five bitriangles can be enumerated}
\label{fig:bipartite-graph-example}
\end{figure}

Identifying the different bitriangles in the graph in \autoref{fig:bipartite-graph-example} can be done manually.
In what follows, we use this graph to illustrate the new definitions.
This small example allows us to realize that identifying and listing bitriangles in large graphs is a challenging task.

\begin{definition}[\acrlong{bt}]\label{def:bt}
Let the triples $\mu=(u_1, u_2, u_3)$ and $\ell=(l_1, l_2,l_3)$ on $U$ and $L$, respectively, i.e.  $\{u_1, u_2, u_3\} \subseteq U$, $\{l_1, l_2,l_3\} \subseteq L$.
The 6-cycle $(u_1,l_1,u_2,l_3,u_3,l_2,u_1)$  is a \textit{\acrfull{bt}} in $G$, denoted by $BT_{\ell}^{\mu} = \bti$.
\end{definition}

\begin{figure}[h!]
\begin{subfigure}[b]{0.5\textwidth}
\centering
\inputtikz{bipartite_bt_a}
\caption{\acrshort{bt} $(1,a,3,c,5,b,1)$}
\end{subfigure}
\begin{subfigure}[b]{0.5\textwidth}
\centering
\inputtikz{bipartite_bt_b}
\caption{\acrshort{bt} $(1,a,7,c,5,b,1)$}
\end{subfigure}
\caption[{[\acrshort{iebt}] Example of bitriangles}]{Bitriangles of the triple $(a,b,c)$ combined with vertices from $U$ $\{1,3,5,7\}$ of \autoref{fig:bipartite-graph-example}. In this case two bitriangles have been built from the \autoref{fig:agg-bt-example}}
\label{fig:bt-a-example}
\end{figure}

A bitriangle $(u_1,l_1,u_2,l_3,u_3,l_2,u_1)$ admits different ways of traversing it depending on the vertex where the traversal starts. This fact gives rise to different feasible permutations of its representation as a 6-cycle.
For example in the \autoref{fig:bt-a-example} the bitriangle $(1,a,3,c,5,b,1)$, can also be traverse using the permutations $(a,1,b,5,c,3,a)$, $(5,c,3,a,1,b,5)$, $\dots$, and so on.
Notice that all these feasible permutations guarantee the bitriangle condition of vertices intercalation from $U$ and $L$.

\begin{definition}[\acrfull{wg}]\label{def:wg}
A \textit{\acrfull{wg}} in $G$ is a triple $(u_1,l,u_2)$, $\{u_1,u_2\}\subseteq U$, $l \in L$ and $\{(u_1,l),$ $(u_2,l)\} \subseteq E$. The vertex $l$ is the middle vertex of the wedge.
\end{definition}

With this definition in place, we can now define an aggregated wedge, which is a compressed form of all the wedges having $l \in L$ as a middle vertex.

\begin{definition}[\acrfull{awg}]\label{def:awg}
An \textit{\acrfull{awg}} is a pair $\la l, W_l \ra$, where $l \in L$, $W_l \subseteq U$ and for all $u\in W_l$, the edge  $(u,l)\in E$.
\end{definition}

\begin{figure}[htp!]
\begin{subfigure}[t]{0.3\textwidth}
\centering
\inputtikz{bipartite_awg_a}
\caption[\acrshort{awg} on $a$]{This is the \acrshort{awg} $\la a, \{1,3,7\}\ra$, the compact representation of wedges}
\label{fig:awedge-example-a}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.3\textwidth}
\centering
\inputtikz{bipartite_awg_b}
\caption[\acrshort{awg} on $b$]{This is the \acrshort{awg} $\la b, \{1,5\}\ra$, the compact representation of wedges}
\label{fig:awedge-example-b}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.3\textwidth}
\centering
\inputtikz{bipartite_awg_c}
\caption[\acrshort{awg} on $c$]{This is the \acrshort{awg} $\la c, \{1,3,5,7,9\}\ra$, the compact representation of wedges}
\label{fig:awedge-example-c}
\end{subfigure}
\caption[{[\acrshort{iebt}] Definitions Examples \acrlong{awg}}]{\acrlong{awg} of nodes $a,b,c$ of \autoref{fig:bipartite-graph-example}}
\label{fig:awedge-example}
\end{figure}

In \autoref{fig:awedge-example}, we are only enumerating the first three lower layer vertices \acrshort{awg}.
Next, we define a more refined structure that will allow us to enrich the \acrshort{awg} in order to lead us to a \acrshort{bt}.
That intermediate structure is an \emph{aggregated double-wedge}. For defining \acrshort{adwg} we first need another structure called \emph{double-wedge}. Intuitively, a \emph{double-wedge}, is similar to a \acrshort{wg} and \acrshort{awg} but relating two vertices in the lower layer $L$.

\begin{definition}[\acrfull{dwg}]
A \textit{\acrfull{dwg}} in $G$ is a path of length 4 $(u_1,l_1,u_2,l_2,u_3)$ where  $\{u_1,$ $u_2,u_3\}\subseteq U$ and $\{l_1,l_2\}\subseteq L$. Vertices $l_1$ and $l_2$ are the middle vertices of \acrshort{dwg}.
\end{definition}

\begin{figure}[htp!]
\begin{subfigure}[t]{0.5\textwidth}
\centering
\inputtikz{bipartite_dw_a}
\caption[\acrshort{dwg} $(u_1,l_l,u_2,l_u,u_3)$]{A \emph{double-wedge} $(u_1,l_l,u_2,l_u,u_3)$}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.5\textwidth}
\centering
\inputtikz{bipartite_wg_a}
\caption[\acrshort{wg} $(u_1,l_m,u_3)$]{A connector \emph{wedge} $(u_1,l_m,u_3)$ with middle vertex $l_m$}
\end{subfigure}\hfill
%
\begin{subfigure}[t]{1\textwidth}
\centering
\inputtikz{bipartite_bt_dw_wg_a}
\caption[\acrshort{bt} with connector]{A \emph{bitriangle} formed by \emph{double-wedge} $(u_1,l_l,u_2,l_u,u_3)$ and connector \emph{wedge} $(u_1,l_m,u_3)$}
\end{subfigure}
\caption[{[\acrshort{iebt}] Example of connector wedge}]{Example on how to build a bitriangle from a \emph{double-wege} plus a \emph{wedge} connector}
\label{fig:wg-dw-example}
\end{figure}

In \autoref{fig:wg-dw-example} the \emph{wedge} $(u_1,l_m,u_3)$, is called the \textit{connector wedge}.
This connector wedge together with the \emph{double-wedge} $(u_1,l_l,u_2,l_u,u_3)$ form a  a bitriangle.

\begin{definition}[\acrfull{adwg}]\label{def:adwg}
Let $U_l = \la I, J, K\ra$ be a triplet such that $I \subseteq U, J \subseteq U$ and $K \subseteq U$, where $I, J$ and $K$ are disjoint sets.
An \textit{\acrfull{adwg}}  is a pair  $\la (l_1, l_2), U_l \ra$, where $\{l_1,l_2\}\subseteq L$ and  for all $u_i \in I, u_j \in J$ and $u_k \in K$, $\{(u_i, l_1), (u_j, l_1), (u_j, l_2), (u_k, l_2)\} \in E$.
\end{definition}

\begin{figure}[htp!]
\centering
\inputtikz{bipartite_adwg_a}
\caption[{[\acrshort{iebt}] Example Aggregated double-wedge}]{\acrlong{adwg} $\la (a,c), \la \emptyset, \{1,3,7\}, \{5,9\}\ra \ra$ of \autoref{fig:bipartite-graph-example}. Upper layer nodes $1,3,7$ and $5,9$ are enclosed in a square indicating the set that they belong to. Remember in \dref{def:adwg} we are forming three sets $U_l = \la I, J, K \ra$. $I = \emptyset$ in this case because they should be disjoint sets and $a$ and $c$ share the same upper layer nodes in $J$.}
\label{fig:agg-double-wedge-example}
\end{figure}

In \autoref{fig:agg-double-wedge-example} we can see the \acrshort{adwg} built by $a$ and $c$ where, according to \dref{def:adwg}, $a = l_1$ and $c = l_2$.
The last aggregated structure that enable the algorithm to build a \acrshort{bt} is the \emph{\acrlong{abt}}.
Intuitively, this intermediate structure is an aggregation of a \acrshort{adwg} with another new \acrshort{awg} related with the former structure.

\begin{definition}[\acrfull{awgc}]\label{def:awgc}
Let $\la (l_1, l_2), U_l \ra$ be an \acrshort{adwg}, where $\{l_1,l_2\}\subseteq L$ and $U_l = \la I, J, K\ra$.
An \acrshort{awg} $\la l, W_l \ra$ is an \textit{\acrfull{awgc}} \emph{if and only if} $l > l_1$ and $l < l_2$ and $W_l \cap (I \cup J) \neq \emptyset$ and $W_l \cap (K \cup J) \neq \emptyset$
\end{definition}

\begin{definition}[\acrfull{abt}]\label{def:abt}
Let $\hat{U}_l=\la I, J, K\ra$, such that $I \subseteq U, J \subseteq U, K \subseteq U$. An \textit{\acrfull{abt}}  is a pair  $\langle \ell, \hat{U}_l\rangle$,
where $\ell=(l_1, l_2, l_3)$ is a triple on $L$, $l_1 < l_2 < l_3$ and for all $\la I, J, K\ra$ and for all $\mu=(u_i, u_j, u_k)$ such that $u_i \in I, u_j \in J, u_k \in K$, $BT_{\ell}^{\mu}\in \bt$.
\end{definition}

\begin{figure}[h!]
\centering
\inputtikz{bipartite_abt_a}
\caption[{[\acrshort{iebt}] Example Aggregated bitriangle}]{%
\acrlong{awgc} $\la b, \{1,5\}\ra$ connects to \acrlong{adwg} $\la (a,c), \la \emptyset, \{1,3,7\}, \{5,9\}\ra \ra$ of \autoref{fig:bipartite-graph-example}, constructing the \acrlong{abt} $\la (a,b,c), \la \{1\},\{1,3,7\},\{1,5\} \ra \ra$.
Note that $\hat{U}_{(a,b,c)}=\{I,J,K\}$ does not contains disjoint sets according to \dref{def:abt}.
}
\label{fig:agg-bt-example}
\end{figure}

As we can see in \autoref{fig:agg-bt-example}, we have all the structures at our disposal to build a \acrshort{bt}.
In \autoref{fig:bt-a-example}, the two possible \acrshort{bt} can be extracted from the \acrshort{abt} presented in \autoref{fig:agg-bt-example} are $(1,a,3,c,5,b,1)$ and $(1,a,7,c,5,b,1)$.

\acrshort{iebt} will allow to enumerate \acrshort{bt} according to some locality criteria.
In this regards, we next define the \emph{query operators} to be used for that purpose.

\begin{definition}[\acrfull{qo}]\label{def:query:match}
a \emph{\acrlong{qo}} $Q$ is a value from the  \textit{sum type}
$\mathcal{P}(U + L) + \mathcal{P}(E)$ producing as a result a possible set of \acrshort{bt} that include any of the vertices or edges given in $Q$.
\end{definition}

In \autoref{table:notation}, we present a summary of the notation used in this chapter.

\begin{table}[!ht]
\centering
\begin{tabular}{|c|l|} \hline
\textbf{Notation} & \textbf{Meaning}\\ \hline
$G=((U\cup L),E)$ & a bipartite graph\\  \hline
$n,m$ & the number of vertices and edges in $G$, resp.\\  \hline
$(u,l)$ & an edge between vertices $u$ and $l$\\  \hline
$(u_1,l,u_2)$ & a wedge with  middle vertex $l$\\  \hline
$\la l, W_l \ra$ & an \acrshort{awg}\\  \hline
$\aw$ & the set of all the possible \acrshort{awg} in $G$\\  \hline
$(u_1,l_1,u_2,l_2,u_3)$ & a \acrshort{dwg} with middle vertices $l_1$ and $l_2$\\  \hline
$\la (l_1, l_2), U_l \ra$ & an \acrshort{adwg}\\  \hline
$\dw$ & the set of all the possible \acrshort{adwg} in $G$\\  \hline
$\dwi$ & Subset of $\dw$, such that $\dwi \subseteq \dw$ \\  \hline
$\la (l_1, l_2,l_3), \hat{U}_l \ra$ & an \acrshort{abt}\\  \hline
$\at$ & the set of all the possible \acrshort{abt} in $G$ \\  \hline
$\ati$ & Subset of $\at$, such that $\ati \subseteq \at$ \\  \hline
$\bti$ & the \acrshort{bt} $u_1,l_1,u_2,l_3,u_3,l_2,u_1$\\  \hline
$\bt$ & the set of all the possible \acrshort{bt} in $G$ \\  \hline
$bt$ & Subset of $\bt$, such that $bt \subseteq \bt$ \\  \hline
$\mathcal{P}$ & Parts of set \\  \hline
\end{tabular}
\caption[{[\acrshort{iebt}] Summary of notations and their meanings}]{This table summarizes all the different object definitions that we have detailed in \autoref{sec:prem:def}. The first column describe the formal term used in the definitions on \autoref{sec:prem:def}. The second column summarize the meaning of each term}
\label{table:notation}
\end{table}

\section{Algorithm Sketch}\label{sub:sec:algo-sketch}
The algorithm for enumerating incrementally bitriangles in a large bipartite graph consists in two main phases.
During the first phase, the bipartite network is received by the $\dpbt$ as a stream of edges and a graph index structure is created.
This graph index is represented by the different structures stored along the filters stages of the $\dpbt$.
This first phase is in charge of constructing the compressed structures \emph{double wedge}, \emph{aggregated double-wedge}, and finally \emph{aggregated bitriangles}.
The second phase is a querying phase. During the querying phase local queries can be submitted to the $\dpbt$.
When a query arrives to the $\dpbt$, the enumeration incremental of bitriangles process --according to the criteria in the submitted  queries-- is launched.
In this phase, enumerated bitriangles are extracted from the different aggregated bitriangles occurring in the graph index, i.e. the aggregated bitriangles stored in filter stages.
More concretely, the Dynamic Pipeline Algorithm for Enumerating Bitriangles ($\dpbt$) is defined in terms of the behavior of its four kinds stages: \textit{Source} ($\ibt$),
\textit{Generator} ($\gbt$),  \textit{Sink} ($\obt$), and \textit{Filter}($\fbt$) stages.
The algorithm considers a \acrlong{bt} as a convenient composition of three wedges as we can see the example in \autoref{fig:bitriangle-example}.
In order to reduce memory footprint, the algorithm aggregates results, i.e. the set of wedges having the same middle vertex is represented as a pair $\la l, W_l \ra$ where $l$ is the middle vertex and $W_l$ is the set of adjacent vertices of $l$ called \acrfull{awg} (see \dref{def:awg}).
The algorithm first collects \acrshort{awg} for every vertex in the $L$ set of the graph. Afterwards it constructs \acrfull{adwg} for every pair of distinct vertices, Finally  constructs \acrshort{abt}  for selected triples of vertices.
The following \autoref{table:channels} describes the different channels that are connecting the stages in $\dpbt$.

\begin{table}[ht!]
\centering
\begin{tabular}{|p{0.2\linewidth}|p{0.8\linewidth}|} \hline
\textbf{Channel} & \textbf{Meaning}\\ \hline
$C = \la IC, OC \ra$ & A Channel pair that connects input and output channel\\ \hline
$OC$ & Set of Output Channels \\ \hline
$C_E$ & Channel of $e \in E$ \\ \hline
$IC_E$ & Input Channels carrying $e \in E$ \\ \hline
$OC_E$ & Output Channels carrying $e \in E$ \\ \hline
$C_{W_l1}$ & Channel of \acrshort{awg} \\ \hline
$IC_{W_l1}$ & Input Channels carrying \acrshort{awg} \\ \hline
$OC_{W_l1}$ & Output Channels carrying \acrshort{awg} \\ \hline
$C_{W_l2}$ & Channel of \acrshort{awg} \\ \hline
$IC_{W_l2}$ & Input Channels carrying \acrshort{awg} \\ \hline
$OC_{W_l2}$ & Output Channels carrying \acrshort{awg} \\ \hline
$C_Q$ & Channel of \acrshort{qo} \\ \hline
$IC_Q$ & Input Channels carrying \acrshort{qo} \\ \hline
$OC_Q$ & Output Channels carrying \acrshort{qo} \\ \hline
$C_{BT}$ & Channel of $\bt$ \\ \hline
$IC_{BT}$ & Input Channels carrying $\bt$ \\ \hline
$OC_{BT}$ & Output Channels carrying $\bt$ \\ \hline
\end{tabular}
\caption[{[\acrshort{iebt}] Summary of Channels used in \acrshort{dpbt}}]{Summary of Channels used in \acrshort{dpbt}. The subindex on the Channel name indicates the element type this channel is carrying, either producing or consuming. Channels prefixed with $C$ are a generic form of denominating a channel independently of it is a producing or consuming. Channels prefixed with $IC$ are Input Channels or Consumers. Channels prefixed with $OC$ are Output Channels or Producers.}
\label{table:channels}
\end{table}

\begin{figure}[h]
\centering
\resizebox{1\textwidth}{!}{%
\inputtikz{btDP}
}
\caption[{[\acrshort{iebt}] $\dpbt$ Initial setup}]{Example of an initial setup of $\dpbt$. The text between dotted lines indicates the incoming data from an external source such as a file, socket, or any other. There are two incoming sources because one of them is carrying the edges and the other the commands \acrshort{qo}. Thick black double lines at the most right indicate the output targe that can be file, socket, screen, or any other. There is no data there because it is the initial step of \acrshort{dpbt}. At this initial step only $\ibt$, $\gbt$, and $\obt$ are set up with initial channels.}
\label{fig:btDP}
\end{figure}

The setup of $\dpbt$ can be appreciated in \autoref{fig:btDP}. $\ibt$ reads from input stream all $(u,l) \in E$ and transfers to the following stage each $(u,l)$ using $C_E$.
For every $(u,l)$ that arrives to $\gbt$, a new $\fbt$ instance with parameter $l \in L$ is spawn. $\fbt$ contains four actors.
First, $\aaa$ receives from $IC_E$ the edges and builds aggregated wedges, when there are no more edges it downstreams it to it's neighbour $\fbt$ using $OC_{W1}$.
Then $\ab$ receives from $IC_{W1}$ aggregated wedges from previous $\fbt$, downstream to next $\fbt$ and $\gbt$ using $OC_{W1}$ and at the same time use its information to see
if it can build an aggregated double-wedge. If an aggregated double-wedge could be constructed, it will be stored in the filter state.
The need of bypassing all the aggregated wedges up to $\gbt$ is for retro feeding all the pipeline with all the aggregated wedges a second time, in order to find an \acrfull{awgc} as we defined in \dref{def:awgc}.
From retro feeding channel $IC_{W2}$, $\ac$ receives all the \acrshort{awg}  and builds \acrshort{abt} storing them in $\st$.
$Q$ Commands are downstream from $\ibt$ to $\ad$ using channel $IC_Q$. $\ad$ receives all the commands, and for each of them, if there is match according to \dref{def:query:match}, it enumerates those \acrshort{bt} extracted from $\st$ and downstream to $\obt$ through $OC_{BT}$.

\section{Dynamic Pipeline for Enumerating Bitriangles}\label{sub:sec:iebt:dpalgo}
In this section, we present all the pseudo-code definitions of each of the stages described in ~\autoref{sub:sec:algo-sketch}.
In spite of this work has been implemented using \acrshort{hs}, and in particular \acrshort{dpfh}, the pseudo-code algorithms presented here are language independent.
Before starting with the details, we introduce in \autoref{table:aux:fn} auxiliary functions that we use in the pseudo-code to help understanding better the desired behavior of the algorithm.

\begin{table}[!htp]
\centering
\begin{tabular}{|p{0.3\linewidth}|p{0.7\linewidth}|} \hline
\textbf{Function} & \textbf{Meaning}\\ \hline
$\sw(F,l,\st)$ & Spawn new filter instance with parameters $F$, $l \in L$ and $\st$ as the State of the Filter\\\hline
$\fd$ & Kill this filter instance because PostCondition is not satisfied\\ \hline
$\fid$ & State after calling $\fd$ on filter. Indicates if Filter is die or not. If it is dead, this filter instance does not participate anymore in the pipeline streaming processor\\ \hline
$\gs$ & Get Current State $\st$ for Filter Instance \\ \hline
$\us(\st)$ & Update Current State $\st$ for Filter Instance \\ \hline
$\p(\mathtt{v}, OC_x)$ & push some value $\mathtt{v}$ to some Output Channel $OC_x$ \\\hline
$\mt(Q, BT)$ & Check if a \acrshort{qo} $Q$ produces $\bt$ \\ \hline
\end{tabular}
\caption[{[\acrshort{iebt}] Summary of auxiliary functions for handling \acrshort{dpbt} internals}]{This table shows some auxiliary functions that are used to help understanding the pseudo-code general behavior. Depending on the language implementation chosen these functions might not exists at all, but we generalize here in order to describe a pseudo-code language-independent. For example in our \acrshort{hs} implementation there is not $\fd$ because functions in \acrshort{hs} clean after execution finish automatically by \acrshort{ghc}}
\label{table:aux:fn}
\end{table}

\begin{algorithm}[h!]
\SetKwInOut{P}{Input Data}
\SetKwInOut{Q}{Input Commands}
\SetKwInOut{IC}{Input Channels}
\SetKwInOut{OC}{Output Channels}
\SetAlgorithmName{}{src}{}
\SetAlgoRefName{[A1]}
\P{$IO_E$: File or Input Stream with Set of Edges $E$}
\Q{$IO_Q$: File or Input Stream with \acrlong{qo} $Q$}
\IC{$IC = \la IC_{W_l2} \ra$}
\OC{$ OC = \la OC_E, OC_{W_l1}, OC_{W_l2}, OC_Q, OC_{BT} \ra $}
\ForAll(\tcp*[f]{Edges to Generator/Filter}){$(u,l) \in IO_E$}
{$\p((u,l), OC_E)$ \label{algo:source:1}
}
\ForAll(\tcp*[f]{Feedback from Generator to Filter}){$\la l, W_l \ra \in IC_{W_l2}$}
{$\p(\la l, W_l \ra, OC_{W_l2})$ \label{algo:source:2}
}
\ForAll(\tcp*[f]{Send Query Commands}){$Q \in IO_Q$}
{$\p(Q, OC_Q)$ \label{algo:source:3}
}
\caption[Source ($\ibt$)]{Source ($\ibt$): It process all the edges from the file or input channel $IO_E$ and send to the following stages. It also receives from $IC_{W_l2}$ all the feedback \acrshort{awg} that is sending back the $\gbt$. At the end it process also from other File or Input stream the Query Command $Q$ to be sent to th filters}
\label{algo:source}
\end{algorithm}

\paragraph{Source $\ibt$} In \autoref{algo:source} we can see in \autoref{algo:source:1} how the edges arriving from the input stream of the graph are downstream to the pipeline.
Another important part as well is \autoref{algo:source:2}, the $\ibt$ is retro feed with \acrshort{awg} stream that is generated during pipeline execution.
This is important to finally build the \acrshort{bt} as we have describe in \autoref{sub:sec:algo-sketch}. Finally, \autoref{algo:source:3} shows how all the queries are downstream as well.

\begin{algorithm}[h!]
\SetKwInOut{P}{Parameter}
\SetKwInOut{IC}{Input Channels}
\SetKwInOut{OC}{Output Channels}
\SetAlgorithmName{}{gen}{}
\SetAlgoRefName{[A2]}
\P{$F$}
\IC{$ IC = \la IC_E, IC_{W_l1}, IC_{W_l2}, IC_Q, IC_{BT} \ra $}
\OC{$ OC = \la OC_{W_l2}, OC_{BT} \ra$}
\ForAll{$(u,l) \in IC_E$}
{$\sw(F, l, \la l, \{u\} \ra)$ \label{algo:gen:1}
}
\ForAll(\tcp*[f]{Feedback channel to retrofit Source}){$\la l, W_l \ra \in IC_{W_l1}$}
{$\p(\la l, W_l \ra, OC_{W_l2})$ \label{algo:gen:2}
}
\ForAll{$\bti \in IC_{BT}$}
{$\p(\bti, OC_{BT})$ \label{algo:gen:3}
}
\caption[Generator ($\gbt$)]{Generator ($\gbt$): For each edge $(u,l)$ it receives from previous stage, it spawn a new filter using $l$ as parameter of the Filter and $\{u\}$ as the state. It also receives all the \acrshort{awg} that previous filters built and sends back to $\ibt$. Finally it sends to the $\obt$ the \acrshort{bt} matched by filters according to Command Query $Q$}
\label{algo:gen}
\end{algorithm}

\begin{figure}[h]
\centering
\resizebox{1\textwidth}{!}{%
\inputtikz{btDP_actor1}
}
\caption[{[\acrshort{iebt}] ]$\dpbt$ With Filter instances}]{This is the evolving state of the $\dpbt$ shows in \autoref{fig:btDP}. This image is showing what happen when a $\fbt$ is spawned. We can see how all the channels are set up in the middle of the spawned $\fbt$ and also between $\fbt$ and $\gbt$.}
\label{fig:btDP_actor1}
\end{figure}

\paragraph{Generator $\gbt$} $\gbt$ also have three main loops. In \autoref{algo:gen:1} it receives each edge $(u,l)$ not consumed by any already spawn  $\fbt$, and
spawns a new $\fbt$ using $l \in L$ as filter parameter and initializing $\st = \la l, \{u\} \ra$ as it can be seen in \autoref{fig:btDP_actor1}.
Spawn assumes that, the implementation will connect the channels to keep the downstream correct. Defines all the input channels of $\gbt$ as input channels of the newly spawn $\fbt$ and set $\fbt$'s output channels as $\gbt$ input channels.
In \autoref{algo:gen:2} we can see how the algorithm is receiving and retro feeding $\ibt$ with all the \acrshort{awg} produced by the $\fbt$ . This also assumes that $OC_{W2}$ in $\gbt$
is connected with $IC_{W2}$ in $\ibt$.
Finally, \autoref{algo:gen:3} sends all the results that $Q$ matches in the different filters to $\obt$.

\begin{algorithm}[h!]
\SetKwInOut{IC}{Input Channels}
\SetKwInOut{O}{Output}
\SetAlgorithmName{}{sink}{}
\SetAlgoRefName{[A3]}
\O{$IO_{BT}$: File or Output Stream with $BT$}
\IC{$IC = \la IC_{BT} \ra$}
\ForAll(\tcp*[f]{Read Results and put in $IO_{BT}$}){$\bti \in IC_{BT}$}
{$put(\bti, IO_{BT})$
}
\caption[Sink ($\obt$)]{Sink ($\obt$): It receives \acrshort{bt} from $\gbt$ and send to the File or Output Stream}
\label{algo:sink}
\end{algorithm}

\paragraph{Sink $\obt$} In \autoref{algo:sink} shows a simple stage that receives all results and sends them to some output handler (file, standard output, etc.).

\begin{algorithm}[h!]
\SetKwInOut{P}{Filter Parameter}
\SetKwInOut{FS}{Filter State}
\SetKwInOut{IC}{Input Channels}
\SetKwInOut{OC}{Output Channels}
\SetKwFunction{actora}{actor1}
\SetKwFunction{actorb}{actor2}
\SetKwFunction{actorc}{actor3}
\SetKwFunction{actord}{actor4}
\SetKwFunction{filter}{filter}
\SetKwProg{df}{def}{:}{end}
\SetAlgorithmName{}{fil}{}
\SetAlgoRefName{[A4]}
\P{$l \in L$}
\FS{$\st = \aw + \mathcal{P}(\dw) + \mathcal{P}(\at)$}
\IC{$ IC = \la IC_E, IC_{W_l1}, IC_{W_l2}, IC_Q, IC_{BT} \ra $}
\OC{$ OC = \la OC_E, OC_{W_l1}, OC_{W_l2}, OC_Q, OC_{BT} \ra $}
\df{\filter{}}{
      $\actora()$\\
      $\actorb()$\\
      $\actorc()$\\
      $\actord()$\\
}
\caption[Filter ($\fbt$)]{Filter ($\fbt$): Call sequentially to all the actors in this filter}
\label{algo:fil}
\end{algorithm}

\paragraph{Filter $\fbt$} In \autoref{algo:fil} we can see the simple call sequence over all the actor functions of the filter template.
The values stored in the \textit{state of the filter} with parameter $l$ are of the sum type $\st = \aw + \mathcal{P}(\dw) + \mathcal{P}(\at)$.

\begin{algorithm}[h!]
\SetKwInOut{P}{Parameter}
\SetKwInOut{IC}{Input Channels}
\SetKwInOut{OC}{Output Channels}
\SetKwInOut{FS}{$\st$}
\SetKwInOut{PC}{Post-Cond}
\SetKwInOut{PrC}{Pre-Cond}
\SetKwFunction{acta}{actor1}
\SetKwProg{df}{def}{:}{end}
\SetAlgorithmName{}{fa1}{}
\SetAlgoRefName{[A5]}
\P{$l \in L$}
\FS{$\la l, W_l \ra$}
\IC{$ IC = \la IC_E, IC_{W_l1}, IC_{W_l2}, IC_Q, IC_{BT} \ra $}
\OC{$ OC = \la OC_E, OC_{W_l1}, OC_{W_l2}, OC_Q, OC_{BT} \ra $}
\PC{$|W_l| > 1 \lor \fid$}
\BlankLine
\df{\acta{}}{
$\la l, W_l \ra \leftarrow \gs$\\
\ForAll{$(u',l') \in IC_E$}
{\uIf{$l = l'$}{
      $W_l \leftarrow W_l \cup \{u'\}$\label{algo:act-1:1}
}\Else{$\p((u',l'),OC_E)$}
}
\uIf{$|W_l| > 1$}{
      $\us(\la l, W_l \ra)$\\ \label{algo:act-1:2}
      $\p(\la l, W_l \ra, OC_{W_l1})$\\
}\Else{$\fd$}
}
\caption[Actor1 ($actor_1$)]{Actor1 ($actor_1$): Build a set of aggregated wedges. This is, $W_l \subseteq U$ adjacent to $l$ Filter parameter. For each received edge $(u',l')$ which $l \neq l'$ by pass the edge to next filters. It updates the State of the filter with $W_l$ if it could build a $W_l$ with more than 1 vertex in $U$.}
\label{algo:act-1}
\end{algorithm}

\paragraph{Filter-$\aaa$}
$\aaa$ receives all edges not consumed by previous $\fbt$.
If the received edge $(u',l')$ is incident to l i.e. $l = l'$,  then $u'$ will be added to the list of \acrshort{awg}. Otherwise the edge is downstream to the next stage.
In \autoref{algo:act-1:2} the state is updated and \acrshort{awg} downstream, if and only if at least 1 wedge was collected. Otherwise, the filter is marked as dead using the  $\fd$ function.

\begin{algorithm}[h!]
\SetKwInOut{P}{Parameter}
\SetKwInOut{IC}{Input Channels}
\SetKwInOut{OC}{Output Channels}
\SetKwInOut{FS}{$\st$}
\SetKwInOut{PC}{Post-Cond}
\SetKwInOut{PrC}{Pre-Cond}
\SetKwFunction{actb}{actor2}
\SetKwProg{df}{def}{:}{end}
\SetAlgorithmName{}{fa2}{}
\SetAlgoRefName{[A6]}
\P{$l \in L$}
\FS{$\la l, W_l \ra$}
\IC{$ IC = \la IC_E, IC_{W_l1}, IC_{W_l2}, IC_Q, IC_{BT} \ra $}
\OC{$ OC = \la OC_E, OC_{W_l1}, OC_{W_l2}, OC_Q, OC_{BT} \ra $}
\BlankLine
\PrC{$W_l \subseteq U, |W_l| > 1$}
\PC{$|\dwi| \geq 1 \lor \fid$}
\df{\actb{}}{
$\la l, W_l \ra \leftarrow \gs$\\
\ForAll{$\la l', W_l' \ra \in IC_{W_l1}$}{
      \tcp*[h]{Send Wedge from previous filters to next one}
      $\p(\la l', W_l \ra, OC_{W_l1})$\\
      $\dwi \leftarrow \emptyset$\\
      \If{$W_l' \cap W_l \neq \emptyset$}{ \label{algo:act-2:1}
            $(l_l, l_u) \leftarrow (\argmin_{l,l'}, \argmax_{l,l'})$\\
            \uIf{$l < l'$}{
                  $(W_{l_l}, W_{l_u}) \leftarrow (W_l, W_l')$
            }\Else{
                  $(W_{l_l}, W_{l_u}) \leftarrow (W_l', W_l)$
            }
            $I \leftarrow W_{l_l} \setminus W_{l_u}$\\ \label{algo:act-2:2}
            $J \leftarrow W_{l_l} \cap W_{l_u}$\\
            $K \leftarrow W_{l_u} \setminus W_{l_l}$\\ \label{algo:act-2:3}
            $U_l \leftarrow \la I, J, K\ra$\\
            $\dwi \leftarrow dw \cup \{\la (l_l, l_u), U_l \ra\}$
      }
}
\uIf{$\dwi = \emptyset$}{$\fd$}
\Else{$\us(\dwi)$}
}
\caption[Actor2 ($actor_2$)]{Actor2 ($actor_2$): Receiving all aggregated wedges from previous filters, build a set of all possible aggregated double-wedges $\dwi = \{\la (l,l'), U_l \ra\}, \dwi \subseteq \dw$, which first component $l$ is smallest between the Parameter of the Filter and the vertices from the incoming wedges. At the end, it updates the State of the filter with $\dwi$ if $\dwi \neq \emptyset$}
\label{algo:act-2}
\end{algorithm}

\paragraph{Filter-$\ab$} In \autoref{algo:act-2} \acrshort{adwg} is built. Following that idea and according to the \dref{def:adwg}, this algorithm will collect all the \acrshort{awg}
from previous filters if an only if the condition in \autoref{algo:act-2:1} is met. Note that the two \acrshort{awg}, $W_l$ and $W_l'$ have different vertices from the set $L$. $W_l$ intersection check is mandatory since if \acrshort{awg} are disjoint, we cannot aggregate them.
After this checking from \autoref{algo:act-2:2} to \autoref{algo:act-2:3}, the algorithm builds three disjoint Sets to separate upper edges in three subsets; those which are incident only of
$l$ and $l'$ which are $I$ and $K$ and those that are shared by both lower layer vertices. This can be appreciated in \autoref{fig:agg-double-wedge-example}.
Once \acrshort{adwg} is built $\st = \dwi$ updating the state for the next $\ac$.

\begin{algorithm}[h!]
\DontPrintSemicolon
\SetKwInOut{P}{Parameter}
\SetKwInOut{IC}{Input Channels}
\SetKwInOut{OC}{Output Channels}
\SetKwInOut{FS}{$\st$}
\SetKwInOut{PC}{Post-Cond}
\SetKwInOut{PrC}{Pre-Cond}
\SetKwFunction{actc}{actor3}
\SetKwProg{df}{def}{:}{end}
\SetAlgorithmName{}{fa3}{}
\SetAlgoRefName{[A7]}
\P{$l \in L$}
\FS{$\dwi \subseteq \dw$}
\IC{$ IC = \la IC_E, IC_{W_l1}, IC_{W_l2}, IC_Q, IC_{BT} \ra $}
\OC{$ OC = \la OC_E, OC_{W_l1}, OC_{W_l2}, OC_Q, OC_{BT} \ra $}
\BlankLine
\PrC{$|\dwi| \geq 1$}
\PC{$|\ati| \geq 1 \lor \fid$}
\df{\actc{}}{
      $\dwi \leftarrow \gs$\\
      $\ati \leftarrow \emptyset$\\
      \ForAll{$\la l', W_l \ra \in IC_{W_l2}$}{
            \tcp*[h]{By pass to be used by following Filters}
            $\p(\la l', W_l \ra, OC_{W_l2})$\\
            \tcp*[h]{For each double wedge in State}\\
            \ForEach{$\la (l_l, l_u), \la I, J, K \ra \ra \in \dwi, l_l < l' \land l_u > l'$}{
                  $I' \leftarrow I \cup J$\\
                  $K' \leftarrow K \cup J$\\
                  \If{$W_l \cap I' \neq \emptyset \land W_l \cap K' \neq \emptyset$}{
                        $I' \leftarrow I' \cap W_l$\\
                        $K' \leftarrow K' \cap W_l$\\
                        $\hat{U}_l  \leftarrow \la I', J, K' \ra$\\
                        $\ati \leftarrow \ati \cup \big\{\la (l_l, l', l_u), \hat{U}_l \ra\big\}$
                  }
            }
      }
      \uIf{$\ati = \emptyset$}{$\fd$}
      \Else{$\us(\ati)$\\}
}
\caption[Actor3 ($actor_3$)]{Actor3 ($actor_3$): Receiving all aggregated wedges that came from feedback channel, build a Set of all possible Aggregated bitriangles $\ati = \{\la (l_l, l_m, l_u), U_l \ra\}, \ati \subseteq \at$, , such that $l = l_l \lor l = l_u$, where $l$ is the Filter Parameter and $l_m$ is the middle vertex of the incoming wedge. At the end, it updates the State of the filter with $\ati$ if $\ati \neq \emptyset$}
\label{algo:act-3}
\end{algorithm}

\paragraph{Filter-$\ac$} $\ac$ focuses on treating elements from feedback channel $IC_{W2}$ which is going to downstream all the \acrshort{awg} of all filters.
This is because in order to build \acrshort{abt} finding all the possibles \acrshort{awgc}. This is what is doing \autoref{algo:act-3} according to definition
\dref{def:awgc} and \dref{def:abt}. If that can be achieved, algorithm sets $\st = \ati$ and $\ad$ can be executed.

\begin{algorithm}[h!]
\SetKwInOut{P}{Parameter}
\SetKwInOut{IC}{Input Channels}
\SetKwInOut{OC}{Output Channels}
\SetKwInOut{FS}{$\st$}
\SetKwInOut{PC}{Post-Cond}
\SetKwInOut{PrC}{Pre-Cond}
\SetKwFunction{actc}{actor4}
\SetKwFunction{butV}{buildBtVertex}
\SetKwFunction{butE}{buildBtEdge}
\SetKwProg{df}{def}{:}{end}
\SetAlgorithmName{}{fa4}{}
\SetAlgoRefName{[A8]}
\P{$l \in L$}
\FS{$\ati \subseteq \at$}
\IC{$ IC = \la IC_E, IC_{W_l1}, IC_{W_l2}, IC_Q, IC_{BT} \ra $}
\OC{$ OC = \la OC_E, OC_{W_l1}, OC_{W_l2}, OC_Q, OC_{BT} \ra $}
\BlankLine
\PrC{$|\ati| \geq 1$}
\df{\actc{}}{
$\ati \leftarrow \gs$\\
\ForAll{$Q \in IC_Q$}{
      \ForEach{$\la (l_l, l_m, l_u), \la I,J,K \ra \ra \in \ati$}{
            \Switch{$Q$}{
                  \Case{$\mathcal{P}(U + L)$}{
                        \If{$\mathcal{P}(U + L) \cap \{l_l, l_m, l_u\} \neq \emptyset \lor \mathcal{P}(U + L) \cap (I \cup J \cup K) \neq \emptyset$}{
                              $\btii \leftarrow \butV(\la (l_l, l_m, l_u), \la I,J,K \ra \ra), \mathcal{P}(U + L))$\\
                              \ForAll{$\bti \in \btii$}{
                                    $\p(\bti, OC_{BT})$
                              }
                        }
                  }
                  \Case{$\mathcal{P}(E)$}{
                        \ForEach{$(u,l) \in \mathcal{P}(E)$}{
                              \If{$(l = l_l \lor l = l_m \lor l = l_u) \land u \in (I \cup J \cup K)$}{
                                    $\btii \leftarrow \butE(\la (l_l, l_m, l_u), \la I,J,K \ra \ra), (u,l))$\\
                                    \ForAll{$\bti \in \btii$}{
                                          $\p(\bti, OC_{BT})$
                                    }
                              }
                        }
                  }
            }
      }
}
}
\caption[Actor4 ($actor_4$)]{Actor4 ($actor_4$): Receives all the \acrlong{qo} $Q$ that arrives from channel $IC_Q$. For each \acrshort{qo} $Q$ it builds the bitriangles from $\ati \subseteq at$ according to the query, and downstream to channel $OC_{BT}$ to be processed by the Sink}

\label{algo:act-4}
\end{algorithm}

\paragraph{Filter-$\ad$} Once the execution reaches $\ad$, it is ready for processing $Q$ \acrlong{qo}.
Since we have a compressed representation of \acrshort{bt}, which is a similar idea exposed here~\cite{Lai}, we need to enumerate incrementally all the \acrshort{bt} present in the compressed format and filter the ones that match the command.
The matches proceeds in the following manner. Given a \acrshort{abt} with the form $\la \ell, \hat{U}_l \ra$, such that $\ell = (l_1,l_2,l_3)$ and $\hat{U}_l = \la I,J,K \ra$, where $I \subseteq U, J \subseteq U, K \subseteq U$, if the \acrshort{qo} contains a command with vertices, those vertices are searched in $\ell$ and $\hat{U}_l$.
There are two posibilities. On the first case, if any of those vertices that belongs to $Q$ matches $\ell$, then forall $u_i \in I, u_j \in J, u_k \in K$ where $ u_i \neq u_j \neq u_k$, a \acrshort{bt} is built using a 6-cycle $(u_i, l_1, u_j, l_3, u_k, l_2, u_i)$ path.
On the second case, if any of those vertices that belongs to $Q$ matches on some vertex in $\hat{U}_l$, build only the \acrshort{bt} in which 6-cycle $(u_i, l_1, u_j, l_3, u_k, l_2, u_i)$ path contains that vertex in $Q$.
Using the same construction process a \acrshort{bt} is built if \acrshort{qo} contains a command with edges and any of those edges matches with any of the possible \acrshort{bt} 6-cycle path.

The following pseudo-code definitions on \autoref{algo:buildBtVertex} and \autoref{algo:buildBtEdge} are auxiliary functions that are called
from $\ad$ if $Q$ pattern match either with $\mathcal{P}(U + L)$ or $\mathcal{P}(E)$.

\begin{algorithm}[h!]
\SetKwInOut{I}{Input}
\SetKwInOut{O}{Output}
\SetKwFunction{but}{buildBtVertex}
\SetKwProg{df}{def}{:}{end}
\SetAlgorithmName{}{a1}{}
\SetAlgoRefName{[A9]}
\I{$\la (l_l, l_m, l_u), \la I,J,K \ra \ra \in \ati, \ati \subseteq \at$}
\I{$\mathcal{P}(U + L)$}
\O{$\btii \subseteq \bt$ or $\emptyset$ if cannot build any $\btii$}
\df{\but{$\la (l_l, l_m, l_u), \la I,J,K \ra \ra$, $\mathcal{P}(U + L)$}}{
      $\btii \leftarrow \emptyset$\\
      \uIf{$\mathcal{P}(U + L) \cap \{l_l, l_m, l_u\}$}{
            \tcp*[h]{If it is in lower i need to build all for this lower triplet}\\
            \ForEach{$i \in I$}{
                  \ForEach{$j \in J, j \neq i$}{
                        \ForEach{$k \in K, k \neq i \land k \neq j$}{
                              $\btii \leftarrow \btii \cup \{BT_{(l_l, l_m,l_u)}^{(i, j, k)}\}$
                        }
                  }
            }
      }\Else{
            \tcp*[h]{Otherwise just build those that are in the this upper $v$}\\
            \ForEach{$i \in I$}{
                  \ForEach{$j \in J, j \neq i$}{
                        \ForEach{$k \in K, k \neq i \land k \neq j \land (\mathcal{P}(U + L) \cap \{i,j,k\} \neq \emptyset$}{
                              $\btii \leftarrow \btii \cup \{BT_{(l_l, l_m,l_u)}^{(i, j, k)}\}$
                        }
                  }
            }
      }
      \Return{$\btii$}
}
\caption[Function \mintinline{shell}{buildBtVertex}]{Function \mintinline{shell}{buildBtVertex}: Given a Set of Vertex in $\mathcal{P}(U + L)$, if any of those vertices matches with some vertex in the parameter of the function $\la (l_l, l_m, l_u), \la I,J,K \ra \ra$, build the set of bitriangles that matches that query}
\label{algo:buildBtVertex}
\end{algorithm}

\begin{algorithm}[h!]
\SetKwInOut{I}{Input}
\SetKwInOut{O}{Output}
\SetKwFunction{but}{buildBtEdge}
\SetKwFunction{he}{hasEdge}
\SetKwProg{df}{def}{:}{end}
\SetAlgorithmName{}{a2}{}
\SetAlgoRefName{[A10]}
\I{$\la (l_l, l_m, l_u), \la I,J,K \ra \ra \in \ati, \ati \subseteq \at$}
\I{$(u,l)$}
\O{$\btii \subseteq \bt$ or $\emptyset$ if cannot build any $\btii$}
\df{\but{$\la (l_l, l_m, l_u), \la I,J,K \ra \ra$, $\mathcal{P}(U + L)$}}{
      $\btii \leftarrow \emptyset$\\
      \ForEach{$i \in I$}{
            \ForEach{$j \in J, j \neq i$}{
                  \ForEach{$k \in K, k \neq i \land k \neq j \land \he((u,l), (l_l,l_m,l_u),(i,j,k))$}{
                        $\btii \leftarrow \btii \cup \{BT_{(l_l, l_m,l_u)}^{(i, j, k)}\}$
                  }
            }
      }
      \Return{$\btii$}
}
\df{\he{$(u,l), (l_l,l_m,l_u), (i,j,k)$}}{
      \If{$(u,l) = (l_l, i) \lor (u,l) = (l_l, j) \lor (u,l) = (l_u, j) \lor (u,l) = (l_u, k) \lor (u,l) = (l_m, i) \lor (u,l) = (l_m, k)$}{
            \Return{True}
      }\Else{
            \Return{False}
      }
}
\caption[Function \mintinline{shell}{buildBtEdge}]{Function \mintinline{shell}{buildBtEdge}: Given a Set of Edges in $\mathcal{P}(E)$, if any of those edges matches with some edge in the parameter of the function $\la (l_l, l_m, l_u), \la I,J,K \ra \ra$, build the set of bitriangles that matches that query}
\label{algo:buildBtEdge}
\end{algorithm}

\iffalse
\clearpage
\section{DRAFT - Complexity Analysis of Algorithm}
The complexity analysis provided in this section will cover each of the principal pseudo-code algorithms described in the previous \autoref{sub:sec:iebt:dpalgo}, this is \autoref{algo:source}, \autoref{algo:gen}, \autoref{algo:sink} and \autoref{algo:fil} (including all actors).

\begin{theorem}[$|\aw| \leq |L|$]\label{theorem:awg}
The size of all possible aggregated wedges is upper bound by $|L|$.
\end{theorem}
\begin{proof}[Anlaysis]
This can be deduced by \dref{def:awg}, because at most each $l \in L$ is participating on a wedge.
\end{proof}

\begin{theorem}[$|\dw| \leq \binom{|L|}{2}$]\label{theorem:adwg}
The size of all possible aggregated double-wedges is upper bound by $\binom{|L|}{2}$.
\end{theorem}
\begin{proof}[Anlaysis]
This can be deduced by \dref{def:adwg}, and by \autoref{theorem:awg}. If we can combine vertices from $L$ taken by $2$ without repetition that forms an \acrshort{adwg}, we have at most that amount of possible aggregated double-wedges combinations.
\end{proof}

\begin{complexity}[Source $\ibt$]\label{prop:comp-src}
The total complexity of $\ibt$ algorithm in worst-case is $O(|E|)$.
\end{complexity}
\begin{proof}[Anlaysis]
In the case of this stage, we have three \texttt{for} loops. The first one is done in the number of edges of the graph $|E|$. The second \texttt{for} loop iterates over
all possible \acrshort{awg} which is at most $|L|$ according to \autoref{theorem:awg}.
And the last \texttt{for} loop is taking the size of all commands $Q$ provided by the user. Then, worst-case complexity is $O(|E| + |L| + |Q|)$.
Since $|L| \leq |E|$ by \dref{def:bt} and $|Q| << |E|$ because otherwise the user will be able to manually inspect the whole graph. Since the algorithm implements a \emph{pay-as-you-go} model
as we have described in \autoref{relate-work}, it has a high cost for the user to ask for an amount of commands near to $|E|$.
Therefore, the dominant term is $|E|$, and the complexity of this part of the algorithm is $O(|E|)$.
\end{proof}

\begin{complexity}[Generator $\gbt$]\label{comp:anal:gen}
The total complexity of $\gbt$ algorithm in worst-case is $O(2|L| + |Q||\ati|)$.
\end{complexity}
\begin{proof}[Anlaysis]
$\gbt$ receives at most $|L|$ edges in in \autoref{algo:gen:1} first loop, since $\aaa$ will collect all the upper vertices from the same $l$ without bypassing those collected edges (\autoref{theorem:awg}).
In \autoref{algo:gen:2}, the second loop, it will receive at most $|L|$ aggregated wedges $W_l$, which also can be deduced by \autoref{theorem:awg}.
Finally, it will receive all matched $\btii \subseteq \bt$. By \autoref{algo:act-4}, \autoref{algo:buildBtEdge} and \autoref{algo:buildBtVertex} the amount of possible $\btii$ matched is $|Q| \times |\ati|$ (see \dref{def:abt}).
Then worst-case complexity is $O(|L| + |L| + |Q||\ati|) = O(2|L| + |Q||\ati|)$.
\end{proof}

\begin{complexity}[Sink $\obt$]
The total complexity of $\obt$ algorithm in worst-case is $O(|Q||\ati|)$.
\end{complexity}
\begin{proof}[Anlaysis]
Obvious by definition of \autoref{algo:sink} and by complexity analysis in \autoref{comp:anal:gen}.
\end{proof}

\paragraph{Filter Consideration}\label{comp:an:filter:note} In the case of complexity analysis of the Filter instances $\fbt$ algorithms, which involves the four actors, we are going to take into consideration the worst-case.
The worst-case is the first filter, because it is the one that is going to receive all the edges from the $\ibt$ at least as we can see in \autoref{sub:sec:algo-sketch}.
If the second filter is spawned, is going to receive at most $|E|-2$ edges, the third $|E|-3$, and so on.

\begin{complexity}[Filter $\fbt$]
The total complexity of $\fbt$ algorithm in worst-case is $O(|E| + |L|^2 \times 6|U| + |Q||\ati|)$.
\end{complexity}
\begin{proof}[Anlaysis]
Since actors in filters are executed sequentially, lets analyze each actor separately.
The complexity of $\aaa$ is $O(|E|)$ by \dref{algo:act-1}.
This worst-case complexity of $|E|$ for $\aaa$ only happens in the first filter as we have explained before in \ref{comp:an:filter:note}.
Then for $\ab$ complexity is $O(|L| \times 6|U|)$. $|L|$ is the upper bound by \autoref{theorem:awg} since the \texttt{for} loop is iterating over all $W_l$ received from the channel.
Inside that \texttt{for} loop there 3 sets operations are taking place. Each of those operations are upper bounded by $|U|$ in the worst-case, since $W_l \subseteq U$ by \dref{def:awg}.
Then, the cost of each loop is $O(6|U|)$ worst-case.
The complexity of $\ac$ is $O(|L| \times |L| \times 6|U|)$. The first $|L|$ is the outer \texttt{for} loop which receives all the possible \acrshort{awg}, which is upper bounded by $|L|$ according to \autoref{theorem:awg}.
$|L|$ is the second \texttt{for} loop that is iterating over all the \acrshort{adwg} that are in this filter, and it is upper bounded by $|L|-1$ because all possible $\dwi \subseteq \dw$ in one filter has the $l \in L$ parameter of the filter, which can be possible combine with the rest of $|L|-1$ lower layer verticesin the worst case.
Then, $6|U|$ is inside of the second \texttt{for} loop and follows the same analisys for the set operations provided in $\ab$. Finally, $\ad$ takes $O(|Q||\ati|)$ as we have described in $\gbt$ complexity analysis.
Gluing everything together the worst-case complexity of four actors in the filter instance is $O(|E| + |L| \times 6|U| + |L| \times |L| \times 6|U| + |Q||\ati|)$.
Since $|L| \times |L| \times 6|U| > |L| \times 6|U|$, we can eliminate that term which is not dominating. Therefore $O(|E| + |L|^2 \times 6|U| + |Q||\ati|)$.

\end{proof}
\fi

\clearpage
\section{Correctness of the Algorithm}
Given a bipartite graph $G$ we prove the algorithm enumerates all the bitriangles in $G$  without duplicating them.
First, we prove that if $BT_{\{l_1,l_2,l_3\}}^{\{u_1,u_2,u_3\}} = (u_1,l_1,u_2,l_3,u_3,l_2,u_1)$ is a bitriangle occurring in $G$, then only one of the feasible permutations of this 6-cycle is contained in an aggregated bitriangle of $\at$  (\autoref{TH-uniqueness}). Second, we prove that  all the bitriangles occuring in $G$ are contained in an aggregate bitriangles of $\at$ (\autoref{TH-all}).
%
\begin{theorem}[Uniqueness] \label{TH-uniqueness}
Given a bipartite graph $G = ((U\cup L),E)$, $\forall \btii\in\bt$  \acrshort{iebt} stores $\btii$ in an $\ati\in \at$ only once.

\end{theorem}
\begin{proof}
Let $\btii\in \bt$, $\btii = (u_1,l_1,u_2,l_3,u_3,l_2,u_1)$. Let us suppose that \acrshort{iebt} stores two different feasible permutations of $\btii$, $\btii$$_1= (u_1',l_1',u_2',l_3',u_3',l_2',u_1')$ and $\btii$$_2 = (u_1'',l_1'',u_2'',l_3'',$ $u_3'',l_2'',u_1'')$ in some aggregated bitriangles. By Definition \ref{def:abt} and $\ac$ line 7, $l_1'<l_2'<l_3'$ and $l_1''<l_2''<l_3''$. The fact that $\btii$$_1 \neq \btii$$_2$ implies that $(l_1',l_2',l_3') \neq (l_1'',l_2'',l_3'')$. This means that  $(l_1',l_2',l_3')$ is a permutation of $(l_1'',l_2'',l_3'')$ (and viceversa). Thus, because of the strict order defined on $L$,  just one, either   $l_1'<l_2'<l_3'$ or $l_1''<l_2''<l_3''$ holds. Therefore, only one of the triples $(l_1',l_2',l_3')$ or $(l_1'',l_2'',l_3'')$  is used in lines 7--16 in $\ac$ to include $\btii = (u_1,l_1,u_2,l_3,u_3,l_2,u_1)$ in an aggregated bitriangle
\end{proof}

\begin{theorem}\label{TH-all}Given a bipartite graph $G$, if the bitriangle $\btii = (u_1,l_1,u_2,l_3,$ $u_3,l_2,u_1)\in \bt$, then $\btii$  can be enumerated.
\end{theorem}

\begin{proof}
We proceed in a constructive way. Let $\btii = (u_1,l_1,u_2,l_3,u_3,l_2,u_1) \in \bt$.
When $\aaa$ running in a filter $\sfilter$ with parameter $l_1$, $F_{l_1}$, ends reading all the edges, the state of $F_{l_1} = \langle l_1,  W_{l_1}\rangle$ and  $\{u_1,u_2\} \subseteq W_{l_1}$. Similarly,  when $\aaa$ running in filter $F_{l_3}$ ends reading all the edges,  $F_{l_3} = \langle l_3,  W_{l_3}\rangle$ and $\{u_2,u_3\} \subseteq W_{l_3}$. This is, wedges $(u_1,l_1,u_2)$ and $(u_2,l_3,u_3)$ are stored in the aggregated wedges (states) of $F_{l_1}$ and $F_{l_3}$, respectively.
When running  $\ab$ in filter $F_{l_1}$ or in filter $F_{l_3}$, in lines 7-12 the pair
$(l_l, l_u) \equiv (\argmin_{l_1,l_3}, \argmax_{l_1,l_3})$ is added to $\mathsf{dw}$. This is, the double wedge $(u_1,l_l,u_2,l_u,u_3)$ is stored either, in the state $F_{l_1}$ or the state of $F_{l_3}$. When running $\ac$, in filter $F_{l_1}$ or in filter $F_{l_3}$, $\ac$ receives in $IC_{W_l2}$, line 4, the connector wedge $(l_2, W_{l_2})$ from the incoming  aggregated BT-connector. In lines 7-16, if the condition in line 10 holds, i.e. non empty intersection, since edges $(u_1,l_2)$ and $(u_3,l_2)$ are in $E$, $\{u_1,u_3\} \subseteq W_{l_2}$. Therefore, $\ac$ adds  $\btii$  to the aggregated bitriangle of the filter. This is, the permutation  $(u_1,l_l,u_2,l_u,u_3,l_2,u_1)$ of the bitriangle $\btii$ is stored in $\ati$ and therefore it can be listed by $\ad$
\end{proof}

\section{\acrshort{dpbt} implementation}\label{sec:iebt:hs:imp}
As we have seen in the previous \autoref{dp-hs}, first we need to define the $\dpbt$ using the \acrshort{dsl}.

\begin{listing}[htp!]
\begin{minted}[fontsize=\small,numbers=left,breaklines,frame=lines,framerule=2pt,framesep=2mm,baselinestretch=1.2,highlightlines={3}]{haskell}

type DPBT = Source (Channel (Edge :<+> W :<+> Q :<+> BT :<+> BTResult :<+> W :<+> Eof))
      :=> Generator (Channel (Edge :<+> W :<+> Q :<+> BT :<+> BTResult :<+> Eof))
      :=> FeedbackChannel (W :<+> Eof)
      :=> Sink

\end{minted}
\caption{[\mintinline{shell}{BTriangle.hs}] Enconding of \acrshort{dpbt}}
\label{src:dpbt:1}
\end{listing}

In \autoref{src:dpbt:1} can be appreciated the use of feedback channel in the highlighted line.
Automatically \acrshort{dpbt} is going to connect this with the $\ibt$.

$\ibt, \obt, \gbt$ are not going to be covered because they are straightforward to follow from the code.
The most sofisticated part of the algorithm, as we have seen in \autoref{sub:sec:iebt:dpalgo}, relies on actor filter.

\begin{listing}[htp!]
\begin{minted}[fontsize=\small,numbers=left,breaklines,frame=lines,framerule=2pt,framesep=2mm,baselinestretch=1.2,highlightlines={16,21}]{haskell}

actor1 :: Edge
      -> ReadChannel (UpperVertex, LowerVertex)
      -> ReadChannel W
      -> ReadChannel Q
      -> ReadChannel BT
      -> ReadChannel BTResult
      -> ReadChannel W
      -> WriteChannel (UpperVertex, LowerVertex)
      -> WriteChannel W
      -> WriteChannel Q
      -> WriteChannel BT
      -> WriteChannel BTResult
      -> WriteChannel W
      -> StateT FilterState (DP st) ()
actor1 (_, l) redges _ _ _ _ _ we ww1 _ _ _ _ = do
 foldM_ redges $ \e@(u', l') -> do
   e `seq` if l' == l then modify $ flip modifyWState u' else push e we
 finish we
 state' <- get
 case state' of
   Adj w@(W _ ws) -> when (IS.size ws > 1) $ push w ww1
   _              -> pure ()

\end{minted}
\caption{[\mintinline{shell}{BTriangle.hs}] $\aaa$}
\label{src:dpbt:2}
\end{listing}

In \autoref{src:dpbt:2} $\aaa$ source code can be appreciated. Since all the actors are inside the same filter context, all of them have access to read and write
channels of all the filters. That explains the number of parameters which is generated by the \acrshort{idl}.
The first highlighted line is the catamorphism (\mintinline{haskell}{foldM_}) of the $IC_E$ channel, which corresponds to \autoref{algo:act-1:1}.
At the last highlighted line the code is downstream the aggregated wedge collected in this filter.

\begin{listing}[htp!]
\begin{minted}[fontsize=\small,numbers=left,breaklines,frame=lines,framerule=2pt,framesep=2mm,baselinestretch=1.2,highlightlines={22,30,36}]{haskell}

actor2 :: Edge
      -> ReadChannel (UpperVertex, LowerVertex)
      -> ReadChannel W
      -> ReadChannel Q
      -> ReadChannel BT
      -> ReadChannel BTResult
      -> ReadChannel W
      -> WriteChannel (UpperVertex, LowerVertex)
      -> WriteChannel W
      -> WriteChannel Q
      -> WriteChannel BT
      -> WriteChannel BTResult
      -> WriteChannel W
      -> StateT FilterState (DP st) ()
actor2 (_, l) _ rw1 _ _ _ _ _ ww1 _ _ _ _ = do
 state' <- get
 case state' of
   Adj (W _ w_t) -> do
     modify $ const $ DoubleWedges mempty
     foldM_ rw1 $ \w@(W l' w_t') -> do
       push w ww1
       buildDW w_t w_t' l l'
     finish ww1
   _ -> pure ()

buildDW :: IntSet -> IntSet -> LowerVertex -> LowerVertex -> StateT FilterState (DP st) ()
buildDW w_t w_t' l l' =
let pair       = Pair (min l l') (max l l')
      paramBuild = if l < l' then (w_t, w_t') else (w_t', w_t)
      ut         = uncurry buildDW' paramBuild
in  if (IS.size w_t > 1) && l /= l' && not (IS.null (IS.intersection w_t w_t')) && not (nullUT ut)
      then modify $ flip modifyDWState (DW pair ut)
      else pure ()

buildDW' :: IntSet -> IntSet -> UT
buildDW' !w_t !w_t' = (w_t IS.\\ w_t', IS.intersection w_t w_t', w_t' IS.\\ w_t)

\end{minted}
\caption{[\mintinline{shell}{BTriangle.hs}] $\ab$}
\label{src:dpbt:3}
\end{listing}

In \autoref{src:dpbt:3}, the interesting part of $\ab$ is the construction of $\dwi$. That is done by the function \mintinline{haskell}{buildDW}
and \mintinline{haskell}{buildDW'} which are building the three subsets required to create aggregated double-wedges $\dwi$.

\begin{listing}[htp!]
\begin{minted}[fontsize=\small,numbers=left,breaklines,frame=lines,framerule=2pt,framesep=2mm,baselinestretch=1.2,highlightlines={27,34-40}]{haskell}
actor3 :: Edge
       -> ReadChannel (UpperVertex, LowerVertex)
       -> ReadChannel W
       -> ReadChannel Q
       -> ReadChannel BT
       -> ReadChannel BTResult
       -> ReadChannel W
       -> WriteChannel (UpperVertex, LowerVertex)
       -> WriteChannel W
       -> WriteChannel Q
       -> WriteChannel BT
       -> WriteChannel BTResult
       -> WriteChannel W
       -> StateT FilterState (DP st) ()
actor3 (_, l) _ _ _ _ _ rfb _ _ _ _ _ wfb = do
  state' <- get
  case state' of
    DoubleWedges dwtt -> do
      modify $ const $ BiTriangles mempty
      foldM_ rfb $ \w@(W l' w_t') -> do
        push w wfb
        when (hasDW dwtt) $ do
          let (DWTT dtlist) = dwtt
          forM_ dtlist $ \(DW (Pair l_l l_u) ut) ->
            let triple = Triplet l_l l' l_u
                result =
                  if l' < l_u && l' > l_l then filterUt w_t' ut else Nothing
            in  maybe (pure ()) (modify . flip modifyBTState . BT triple) result
      finish wfb
    _ -> pure ()

filterUt :: IntSet -> UT -> Maybe UT
filterUt wt (si, sj, sk) =
  let si' = IS.filter (`IS.member` wt) si
      sj' = IS.filter (`IS.member` wt) sj
      sk' = IS.filter (`IS.member` wt) sk
      sij' = si' `IS.union` sj'
      sjk' = sk' `IS.union` sj'
      wtInSome = not (IS.null sij' || IS.null sjk')
  in  if wtInSome then Just (sij', sj, sjk') else Nothing
\end{minted}
\caption{[\mintinline{shell}{BTriangle.hs}] $\ac$}
\label{src:dpbt:4}
\end{listing}

$\ac$ in \autoref{src:dpbt:4}, shows the process of collecting aggregated bitriangles $\ati$. In highlighted lines on \autoref{src:dpbt:4} we can see the algorithm to detect when the \acrshort{awg}
received in the feedback channel are \acrshort{awgc} according to \dref{def:awgc}.

\begin{listing}[htp!]
\begin{minted}[fontsize=\small,numbers=left,breaklines,frame=lines,framerule=2pt,framesep=2mm,baselinestretch=1.2,highlightlines={21-22,27-31}]{haskell}
actor4 :: Edge
       -> ReadChannel (UpperVertex, LowerVertex)
       -> ReadChannel W
       -> ReadChannel Q
       -> ReadChannel BT
       -> ReadChannel BTResult
       -> ReadChannel W
       -> WriteChannel (UpperVertex, LowerVertex)
       -> WriteChannel W
       -> WriteChannel Q
       -> WriteChannel BT
       -> WriteChannel BTResult
       -> WriteChannel W
       -> StateT FilterState (DP st) ()
actor4 (_, l) _ _ query _ rbtr _ _ _ wq _ wbtr _ = do
  state' <- get
  case state' of
    BiTriangles bttt -> do
      rbtr |=> wbtr
      foldM_ query $ \e -> do
        push e wq
        unless (hasNotBT bttt) $ sendBts bttt e wbtr
    _ -> pure ()

sendBts :: MonadIO m => BTTT -> Q -> WriteChannel BTResult -> m ()
sendBts (BTTT bttt) q@(Q c _ _) wbtr = case c of
  ByVertex vx    -> forM_ bttt (\bt -> filterBTByVertex bt vx (flip push wbtr . RBT q))
  ByEdge   edges -> forM_ bttt (\bt -> filterBTByEdge bt edges (flip push wbtr . RBT q))
  AllBT          -> forM_ bttt (R.mapM_ (flip push wbtr . RBT q) . buildBT)
  Count          -> forM_ bttt (flip push wbtr . RC q . R.length . buildBT)
  _              -> pure ()
\end{minted}
\caption{[\mintinline{shell}{BTriangle.hs}] $\ad$}
\label{src:dpbt:5}
\end{listing}

$\ad$ in \autoref{src:dpbt:5} shows the pattern match done over the query $Q$ sum type and the construction
of the final \acrshort{bt} to be downstream to the $\obt$.

\begin{listing}[htp!]
\begin{minted}[fontsize=\small,numbers=left,breaklines,frame=lines,framerule=2pt,framesep=2mm,baselinestretch=1.2,highlightlines={}]{haskell}
filterBTByVertex :: MonadIO m => BT -> IntSet -> ((Int, Int, Int, Int, Int, Int, Int) -> IO ()) -> m ()
filterBTByVertex bt vertices f =
  if inLower bt vertices then
     liftIO
     . mapConcurrently_ f
     . buildBT
     $ bt
  else
    if inUpper bt vertices
      then buildBT' bt vertices f
      else pure ()

filterBTByEdge :: MonadIO m => BT -> Set Edge -> ((Int, Int, Int, Int, Int, Int, Int) -> IO ()) -> m ()
filterBTByEdge bt edges f =
  when (getAny $ foldMap (`hasEdge` bt) edges)
   $ liftIO . mapConcurrently_ f . R.filter (isInSetEdge edges) . buildBT $ bt

\end{minted}
\caption{[\mintinline{shell}{Edges.hs}] \texttt{filterBTByVertex} and \texttt{filterBTByEdge}}
\label{src:dpbt:6}
\end{listing}

Finally, definition of \mintinline{haskell}{filterBTByEdge} and \mintinline{haskell}{filterBTByVertex} can be seen in \autoref{src:dpbt:6} and differs
from the pseudo-code presented in \autoref{algo:buildBtEdge} and \autoref{algo:buildBtVertex} because it is using \acrshort{hs} specific combinators and
for-comprehension lists to take advantage of the non-strictness of the language as well as some concurrency primitives.

\section{Chapter Summary}
In this chapter we first introduced some basic definitions. These definitions are the foundation to represent the graph index created from the input bipartite graph and to define the \acrshort{iebt} algorithm. Second, we presented a general view of the algorithm. Third, we deeply described the definitions of the  stages  of the $\dpbt$. Then, we provided a correctness proof  of the \acrshort{iebt} algorithm. Finally, we give the main details of the implementation of the \acrshort{iebt} algorithm using \acrshort{dpfh}.


\section{Information Theory,  Fitting Constraints}

 \begin{slide}
   \small
   \begin{cenpage}{80mm}\setlength{\baselineskip}{10pt}
     \vfill
     \begin{center}
     {\Huge\BlueEmph{ Information Theory, Generalized Variables   }}

     \end{center}
     \vfill
   \end{cenpage}
 \end{slide}

\begin{slide}{EXAFS Analysis and Information Theory}

    The basic difficulties with EXAFS Analysis are:

    \begin{enumerate}
    \item The basis functions (Paths, Shells, \ldots) are not very well
      resolved, and their number grows exponentially with $R$.

    \item The scattering factors \feffc{f(k)}, \feffc{\delta(k)} are not simple to
      calculate.

    \item There's not much information in a real measurement:
      \[ N_{\rm idp} \approx \frac{2\Delta k\Delta R}{\pi} \]

    \end{enumerate}

    \hrule \vmm

    Quantitative EXAFS analysis procedures address these with methods to:

    \begin{enumerate}
    \item {reduce} the number of Paths to consider (Fourier analysis).

    \item re-use or parametrize {\emph{ab initio}} calculations of
      \feffc{f(k)}, \feffc{\delta(k)} (using {\scshape{FEFF}} or something similar).

    \item recycle parameters to cut down the number of independent
      variables in the fit, while keeping a meaningful analysis.

    \end{enumerate}

    \hrule \vmm

    Generally, we parametrize the EXAFS with a physical model, and then
    put {\RedEmph{Constraints}} and {\RedEmph{Restraints}} on the
    parameters in a least-squares analysis.

    \vmm
\vfill
\end{slide}

%%%%%%%%%%%%%%%%%%%%%%
\begin{slide}{Constraints: why we need them}

    The EXAFS Equation (slightly simplified here):

%     \[
%     \displaystyle{{\chi_j(k) = \sum_j
%         \frac{\feffc{f(k)} \feffc{N}\pthpar{S02} e^{(-2p''\reff - 2p^2\pthpar{sigma2})}}
%         {k(\reff + \pthpar{delR})^2}
%         \bsin[ 2k\reff + \feffc{\delta(k)}
%           + 2p\pthpar{delR} ]
%         }}
%     \]

    \vmm has 7 {\Blue{Path Parameters}} that could be refined per Path $j$:
%     \begin{center}

%        \pthpar{S02}, \pthpar{e0}, \pthpar{ei},
%        \pthpar{delR}, \pthpar{sigma2}, \pthpar{third}, and \pthpar{fourth}.
%     \end{center}

    \vmm

    But the information content of EXAFS is low:
%     \begin{center} \[ N_{\rm idp} = 8 \> \> \>
%       {\rm for} \> \Delta R = 1 \, {\rm \AA} \>\> {\rm and} \>\> \Delta k = 12.5\,
%     {\rm \AA^{-1} } \] \end{center}

    \vmm \hrule \vmm

    For well-isolated single path problems, it's fine to just fit $N$, $R$,
    $\sigma^2$, and $E_0$.

    \vmm
    But for more complicated problems, including all the ``multiples'':

    \vmm

    {\Red{
        \begin{tabular}{clcl}
          \hspace{3mm}
          &multiple neighboring species  &&  multiple sites for central atom \\
          &multiple scattering paths     &&  multiple polarizations \\
          &multiple data sets            &\hspace{4mm}&  multiple edges to co-refine   \\
        \end{tabular}
        }}

    \vmm
    we need to limit the number of parameters, and want to impose a
    physical model to get more meaningful results.

\vfill
\end{slide}

%%%%%%%%%%%%%%%%%%%%%%
\begin{slide}{Constraints, continued}

    {\BlueEmph{Constraints}} allow us to impose relationships between the
    parameters, and create physical models about our data.  For example:

    \begin{itemize}
    \item force the same $E_0$ for all Paths (may not always \ldots).

    \item allow mixed coordination shell: $x$ (Fe-O) + (1-$x$)(Fe-Fe).

    \item make $\sigma^2$ follow a simple temperature law (e.g,  an
      Einstein model).

    \item force one $R$ for the same bond for data taken from different
      edges.

    \item model complex distortions (height of a sorbed atom above a surface).

    \end{itemize}

    \hrule \vmm \vmm

    {\ifeffit} uses {\RedEmph{Generalized Variables}} for Constraints in
    the Fit:

    \vmm\vmm

    The {{Path Parameters}} \pthpar{delR}, \pthpar{S02}, \pthpar{sigma2},
    \pthpar{e0}, \ldots are not adjusted directly.

    \vmm Instead, the Path Parameters are written as mathematical
    expressions of the Generalized Variables.  This makes it easy to share
    Variables across:

    \begin{itemize}
    \item different Path Parameters for a Path
    \item different Scattering Paths (including Multiple Scattering)
    \item different data sets (different temperature, edge, polarization
      \ldots)
    \end{itemize}

\vfill
\end{slide}

% %%%%%%%%%%%%%%%%%%%%%%
% \begin{slide}{Generalized Variables and DEF in IFEFFIT }

%     {\ifeffit} has user-defined Generalized Variables for easy
%     model-building (ie, putting constraints on Path Parameters).

%      \vmm

%      \begin{tabular}{ll}
%        \hspace{-5mm} \begin{minipage}{44mm}
%             \begin{VerbSBox}{40mm} \bfseries
%   {{set   S02     = 0.80 }}
%   {\Blue{guess N_FeO   = 5.0 }}

%   {\Red{def    N_FeS     = 6 - N_FeO  }}
%   {\Red{def    Amp_Path1 = N_FeO * S02 }}
%   {\Red{def    Amp_Path1 = N_FeS * S02 }}

%   path(1,  feff=feff_feo.dat, degen=1,
%        s02= Amp_Path1 )

%   path(2,  feff=feff_fes.dat, degen=1,
%        s02= Amp_Path2)

%   feffit(1,2, chi=data.chi,....)
%     \end{VerbSBox}

%     \end{minipage}
%     &

%     \begin{minipage}{56mm}

%       The $S_0^2$ parameter is set to 0.80
%       \vmm
%       {\Blue{\tt{N\_FeO}}} will be varied in the fit, adjusting the
%       amplitude for Paths 1 (Fe-O) and 2 (Fe-S).
%       \vmm

%      Both {\Blue{\tt{N\_FeO}}} and {\Red{\tt{N\_FeS}}} can change, but not
%      independently: they must sum to 6.

%      \vmm

%      {\Red{\tt{N\_FeS}}}, {\Red{\tt{Amp\_Path1}}} and
%      {\Red{\tt{Amp\_Path2}}} are {\RedEmph{defined}} parameters: their
%      values will adjust in the fit according to the formula provided.
%
%   \end{minipage}\\
%  \end{tabular}

%   \vmm

%   {\Blue{\tt{N\_FeO}}}, {\Red{\tt{N\_FeS}}}, {\Red{\tt{Amp\_Path1}}}, etc
%   are {\RedEmph{Generalized Variables}}.  These can be:

%   \begin{description}
%   \item[{\Blue{Normal variables}}] ({\tt{S02}}), that are set to take a fixed value
%   \item[{\Blue{Fitting variables}}] ({\tt{N\_FeO}}), that are freely adjusted in the
%     fit.
%   \item[{\Blue{Defined variables}}] ({\tt{N\_FeS}}), that are functions of other
%     Variables whose values will adjust according to the formula provided.

%   \end{description}

%   \vmm

%   \vmm

%  \end{slide}

% %%%%%%%%%%%%%%%%%%%%%%
% \begin{slide}{Some Examples of Generalized Variables}

%     Examples of Path Parameters written as functions of the Generalized
%     Variables:

%     \vmm

%     \begin{tabular}{ll}
%       \hspace{-5mm} \begin{minipage}{40mm}
%            \begin{VerbSBox}{38mm} \bfseries
%  {\Blue{# simplest case: Parameter=Variable}}

%  {\Red{guess e0    = 1.0 }}

%  path(1,  e0 = e0)
%  path(2,  e0 = e0)
%    \end{VerbSBox}

%    \vspace{-2.5mm}

%    \begin{VerbSBox}{38mm} \bfseries
% {\Blue{# mixed coordination shell}}
%  set S02   = 0.80

%  {\Red{guess x   = 0.5}}

%  path(1,  S02 = S02 * x )

%  path(2,  S02 = S02 * (1-x) )
%    \end{VerbSBox}
%    \end{minipage}
%    &
%    \begin{minipage}{59mm}
%    \begin{VerbSBox}{58mm} \bfseries
% {\Blue{# Einstein Temperature }}

%  set   factor  = 24.254337   {\Blue{#= (hbar*c)^2/(2 k_boltz)}}
% {\Blue{# mass and reduced mass in amu}}
%  set   mass1 = 63.54,  mass2 = 63.54
%  set   r_mass =  1/ (1/mass1 +  1/mass2)

% {\Blue{# the Einstein Temp will be adjusted in the fit!}}
%  {\Red{guess thetaE = 200}}
% {\Blue{# use for data set 1, T=77}}
%  set   temp1 = 77
%  {\Red{def ss2_path1 = factor*coth(thetaE/(2*temp1))/r_mass )}}
%  path(101,  sigma2 = ss2_path1   )

% {\Blue{# use for data set 2, T=300}}
%  set   temp2 = 300
%  {\Red{def ss2_path2 = factor*coth(thetaE/(2*temp2))/r_mass )}}
%  path(201,  sigma2 = ss2_path2   )
%     \end{VerbSBox}
%     \end{minipage}\\

%   \end{tabular}

% %% \sigma^2 = {{(\hbar c)^2} \over{2k_{B}}} { {\coth(\theta /2T) }
% %% \over{M_{\rm R}\theta}}

%   \vmm\vmm

%   Constraints and Defined Variables are one kind of ``Prior Knowledge'',
%   allowing us to build and compare simple physical models for our data.

%   \vmm

% \vfill
% \end{slide}


\chapter{Discussion and Conclusions}

	With what has been developed throughout this thesis work, it has given us a satisfactory understanding of the knowledge necessary to apply spectral methods in solving initial value problems of nonlinear partial differential equations, which with practice it will be possible to acquire the ability to study and attack other more complex problems. \\

	The practice that we have obtained when developing Chapter \ref{Chapter_3} with the help of the tools examined in Chapter \ref{Chapter_2}, can be useful to construct and study some other spectral method using another family of polynomials that allows us to approximate more precisely a function, and to construct practical methods for its implementation solving this same problem or another one of interest. Although, as expected based on theory, we were able to obtain good approximations using Fourier methods to solve linear problems. But, the situation was different when we considered the non-linear problem since it was not possible to obtain the solutions directly, forcing us to develop more carefully the numerical methods that finally gave us a good numerical approximation and with practical mathematical expressions for its implementation.  \\

	However, we noted that very small steps in time were required to ensure that the methods worked correctly, which greatly increased the computation time, but this was due to the presence of the aliasing error caused by the nonlinear term, which manifested itself unfavorably with small values of $\alpha$, and they should be handled more carefully, possibly considering another class of polynomials. Even so, we were able to satisfactorily detect the problems that can arise when using these tools, and it is interesting to delve into this with the help of the appropriate tools that allow us to characterize and know the convergence of these methods. \\

	Regarding the spectral method that we developed in the Chapter \ref{Chapter_4}, the problem that arises is due to the number of calculations that are necessary to obtain the numerical approximations, but nevertheless, the implementation is easy to understand and can be studied with more detail to propose a more efficient and less expensive algorithm to calculate, mainly attacking the calculation of the integrals involved in the process. For this type of problem, it is worth investigating an implementation that allows you to write parallel code in some programming language to optimize memory and computation times. \\

	Even so, it was possible to successfully carry out some numerical experiments that allowed us to observe a theoretical result that characterized the stability of this method, and perhaps could allow us to investigate other characteristics of interest, such as its convergence. Thanks to the practice developed in this chapter we have acquired the ability to implement a spectral method to carry out numerical studies that may be useful for its analysis, which could also be interesting to implement these same techniques in other problems that are considered important. \\

	Therefore, we can conclude that spectral methods are a good option to study partial differential equations, either deterministic or stochastic since in addition to giving us the possibility of obtaining good precision in their approximations provided they are implemented correctly, they are also an excellent alternative to investigate the nature of these problems, be they physical or mathematical, opening different paths, such as some of those already mentioned, that may be interesting for developing future research. \\

	Finally, from a personal perspective, it considers of great importance the knowledge about the use of programming languages to build quality algorithms, that is, that they manage to have a code structure that can be a base to solve other problems. In addition, good training in scientific computing is of great advantage when producing research on the development of algorithms that allow to satisfactorily show the precision of some numerical method, since it must be considered that this depends strongly on the quality of the computation and could interfere considerably with the evaluation of our experiments.


\section{Dictionaries} % (fold)
\label{sec:dictionaries}
\begin{questions}
\titledquestion{Printing a dictionary} % (fold)
\label{sub:printing_a_dictionary}

Write a function that prints key-value pairs of a dictionary.

% titledquestion printing_a_dictionary (end)

\titledquestion{Histogram} % (fold)
\label{sub:histogram}

Write a function that takes a list, and returns a dictionary with
keys the elements of the list and as value the number of occurances of
that element in the list.

After you are done, look up `python collections counter' in Google.
Could you use a counter instead?

% titledquestion histogram (end)

\titledquestion{Get method} % (fold)
\label{sub:get_method}

Dictionaries have a \texttt{get} method, which takes a key and a default value.
If the key is in the dictionary, it returns the value, otherwise, it returns the
default value.

Rewrite your code from the previous problem to make use of this \texttt{get} method.

% titledquestion get_method (end)

\titledquestion{Random text generator}
\label{sub:random_text_generator}

In this question we will start implementing a random text generator.
The generated phrases somewhat resemble English, but are usually nonsense.
Next week, after we learn about file I/O, we are ready to complete the code.

To generate sentences, we first construct a so-called Markov chain based on actual text data.
This is a very basic language model.
We can then sample paths from this Markov chain to create new phrases.
This is actually easier than it sounds:

A Markov chain consists of states, and transition probabilities between states:
i.e. when I am in state A, what is the probability that I'll go to state B?

We focus on the simple case where the state will be the current word.
Consider the example sentence `the fire and the wind.'
Then, the states that we move through are
\[
    \text{BEGIN} \to \text{the} \to \text{fire} \to \text{and} \to \text{the} \to \text{wind.} \to \text{END}
\]
where `BEGIN' and `END' are special states for the beginning and the end of the sentence.
To find the transition probabilities, we go over a large body of text and
record current word and the next word.
In the above example, `BEGIN' is followed by `the', and `the' is followed by
`fire' and `wind'.

We won't be computing actual probabilities.
Instead, we create a dictionary that,
for every word, contains all the words that follow it.
To generate a phrase, we start at the `BEGIN' state, and pick randomly one word
from the list of words that follows the `BEGIN' state.
Then we look up which words follow that word, and again pick one word at random, until
we hit the `END' state, which signals that we are done.

Before you get started, download the starter code by using\\
\texttt{\$ git clone https://github.com/schmit/Markov-chain-startercode.git} (if you have git / use Cloud9)
    or dowload the code directly:\\ \url{https://github.com/schmit/Markov-chain-startercode/archive/master.zip}

Then
\begin{itemize}
    \item Implement the \texttt{process\_line} function, which takes a line as input,
    and returns a list with tuples with the current state, and the next state.
    \item Implement the \texttt{process\_textfile} function, which loops over
    text, calls \texttt{process\_line} to extract the transitions, and adds these to
    a dictionary. For now, do not worry about reading data from a file, the lines
    are given as elements of the list \texttt{f}.
    \item Implement the \texttt{generate\_line} function, which generates random
    phrases based on a dictionary with transitions.
\end{itemize}
See \texttt{markov.py} for a more detailed description of each function.

To run the code, use \texttt{python markov.py <filename>}.
Since we are not using any files yet, replace \texttt{<filename>} with a random word.

\titledquestion{Vector functions} % (fold)
\label{sub:vector_multiply}

Let's implement some vector functions.
There are two types of vectors, normal or dense vectors, which
we can represent using lists.
For sparse vectors, where many of the elements are zero,
this is inefficient.
Instead, we use a dictionary with keys the indices of non-zero values, and then the value
corresponding to the key is the value of the vector at that index.
Hence, the vector $[1, 2, 4]$ can be stored as a list: \texttt{[1, 2, 4]}
or as a dictionary \texttt{\{0:1, 1: 2, 2: 4\}}.

\begin{parts}
    \part Write a function that adds two (dense) vectors
    \part Write a function that multiplies (i.e. inner product) two (dense) vectors
    \part Write a function that adds two sparse vectors
    \part Write a function that multiplies two sparse vectors
    \part Write a function that adds a sparse vector and a dense vector
    \part Write a function that multiplies a sparse vector and a dense vector
\end{parts}

% titledquestion vector_multiply (end)

\titledquestion{Reverse look-up} % (fold)
\label{sub:reverse_look_up}

Dictionaries are made to look up values by keys.
Suppose however, we want to find the key that is associated with some value.
Write a function that takes a dictionary and a value, and returns the key
associated with this value.

What challenges do you face?
How would you deal with those challenges?

% titledquestion reverse_look_up (end)

\end{questions}
% section dictionaries (end)