\documentclass{article}
    \usepackage{subcaption}
    \usepackage{amsmath}
    \usepackage{amssymb}
    \usepackage[table]{xcolor}
    \usepackage{graphicx}
    \usepackage{algorithm, algpseudocode}
    \usepackage{caption, wrapfig}
    \usepackage{varwidth}
    \setlength{\parskip}{\baselineskip}

    \usepackage[hidelinks]{hyperref}
    \usepackage[margin=.8in, tmargin=.3in]{geometry}
    \renewcommand{\baselinestretch}{1.2}

    % \usepackage[
    %     backend=biber,
    %     % citestyle=authoryear,
    %     sorting=ynt,
    %     doi=false,
    %     isbn=false,
    %     url=false,
    %     giveninits=true,
    %     maxbibnames=2,
    %     style=chicago-authordate
    % ]{biblatex}
    % \AtEveryBibitem{
    %     \clearfield{urlyear}
    %     \clearfield{urlmonth}
    % }
    % graphs
    \usepackage{tikz}
    \usetikzlibrary{bayesnet}
    \usetikzlibrary{patterns}

    \newcommand{\prob}{\text{P}}
    \newcommand{\pr}{\prob}
    \newcommand{\pre}{\text{pre}_i}
    \newcommand{\pa}{\text{pa}_i}
    \renewcommand{\vec}[1]{\mathbf{#1}}
    \newcommand{\bx}{\vec{x}}
    \newcommand{\bu}{\vec{u}}
    \newcommand{\bfu}{\vec{f}}
    \newcommand{\bA}{\vec{A}}
    \newcommand{\bB}{\vec{B}}
    \newcommand{\bb}{\vec{b}}
    \newcommand{\bQ}{\vec{Q}}
    \newcommand{\bR}{\vec{R}}
    \newcommand{\bN}{\vec{N}}
    \newcommand{\bP}{\vec{P}}
    \newcommand{\bS}{\vec{S}}
    \newcommand{\bK}{\vec{K}}
    \newcommand{\bH}{\vec{H}}
    \newcommand{\by}{\vec{y}}
    \newcommand{\bz}{\vec{z}}
    \newcommand{\bW}{\vec{W}}
    \newcommand{\bq}{\vec{q}}
    \newcommand{\bl}{\mathbf{\lambda}}
    \newcommand{\giv}{\ |\ }
    \newcommand{\dotvec}[1]{\mathbf{\dot{#1}}}
    \renewcommand{\d}[1]{\mathrm{d#1}}
    \newcommand{\norm}[1]{\left\lVert#1\right\rVert}
    \newcommand{\der}[2]{\frac{\partial #1}{\partial #2}}
    \newcommand{\refx}{\bx^{(0)}}
    \newcommand{\refu}{\bu^{(0)}}
    \newcommand{\delx}{\delta \bx}
    \newcommand{\delu}{\delta \bu}
    \newcommand{\red}[1]{\textcolor{red}{#1}}
    \newcommand{\blue}[1]{\textcolor{blue}{#1}}
    \def\code#1{\texttt{#1}}

    \bibliography{MEMMO, Robotics, rl}

    \usepackage{mathtools}
    \DeclarePairedDelimiter\ceil{\lceil}{\rceil}
    \DeclarePairedDelimiter\floor{\lfloor}{\rfloor}

    \DeclareMathOperator*{\argmax}{arg\,max}
    \DeclareMathOperator*{\argmin}{arg\,min}

    % \newcommand\indep{\protect\mathpalette{\protect\independenT}{\perp}}
    % \def\independenT#1#2{\mathrel{\rlap{$#1#2$}\mkern2mu{#1#2}}}
    \newcommand\indep{\!\perp\!\!\!\perp}
    \newcommand\notindep{\not\!\perp\!\!\!\perp}
    % \newcommand{\l}{\left(}

    \newcommand{\xor}{\oplus}
    \newcommand{\ex}{\mathbb{E}}

    \setlength\parindent{0pt}
    \captionsetup{justification=centering}

    \title{Probabilistic Modeling and Reasoning}
    \date{\today}
    \author{Traiko Dinev \textless traiko.dinev@gmail.com\textgreater}

\begin{document}

\maketitle
\textit{NOTE: This partially follows Probabilistic Modeling and Reasoning, a masters level course at the University of Edinburgh.}

\textit{NOTE: Note this "summary" is NOT a reproduction of the course materials nor is it copied from the corresponding courses. It was entirely written and typeset from scratch.}

\textit{License: Creative Commons public license; See README.md of repository}

\section{Probability Identities}
Non-exhaustive list of identities useful for the rest of this cheatsheet:
%
\begin{gather*}
    \pr(A,B) = \pr(A \giv B)\ \pr(B)
        \qquad \tag*{\blue{product rule}}\\
    \pr(A) = \sum_B \pr(A \giv B)\ \pr(B) =
        \sum_B \pr(A,B) \qquad \tag*{\blue{sum rule}} \\
    x \indep y \iff \pr(x, y) = \pr(x)\ \pr(y) \\
    x_1 \indep x_2 \indep, \dots , x_N \iff
        \pr(x1, \dots, x_N)  = \prod_i \pr(x_i)
\end{gather*}
%
We can store discrete distributions as tables of data. Conditional independece allows us to save space. Consider the conditional independence rules first:
%
\begin{gather*}
    \pr(x, y \giv z) = \pr(x \giv z)\ \pr(y \giv z)
        \iff x \indep y \giv z \\
    \pr(x \giv y, z) = \pr(x \giv z)
\end{gather*}
%
Then to store $\pr(x, y, z) = \pr(x) \pr(y) \pr(z)$ we would need $dim(x) \times dim(y) \times dim(z)$ space. If they are all equal this means $k^{3d} - 1$ entries. The factorization allows us to have $3(k^d - 1)$ entries instead.

\section{Directed Graphical Models}
From the chain rule, we can factorize any distribution as:
%
\begin{equation*}
    \pr(\bx) = \prod_i \pr(x_i \giv \pi_i),
        \quad \pi_i = \{ x_1, \dots, x_{i-1} \}
\end{equation*}
%
We can prove that the following holds true by induction:
\begin{equation*}
    \pr(\bx) = \prod_i \pr(x_i \giv \pi_i)
        \iff x_i \indep (\pre \setminus \pi_i) \giv \pi_i, \ \forall i
\end{equation*}
%
where $\pi_i$ is some subset of elements. This is to say that the factorization implies independence and a set of independences implies a factorization.

Thus we can visualize a distribution by drawing a DAG (directed acyclic graph) where the parents are the above $\pi_i$'s. Thus if:
%
\begin{equation*}
    \pr(\bx) = \pr(x_1)\ \pr(x_2)\ \pr(x_3 \giv x_1,x_2)\ \pr(x_4 \giv x_3)\ \pr(x_5 \giv x_2)
\end{equation*}
%
then the DAG is in \autoref{fig:dag1}.

\begin{figure}[h]
    \centering
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x1) {$x_1$};
        \node[latent, right=of x1]  (x2) {$x_2$};

        \node[latent, below=of x1, xshift=1.2cm]  (x3) {$x_3$};
        \node[latent, right=of x3]  (x5) {$x_5$};

        \node[latent, below=of x3]  (x4) {$x_4$};

        \edge{x1, x2}   {x3}
        \edge{x3}       {x4}
        \edge{x2}       {x5}
    \end{tikzpicture}
    \caption{Simple DAG}
    \label{fig:dag1}
\end{figure}

A graph can be generated from a distribution and a (topological) ordering of the elements. A topological ordering is one where the parents come before the children. Note that different orderings may generate different graphs.

\subsection{Examples}

\textbf{Markov Models} (of order 1) or chains are a series of serial connections (\autoref{fig:markov}).

\begin{figure}[h]
    \centering
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x1) {$x_1$};
        \node[latent, right=of x1]  (x2) {$x_2$};
        \node[latent, right=of x2]  (x3) {$x_3$};

        \edge{x1}   {x2}
        \edge{x2}   {x3}
    \end{tikzpicture}
    \caption{Markov Chain}
    \label{fig:markov}
\end{figure}

\textbf{Hidden Markov Models} contain a markov chain that is not observed. Each hidden $\mathbf{h}$ influences an observable. $\mathbf{x}$'s are often at different timesteps, making the chain represent a time series. \autoref{fig:hidden_markov}.

\begin{figure}[h]
    \centering
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (h1) {$h_1$};
        \node[latent, right=of h1]  (h2) {$h_2$};
        \node[latent, right=of h2]  (h3) {$h_3$};
        \node[latent, below=of h1]  (v1) {$v_3$};
        \node[latent, below=of h2]  (v2) {$v_3$};
        \node[latent, below=of h3]  (v3) {$v_3$};

        \edge{h1}   {h2}
        \edge{h2}   {h3}
        \edge{h1}   {v1}
        \edge{h2}   {v2}
        \edge{h3}   {v3}
    \end{tikzpicture}
    \caption{Hidden Markov Model}
    \label{fig:hidden_markov}
\end{figure}

\textbf{Probabilistic PCA/ Independent Component Analysis} are methods that both use tha same graphical model. Here the latents (hiddens) variables are not connected. At the same time they influence all of the observables. \autoref{fig:ppca_lca}

\begin{figure}[h]
    \centering
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (h1) {$h_1$};
        \node[latent, right=of h1]  (h2) {$h_2$};
        \node[latent, below=of h1, xshift=-1.2cm]  (v1) {$v_3$};
        \node[latent, right=of v1]  (v2) {$v_3$};
        \node[latent, right=of v2]  (v3) {$v_3$};

        \edge{h1, h2}   {v1}
        \edge{h1, h2}   {v2}
        \edge{h1, h2}   {v3}
    \end{tikzpicture}
    \caption{PPCA/ ICA graphical model}
    \label{fig:ppca_lca}
\end{figure}

\subsection{D-Separation}
The main reason for using graphical models is to more easily determinite independencies between variables. In a DAG a tool for using this is D-separation. We start by examining the three possible trail connections in a DAG. Note that these are not all possible connections between three elements, but rather \textit{all possible connections when following a trail}.

\textbf{Serial Connections} are like Markov chains.

\begin{figure}[h]
    \centering
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x) {$x$};
        \node[latent, right=of x]   (z) {$z$};
        \node[latent, right=of z]   (y) {$y$};
        \edge{x}   {z}
        \edge{z}   {y}
    \end{tikzpicture}
    \caption{Serial Connecton}
    \label{fig:serial}
\end{figure}

Importantly we have:
\begin{gather*}
    \pr(x, z, y) = \pr(x) \pr(z \giv x) \pr(y \giv z) \\
    x \indep y \giv z \quad\quad x \notindep y
\end{gather*}

This means that if we know the variable $z$, x and y, \textbf{and all their parents and children} that are not connected to $z$ are independent of each other.

\textbf{Diverging Connections}
\begin{figure}[h]
    \centering
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x) {$x$};
        \node[latent, right=of x]   (z) {$z$};
        \node[latent, right=of z]   (y) {$y$};
        \edge{z}   {x}
        \edge{z}   {y}
    \end{tikzpicture}
    \caption{Diverging Connecton}
    \label{fig:divering}
\end{figure}

The same property of independence holds true here:
\begin{gather*}
    \pr(x, z, y) = \pr(z) \pr(x \giv z) \pr(y \giv z) \\
    x \indep y \giv z \quad\quad x \notindep y
\end{gather*}

\textbf{Converging Connections (Colliders)}
\begin{figure}[h!]
    \centering
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x) {$x$};
        \node[latent, right=of x]   (z) {$z$};
        \node[latent, right=of z]   (y) {$y$};
        \edge{x}   {z}
        \edge{y}   {z}
    \end{tikzpicture}
    \caption{Collider}
    \label{fig:collider}
\end{figure}

For colliders if we \textbf{do not know $z$}, $x$ and $y$ are independent:
\begin{gather*}
    \pr(x, z, y) = \pr(z) \pr(x \giv z) \pr(y \giv z) \\
    x \indep y \quad\quad x \notindep y \giv z
\end{gather*}

This is true since $\pr(\cdot) = $...

\textbf{D-Separation}
Sets $X$ and $Y$ are d-separated by $Z$ iff all trails are blocked by $Z$. One of the following needs to be true for a trail to be blocked:

\textbf{1)} Either $b$ is in a head-tail or tail-tail configuration

\qquad
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x) {$x$};
        \node[latent, right=of x]   (b) {$b$};
        \node[latent, right=of b]   (y) {$y$};
        \edge{x}   {b}
        \edge{b}   {y}
    \end{tikzpicture} \qquad
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x) {$x$};
        \node[latent, right=of x]   (b) {$b$};
        \node[latent, right=of b]   (y) {$y$};
        \edge{b}   {x}
        \edge{b}   {y}
    \end{tikzpicture}

and $b$ is in $Z$.

\textbf{2)} $b$ is a part of a collider

\qquad \begin{tikzpicture}[x=1.6cm,y=1cm]
    % Nodes
    \node[latent]               (x) {$x$};
    \node[latent, right=of x]   (b) {$b$};
    \node[latent, right=of b]   (y) {$y$};
    \node[latent, below=of b]   (e) {};
    \edge{x}   {b}
    \edge{y}   {b}
    \edge{b}   {e}
\end{tikzpicture}

and neither b or its descendents are in $z$. \textbf{Then} $X \indep Y \giv Z$.

\subsection{I-Maps}
A graph is an I-map for a set of independencies $I$ iff all independencies asserted by the graph are part of $I$. A graph can thus have fewer independencies than the set. A fully-connected graph is a trivial I-map for all sets $I$.

\subsection{Directed Local Markov Property}
\begin{equation*}
    \bx_i \indep (\pre \setminus \pa) \giv \pa \leftrightarrow \bx_i \indep (\text{nondesc}(\bx_i) \setminus \pa) \giv \pa
\end{equation*}

This [todo] figure.

\subsection{Gloabl directed Markov Property}
All independencies asserted by D-separation.

\subsection{Markov Blanket}
By definition:
$$
    x \indep (\text{all} \setminus \bx \setminus \text{MB}(\bx)) \giv \text{MB}(\bx)
$$
And for DAGs we get:
$$
    \text{MB}(\bx) = \text{pa}(\bx)\ \cup\ \text{children} (\bx) \ \cup\ \text{co-parents}(\bx)
$$

\section{Undirected Graphical Models}
Firstly, we note the following. For non-negative functions $a$ and $b$:

\begin{gather*}
    x \indep y \giv z \leftrightarrow \pr(x, y, z) = a(x, z) \times b(y, z) \\
    x \indep y \leftrightarrow \pr(x, y) = a(x) \times b(y) \\
    \sum_{x, y, z} a(x, z) b(y, z) = 1 \\
    \textbf{if} \   p(x, y, z) = \frac{1}{Z} \phi_A(x, z) \phi_B(y, z), \quad
    Z = \sum_{x, y, z} \phi_A(x, z) \phi_B(y, z)
\end{gather*}

\subsection{Gibbs Distribution}
.. is a distribution that factorizes as:
$$
    \pr(\bx) = \frac{1}{Z} \prod_c \phi_C(\mathcal{X}_C), \hspace{0.2in}
    \mathcal{X}_C \subseteq \{ x_1, \dots, x_d \}
$$

\subsection{Energy-Based Model}
If in the above $\phi_C(\mathcal{X}_C) = \text{exp} (- \text{E}_c(\mathcal{X}_c))$. Then:

$$
    \pr(\bx) = \frac{1}{Z} =
        \frac{1}{Z} exp \bigg[ - \sum_c \text{E}_c(\mathcal{X}_c) \bigg] =
        \frac{1}{Z} \prod_c \underbrace{\text{exp}^{
            -\text{E}_c(\mathcal{X}_c)
        }}_{\phi_c(\mathcal{X}_c)}
$$

\subsection{Undirected Graphs}
Assuming a distribution (up to a constant) factorizes as:

\begin{equation*}
    \pr(\bx) \propto \phi_1(x_1, x_2, x_3) \phi_2(x_2, x_3, x_4) \phi_3(x_3, x_5) \phi_4(x_5, x_6)
\end{equation*}

then we visualize it as the following graph:
\begin{figure}[h!]
    \centering
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x1) {$x_1$};
        \node[latent, right=of x1, xshift=.5cm]   (x4) {$x_4$};
        \node[latent, above=of x1, xshift=1.25cm, yshift=-.5cm]   (x2) {$x_2$};
        \node[latent, below=of x2, yshift=-.25cm]   (x3) {$x_3$};
        \node[latent, left=of x3, yshift=-1cm,xshift=1cm]   (x5) {$x_5$};
        \node[latent, right=of x5, xshift=.5cm]   (x6) {$x_6$};
        \edge[-]{x2, x3}   {x1}
        \edge[-]{x3, x4}   {x2}
        \edge[-]{x4, x5, x6}   {x3}
    \end{tikzpicture}
    \caption{Undirected Graph}
    \label{fig:ug}
\end{figure}

We form cliques for all variables in each factor $\phi_i$.

\subsection{Independencies in Undirected Models}
If
$$
    \pr(\bx) \propto \phi_1(x_1, x_2) \phi_2(x_2, x_3) \phi_3(x_4)
$$

Then the corresponding graph is:

\begin{center}
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x1) {$x_1$};
        \node[latent, right=of x1]  (x2) {$x_2$};
        \node[latent, right=of x2]  (x3) {$x_3$};
        \node[latent, right=of x3]  (x4) {$x_4$};
        \edge[-] {x1} {x2}
        \edge[-] {x2} {x3}
    \end{tikzpicture}
\end{center}

This directly implies that
\begin{gather*}
    x_4 \indep x_1, x_2, x_3 \\
    x_1 \indep x_3 \giv x_2
\end{gather*}

In other words, a trail is blocked if there are no paths between the two nodes. Thus D-separation is more easily done here.

\subsection{Graph $\to$ Distribution}
Since we built undirected graphs by connecting cliques, it follows that given a graph we look at the maximum cliques to recover the distribution. \textbf{I-maps} are defined as before.

\subsection{Local Markov Property}
An edge is independent of all other edges given its neighbors. In the below graph the neighbors are colored in red. Formally, if $\text{ne}(\alpha)$ are the neighbors of $\alpha$, then:

$$
    \alpha \indep X \setminus (\alpha\ \cup\ \text{ne}(\alpha)) \giv \text{ne}(\alpha), \quad \forall \alpha \in X
$$

\definecolor{faintred}{RGB}{255,139,137}
\begin{center}
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (a) {$\alpha$};
        \node[latent, right=of a,fill=faintred]  (r) {};
        \node[latent, left=of a,fill=faintred]  (l) {};
        \node[latent, above=of a,fill=faintred]  (t) {};
        \node[latent, below=of a,fill=faintred]  (b) {};
        \node[latent, right=of r]  (r2) {};
        \node[latent, right=of t, xshift=0cm, yshift=1cm]  (t2) {};
        \node[latent, left=of b, xshift=.25cm, yshift=.25cm]  (bl) {};
        \node[latent, left=of l, yshift=.5cm]  (l2) {};
        \edge[-]{r, l, t, b} {a}
        \edge[-]{r2} {r}
        \edge[-]{t2} {t}
        \edge[-]{l, b} {bl}
        \edge[-]{l2} {l}
    \end{tikzpicture}
\end{center}

\subsection{Pairwise Markov Property}
\begin{equation*}
    \alpha \indep \beta \giv \underbrace{X}_{\text{all nodes}} \setminus \{ \alpha, \beta \}
\end{equation*}

for all non-neighboring $\alpha, \beta \in X$.

\definecolor{faintred}{RGB}{255,139,137}
\begin{center}
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (a) {$\alpha$};
        \node[latent, right=of a,fill=faintred]  (r) {};
        \node[latent, left=of a,fill=faintred]  (l) {};
        \node[latent, above=of a,fill=faintred]  (t) {};
        \node[latent, below=of a,fill=faintred]  (b) {};
        \node[latent, right=of r]  (r2) {$\beta$};
        \node[latent, right=of t, xshift=0cm, yshift=1cm,fill=faintred]  (t2) {};
        \node[latent, left=of b, xshift=.25cm, yshift=.25cm,fill=faintred]  (bl) {};
        \node[latent, left=of l, yshift=.5cm,fill=faintred]  (l2) {};
        \edge[-]{r, l, t, b} {a}
        \edge[-]{r2} {r}
        \edge[-]{t2} {t}
        \edge[-]{l, b} {bl}
        \edge[-]{l2} {l}
    \end{tikzpicture}
\end{center}

\subsection{Markov Blanket}
For undirected graph, the markov blanket of $x$ is the neighbors of $x$.

\noindent\fbox{%
    \parbox{\textwidth}{%
        N.B. All of the above properties (Markov properties) are equivalent for both directed and undirected graphical models. This means if one of them is true \textbf{for a distribution}, then all of them are true.
    }%
}

\textbf{TODO: Add Bishop plots with repetition boxes.}

\subsection{Minimal I-map}
\begin{itemize}
    \item[-] If an edge is removed, it ceases to be an imap.
    \item[-] A graph is an I-map if $\pr(\cdot)$ factorizes over the graph.
\end{itemize}

\begin{varwidth}[t]{.5\textwidth}
    \textbf{Undirected Models}
    \begin{itemize}
        \item[-] For all $\bx_i$ find $MB(\bx_i)$ and connect.
    \end{itemize}
\end{varwidth}
\hspace{4em}
\begin{varwidth}[t]{.5\textwidth}
    \textbf{Directed Models}
    \begin{itemize}
        \item[-] For all $\bx_i$ find $\pi_i \subseteq \pre$ such that $\bx_i \indep \{ \pre \setminus \pi_i \} \giv \pi_i$
        \item[-] Set $\pa = \pi_i$
    \end{itemize}
\end{varwidth}

\section{Equivalence and Conversion Between Models}
Two directed graphs $G_1$ and $G_2$ are $I$-equivalent if they have the same set of immoralities and the same skeleton. An immorality is a collider without covering edge. \textbf{Look for colliders that don't match.} Since serial (head-tail) and diverging (tail-tail) connections imply the same independencies, we only need to look for converging (head-head) connections that don't match.

\subsection{Directed $\to$ Undirected}
We have:
\begin{equation*}
    \pr(\cdot) = \prod_i \pr(\bx_i \giv \pa) =
        \prod_i \underbrace{\phi_i(\bx_i, \pa)}_{\blue{cliques}}
\end{equation*}

\begin{itemize}
    \item[$\blacktriangleright$] This is called \textbf{moralization} and we obtain a \textbf{moral graph}
    % \item[$\vartriangleright$] This is called \textbf{moralization} and we obtain a \textbf{moral graph}
    \item[$\blacktriangleright$] This it \textbf{NOT} an undirected I-map for the distribution. Most notably
        we can not represent collider independencies.
\end{itemize}

\subsection{Undirected $\to$ Directed}
(See example below)
\begin{itemize}
    \item[$\blacktriangleright$] Choose an ordering.
    \item[$\blacktriangleright$] Read independencies off of the graph and find $\pi_i$ for each $i$.
    \item[$\blacktriangleright$] Connect $\pi_i \to \bx_i$
\end{itemize}

\subsection{Non-equivalent Trails}
\textbf{Colliders can not be represented by undirected graphs.}
\begin{center}
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x1) {$x_1$};
        \node[latent, right=of x1]  (x2) {$x_2$};
        \node[latent, right=of x2]  (x3) {$x_3$};
        \edge {x1} {x2}
        \edge {x3} {x2}
    \end{tikzpicture}
    \hspace{.5in}
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x1) {$x_1$};
        \node[latent, below=of x1]  (x2) {$x_2$};
        \node[latent, right=of x2]  (x3) {$x_3$};
        \edge[-] {x1} {x2}
        \edge[-] {x3} {x2}
        \edge[-] {x1} {x3}
    \end{tikzpicture}
\end{center}
In the moralized graph on the
right the independence $\bx_1 \indep \bx_3$ is lost.

\textbf{Closed loops can not be represented by directed graphs.}
\begin{center}
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x) {$x$};
        \node[latent, below=of x1, xshift=-1cm, yshift=.5cm]  (u) {$u$};
        \node[latent, below=of x1, xshift=1cm, yshift=.5cm]  (y) {$y$};
        \node[latent, below=of x1, yshift=-.5cm]  (z) {$z$};
        \edge[-] {u, y} {x}
        \edge[-] {u, y} {z}
    \end{tikzpicture}
    \hspace{.5in}
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x) {$x$};
        \node[latent, below=of x1, xshift=-1cm, yshift=.5cm]  (u) {$u$};
        \node[latent, below=of x1, xshift=1cm, yshift=.5cm]  (y) {$y$};
        \node[latent, below=of x1, yshift=-.5cm]  (z) {$z$};
        \edge {x} {y}
        \edge {x, y} {u}
        \edge {u, y} {z}
    \end{tikzpicture}
\end{center}

Consider the left graph. Let's review the process of creating the directed graph:
\begin{itemize}
    \item[$\blacktriangleright$] Choose an ordering: $x, y, u, z$.
    \item[$\blacktriangleright$] For each element, consider the parent set and read
        independencies off the directed graph.
    \item[$\blacktriangleright$] Start with $y \notindep x$, which implies the edge $x \to y$.
    \item[$\blacktriangleright$] Find the minimal set for which $u$ is independent of the parents $\pi_u$.
        This is $x, y$. Hence $x, y \to u$.
    \item[$\blacktriangleright$] For $z$ this set is $u, y$. Hence $u \to z$ and $y \to z$
\end{itemize}

\textbf{We no longer have the independence} $u \indep y \giv x, z$.

\section{Factor Graphs}
\begin{equation}
    \pr(\bx_1, \bx_2, \bx_3, \bx_4) = \frac{1}{Z} \phi_1(\bx_1, \bx_2, \bx_3)
        \phi_2(\bx_3, \bx_4) \phi_3(\bx_4)
\end{equation}

\begin{figure}[h]
    \centering
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (x1) {$x_1$};
        \factor[right=of x1, pattern=north east lines, draw=black] {phi1} {$\phi_1$} {} {};
        \node[latent, below=of phi1]  (x2) {$x_2$};
        \node[latent, right=of phi1, xshift=-2.5em]  (x3) {$x_3$};
        \factor[right=of x3, pattern=north east lines, draw=black] {phi2} {$\phi_2$} {} {};
        \node[latent, right=of phi2, xshift=-2.5em]  (x4) {$x_4$};
        \factor[right=of x4, pattern=north east lines, draw=black] {phi3} {$\phi_3$} {} {};
        \edge[-] {x1, x2, x3} {phi1}
        \edge[-] {x3, x4} {phi2}
        \edge[-] {x4} {phi3}
    \end{tikzpicture}
    \caption{Factor Graph}
    \label{fig:factor_graph}
\end{figure}

\section{Exact Inference in Factor Graphs}
Assume discrete variables. The task is to compute $\pr(\bx_k = k)$ for all $k$.

We can group terms in the leaves of the tree. Consider:
\begin{equation}
    \pr(\cdot) \propto \phi_A(\bx_1) \phi_B(\bx_2) \phi_C(\bx_1, \bx_2, \bx_3)
        \phi_D(\bx_3, \bx_4) \phi_E(\bx_3, \bx_5) \phi_F(\bx_5)
\end{equation}

which is:
\begin{center}
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \factor[pattern=north east lines, draw=black] {phia} {$\phi_A$} {} {};
        \node[latent, right=of phia, xshift=-2.5em] (x1) {$x_1$};
        \factor[right=of x1, pattern=north east lines, draw=black, label=below:$\phi_C$] {phic} {} {} {};
        \node[latent, right=of phic, xshift=-2.5em] (x3) {$x_3$};
        \node[latent, above=of phic] (x2) {$x_2$};
        \factor[above=of x2, pattern=north east lines, draw=black] {phib} {$\phi_B$} {} {};

        \factor[right=of x3, pattern=north east lines, draw=black, yshift=2em] {phid} {$\phi_D$} {} {};
        \factor[right=of x3, pattern=north east lines, draw=black, yshift=-2em] {phie} {$\phi_E$} {} {};

        \node[latent, right=of phid, xshift=-2.5em] (x4) {$x_4$};
        \node[latent, right=of phie, xshift=-2.5em] (x5) {$x_5$};
        \factor[right=of x5, pattern=north east lines, draw=black] {phif} {$\phi_F$} {} {};

        \edge[-] {x1} {phia}
        \edge[-] {x1, x2, x3} {phic}
        \edge[-] {x2} {phib}
        \edge[-] {x3, x4} {phid}
        \edge[-] {x3, x5} {phie}
        \edge[-] {x5} {phif}
    \end{tikzpicture}
\end{center}

We iteratively "eliminate" variables by summing or integrating them out. Firstly, we eliminate $\bx_5$:

\begin{align*}
    \pr(\bx_1, \dots, \bx_4) &= \sum_{\bx_5} \pr(\bx_1, \dots, \bx_5) \\
    &\propto \sum_{\bx_5} \phi_A(\bx_1) \phi_B(\bx_2) \phi_C(\bx_1, \bx_2, \bx_3)
    \phi_D(\bx_3, \bx_4) \phi_E(\bx_3, \bx_5) \phi_F(\bx_5) \\
    &\propto \phi_A(\bx_1) \phi_B(\bx_2) \phi_C(\bx_1, \bx_2, \bx_3)
    \phi_D(\bx_3, \bx_4) \sum_{\bx_5} \phi_E(\bx_3, \bx_5) \phi_F(\bx_5) \\
    &\propto \phi_A(\bx_1) \phi_B(\bx_2) \phi_C(\bx_1, \bx_2, \bx_3)
    \phi_D(\bx_3, \bx_4) \widetilde{\phi}_5(\bx_3)
\end{align*}

The idea is that we have reduced the factors above. Numerically, we would have the following:

\begin{equation*}
    \widetilde{\phi}_5(\bx_3) = \begin{cases}
        a \qquad \bx_3 = 1 \\
        \dots \\
        z \qquad \bx_3 = N \\
    \end{cases}
\end{equation*}

For each value of the factor. This is pre-computing all of the values, which in this case costs $O(N^2)$. For each value of $\bx_3$ we need to sum over $\bx_5$.

If we keep on doing this, the total cost will be greatly reduced from the $O(N^4)$ that is needed to sum over $\bx_2, \bx_3, \bx_4, \bx_5$.

The above operation is represented by the following \textbf{reduced} factor graph:

\begin{center}
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \factor[pattern=north east lines, draw=black] {phia} {$\phi_A$} {} {};
        \node[latent, right=of phia, xshift=-2.5em] (x1) {$x_1$};
        \factor[right=of x1, pattern=north east lines, draw=black, label=below:$\phi_C$] {phic} {} {} {};
        \node[latent, right=of phic, xshift=-2.5em] (x3) {$x_3$};
        \node[latent, above=of phic] (x2) {$x_2$};
        \factor[above=of x2, pattern=north east lines, draw=black] {phib} {$\phi_B$} {} {};

        \factor[right=of x3, pattern=north east lines, draw=black, yshift=2em] {phid} {$\phi_D$} {} {};
        \factor[right=of x3, pattern=north east lines, draw=black, yshift=-2em] {phi5tilde} {$\widetilde{\phi}_5$} {} {};

        \node[latent, right=of phid, xshift=-2.5em] (x4) {$x_4$};

        \edge[-] {x1} {phia}
        \edge[-] {x1, x2, x3} {phic}
        \edge[-] {x2} {phib}
        \edge[-] {x3, x4} {phid}
        \edge[-] {x3} {phie}
    \end{tikzpicture}
\end{center}

\subsection{Message Passing}
\textbf{TODO: Graphs}
\begin{equation*}
    \mu_{\phi \to \bx}(\bx) = \sum_{\bx_1, \dots, \bx_j} \phi(\bx_1, \dots, \bx_j, x)
        \prod_{i = 1}^j \mu_{\bx_i \to \phi}(\bx_i)
\end{equation*}

\begin{equation*}
    \mu_{\bx \to \phi}(\bx) = \prod_{i = 1} ^ {j} \mu_{\phi_i \to \bx}
\end{equation*}

\begin{equation*}
    \prob(\bx) \propto \prod_{i = 1}^j \mu_{\phi_i \to \bx}(\bx)
\end{equation*}

\begin{equation*}
    \prob(\bx_1, \dots, \bx_j) \propto \phi(\bx_1, \dots, \bx_j)
        \prod_{i = 1}^{j} \mu_{\bx_i \to \phi} (\bx_i)
\end{equation*}

\section{Inference for Markov Chains}
\textbf{TODO: Just scan this..}
\subsection{$\alpha$-recursion}
\subsection{Smoothing}
\subsection{$\alpha$-$\beta$ Recursion}

\section{Model-Based Learning}
\begin{itemize}
    \item[--] Probabilistic model: table for a distribution $\prob(\bx)$
    \item[--] Statistical model: set of probabilistic models: $\{ \prob(\bx; \theta) \}$
    \item[--] Bayesian model: prior on theta, replace "parametrized by" with "conditioned on"
        $\prob(\bx) = \int \prob(\bx \giv \theta)\ \prob(\theta)$. To get the probability distribution
        of $\bx$ as above, we need to integrate over all possible values of $\theta$.
\end{itemize}

\section{Moment Matching}
Assume we have a parametric statistical model:

\begin{equation*}
    \prob(\bx; \theta) = \frac{
        \prob^*(\bx; \theta)
    } {
        Z(\theta)
    }
\end{equation*}

where $Z(\theta) = \int \prob^*(\bx; \theta)\ \text{d}\bx$ is the partition function. Maximum likelihood returns:

\begin{equation*}
    \theta^* = \argmax_\theta L(\theta) =
        \argmax_\theta \sum_i \log \prob(\bx_i; \theta)
\end{equation*}

the last being true for i.i.d. data only. This directly implies that at the maximum, the derivative is $0$:

\begin{equation*}
    \nabla_\theta L(\theta) \vert_{\theta^*} = 0
\end{equation*}

We define the moments to be:
\begin{equation*}
    m(\bx; \theta) = \nabla_\theta \log \prob^*(\bx; \theta)
\end{equation*}

Then:

\begin{align*}
    \nabla_\theta L(\theta) &= \nabla_\theta \sum_i \log \prob(\bx_i; \theta) = \nabla_\theta \sum_i \log \frac{\prob^*(\bx_i; \theta)}{Z(\theta)} \\
    %
    &= \bigg[\nabla_\theta \sum_i \log\prob^*(\bx_i; \theta) \bigg] - \nabla_\theta n \times \log Z(\theta) \\
    %
    &= \sum_i \nabla_\theta \prob^*(\bx_i; \theta) - n \nabla_\theta \log Z(\theta) \\
    %
    &= \sum_i m(\bx_i; \theta) - n \nabla_\theta \log Z(\theta)
\end{align*}

then for the second term we have:

\begin{align*}
    \nabla_\theta \log Z(\theta) &= \frac{1}{Z(\theta)} \nabla_\theta Z(\theta) = \frac{1}{Z(\theta)} \nabla_\theta \int \prob^*(\bx; \theta) \text{d} \bx = \\
    %
    &= \frac{
        \int \nabla_\theta \prob^*(\bx; \theta) \text{d} \bx
    } {
        Z(\theta)
    } = \\
    %
    &= \frac{
        \int \nabla_\theta \big[ \log \prob^*(\bx; \theta) \big] \prob^*(\bx; \theta) \text{d}\theta
    } {
        Z(\theta)
    } = \\
    %
    &= \int \nabla_\theta \big[ \log \prob^*({\bx; \theta}) \big] \prob(\bx; \theta) \text{d}\bx \\
    %
    &= \int m(\bx; \theta) \prob(\bx; \theta) \text{d}\bx
\end{align*}

which implies that:
\begin{equation*}
    \nabla_\theta L(\theta) = \sum_i m(\bx_i; \theta) - n \int m(\bx; \theta) \prob(\bx; \theta) \text{d} \theta
\end{equation*}

which means that at $\theta^*$, where $\nabla_\theta L(\theta) = 0$:
\begin{equation*}
    \frac{1}{n} \sum_i m(\bx_i; \theta) = \int m(\bx; \theta) \prob(\bx; \theta) \text{d} \theta
\end{equation*}

This proves that maximumum likelihood can be thought of as moment matching. We are matching the empirical moments to their expectation over the normalized data distribution $\prob(\bx; \theta)$. Note we don't have access to $\prob(\bx; \theta)$, but only to $\prob^*(\bx; \theta)$, the unnormalized distribution.

\section{Bayesian Inference}
This:
\begin{align*}
    \prob(\theta \giv \mathcal{D}) &= \frac{
        \prob(\theta; \mathcal{D})
    } {
        \prob(\mathcal{D})
    } = \frac{
        \prob(\mathcal{D} \giv \theta) \prob(\theta)
    } {
        \prob(\mathcal{D})
    } \\
    &\propto \text{Likelihood}(\theta) \prob(\theta) \propto \underbrace{\bigg[
        \prod_i \prob(\bx_i \giv \theta)
    \bigg]}_{\text{for i.i.d. data}} \prob(\theta)
\end{align*}

\section{Factor Analysis}
\begin{figure}
    \centering
    \begin{tikzpicture}[x=1.6cm,y=1cm]
        % Nodes
        \node[latent]               (h1) {$h_1$};
        \node[latent, right=of h1]  (h2) {$h_2$};
        \node[latent, below=of h1, xshift=-1.2cm]  (v1) {$v_3$};
        \node[latent, right=of v1]  (v2) {$v_3$};
        \node[latent, right=of v2]  (v3) {$v_3$};

        \edge{h1, h2}   {v1}
        \edge{h1, h2}   {v2}
        \edge{h1, h2}   {v3}
    \end{tikzpicture}
    \caption{FA/ICA graphical model}
    \label{fig:fa_ica}
\end{figure}

Both FA and ICA have the same (parametrized) graphical model. See \autoref{fig:fa_ica}. We have $D$ visible and $H$ hidden units:

$$
    \mathbf{h} \sim \mathcal{N}(\mathbf{0}, \mathbb{I}) \qquad H < D
$$
$$
    \prob(\mathbf{v} \giv \mathbf{\mathbf{h}}; \theta) = \mathcal{N}(\mathbf{v}; \mathbf{F} \mathbf{h} + \mathbf{c}, \psi)
$$

where:
$$
    \theta = \{
        \underbrace{\mathbf{c}}_{\text{mean of } \mathbf{x}},
        \mathbf{F} =
            \underbrace{(f_1, \dots, f_H)}_{\text{factors of size D; F is DxH}},
        \Psi =
            \underbrace{\text{diag}(\psi_1, \dots, \psi_D)}_{\text{noise variance}}
    \}
$$

This means that the model is:
\begin{align*}
    \mathbf{v} &= \mathbf{F} \mathbf{h} + \mathbf{c} + \mathbf{\epsilon} \qquad \epsilon \sim \mathcal{N}(0, \Psi)\\
    &= \sum_{i = 1}^H \mathbf{f}_i h_i + \mathbf{c} + \mathbf{\epsilon}
\end{align*}

\subsection{An important identity}
\begin{gather*}
    \text{if}\ \bx \sim \mathcal{N}(\mu_x, \mathbf{C}_x);
        \qquad \mathbf{z} \sim \mathcal{N} (\mu_z, \mathbf{C}_z) \\
    \mathbf{y} = \mathbf{A} \mathbf{x} + \mathbf{z},\
    \text{then} \qquad y \sim \mathcal{N}(
        \mathbf{A} \mu_x + \mu_z,
        \mathbf{A} \mathbf{C}_x \mathbf{A}^{T} + C_z
    )
\end{gather*}

\subsection{Factor Rotation Problem}
\begin{align*}
    \mathbf{v} &= \mathbf{F} \mathbf{h} + \mathbf{c} + \mathbf{\epsilon} = \mathbf{F} (\mathbf{R} \mathbf{R}^{T}) \mathbf{h} + \mathbf{c} + \mathbf{\epsilon} \\
    &= (\mathbf{F} \mathbf{R}) (\mathbf{R}^{T} \mathbf{h}) + \mathbf{c} + \mathbf{\epsilon} =
    (\mathbf{F} \mathbf{R}) \bar{\mathbf{h}} + \mathbf{c} + \mathbf{\epsilon}
\end{align*}

Which means that:
$$
    \prob(\bar{\mathbf{h}}) = \mathcal{N}(
        \bar{\mathbf{h}}; 0, \mathbb{I}
    )
$$
since $\mathbf{R}$ is an orthogonal matrix.

\section{Independent Component Analysis (ICA)}
ICA uses the same model as FCA but makes fewer assumptions. The joint of the hidden is:
%
\begin{equation*}
    \prob_h(\mathbf{h}) = \prod_i p_h(h_i)
\end{equation*}
%
with the observed (visible) following:
%
\begin{equation*}
    \mathbf{v} = \mathbf{F} \mathbf{h} + \mathbf{c} + \epsilon
\end{equation*}

\section{Intractable Likelihood Functions}
We'll look at three scenarios. Firstly, unobserved varibles. Assuming the following likelihood form:
%
\begin{equation*}
    \text{L}(\theta) = \prob(\mathcal{D}; \theta) =
        \int_u p(\underbrace{\mathbf{u}}_\text{Unobserved}, \mathcal{D}; \theta)\ \text{d}\mathbf{u}
\end{equation*}
%
This is intractable, since we need to integrate over the observed, which are ... unobserved. We can use monte-carlo integration (below) if we know their distribution. However, it turns out that the gradient of the likelihood we can compute in a more convenient way:
%
\begin{equation*}
    \nabla_\theta l(\theta) = \mathbb{E}_{\bu \sim \prob(\bu \giv \mathcal{D}; \theta)} \bigg[ \nabla_\theta \log \prob(\bu, \mathcal{D}; \theta) \giv \mathcal{D}; \theta \bigg]
\end{equation*}
%
It is in most cases that we can compute the joint $p(\bu, \mathcal{D}; \theta)$ and conditional $p(\bu \giv \mathcal{D}; \theta)$. We can sample from the conditional and use Monte-Carlo integration.

TODO: proof

\subsection{Intractable Partition Function}
If the partition $Z(\theta)$ is intractable, we generally have the distribution up to a constant:
%
\begin{equation*}
    \prob(\bx; \theta) = \frac{
        \prob^*(\bx; \theta)
    } {
        Z(\theta)
    }
\end{equation*}

Then we can use moment matching.

\begin{align*}
    \nabla_\theta l(\theta) &=  \sum_i m(\bx_i; \theta) - n \int m(\bx; \theta) \prob(\bx; \theta) \text{d} \theta \\
    &\propto \frac{1}{n} \sum_i m(\bx_i; \theta) - \mathbb{E} \big[ m(\bx; \theta) \big]
\end{align*}

where $m(\bx; \theta) = \nabla_\theta \prob^*(\bx; \theta)$

\section{Score Matching}
\section{Sampling and Monte Carlo}

\section{Sampling from Continuous Distributions}
\subsection{Rejection Sampling}
\subsection{Ancestral Sampling}
\subsection{Gibbs Sampling}

\section{Variational Inference}

% \printbibliography

\end{document}


% Like most advanced LaTeX files, this one begins with a lot of
% boilerplate. You don't need to understand (or even read) most of it.
% All you need to do is fill in your name, UMN ID, email address,
% and the number of the pset. (Search for "METADATA" to find the place
% for this.) Then, you can go straight to the "EXERCISE 1"
% section and start writing your solutions.
% The "VARIOUS USEFUL COMMANDS" section is probably worth taking a
% look at at some point.

%----------------------------------------------------------------------------------------
%	PACKAGES AND OTHER DOCUMENT CONFIGURATIONS
%----------------------------------------------------------------------------------------
\documentclass[paper=a4, fontsize=12pt]{scrartcl} % A4 paper and 12pt font size
\usepackage[T1]{fontenc} % Use 8-bit encoding that has 256 glyphs
\usepackage[english]{babel} % English language/hyphenation
\usepackage{amsmath,amsfonts,amsthm,amssymb} % Math packages
\usepackage{mathrsfs}    % More math packages
\usepackage{sectsty}  % Allows customizing section commands
\allsectionsfont{\centering \normalfont\scshape} % Make all section titles centered, the default font and small caps %remove this to left align section tites
\usepackage{hyperref} % Turns cross-references into hyperlinks,
                      % and defines \url and \href commands.
\usepackage{graphicx} % For embedding graphics files.
\usepackage{framed}   % For the "leftbar" environment used below.
\usepackage{ifthen}   % Used for the \powset command below.
\usepackage{lastpage} % for counting the number of pages
\usepackage[headsepline,footsepline,manualmark]{scrlayer-scrpage}
\usepackage[height=10in,a4paper,hmargin={1in,0.8in}]{geometry}
\usepackage[usenames,dvipsnames]{xcolor}
\usepackage{tikz}     % This is a powerful tool to draw vector
                      % graphics inside LaTeX. In particular, you can
                      % use it to draw graphs.
\usepackage{verbatim} % For the "verbatim" environment, in which
                      % special symbols can be used freely without
                      % confusing the compiler. (And it's typeset in
                      % a constant-width font.)
                      % Useful, e.g., for quoting code (or ASCII art).

%\numberwithin{table}{section} % Number tables within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)

\setlength\parindent{20pt} % Makes indentation for paragraphs longer.
                           % This makes paragraphs stand out more.

%----------------------------------------------------------------------------------------
%	VARIOUS USEFUL COMMANDS
%----------------------------------------------------------------------------------------
% The commands below might be convenient. For example, you probably
% prefer to write $\powset[2]{V}$ for the set of $2$-element subsets
% of $V$, rather than writing $\mathcal{P}_2(V)$.
% Notice that you can easily define your own commands like this.
% Caveat: Some of these commands need to be properly "guarded" when
% they occur in subscripts or superscripts. So you should not write
% $K_\CC$, but rather $K_{\CC}$.
\newcommand{\CC}{\mathbb{C}} % complex numbers
\newcommand{\RR}{\mathbb{R}} % real numbers
\newcommand{\QQ}{\mathbb{Q}} % rational numbers
\newcommand{\NN}{\mathbb{N}} % nonnegative integers
\newcommand{\DD}{{\mathbb{D}}} % dual numbers
\newcommand{\PP}{\mathbb{P}} % positive integers
\newcommand{\Z}[1]{\mathbb{Z}/#1\mathbb{Z}} % integers modulo k
                                            % (syntax: "\Z{k}")
\newcommand{\ZZ}{\mathbb{Z}} % integers
\newcommand{\id}{\operatorname{id}} % identity map
\newcommand{\lcm}{\operatorname{lcm}}
% Lowest common multiple. For historical reasons, LaTeX has a \gcd
% command built in, but not an \lcm command. The preceding line
% rectifies that.
\newcommand{\set}[1]{\left\{ #1 \right\}}
% $\set{...}$ compiles to {...} (set-brackets).
\newcommand{\abs}[1]{\left| #1 \right|}
% $\abs{...}$ compiles to |...| (absolute value, or size of a set).
\newcommand{\tup}[1]{\left( #1 \right)}
% $\tup{...}$ compiles to (...) (parentheses, or tuple-brackets).
\newcommand{\ive}[1]{\left[ #1 \right]}
% $\ive{...}$ compiles to [...] (Iverson bracket, aka truth value; also, set of first n integers).
\newcommand{\floor}[1]{\left\lfloor #1 \right\rfloor}
% $\floor{...}$ compiles to |_..._| (floor function).
\newcommand{\underbrack}[2]{\underbrace{#1}_{\substack{#2}}}
% $\underbrack{...1}{...2}$ yields
% $\underbrace{...1}_{\substack{...2}}$. This is useful for doing
% local rewriting transformations on mathematical expressions with
% justifications. For example, try this out:
% $ \underbrack{(a+b)^2}{= a^2 + 2ab + b^2 \\ \text{(by the binomial formula)}} $
\newcommand{\powset}[2][]{\ifthenelse{\equal{#2}{}}{\mathcal{P}\left(#1\right)}{\mathcal{P}_{#1}\left(#2\right)}}
% $\powset[k]{S}$ stands for the set of all $k$-element subsets of
% $S$. The argument $k$ is optional, and if not provided, the result
% is the whole powerset of $S$.
\newcommand{\mapeq}[1]{\underset{#1}{\equiv}}
% $\mapeq{f}$ yields $\underset{f}{\equiv}$.
% This is the "equality upon applying $f$" relation.
\newcommand{\eps}{\varepsilon}
% Just a shorthand for a commonly-used symbol.
\newcommand{\horrule}[1]{\rule{\linewidth}{#1}} % Create horizontal rule command with 1 argument of height
\newcommand{\nnn}{\nonumber\\} % Don't number this line in an "align" environment, and move on to the next line.

%----------------------------------------------------------------------------------------
%	MAKING SUMMATION SIGNS ALWAYS PUT THEIR BOUNDS ABOVE AND BELOW
%	THE SIGN
%----------------------------------------------------------------------------------------
% The following are hacks to ensure that sums (such as
% $\sum_{k=1}^n k$) always put their bounds (i.e., the $k=1$ and the
% $n$) underneath and above the sign, as opposed to on its right.
% Same for products (\prod), set unions (\bigcup) and set
% intersections (\bigcap). Remove the 8 lines below if you do not want
% this behavior.
\let\sumnonlimits\sum
\let\prodnonlimits\prod
\let\cupnonlimits\bigcup
\let\capnonlimits\bigcap
\renewcommand{\sum}{\sumnonlimits\limits}
\renewcommand{\prod}{\prodnonlimits\limits}
\renewcommand{\bigcup}{\cupnonlimits\limits}
\renewcommand{\bigcap}{\capnonlimits\limits}

%----------------------------------------------------------------------------------------
%	ENVIRONMENTS
%----------------------------------------------------------------------------------------
% The incantations below define how theorem environments
% (\begin{theorem} ... \end{theorem}) and their likes will look like.
\newtheoremstyle{plainsl}% <name>
  {8pt plus 2pt minus 4pt}% <Space above>
  {8pt plus 2pt minus 4pt}% <Space below>
  {\slshape}% <Body font>
  {0pt}% <Indent amount>
  {\bfseries}% <Theorem head font>
  {.}% <Punctuation after theorem head>
  {5pt plus 1pt minus 1pt}% <Space after theorem headi>
  {}% <Theorem head spec (can be left empty, meaning `normal')>

% Environments which make the text inside them slanted:
\theoremstyle{plainsl}
  \newtheorem{theorem}{Theorem}[section]
  \newtheorem{proposition}[theorem]{Proposition}
  \newtheorem{lemma}[theorem]{Lemma}
  \newtheorem{corollary}[theorem]{Corollary}
  \newtheorem{conjecture}[theorem]{Conjecture}
% Environments that don't:
\theoremstyle{definition}
  \newtheorem{definition}[theorem]{Definition}
  \newtheorem{example}[theorem]{Example}
  \newtheorem{exercise}[theorem]{Exercise}
  \newtheorem{examples}[theorem]{Examples}
  \newtheorem{algorithm}[theorem]{Algorithm}
  \newtheorem{question}[theorem]{Question}
 \theoremstyle{remark}
  \newtheorem{remark}[theorem]{Remark}
\newenvironment{statement}{\begin{quote}}{\end{quote}}
\newenvironment{fineprint}{\begin{small}}{\end{small}}

%----------------------------------------------------------------------------------------
%	METADATA
%----------------------------------------------------------------------------------------
\newcommand{\myname}{Darij Grinberg} % ENTER YOUR NAME HERE
\newcommand{\myid}{00000000} % ENTER YOUR UMN ID HERE
\newcommand{\mymail}{dgrinber@umn.edu} % ENTER YOUR EMAIL HERE
\newcommand{\psetnumber}{4} % ENTER THE NUMBER OF THIS PSET HERE

%----------------------------------------------------------------------------------------
%	HEADER AND FOOTER
%----------------------------------------------------------------------------------------
\ihead{Solutions to homework set \#\psetnumber} % Page header left
\ohead{page \thepage\ of \pageref{LastPage}} % Page header right
\ifoot{\myname, \myid} % left footer
\ofoot{\mymail} % right footer

%----------------------------------------------------------------------------------------
%	TITLE SECTION
%----------------------------------------------------------------------------------------
\title{
\normalfont \normalsize
\textsc{University of Minnesota, School of Mathematics} \\ [25pt] % Your university, school and/or department name(s)
\horrule{0.5pt} \\[0.4cm] % Thin top horizontal rule
\huge Math 4281: Introduction to Modern Algebra, \\
Spring 2019:
Homework \psetnumber\\% The assignment title
\horrule{2pt} \\[0.5cm] % Thick bottom horizontal rule
}
\author{\myname}

\begin{document}

\maketitle % Print the title

\begin{center} % Delete this if you want to save space!
{\large due date: \textbf{Wednesday, 13 March 2019} at the beginning of class, \\
or before that by email or canvas.

Please solve \textbf{at most 3 of the 6 exercises}!}
\end{center}

%----------------------------------------------------------------------------------------
%	EXERCISE 1
%----------------------------------------------------------------------------------------
\horrule{0.3pt} \\[0.4cm]

\section{Exercise 1: Equivalence relations always come from maps}

\subsection{Problem}

Let $S$ be a set.

Recall that if $T$ is a further set, and if $f : S \to T$
is a map, then $\mapeq{f}$
% The "\mapeq" command is not standard LaTeX.
% It is a macro I've defined in the header of this file.
% "\mapeq{f}" expands to "\underset{f}{\equiv}", which is
% a congruence sign with an "f" under it.
denotes the relation on $S$ defined by
\[
\tup{ a \mapeq{f} b }
\iff % This is the "if and only if" sign.
\tup{ f\tup{a} = f\tup{b} }.
\]
This is an equivalence relation, called ``equality upon
applying $f$'' or ``equality under $f$''.

Now, let $\sim$ be \textbf{any} equivalence relation on $S$.
Prove that $\sim$ has the form $\mapeq{f}$ for a properly
chosen set $T$ and a properly chosen $f : S \to T$.

More precisely, prove that $\sim$ equals $\mapeq{f}$,
where $T$ is the quotient set $S / \sim$ and where
$f : S \to T$ is the projection map
$\pi_\sim : S \to S / \sim$.

[\textbf{Hint:} To prove that two relations $R_1$ and $R_2$
on $S$ are equal, you need to check that every pair
$\tup{a, b}$ of elements of $S$ satisfies the equivalence
$\tup{a R_1 b} \iff \tup{a R_2 b}$.]

\subsection{Solution}

[...]

%----------------------------------------------------------------------------------------
%	EXERCISE 2
%----------------------------------------------------------------------------------------
\horrule{0.3pt} \\[0.4cm]

\section{Exercise 2: Totient-related sum}

\subsection{Problem}

Let $n > 1$ be an integer. Prove that
\[
\sum_{\substack{i \in \set{1, 2, \ldots, n}; \\ i \perp n}} i
= n \phi\tup{n} / 2 .
\]

\subsection{Solution}

[...]

%----------------------------------------------------------------------------------------
%	EXERCISE 3
%----------------------------------------------------------------------------------------
\horrule{0.3pt} \\[0.4cm]

\section{Exercise 3: Dual numbers}

\subsection{Problem}

Recall that complex numbers were defined as pairs
$\tup{a, b}$ of real numbers, with entrywise addition
and subtraction and a certain weird-looking
multiplication.

Let me define a different kind of ``numbers'': the
\textit{dual numbers}.
(The word ``numbers'' may appear a bit inappropriate
for them, but it is not exactly a trademark...)

We define a \textit{dual number} to be a pair
$\tup{a, b}$ of two real numbers $a$ and $b$.

We let $\DD$ % a macro for $\mathbb{D}$
be the set of all dual numbers.

For each real number $r$, we denote the dual number
$\tup{r, 0}$ by $r_\DD$.

We let $\eps$ denote the dual number $\tup{0, 1}$.

Define three binary operations $+$, $-$ and $\cdot$
on $\DD$ by setting
\begin{align}
\tup{a, b} + \tup{c, d} &= \tup{a+c, b+d}, \\
\tup{a, b} - \tup{c, d} &= \tup{a-c, b-d}, \\
\tup{a, b} \cdot \tup{c, d} &= \tup{ac, ad+bc}
\end{align}
for all $\tup{a, b} \in \DD$ and $\tup{c, d} \in \DD$.

(Note that the only difference to complex numbers
is the definition of $\cdot$, which is lacking
a $-bd$ term.)

Again, we are following standard PEMDAS rules\footnote{
\url{https://en.wikipedia.org/wiki/Order_of_operations}} for
the order of operations, and we abbreviate $\alpha \cdot \beta$
as $\alpha \beta$.

\begin{enumerate}

\item[\textbf{(a)}]
Prove that
$\alpha \cdot \tup{\beta \cdot \gamma}
= \tup{\alpha \cdot \beta} \cdot \gamma$
for any $\alpha, \beta, \gamma \in \DD$.

\end{enumerate}

You can use the following properties of dual numbers
without proof (they are all essentially obvious):

\begin{itemize}

\item We have $\alpha + \beta = \beta + \alpha$
for any $\alpha, \beta \in \DD$.

\item We have $\alpha + \tup{\beta + \gamma}
= \tup{\alpha + \beta} + \gamma$
for any $\alpha, \beta, \gamma \in \DD$.

\item We have
$\alpha + 0_\DD = 0_\DD + \alpha = = \alpha$
for any $\alpha \in \DD$.

\item We have
$\alpha \cdot 1_\DD = 1_\DD \cdot \alpha = \alpha$
for any $\alpha \in \DD$.

\item We have $\alpha \cdot \beta = \beta \cdot \alpha$
for any $\alpha, \beta \in \DD$.

\item We have $\alpha \cdot \tup{\beta + \gamma}
= \alpha \beta + \alpha \gamma$
and $\tup{\alpha + \beta} \cdot \gamma
= \alpha \gamma + \beta \gamma$
for any $\alpha, \beta, \gamma \in \DD$.

\item We have
$\alpha \cdot 0_\DD = 0_\DD \cdot \alpha = 0_\DD$
for any $\alpha \in \DD$.

\item If $\alpha, \beta, \gamma \in \DD$,
then we have the equivalence
$\tup{\alpha - \beta = \gamma}
\iff \tup{\alpha = \beta + \gamma}$.
\end{itemize}

We shall identify each real number $r$ with the dual
number $r_\DD = \tup{r, 0}$.

\begin{enumerate}

\item[\textbf{(b)}]
Prove that $a + b\eps = \tup{a, b}$
for any $a, b \in \RR$.

\end{enumerate}

An \textit{inverse} of a dual number
$\alpha \in \DD$ means a dual number $\beta$ such
that $\alpha \beta = 1_\DD$.
This inverse is unique, and is called $\alpha^{-1}$.

\begin{enumerate}

\item[\textbf{(c)}]
Prove that a dual number $\alpha = a + b\eps$
(with $a, b \in \RR$) has an inverse if and only
if $a \neq 0$.

\item[\textbf{(d)}]
If $a, b \in \RR$ satisfy $a \neq 0$, prove
that the inverse of the dual number $a + b\eps$
is $\dfrac{1}{a} - \dfrac{b}{a^2} \eps$.

\end{enumerate}

We define finite sums and products of dual numbers
in the usual way (i.e., just as finite sums and products
of real numbers were defined).
Here, an empty sum of dual numbers is always understood
to be $0_\DD$, whereas an empty product of dual numbers
is always understood to be $1_\DD$.

If $\alpha$ is a dual number and $k \in \NN$, then
the \textit{$k$-th power of $\alpha$} is defined to be the
dual number
$\underbrace{\alpha \alpha \cdots \alpha}_{k \text{ factors}}$.
This $k$-th power is denoted by $\alpha^k$.
Thus, in particular,
$\alpha^0 =
 \underbrace{\alpha \alpha \cdots \alpha}_{0 \text{ factors}}
= \tup{\text{empty product}} = 1_\DD$.

\begin{enumerate}

\item[\textbf{(e)}]
Let $P\tup{x} = a_k x^k + a_{k-1} x^{k-1} + \cdots + a_0$
be a polynomial with real coefficients.
Prove that
\[
P \tup{a + b \eps}
= P\tup{a} + b P^\prime\tup{a} \eps
\qquad \text{for any } a, b \in \RR .
\]
Here, $P^\prime$ denotes the derivative of $P$, which
is defined by
\[
P^\prime\tup{x}
= k a_k x^{k-1} + \tup{k-1} a_{k-1} x^{k-2} + \cdots + 1 a_1 x^0 .
\]

\end{enumerate}

\subsection{Remark}

The dual number $\eps$ is one of the simplest
``rigorous infinitesimals'' that appear in mathematics.
Part \textbf{(e)} of the exercise shows that we can
literally write
$P \tup{a + \eps}
= P\tup{a} + P^\prime\tup{a} \eps$
when $P$ is a polynomial,
without having to compute any limits.
It is tempting to ``solve'' this equation for
$P^\prime\tup{a}$, thus obtaining something like
$P^\prime\tup{a} = \dfrac{P \tup{a + \eps} - P\tup{a}}{\eps}$.
However, this needs to be taken with a grain of salt,
since $\eps$ has no inverse and the fraction
$\dfrac{P \tup{a + \eps} - P\tup{a}}{\eps}$
is not uniquely determined.
There are other, subtler ways to put infinitesimals
on a firm algebraic footing, but dual numbers are already
useful in some situations.

Note that dual numbers have \textit{zero-divisors}:
i.e., there exist nonzero dual numbers $a$ and $b$ such
that $ab = 0$. The simplest example is probably
$\eps^2 = 0$ (despite $\eps \neq 0$).

\subsection{Solution}

[...]

%----------------------------------------------------------------------------------------
%	EXERCISE 4
%----------------------------------------------------------------------------------------
\horrule{0.3pt} \\[0.4cm]

\section{Exercise 4: $\ZZ\ive{\sqrt2}$}

\subsection{Problem}

Let $\ZZ\ive{\sqrt2}$ denote the set of all reals of the
form $a + b \sqrt2$ with $a, b \in \ZZ$.
We shall call such reals \textit{$\sqrt2$-integers}.

\begin{enumerate}

\item[\textbf{(a)}]
Prove that any $\alpha, \beta \in \ZZ\ive{\sqrt2}$ satisfy
$\alpha + \beta \in \ZZ\ive{\sqrt2}$ and
$\alpha - \beta \in \ZZ\ive{\sqrt2}$ and
$\alpha \beta \in \ZZ\ive{\sqrt2}$.

\item[\textbf{(b)}]
Prove that every element of $\ZZ\ive{\sqrt2}$ can be
written as $a + b \sqrt2$ for a \textbf{unique} pair
$\tup{a, b}$ of integers.
(In other words, if four integers $a, b, c, d$
satisfy $a + b \sqrt2 = c + d \sqrt2$, then $a = c$
and $b = d$.)

\end{enumerate}

For any $\alpha\in \ZZ\ive{\sqrt2}$, define the
\textit{$\sqrt2$-norm} $N_2\tup{\alpha}$ of $\alpha$ by
$N_2\tup{\alpha} = a^2 - 2b^2$,
where $\alpha$ is written in the form $\alpha = a + b \sqrt2$
with $a, b \in \ZZ$.
This is well-defined by part \textbf{(b)} of this
exercise.

\begin{enumerate}

\item[\textbf{(c)}]
Prove that
$N_2\tup{\alpha \beta} = N_2\tup{\alpha} N_2\tup{\beta}$
for all $\alpha, \beta \in \ZZ\ive{\sqrt2}$.

\end{enumerate}

Let $\tup{p_0, p_1, p_2, \ldots}$ be the sequence of
nonnegative integers defined recursively by
\[
p_0 = 0, \qquad
p_1 = 1, \qquad
\text{and } \qquad
p_n = 2p_{n-1} + p_{n-2} \text{ for all } n \geq 2.
\]
(Thus, $p_2 = 2$ and $p_3 = 5$ and $p_4 = 12$ and so on.)

\begin{enumerate}

\item[\textbf{(d)}]
Prove that $p_{n+1} p_{n-1} - p_n^2 = \tup{-1}^n$
for each $n \geq 1$.

\item[\textbf{(e)}]
Prove that
$\tup{p_{n-1} + p_n + p_n \sqrt2} \cdot \tup{p_{n-1} + p_n - p_n \sqrt2} = \tup{-1}^n$
for each $n \geq 1$.

\end{enumerate}

[\textbf{Hint:} For \textbf{(d)}, use induction.]

\subsection{Remark}

The set $\ZZ\ive{\sqrt2}$ of $\sqrt2$-integers
is rather similar to the set
$\ZZ\ive{i}$ of Gaussian integers: the former has elements
of the form $a + b\sqrt2$ with $a, b \in \ZZ$,
while the latter has elements of the form
$a + b\sqrt{-1}$ with $a, b \in \ZZ$.
The $\sqrt2$-norm on $\ZZ\ive{\sqrt2}$ is an analogue of
the (usual) norm on $\ZZ\ive{i}$.
However, visually speaking, the latter set is ``spread
out'' in the Euclidean plane, while the former is
``concentrated'' on the real line (and actually everywhere
dense on it -- i.e., every little interval on the real
line has a $\sqrt2$-integer inside it).
The difference has algebraic consequences; in particular,
there are only four units ($1, -1, i, -i$) in $\ZZ\ive{i}$,
whereas $\ZZ\ive{\sqrt2}$ has infinitely many units
(namely, part \textbf{(e)} of the exercise shows that
$p_{n-1} + p_n + p_n \sqrt2$ is a unit for each $n \geq 1$).

\subsection{Solution}

[...]

%----------------------------------------------------------------------------------------
%	EXERCISE 5
%----------------------------------------------------------------------------------------
\horrule{0.3pt} \\[0.4cm]

\section{Exercise 5: Euler's theorem for non-coprime integers}

\subsection{Problem}

Let $a$ be an integer, and let $n$ be a positive integer.
Prove that
$a^n \equiv a^{n - \phi\tup{n}} \mod n$.

\subsection{Solution}

[...]

%----------------------------------------------------------------------------------------
%	EXERCISE 6
%----------------------------------------------------------------------------------------
\horrule{0.3pt} \\[0.4cm]

\section{Exercise 6: Wilson strikes again}

\subsection{Problem}

Let $p$ be an odd prime.
Write $p$ in the form $p = 2k+1$ for some $k \in \NN$.
Prove that $k!^2 \equiv - \tup{-1}^k \mod p$.

[\textbf{Hint:} Each $j \in \ZZ$ satisfies $j \tup{p-j} \equiv -j^2 \mod p$.]

\subsection{Solution}

[...]

\begin{thebibliography}{99999999}                                                                                %

% Feel free to add your sources -- or copy some from the source code
% of the class notes ( http://www.cip.ifi.lmu.de/~grinberg/t/19s/notes.tex ).

% This is the bibliography: The list of papers/books/articles/blogs/...
% cited. The syntax is: "\bibitem[name]{tag}Reference",
% where "name" is the name that will appear in the compiled
% bibliography, and "tag" is the tag by which you will refer to
% the source in the TeX file. For example, the following source
% has name "Grinbe19" (so you will see it referenced as
% "[Grinbe19]" in the compiled PDF) and tag "detnotes" (so you
% can cite it by writing "\cite{detnotes}").

\bibitem[Grinbe19]{detnotes}Darij Grinberg,
\textit{Notes on the combinatorial fundamentals of algebra},
10 January 2019. \\
\url{http://www.cip.ifi.lmu.de/~grinberg/primes2015/sols.pdf}
\\
The numbering of theorems and formulas in this link might shift
when the project gets updated; for a ``frozen'' version whose
numbering is guaranteed to match that in the citations above, see
\url{https://github.com/darijgr/detnotes/releases/tag/2019-01-10} .

\end{thebibliography}

\end{document}


\chapter{Online reprocessing modeling approach}
\section{Fuel salt reprocessing overview} \label{sec:reproc-plant}
Removing specific chemical elements from a molten salt is a complicated
task that requires intentional design (e.g., chemical separations equipment
design, fuel salt flows to equipment). This section contains a brief overview
of a generic \gls{MSR} fuel salt reprocessing system; modeling such systems is
the focus of the current dissertation.

\subsection{Gas separation system} \label{sec:gas-separ}
Gaseous fission products (e.g., Xe) must be removed from the fuel salt
to avoid reactor poisoning, especially during startup and power maneuvering.
This is particularly true for $^{135}$Xe, with its strong neutron capture
cross section ($\approx10^6\dots10^7$ b in a thermal energy range). $^{135}$Xe
is produced directly from fission in about 0.2\% of $^{235}$U fissions
($\gamma_{_{^{135}Xe}}$), but an even larger fraction of $^{135}$Xe is
produced by the decay of $^{135}$I and $^{135}$Te (Table~\ref{tab:xe_gain}).
$^{135}$I and $^{135}$Te yields from fission are
$\gamma_{_{^{135}I}}\!=3.1$\% and $\gamma_{_{^{135}Te}}\!=3.3$\%,
respectively. Thus, the total $^{135}$Xe production from fission is about
6.6\% of fissions (of $^{235}$U), most of this is from $^{135}$I and
$^{135}$Te decay. Noble gases (e.g., tritium, xenon, and krypton) can be
removed from the fuel salt as follows:
\begin{enumerate}[label=(\alph*)]
	\item a bubble generator injects helium bubbles in the salt stream;
	\item noble gases migrate to the helium bubbles due to their insolubility
	in the salt \cite{robertson_conceptual_1971};
	\item and a gas separator discharges the fission-product-rich bubbles from
	the salt to the off-gas system.
\end{enumerate}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{table}[ht!]
	\caption{$^{135}$Xe production sources and principal rate constants
		involved (reproduced from Kedl \emph{et al.}
		\cite{kedl_development_1967}).}
	\centering
	\begin{tabularx}{\textwidth}{b  b}
		\hline \textbf{$^{135}$Xe gain mechanism} & \textbf{Principal rate
			parameters involved}  	\\ [5pt] \hline
		Direct from fission & $\Sigma_f \gamma_{_{^{135}Xe}}\phi$ (for
		$^{235}$U fission) \\
		yield $\gamma_{_{^{135}Xe}}\!\!\!=0.0022$ & \\ [5pt] \hline
		$^{135}$I decay     & $\Sigma_f \gamma_{_{^{135}I}}\phi$ (for
		$^{235}$U fission) \\
		yield $\gamma_{_{^{135}Xe}}\!\!\!=0.031$, it decays to $^{135}$Xe with
		$\tau_{1/2}=6.68$ h & 			                    \\	[5pt]	\hline
		$^{135}$Te decay    & $\Sigma_f \gamma_{_{^{135}Te}}\phi$ (for
		$^{235}$U 		fission) \\
		yield $\gamma_{_{^{135}Xe}}\!\!\!=0.033$,
		it decays to $^{135}$I with $\tau_{1/2}=19$ s
		& 			                    \\ [5pt]	\hline
	\end{tabularx}
	\label{tab:xe_gain}
\end{table}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Figure~\ref{fig:xe_diagram} shows the key pathways for xenon production,
accumulation, and removal in a typical \gls{MSR}.
Figure~\ref{fig:gas_removal_system} shows the conceptual design of the
\gls{MSBR} gas separation system. In that system, helium bubbles of a specific
size are introduced in a salt stream via the primary pump bowl. These bubbles
absorb noble gases before being separated from the salt by a gas separator.
\gls{ORNL} suggested that the \gls{MSBR} off-gas system would inject
$d=0.508mm$ helium bubbles in the pump bowl, redirect 10\% of the fuel salt
flow through a bubble separator to remove the bubbles, and then return the
flow into the pump suction. Robertson \emph{et al.} reported that the helium
bubble size was approximately 25\% of the throat width (blue circle on
Figure~\ref{fig:bubble_separator}) and was independent of the gas flow rate
\cite{robertson_conceptual_1971}. Consequently, it is possible to regulate the
helium bubble size by changing the throat width in the bubble generator.
\begin{figure}[htp!] % replace 't' with 'b' to
	\centering
	\includegraphics[width=1.01\textwidth]{ch2/xe_diagram.pdf}
	\caption{Schematic of $^{135}$Xe circulation in a generic \gls{MSR}. $x$
	is the fraction of fuel salt flow from the pump discharge redirected to
	the gas separation system, while $\epsilon_m$ and $\epsilon_{es}$ are the
	efficiencies of migration (of $^{135}$Xe to the helium bubbles in the
	sparger) and separation (of gas in the entrainment separator),
	respectively. The orange color represents the fuel salt in the primary
	loop, the blue color represents the gas separation system, and the gray
	color is the moderator in the core. Fission yields assume $^{235}$U
	fission 	only.}
	\label{fig:xe_diagram}
\end{figure}
\begin{figure}[htp!] % replace 't' with 'b' to
	\centering
	\includegraphics[width=1.02\textwidth]{ch2/gas_separation.pdf}
	\caption{Schematic flow diagram of the \gls{MSBR} gas separation system
		(reproduced from Robertson \emph{et al.}
		\cite{robertson_conceptual_1971}).}
	\label{fig:gas_removal_system}
\end{figure}
\begin{figure}[htp!] % replace 't' with 'b' to
	\centering
	\includegraphics[width=0.77\textwidth]{ch2/msbr_bubble_generator.png}
	\caption{Preliminary concept of an \gls{MSBR} bubble generator (reproduced
	from Robertson \emph{et al.} \cite{robertson_conceptual_1971}).
	The blue circle shows throat width, which determines bubble size.}
	\vspace{-0.25in}
	\label{fig:bubble_separator}
\end{figure}
\begin{table}[t]
	\caption{$^{135}$Xe loss terms and principal rate constants involved
		(reproduced from Kedl \emph{et al.} \cite{kedl_development_1967}).}
	\centering
	\begin{tabularx}{\textwidth}{b b}
		\hline \textbf{$^{135}$Xe loss mechanism}      & \textbf{Principal
			rate parameters involved}  	\\[5pt]
		\hline Decay of dissolved $^{135}$Xe ($\tau_{1/2}=9.1$ h)  & Decay
		constant	($\lambda$)		\\[5pt]
		\hline $^{135}$Xe burnup              &  Neutron flux
		($\phi$)		 					\\[5pt]
		dissolved $^{135}$Xe burnup as it passes through the core
		& 			            \\
		\hline $^{135}$Xe migrated to
		helium bubbles & Removal efficiency
		($\epsilon_m$)		\\[5pt]
		\hline $^{135}$Xe transferred into circulating He bubbles; this xenon
		will eventually be burnup, decay, or stripped via bubble separator &
		Mass transfer coefficient ($h$), decay constant ($\lambda$),
		neutron flux ($\phi$), bubble removal efficiency
		($\epsilon_{es}$)		\\[5pt]
		\hline
	\end{tabularx}
	\label{tab:xe_loss}
\end{table}

To realistically model the gas separation system, we need a mathematical model
that describes noble gas extraction efficiency during reactor operation.
Particularly, a model of xenon extraction efficiency as a function of sparger
design parameters is needed to accurately model the $^{135}$Xe removal in a
fuel salt depletion simulation. The gain and loss terms for $^{135}$Xe
dissolved in the fuel salt are listed in Tables~\ref{tab:xe_gain} and
\ref{tab:xe_loss}. The removal efficiency for the xenon in the pump bowl was
measured during \glsfirst{MSRE} operation. However, the technical report
ORNL-4069 by Kedl-Houtzeel only stated its range (from 50 to 100\%) and
concluded, ``It is probably a complex parameter like the circulating-void
fraction and depends on many reactor operational variables''
\cite{kedl_development_1967}. %$^{135}$Xe burnup and decay rates are well
%known.

Peebles \emph{et al.} in ORNL-TM-2245 has reported xenon removal efficiency
($\epsilon_{Xe}$) in a gas separation system as a function of many parameters
\cite{peebles_removal_1968}:
\begin{align}\label{eq:gas_eff}
& \epsilon_m = \frac{1-e^{-\beta}}{1+\alpha}
\intertext{where}
\alpha &= \frac{RT}{H}\cdot\frac{Q_{salt}}{Q_{He}} \\
\beta &= \frac{K_L a A_C L (1+\alpha)}{Q_{salt}} \\
R &= \mbox{universal gas constant $[L\cdot Pa\cdot mol^{-1}\cdot K^{-1}]$}
\nonumber \\
T &= \mbox{salt temperature $[K]$} \nonumber \\
H &= \mbox{Henry's law constant for solute gas $[Pa\cdot mol^{-1}\cdot L]$}
\nonumber \\
Q_{salt}&= \mbox{volumetric salt flow rate $[m^3/s]$} \nonumber \\
Q_{He}&= \mbox{volumetric helium flow rate $[m^3/s]$} \nonumber \\
K_L &= \mbox{liquid phase mass transfer coefficient $[m/s]$} \nonumber \\
a &= \mbox{gas-liquid interfacial area per unit volume $[m^{-1}]$} \nonumber \\
A_C &= \mbox{contactor cross section $[m^2]$} \nonumber \\
L &= \mbox{contactor length $[m].$} \nonumber
\end{align}
Most of the input parameters for that correlation are obvious and easy to
obtain from the system component design. The mass transfer coefficient for
transferring xenon into helium bubbles ($K_L$) can be estimated
experimentally, but published information is currently insufficient
to inform an accurate mathematical model appropriate for \gls{CFD}. Thus,
Peebles \emph{et al.} reported the mass transfer coefficient correlation for
the \gls{MSBR} salt (LiF-BeF$_2$-ThF$_4$-UF$_4$) but for a limited case. While
it is out of the scope of this work to accurately estimate mass transfer
coefficient, this work seeks to provide a tool which would allow the user to
specify any mathematical model for a separation efficiency. Provided a mass
transfer coefficient, the user can incorporate it into the model.

Equation~\ref{eq:gas_eff} would apply to other noble gases (e.g., Kr), but
Henry's law constant ($H$) varies by element. Current effort at the University
of Illinois at Urbana-Champaign, namely ``Enabling Load Following Capability
in the Transatomic Power \gls{MSR}," \cite{huff_enabling_2018} has a goal to
determine mass transfer coefficients for various gaseous fission products (Ar,
Kr, Xe) using experiments, enabling \gls{CFD} and multi-physics simulations of
such reactors. As a result, the obtained mathematical model for gas removal
efficiency might be employed to inform a realistic physics-based fuel
reprocessing model in SaltProc.

\subsection{Insoluble fission product filtering}
Approximately 40\% of \glspl{FP} have gaseous elements in their decay chains.
Some of the non-gaseous \glspl{FP} produced in the \gls{MSR} core (e.g.,
noble and semi-noble metals) have negligible solubility in the molten salt.
Some fraction of noble and semi-noble solid \glspl{FP} plate out onto
the internal surfaces of the primary loop equipment, complicating their
removal \cite{briggs_molten-salt_1964}. The remaining noble and semi-noble
metals can be removed along with corrosion products using a mechanical
filtration system, which ``consists largely of a high surface area mechanical
filter, likely a nickel mesh, to promote deposition of suspended, undissolved
fission and corrosion products,'' according to Holcomb \emph{et al.}
\cite{holcomb_instrumentation_2018}. The filter is manufactured from porous
metal, has limited capacity, and needs periodic replacement. The filter
replacement must be done using remote-controlled equipment due to high
radiotoxicity of undissolved \glspl{FP} and residual fuel salt remaining on
the filter \cite{mcfarlane_review_2019}.

The historic \gls{MSRE} program provided basic information design and
performance of the large mechanical filter.
Figure~\ref{fig:large_filter_layout} shows the piping layout of the filter,
storage, and processing tanks. The filter pressure vessel is made of
high-nickel alloy (Inconel) and accommodates 40-$\mu m$ pore size sintered
Inconel fibers. This large molten salt filter had a total filtering area of
0.8${m^2}$ and was designed to filter approximately 1 kg of the molten salt
per minute, but the removal efficiency has never been reported. Also, the
design of the filter, the filter holder, and the remotely operated equipment
for the filter replacement for commercial-scale \gls{MSR} designs presents a
significant engineering challenge \cite{mcfarlane_review_2019}.
\begin{figure}[htp!] % replace 't' with 'b' to
	\centering
	\includegraphics[width=.75\textwidth]{ch2/large_molten_salt_filter_layout.png}
	\caption{Schematic flow diagram of the large molten salt mechanical filter
		designed and operated during the \gls{MSRE} (reproduced from
		Lindauer \emph{et al.} \cite{lindauer_design_1969}).}
	\label{fig:large_filter_layout}
\end{figure}

In this work, we assumed ideal, constant separation efficiency in the
filtering system. However, in the future, a physics-driven mathematical
formula can be used when the experimental data or analytical model will be
available.

\subsection{Fuel chemical processing facility} \label{sec:chemical_processing}
In addition to noble gases, noble metals, and semi-noble metals, the fuel salt
reprocessing system must extract other \glspl{FP} such as the lanthanides.
These absorb fewer neutrons than $^{135}$Xe, but their removal is crucial to
guarantee normal operation. Unfortunately, lanthanides have relatively high
solubility in the carrier salt and must be removed by chemical extraction.

In thorium-fueled \gls{MSR} designs, $^{232}$Th in the fuel salt absorbs
thermal neutrons and produces $^{233}$Pa, which then decays into the fissile
$^{233}$U (Figure~\ref{fig:th_u_reaction}). Protactinium presents a challenge
since it has a large absorption cross section in the thermal energy spectrum.
Accordingly, $^{233}$Pa is continuously removed from the fuel salt into a tank
in which $^{233}$Pa decays to $^{233}$U without poisoning the reactor. This
feature allows the thorium-fueled \gls{MSR} to avoid neutron losses to
protactinium, keeps \glspl{FP} on a trace level, and increases the efficiency
of $^{233}$U breeding.
\begin{figure}[htp!] % replace 't' with 'b' to
	\centering
	\includegraphics[width=0.57\textwidth]{ch2/th_u_cycle.jpg}
	\caption{Production of $^{233}$U from $^{232}$Th  (reproduced from
	Sorensen \cite{sorensen_one-fluid_2006}).}
	\label{fig:th_u_reaction}
\end{figure}

Many authors report that a liquid-liquid reductive extraction process is the
best option for removing protactinium and soluble \glspl{FP} from
molten fluoride salts \cite{briggs_molten-salt_1969, delpech_molten_2010,
doligez_coupled_2014}. In that process, the protactinium or lanthanides can be
selectively stripped from the salt into liquid bismuth due to different
chemical potentials. Moreover, the \gls{MSRE} experience indicated that the
extraction could be carried out rapidly and continuously
\cite{whatley_engineering_1970}.

The principal scheme of the \gls{MSBR} reprocessing facility concept is shown
in Figure~\ref{fig:material_flow}. The fuel salt is first temporarily stored
for cooling and decay of the shortest-lived fission products, then it is
directed to the primary fluorinator. There, most of the uranium is removed by
fluorination to UF$_6$. After that, the salt is routed to an extraction column
where it is combined with a mixture containing metallic bismuth, lithium, and
thorium reductants. The remaining uranium and protactinium are reductively
extracted to a bismuth solution, leaving a salt that only contains fission
products dissolved in carrier salt (base composition LiF-BeF$_2$-ThF$_4$). The
salt then goes through a reduction column where UF$_6$ is reduced to UF$_4$,
preparing it for return to the reactor. BeF$_2$ and ThF$_4$ are also added and
all residual bismuth is removed from the salt. After a final cleanup step and
valence adjustment, the purified salt returns to the reactor
\cite{carter_design_1972, sorensen_one-fluid_2006}.
\begin{figure}[htp!] % replace 't' with 'b' to
	\centering
	\includegraphics[width=1.01\textwidth]{ch2/flowsheet.pdf}
	\caption{Simplified block diagram of chemical processing scheme for
	a single-fluid \gls{MSBR} (reproduced from Sorensen
		\cite{sorensen_one-fluid_2006}). \emph{RE} represents the rare
		earth elements extracted from the salt.}
	\label{fig:material_flow}
\end{figure}

The bismuth, accommodating some uranium and protactinium, is routed to a
hydrofluorination column where metallic solutes in the bismuth are oxidized
into their fluoride forms in the presence of a decay salt\footnote{The decay
salt contains UF$_4$, PaF$_4$, ThF$_4$ and \glspl{FP}. Uranium produced
after $^{233}$Pa decay is extracted and directed back into the reactor. Decay
salt is the precursor for the waste salt as it was periodically discarded
every 220 days \cite{robertson_conceptual_1971}.}. The decay salt, containing
UF$_4$, PaF$_4$, and ThF$_4$, passes into a decay tank where $^{233}$Pa
decays to $^{233}$U. The uranium generated by protactinium decay is removed
through fluorination to UF$_6$ and directed to the reduction column to refuel
the purified fuel salt. A  hydrofluorinator and a fluorinator can remove
approximately 95\% of the uranium from the stream
\cite{robertson_conceptual_1971}.

To maintain or adjust the fissile material concentration in the reactor
(and, consequently, control the reactivity), $^{233}$U is added from the
protactinium decay tank to fully processed salt on its way back to the
reactor. Adding fissile material is performed by sparging the salt with UF$_6$
and hydrogen to produce UF$_4$ in the salt and HF gas
\cite{robertson_conceptual_1971}.

After these separation steps, the fuel salt stream from the protactinium
isolation system contains only traces of protactinium and uranium but contains
practically all of the rare earths. A fraction of this salt stream is
redirected to a reductive extraction process for removing rare earths.  The
principal scheme of a rare earth removal system is shown in
Figure~\ref{fig:rare-earth-removal}. A molten salt flow that contains
rare earth fluorides is fed to the center of an extraction column. The salt
flows countercurrent to a liquid bismuth stream, which contains thorium and
lithium. In the upper part of the column, the rare earths are reduced and
transferred to the downflowing liquid metal stream. Below the feed point, the
rare earth concentration is increased in the salt and metal streams in order
to produce a concentration high enough for disposal
\cite{briggs_molten-salt_1969}.
\begin{figure}[htbp!]
	\centering
	\includegraphics[width=0.45\textwidth]{ch2/rare-earths-removal-system.png}
	\caption{Rare earth removal from a fuel salt by reductive extraction
		(reproduced from Briggs \emph{et al.}
		\cite{briggs_molten-salt_1969}).}
	\label{fig:rare-earth-removal}
\end{figure}

While it is out of the scope of this work to derive the accurate
chemistry-based mathematical formula for rare earths and protactinium
separation efficiency, this work seeks to provide a flexible tool that is able
to simulate chemical processes in significant detail concerning vital system
design parameters.

\section{Serpent overview}
Serpent is a continuous-energy Monte Carlo neutronics software capable of
solving the neutron transport problem by tracking individual neutrons within
the problem geometry and using the stochastic method to determine the chain of
events for each neutron \cite{leppanen_serpent_2014}. Serpent is under active
development at the VTT Technical Research Centre of Finland since 2004, where
it was initially conceived as a tool to simplify group constant generation in
a high-fidelity Monte Carlo environment. Serpent is now widely used by more
than 500 registered individuals in 155 organizations located in 37 countries
around the world. The burnup calculation capability in Serpent is based on
built-in calculation routines without using any external solvers. A restart
feature enables fuel shuffling simulation or applying any modifications to the
input by dividing the calculation into several parts, which is crucial for
online reprocessing simulations.

The latest version, Serpent 2, supports advanced geometries and universe
transformation that are necessary for neutronics modeling of advanced nuclear
reactors. Additionally, multi-physics simulations using Serpent 2 have been
demonstrated, including  calculations with thermal-hydraulics, \gls{CFD}, and
fuel performance codes \cite{leppanen_numerical_2015}.

Serpent 2 can be effectively run in parallel on computer clusters and
multi-core workstations. Parallelization is handled by thread-based OpenMP,
which enables all processors to use shared memory space. Calculations can be
divided into several nodes by distributed-memory \gls{MPI} parallelization.
Serpent 2  is an improvement upon Serpent 1 and contains a complete redesign
of memory management using hybrid OpenMP \cite{dagum_openmp_1998} + \gls{MPI}
parallelization.  This hybrid parallelization is substantial for depletion
calculations using computer clusters with multiple nodes and allows us to
achieve significant speed-up in depletion calculations on computer clusters
with more than 4,000 cores \cite{leppanen_serpent_2014}.

Simulations herein were performed using Serpent 2 version 2.1.31 on both the
National Center for Supercomputing Applications' Blue Waters and Idaho
National Laboratory's Falcon supercomputers. The JEFF-3.1.2
\cite{oecd/nea_jeff-3.1.2_2014} and ENDF/B-VII.1
\cite{chadwick_endf/b-vii.1_2011} libraries provided nuclear data
for all calculations in this dissertation.

\section{Simulation tool design and capabilities}\label{sec:tool_design}
The first version of the SaltProc tool for calculating \gls{MSR} fuel
composition evolution, taking into account an online reprocessing system,
was developed in 2018 as a part of my M.S. thesis
\cite{rykhlevskii_arfc/saltproc_2018, rykhlevskii_advanced_2018}. The tool was
designed to expand Serpent 2 depletion capabilities for modeling liquid-fueled
\glspl{MSR} with online fuel reprocessing systems. SaltProc v0.1 uses HDF5
\cite{the_hdf_group_hierarchical_1997} to store data and the PyNE Nuclear
Engineering Toolkit \cite{scopatz_pyne_2012} for Serpent 2 output file parsing
and nuclide naming. SaltProc v0.1 is an open-source Python package that uses a
batch-wise approach to simulate continuous feeds and removals in \glspl{MSR}.

SaltProc v0.1 only allows 100\% separation efficiency for either specific
elements or groups of elements at the end of the specific ``cycle
time''\footnote{The \gls{MSBR} program defined ``cycle time'' as the time
required to remove 100\% of a target nuclide from a fuel salt
\cite{robertson_conceptual_1971}.}. Capabilities of the developed tool,
working with the Monte Carlo software Serpent 2, were demonstrated using the
full-core MSBR design for a simplified case with ideal removal efficiency
(100\% of mass for target elements removed) \cite{rykhlevskii_modeling_2019}.
Chapter 3 covers the summary of this effort.
The SaltProc v0.1 architecture and the principal structure were not designed
for flexible implementation of sophisticated online reprocessing systems,
including realistic variable extraction efficiencies.

In the current work, SaltProc v0.1 was completely refactored using \gls{OOP}
to create a comprehensive generic tool to realistically model complex
\gls{MSR} fuel reprocessing systems while taking into account variable
extraction efficiencies, time-dependent core geometry, and the mass balance
between the core and the reprocessing plant.

\subsection{Software architecture}

The SaltProc v1.0 Python toolkit couples directly with Serpent 2 input
and output files, to couple the reprocessing system to depletion calculation.
Python 3 \gls{OOP} standard features are used to create a flexible,
user-friendly tool with great potential for further improvement and
collaboration. Figure~\ref{fig:saltproc_class} shows the SaltProc v1.0 class
structure which includes 4 main classes:
\begin{figure}[ht!] % replace 't' with 'b' to \centering
	\includegraphics[width=1.01\textwidth]{ch2/saltproc_class_diagram.png}
	\vspace{-0.15in}
	\caption{SaltProc v1.0 Python package class diagram in UML
	notation with examples of object instances.}
	\label{fig:saltproc_class}
\end{figure}
\paragraph*{Depcode.}\textit{Depcode} class contains attributes and methods
for reading the user's input file for the depletion software, initial material
(e.g., fuel and/or fertile salt) composition, principal parameters for burnup
simulation (e.g., neutron population and number of cycles for Monte Carlo
neutron transport), and running the depletion code.
\paragraph*{Simulation.}\textit{Simulation} class runs a depletion
step, creates and writes an HDF5 database, tracks time, and converts isotopic
composition vector nuclide names from a depletion code format to
human-readable format.
\paragraph*{MaterialFlow.}Each \textit{MaterialFlow} object represents the
material flowing between \textit{Process} objects
(Figures~\ref{fig:matflow_obj} and \ref{fig:matflow_obj_tap}). All instances
of this class contain an isotopic composition vector stored in PyNE Material
object \cite{scopatz_pyne_2012}, mass flow rate, temperature, density, volume,
and void fraction.
Existing PyNE Material capabilities convert the units of the isotopic
composition vector (e.g., from the atomic density provided by Serpent to a
mass fraction or absolute mass in desired units) and decay the material (i.e.,
model the \gls{MSBR} protactinium decay tank). The main idea of the
\textit{MaterialFlow} object is to pass detailed information about the salt
starting at the \gls{MSR} vessel outlet throughout reprocessing components
(\textit{Processes}), which modify the \textit{MaterialFlow} object before
depleting the material in the next depletion step.
\begin{figure}[ht!] % replace 't' with 'b' to
	\centering
	\includegraphics[width=0.7\textwidth]{ch2/materialflow.pdf}
	\caption{Schematic for passing material data between fuel processing
		system components shown for a general case.}
	\label{fig:matflow_obj}
\end{figure}
\begin{figure}[ht!] % replace 't' with 'b' to
	\centering
	\includegraphics[width=0.9\textwidth]{ch2/tap_materialflow.pdf}
	\caption{Schematic for passing material data between fuel processing
		system components shown for the \gls{TAP} concept.}
	\label{fig:matflow_obj_tap}
\end{figure}

\paragraph*{Process.}Each \textit{Process} object represents a
realistic fuel processing step characterized by its throughput rate,
volumetric capacity, extraction efficiency for each target element (can be
a function of many parameters), waste streams, and other process-specific
parameters. The feed \textit{Process} injects fresh fuel salt
\textit{MaterialFlow} directly into the reactor core (e.g., adding fissile
material with a specific mass flow rate to \textit{MaterialFlow} after
performing all removals).\\

Such a class structure provides outstanding flexibility in simulating
various \gls{MSR} fuel processing system designs. I created a library of
various \textit{MaterialFlow} (e.g., fuel salt flow, fertile salt flow,
refueling salt flow) and \textit{Process} (e.g., helium sparging facility, gas
separator, nickel filter) object examples to help a user to create a model of
a desired reprocessing scheme quickly. At runtime, the user
should connect \textit{Process} objects in series, parallel, or both with
\textit{MaterialFlow} objects to form a comprehensive reprocessing system. To
make the reprocessing system definition self-explanatory and straightforward,
I employed a standardized graph description language, \emph{dot}, which is
widely used in computer science for describing undirected and directed graphs
\cite{gansner_drawing_1996}. A directed graph is a set of objects
(vertices) connected by edges, where the edges have a direction associated
with them. In the context of this work, a vertex is \textit{Process} object
(a component of the fuel processing systems), and a directed graph edge is
\textit{MaterialFlow} object (salt flow).
\begin{figure}[hbt!] % replace 't' with 'b' to
	\centering
	\includegraphics[width=0.2\textwidth]{ch2/digraph.png}
	\vspace{-0.1in}
	\caption{General example of directed graph with a source (e.g., a reactor
		outlet) and sink (e.g., a reactor inlet).}
	\label{fig:digraph}
\end{figure}

The reprocessing plant structure described with \emph{dot} can be simply
plotted using Graphviz \cite{ellson_graphviz_2003} and those plots can be used
for analysis, optimization, and publication purposes. The user also has the
flexibility to create custom objects with desired attributes and methods and
contribute back to the code package using GitHub
(\url{https://github.com/arfc/saltproc}).

\subsection{Tool flowchart}
Figure~\ref{fig:saltproc_flow} illustrates the online reprocessing simulation
algorithm coupling SaltProc v1.0 and Serpent. A \emph{json}-compatible
user input file for SaltProc contains depletion software parameters such as
paths to the depletion software executable, neutron population and number of
criticality cycles, and total heating power. Additionally, the input file
contains reprocessing system parameters such as structure of reprocessing
system, capacity and efficiency of the system components, and molten salts
thermophysical properties. To perform a depletion step, SaltProc v1.0 reads a
user-defined Serpent template file. This file contains input parameters such
as the path to a nuclear data library, material isotopic composition at
startup, burnup calculation parameters, and boundary conditions. SaltProc v1.0
fills in the template file and runs Serpent single-step depletion.

After the depletion calculation, SaltProc v1.0 reads the depleted fuel
composition file into the \textit{MaterialFlow} object
(\textit{core\textunderscore outlet} in Figure~\ref{fig:saltproc_flow}). This
object contains an isotopic composition vector, total volume of material,
total mass, mass flow rate, density, temperature, void fraction, etc. For the
simplest reprocessing case, if all fuel processing components are connected in
series (100\% of total material flow goes through a chain of separation
components), the \textit{core\textunderscore outlet} object flows sequentially
between \textit{Processes}, and each \textit{Process} removes a mass fraction
of target elements with specified extraction efficiency.
After that the removed material mass is compensated by fresh fuel salt to
maintain the salt inventory in a primary loop. Finally, the resulting isotopic
composition after reprocessing is stored in the HDF5 database and dumped in a
new composition file for the next Serpent depletion run. SaltProc v1.0 also
stores the isotopic composition before reprocessing and waste stream from each
fuel processing component in the HDF5 database.
\begin{figure}[ht!] % replace 't' with 'b' to \centering
	\centering
	\includegraphics[width=\textwidth]{ch2/saltproc_flowchart.pdf}
	\caption{Flow chart for the SaltProc v1.0 Python package.}
	\label{fig:saltproc_flow}
\end{figure}

For a more general case with multiple concurrent extraction processes,
separate \textit{MaterialFlow} objects are created for each branch with a
user-defined mass flow branching percentage (e.g., 90\% of total mass flow
rate flows via left branch and 10\% throughout a right branch). The total mass
and isotopic composition vector for each \textit{MaterialFlow} object are
calculated as a fraction of incoming \textit{core\textunderscore outlet} flow.
Then each \textit{MaterialFlow} object is passed via a cascade of
\textit{Processes} to separate selected chemical elements with specific
efficiency. Finally, the left-hand-side \textit{MaterialFlow} object is merged
with the right-hand-side, and similarly to the previous case, fresh fuel salt
feed compensates for mass losses in the \textit{Processes} and keeps the fuel
salt mass in the primary loop constant.

The UML diagram (Figure~\ref{fig:saltproc_class}) allows the user to model a
complex, multi-zone, multi-fluid \gls{MSR} operation and is sufficiently
general to represent myriad reactor systems. SaltProc v1.0 only stores and
changes the isotopic composition of the fuel stream, which makes it a flexible
tool to model any geometry: an infinite medium, a unit cell, a multi-zone
simplified assembly, or a full-core. This flexibility allows the user to
perform simulations of varying fidelity and computational intensity.
SaltProc v1.0 is an open-source tool available on GitHub. Although the user
needs Serpent $\geq$2.1.31 installed on his computer to use SaltProc. The tool
leverages unit tests and continuous integration crucial for software
sustainability \cite{krekel_pytest_2004, travis_travis-ci/travis-api_2016}.
The documentation automatically generated using Sphinx
\cite{brandl_sphinx:_2009} is available here:
\url{https://arfc.github.io/saltproc/}. In summary, the
development approach of SaltProc v1.0 is focused on producing a generic,
flexible and expandable tool that extends Serpent 2 Monte Carlo code for
advanced in-reactor fuel cycle analysis as well as simulate many online
refueling and fuel reprocessing systems.

\subsection{Reactivity control module}
Simulation of specific \gls{MSR} concepts requires changing the reactor core
geometry during lifetime-long operation modeling. For instance, the \gls{TAP}
concept aims to increase the core lifetime by using continuous fresh fuel
feeds, removing \glspl{FP}, and reconfiguring moderator rod assemblies to
compensate for negative reactivity insertion due to fissile material burnup.
The concept proposes maintaining reactivity in the long term by replacing
stationary moderator assemblies with denser lattices to increase the
moderator-to-fuel ratio \cite{betzler_assessment_2017-1}.
SaltProc v1.0 can switch from one core geometry to another core geometry
(e.g., with larger moderator-to-fuel ratio) to mimic moderator or absorber
movement if the effective multiplication factor, $k_{eff}$, falls below 1.
This unique capability allows SaltProc v1.0 to
analyze the fuel cycle performance of any liquid-fueled \gls{MSR} system,
including advanced designs with a moving moderator (e.g., \gls{TAP} \gls{MSR}).

%\section{Multi-component fuel salt reprocessing system}
\section{Concluding remarks}
This chapter presented an overview of an fuel salt reprocessing in
\glspl{MSR}. It described various components of the plant and the physical
or chemical mechanism responsible for neutron poison extraction from the
salt. General core physics aspects and Serpent 2 depletion software
capabilities have then been discussed. It also introduced SaltProc, a Python
package developed and used to simulate continuous feeds and removals in
various \gls{MSR} designs.

In the following chapters, SaltProc v1.0 will be demonstrated and validated
for two liquid-fueled \gls{MSR} designs: the \gls{MSBR} and the \gls{TAP}
\gls{MSR}.