Inverse Problems and Data Assimilation

Abstract: We provide a clear and concise introduction to the subjects of inverse problems and data assimilation, and their inter-relations. The first part of our notes covers inverse problems; this refers to the study of how to estimate unknown model parameters from data. The second part of our notes covers data assimilation; this refers to a particular class of inverse problems in which the unknown parameter is the initial condition (and/or state) of a dynamical system, and the data comprises partial and noisy observations of the state. The third and final part of our notes describes the use of data assimilation methods to solve generic inverse problems by introducing an artificial algorithmic time. Our notes cover, among other topics, maximum a posteriori estimation, (stochastic) gradient descent, variational Bayes, Monte Carlo, importance sampling and Markov chain Monte Carlo for inverse problems; and 3DVAR, 4DVAR, extended and ensemble Kalman filters, and particle filters for data assimilation.Each of parts one and two starts with a chapter on the Bayesian formulation, in which the problem solution is given by a posterior distribution on the unknown parameter. Then the following chapter specializes the Bayesian formulation to a linear-Gaussian setting where explicit characterization of the posterior is possible and insightful. The next two chapters explore methods to extract information from the posterior in nonlinear and non-Gaussian settings using optimization and Gaussian approximations. The final two chapters describe sampling methods that can reproduce the full posterior in the large sample limit. Each chapter closes with a bibliography containing citations to alternative pedagogical literature and to relevant research literature. We also include a set of exercises at the end of parts one and two. Our notes are thus useful for both classroom teaching and self-guided study.

\part*{\Large{\sffamily{Introduction}}}

\noindent{\bf Overview of the Notes\\}
\vspace{0.05in}

These notes were first developed out of Caltech course ACM 159 in Fall 2017, and then substantially modified for the University of Chicago course STAT 31550 in Winter 2019. The aim of the notes is to provide a clear and concise introduction
to the subjects of Inverse Problems and Data Assimilation, and their
inter-relations, together with citations to some relevant literature
in this area

In its most basic form, inverse problem theory is the study of how to estimate model parameters from data. Often the data provides indirect information about these parameters, corrupted by noise. The theory of inverse problems,
however, is much richer than just parameter estimation. For example,
the underlying theory can be used to
determine the effects of noisy data on the accuracy of the solution; it can
be used to determine what kind of observations are needed to accurately
determine a parameter; and it can be used to study the uncertainty in a
parameter estimate and, relatedly, is useful, for example,
in the design of strategies for control or optimization under
uncertainty, and for risk analysis. The theory
thus has applications in many fields of science and engineering.\\

To apply the ideas in these notes, the starting point is a mathematical model
mapping the unknown parameters to the observations: termed the ``forward'' or
``direct'' problem, and often a subject of research in its own right. A good forward model will not only identify how the data is dependent on parameters, but
also what sources of noise or model uncertainty are present in the postulated
relationship between unknown parameters and data. For example, if  the desired
forward problem cannot be solved analytically, then the forward model may be approximated by a simulation; in this case
discretization may be considered as a source of error. Once a relationship between model parameters, sources of error, and data is clearly defined, the inverse problem of estimating parameters from data can be addressed. The theory of
inverse problems can be separated into two cases: (1) the ideal case where data is not corrupted by noise and is derived from a known perfect model; and (2) the practical case where data is incomplete and imprecise. The first case is useful for classifying inverse problems and determining if a given set of observations
can, in principle, provide exact solutions; this provides insight into
conditions needed for existence, uniqueness, and stability of a solution.
The second case is useful for the formulation of practical algorithms
to learn about parameters, and uncertainties in their estimates, and will
be the focus of these notes.\\

A model that has the properties: (a) a solution map exists, (b) is unique, and (c) its behavior changes continuously with input (stability) is termed ``well-posed''. Conversely, a model lacking any of these properties is termed ``ill-posed''. Ill-posedness is present is many inverse problems,
and mitigating it is an extensive part of the subject.
Out of the different approaches for formulating an inverse problem, the Bayesian framework naturally offers the ability to assess quality in parameter estimation,
and also leads to a form of well-posedness at the level of probability
distributions describing the solution. The goal of the Bayesian framework is to find a probability measure that assigns a probability to each possible solution for a parameter $u$, given the
data $y$. Bayes formula states that
\begin{equation*}
\operatorname{\mathbbm{P}}(u \vert y) = \frac{1}{\operatorname{\mathbbm{P}}(y)} \operatorname{\mathbbm{P}}(y \vert u)\operatorname{\mathbbm{P}}(u).
\end{equation*}
It enables calculation of the posterior probability on
$u\vert y$, $\operatorname{\mathbbm{P}}(u \vert y) $, in terms of the
product of the data likelihood $\operatorname{\mathbbm{P}}(y \vert u)$ and the
prior information on the parameter encoded in $\operatorname{\mathbbm{P}}(u)$. The likelihood
describes the probability of the observed data $y$, if the input parameter
were set to be $u$; it is determined by the forward model, and the
structure of the noise. The normalization constant $\operatorname{\mathbbm{P}}(y)$ ensures that
$\operatorname{\mathbbm{P}}(u \vert y)$  is a probability measure. There are five primary benefits to this framework: (a) it provides a clear theoretical setting in which the
forward model choice, noise model and a priori information are explicit;
(b) it provides information about the entire solution space for possible
input parameter choices; (c) it naturally leads to quantification of
uncertainty and risk in parameter estimates; (d) it is generalizable to a wide class of inverse problems, in finite and infinite dimension and comes
with a well-posedness theory useful in these contexts.

The first half of the notes is dedicated to studying the Bayesian framework for inverse problems. Techniques such as importance sampling and Markov
Chain Monte Carlo (MCMC) methods are introduced; these methods have the
desirable property that in the limit of an infinite number of samples
they reproduce the full posterior distribution.  Since it is often
computationally intensive to implement these methods, especially in
high dimensional problems, approximate techniques such as approximating
the posterior by a Dirac or a Gaussian distribution are discussed.

The second half of the notes covers data assimilation.
This refers to a particular class of inverse problems in which
the unknown parameter is the initial condition of a dynamical system,
and in the stochastic dynamics case the subsequent states of the system,
and the data comprises partial and noisy observations of that
(possibly stochastic) dynamical system. A primary use of data assimilation is in forecasting, where the purpose is to provide better future estimates than can be obtained using
either the data or the model alone.
All the methods from the first half of the course
may be applied directly, but there are other new methods which
exploit the Markovian structure to update the state of the system
sequentially, rather than to learn about the initial condition. (But
of course knowledge of the initial condition may be used to inform
the state of the system at later times.)
We will also demonstrate that methods developed in
data assimilation may be employed to study generic inverse problems,
by introducing an artificial time to generate a sequence of
probability measures interpolating from the prior to the posterior.\\

\noindent{\bf Notation}
\vspace{0.05in}

Throughout the notes we use $\mathbb{N}$ to denote the positive integers $\{1,2,3, \cdots \}$
and $\mathbb{Z}^+$ to denote the non-negative integers $\mathbb{N} \cup \{0\}=\{0,1,2,3, \cdots \}.$
The matrix $I_d$ denotes the identity on $\mathbbm{R}^\Du.$
We use $|\cdot|$ to denote the Euclidean norm corresponding to the
inner-product $\langle \cdot, \cdot \rangle.$   A square matrix $A$
is positive definite (resp. positive semi-definite) if the quadratic
form $\langle u, Au \rangle$ is positive (resp. non-negative)
for all $u \ne 0$. By
$| \cdot |_A $ we denote the weighted norm defined by $|v|_A^2  = v^T A^{-1} v $.
The corresponding weighted Euclidean inner-product is given by
$\langle \cdot,  \cdot \rangle_{A}$ and $\langle \cdot, A^{-1}\cdot \rangle.$
We use $\otimes$ to denote the outer product between
two vectors: $(a \otimes b)c=\langle b,c \rangle a.$
We let $B(u,\delta)$ denote the open ball of radius $\delta$ at $u$, in the
Euclidean norm.

Throughout we denote by $\operatorname{\mathbbm{P}} (\cdot), \operatorname{\mathbbm{P}}(\cdot | \cdot )$ the pdf of
a random variable and its conditional pdf, respectively.
We write
$$ \rho(f)  =\operatorname{\mathbb{E}}^\rho[f]=\int_{\mathbbm{R}^\Du}f(u)\rho(u)du$$
to denote expectation of $f: \mathbbm{R}^\Du \mapsto  \mathbb{R}$
with respect to probability measure with probability density function (pdf)
$\rho$ on $\mathbbm{R}^\Du.$
The distribution of the random variables in this book will often have density with respect to Lebesgue
measure, but occasional use of Dirac masses will be required; we will
use the notational convention that Dirac mass at point $v$
has ``density'' $\delta(\cdot-v)$ or $\delta_v(\cdot).$ When a random
variable $u$ has pdf $\rho$
we will write $u \sim \rho.$ We use $\Rightarrow$ to denote weak
convergence of probability measures,  that is, $\rho_n \Rightarrow \rho$ if $\rho_n(f) \to \rho(f)$ for all bounded and continuous $f: \mathbbm{R}^\Du \mapsto  \mathbbm{R}$. \\

\noindent{\bf Acknowledgements}
\vspace{0.05in}

These notes were first created out of Caltech course ACM 159 in Fall 2017 and were further developed for the University of Chicago course STAT 31550 in Winter 2019. The
notes were created in \LaTeX \, by the students in ACM 159,
based on lectures presented by the instructor Andrew Stuart, and
on input from the course TA Armeen Taeb. The individuals responsible for the notes listed in alphabetic order are: Blancquart, Paul; Cai, Karena; Chen, Jiajie; Cheng, Richard; Cheng, Rui; Feldstein, Jonathan; Huang, De; Id{\'i}ni, Benjamin; Kovachki, Nikola; Lee, Marcus; Levy, Gabriel; Li, Liuchi; Muir, Jack; Ren, Cindy; Seylabi, Elnaz, Sch{\"a}fer, Florian; Singhal, Vipul; Stephenson, Oliver; Song, Yichuan;  Su, Yu; Teke, Oguzhan; Williams, Ethan; Wray, Parker; Zhan, Eric; Zhang, Shumao; Xiao, Fangzhou. Furthermore, the following students have added content beyond the class materials: Parker Wray -- the Overview, Jiajie Chen -- alternative proof of Theorem 1.10 and proof idea for Theorem 14.3, Fangzhou Xiao -- numerical simulation of prior, likelihood $\&$ posterior, Elnaz Seylabi $\&$ Fangzhou Xiao -- catching typographical errors in a draft of these notes, Cindy Ren -- numerical simulations to enhance understanding of importance sampling in Examples 6.2 and 6.5,  Cindy Ren $\&$ De Huang -- improving the constants in Theorem 6.3 regarding the approximation error of importance sampling, Richard Cheng $\&$ Florian Sch{\"a}fer -- illustrations to enhance understanding of the coupling argument used to study convergence of MCMC algorithms by presenting the finite state-space case, and Ethan Williams $\&$ Jack Muir -- numerical simulations and illustrations of Ensemble Kalman Filter and Extended Kalman Filter.
We would also like to thank Tapio Helin (Helsinki) who used the notes in his own course and
provided very helpful feedback on an early draft.

The work of AS has been funded by the EPSRC (UK), ERC (Europe) and by AFOSR, ARL, NIH, NSF and ONR (USA). The work of DSA is funded by NSF. This funded research has helped to shape the
presentation of the material here and is gratefully acknowledged. \\[0.1in]

\vspace{0.05in}
\noindent{\bf Warning}

\vspace{0.05in}
These are rough notes, far from being perfected. They are likely
to contain mathematical errors, incomplete bibliographical information,
inconsistencies in notation and typographical errors. We hope that the notes are nonetheless useful. Please contact the authors with any feedback from typos, through mathematical errors and bibliographical omissions, to
comments on the structural organization of the material.
\section{\Large{\sffamily{The Linear-Gaussian Setting}}}
Recall the stochastic dynamics and data models introduced in the previous chapter:
\begin{align}\label{eq:models}
\begin{split}
        v_{j+1} &= \Psi(v_j) + \xi_j,   \quad \quad \,\, \, \, \;  \xi_j \sim \mathcal{N}(0, \Sigma) \text{ i.i.d.},  \\
        y_{j+1} &= h(v_{j+1}) + \eta_j,  \quad \quad \, \eta_j \sim \mathcal{N}(0, \Gamma) \text{ i.i.d.},
\end{split}
\end{align}
with \(v_0 \sim \mathcal{N}(m_0, C_0),\) $C_0, \Sigma$ and $\Gamma$ positive definite and $v_0 \perp \{\xi_j\} \perp \{\eta_j\}.$
Here we study the filtering and smoothing problems under the assumption that both the state-transition operator \(\Psi(\cdot)\) and the observation operator \(h(\cdot)\) are  linear. Throughout, we will assume the following:
\begin{assumption}
The stochastic dynamics and the data models defined by equation \eqref{eq:models} hold. Moreover,
\label{a:kf}
\begin{description}
	\item[1] Linear dynamics:  $v_{j+1} = Mv_j + \xi_j$ for some $\mtx{M}\in\mathbbm{R}^{d \times d}$;
    \item[2] Linear obervation: $y_{j+1} = H v_{j+1} + \eta_{j+1}$ for some $H \in \mathbbm{R}^{k\times d}.$
\end{description}
\end{assumption}
We will be mostly concerned with the case where $d>k.$
Under the \emph{linear-Gaussian} assumption, the filtering and smoothing distributions are  Gaussian and therefore are fully characterized by their mean and covariance.
We consider first the \emph{Kalman filter}, which gives explicit formulae for the iterative update of the mean and covariance of the filtering distribution, and then the {\emph{Kalman smoother}, which characterizes the smoothing distribution.
While the Kalman filter and the Kalman smoother only characterize the filtering and smoothing distributions in the linear-Gaussian setting, their importance extends beyond this setting, as will be demonstrated in the next two chapters.

\subsection{Kalman Filter}
The {\em filtering problem} is to estimate the state at time \(j\) given the data from the past up to the present time \(j\). That is, we want to determine the pdf \(\pi_j = \operatorname{\mathbbm{P}}(v_j|Y_j)\), where $Y_j := \{y_1,\hdots,y_j\}.$
We  define $\hat{\pi}_{j+1}=\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)$ and recall
the evolution $$\pi_{j+1} = {\mathcal{A}}_j {\mathcal{P}} \pi_j, \quad \pi_0 = \mathcal{N}(m_0, C_0),$$
which can be decomposed in terms of the prediction and analysis steps \eqref{eq:pna}.
Note that \({\mathcal{P}}\) does not depend on $j$ because the same Markov process defined by the state dynamics governs each prediction step, whereas \({\mathcal{A}}_j\) depends on $j$ because at each step $j$ the likelihood sees different data. The linear dynamics assumption implies that applying the operator ${\mathcal{P}}$ to a Gaussian distribution gives again a Gaussian, and the linear observation assumption implies that applying the operator ${\mathcal{A}}_j$ to a Gaussian gives again a Gaussian. Thefore we have the following:

\begin{theorem}(Gaussianity of Filtering Distributions)
	Under Assumption \ref{a:kf},
$\pi_0$, \(\{\pi_{j+1}\}_{j \in \mathbbm{Z}^+}\) and \(\{\hat{\pi}_{j+1}\}_{j \in \mathbbm{Z}^+}\) are all Gaussian distributions.
\end{theorem}

As a consequence, the filtering distributions can be entirely characterized by
their mean and covariance. We write
\begin{align*}
	\hat{\pi}_{j+1} &= \operatorname{\mathbbm{P}}(v_{j+1}|Y_j) = \mathcal{N}(\hat{m}_{j+1}, \hat{C}_{j+1}), && \text{(prediction)}\\
	\pi_{j+1} &= \operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1}) = \mathcal{N}(m_{j+1}, C_{j+1}), && \text{(analysis)}
\end{align*}
and aim to find update formulae for these means and covariances.
The Kalman filter algorithm achieves this.

\begin{theorem}[Kalman filter]
	\label{kalmanfilter}
Under Assumption \ref{a:kf}
for all $j \in \mathbbm{Z}^+$, $C_j$ is positive definite and
\begin{align*}
\hat{m}_{j+1} &= Mm_j, \\
\hat{C}_{j+1} &= MC_jM^T+\Sigma, \\
C_{j+1}^{-1} &= (MC_jM^T + \Sigma)^{-1} + H^T \Gamma^{-1} H,\\
C_{j+1}^{-1}m_{j+1} &= (MC_jM^T + \Sigma)^{-1} M m_j + H^T \Gamma^{-1} y_{j+1}.
\end{align*}
\end{theorem}

\begin{proof}
The proof proceeds by breaking the Kalman filter step above into the prediction and the analysis steps.

\noindent \textbf{Prediction}: The mean and variance of the prediction
step may be calculated as follows. The mean is given by:
\begin{align*}
	\hat{m}_{j+1} &= \operatorname{\mathbb{E}} \big[ v_{j+1} | Y_j \big] \\
	&= \operatorname{\mathbb{E}} \big[ M v_j + \xi_j | Y_j \big]  \\
	&= M \operatorname{\mathbb{E}} \big[ v_j | Y_j \big] + \operatorname{\mathbb{E}} \big[\xi_j | Y_j \big] \\
	&= Mm_j,
\end{align*}
where we used that \(\xi_j\) and \(Y_j\) are independent.
The covariance is given by:
\begin{align*}
	\hat{C}_{j+1} =& \operatorname{\mathbb{E}} \big[ (v_{j+1} - \hat{m}_{j+1}) \otimes (v_{j+1} - \hat{m}_{j+1}) | Y_j \big] \\
	=& \operatorname{\mathbb{E}} \big[ M(v_j - m_j) \otimes M(v_j - m_j) | Y_j \big] + \operatorname{\mathbb{E}} \big[ \xi_j \otimes\xi_j | Y_j \big] \\
	&+ \operatorname{\mathbb{E}} \big[ \xi_j \otimes M(v_j - m_j) | Y_j \big] + \operatorname{\mathbb{E}} \big[ M(v_j - m_j) \otimes\xi_j | Y_j \big] \\
	=& M \operatorname{\mathbb{E}} \big[(v_j - m_j) \otimes (v_j - m_j) | Y_j \big] M^T + \Sigma   \\
	=& M C_j M^T + \Sigma,
\end{align*}
where we used that \(\xi_j\) and \(v_j\) are independent.
Thus in the linear Gaussian setting
the prediction operator \({\mathcal{P}}\) from \(\pi_j = \mathcal{N}(m_j, C_j)\) to \(\hat{\pi}_{j+1} = \mathcal{N}(\hat{m}_{j+1}, \hat{C}_{j+1})\) is given by
\begin{align*}
	\hat{m}_{j+1} &= M m_j, \\
	\hat{C}_{j+1} &= M C_j M^T + \Sigma.
\end{align*}

\noindent \textbf{Analysis}: The analysis step may be derived as follows,
using Bayes Theorem:
\begin{align*}
	\operatorname{\mathbbm{P}}(v_{j+1} | Y_{j+1}) &= \operatorname{\mathbbm{P}}(v_{j+1} | y_{j+1},Y_{j})\\
& \propto \operatorname{\mathbbm{P}}(y_{j+1} | v_{j+1}, Y_j) \operatorname{\mathbbm{P}}(v_{j+1} | Y_j) \\
	& = \operatorname{\mathbbm{P}}(y_{j+1} | v_{j+1}) \operatorname{\mathbbm{P}}(v_{j+1} | Y_j).
\end{align*}
This gives
\begin{align}\label{eq:smoot}
\begin{split}
\operatorname{\mathbbm{P}}(v_{j+1} | Y_{j+1}) &\propto \exp\left(-\frac{1}{2} |v_{j+1} - m_{j+1}|_{C_{j+1}}^2 \right)  \\
&\propto \exp\left(-\frac{1}{2} |y_{j+1} - H v_{j+1}|_{\Gamma}^2 \right) \exp\left(-\frac{1}{2} |v_{j+1} - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2 \right) \\
	&= \exp\left(-\frac{1}{2} |y_{j+1} - H v_{j+1}|_{\Gamma}^2 -\frac{1}{2} |v_{j+1} - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2 \right).
	\end{split}
\end{align}
Taking logartihms and
matching quadratic and linear terms in $v_{j+1}$ from either side of this
identity gives the update operator \({\mathcal{A}}_j\) from \(\hat{\pi}_{j+1} = \mathcal{N}(\hat{m}_{j+1}, \hat{C}_{j+1})\) to \(\pi_{j+1} = \mathcal{N}(m_{j+1}, C_{j+1})\):
\begin{align*}
	C_{j+1}^{-1} &= \hat{C}_{j+1}^{-1} + H^T \Gamma^{-1} H, \\
	C_{j+1}^{-1} m_{j+1} &= \hat{C}_{j+1}^{-1} \hat{m}_{j+1} + H^T \Gamma^{-1} y_{j+1}.
\end{align*}
Combining the prediction operator \({\mathcal{P}}\) and update operator \({\mathcal{A}}_j\) yields the desired update formulae.

\noindent \textbf{Positive-definiteness of \(C_j\)}: It remains to show that \(C_j > 0\) for all \(j \in \mathbbm{Z}^+\). We will use induction. By assumption the
result holds true for \(j = 0\). Assume that it is true for \(C_j\). For
the prediction operator \({\mathcal{P}}\) we have, for $u \ne 0$,
\begin{align*}
	\langle u, \hat{C}_{j+1} u \rangle &= \langle u, M C_j M^T u \rangle + \langle u, \Sigma u \rangle \\
	&= \langle M^T u, C_j  M^T u \rangle + \langle u, \Sigma u \rangle \\
        & \ge \langle u, \Sigma u \rangle \\
	&> 0,
\end{align*}
where we used that $C_j>0$ and $\Sigma>0.$
Therefore \(\hat{C}_{j+1}, \hat{C}_{j+1}^{-1} > 0\). Then for the update operator \({\mathcal{A}}_j\):
\begin{align*}
	\langle u, C_{j+1}^{-1} u \rangle &= \langle u, \hat{C}_{j+1}^{-1} u \rangle + \langle u, H^T \Gamma^{-1} H u \rangle \\
	&= \langle u, \hat{C}_{j+1}^{-1} u \rangle + \langle H u, \Gamma^{-1} H u \rangle \\
        &\ge \langle u, \hat{C}_{j+1}^{-1} u \rangle \\
	&> 0,
\end{align*}
where we used that $\Gamma>0.$ Therefore, \(C_{j+1}, C_{j+1}^{-1}> 0\), which concludes the proof.
\end{proof}

\begin{remark}
The previous proof reveals two interesting facts about the structure of the Kalman filter updates. The first is that the covariance update does not involve the observed data; this can be thought of as a consequence of the fact that the posterior covariance in the linear-Gaussian setting for inverse problems does not depend on the observed data, as noted in Chapter 2.  The second is that the update formulae for the covariance are linear in the prediction step, but nonlinear in the analysis step; specifically the analysis step is linear in the precisions (inverse covariances).
\end{remark}

\subsubsection{Kalman Filter: Alternative Formulation}
We now rewrite the Kalman filter in an alternative form. This formulation
is written in terms of the covariances directly, and does not involve
the precisions. Furthermore, the formulation in Theorem \ref{kalmanfilter} involves a matrix inversion in the state space while the one that we will present here requires
only inversion the data space, namely \(S_{j+1}^{-1}\) in what follows.
In many applications the observation space dimension is much smaller
than the state space dimension (\(k \ll d\)), and then the
formulation given in this section is
much cheaper to compute than the one given in Theorem \ref{kalmanfilter}.

\begin{corollary}[Standard Form of Kalman Filter]
\label{cor:k2}
	Under Assumption \ref{kalmanfilter}, the Kalman
update formulae may be written as
	\begin{align}\label{eq:kf update}
	\begin{split}
		m_{j+1} &= \hat{m}_{j+1} + K_{j+1} d_{j+1}, \\
		C_{j+1} &= (I - K_{j+1} H) \hat{C}_{j+1},
		\end{split}
	\end{align}
where
	\begin{align*}
		d_{j+1} &= y_{j+1} - H \hat{m}_{j+1}, \\
		S_{j+1} &= H \hat{C}_{j+1} H^T + \Gamma, \\
		K_{j+1} &= \hat{C}_{j+1} H^T S_{j+1}^{-1},
	\end{align*}
and where \(\hat{m}_{j+1}, \hat{C}_{j+1}\) are defined as in the proof of Theorem \ref{kalmanfilter}.
\end{corollary}

\begin{remark} The vector \(d_{j+1}\) is known as the {\em innovation}
and the matrix \(K_{j+1}\) as the {\em Kalman gain}. Note that
\(d_{j+1}\) measures the mismatch of the predicted state from the given data.
\end{remark}

The corollary may be proved by an application of the following:

\begin{lemma}
	(Woodbury Matrix Identity) Let \(A \in \mathbbm{R}^{p \times p}\), \(U \in \mathbbm{R}^{p \times q}\), \(B \in \mathbbm{R}^{q \times q}\), \(V \in \mathbbm{R}^{q \times p}\). If \(A, B > 0\), then \(A + UBV\) is invertible and
	\[(A + UBV)^{-1} = A^{-1} - A^{-1}U (B^{-1} + VA^{-1}U)^{-1} V A^{-1}.\]
\end{lemma}

Combining the form of $d_{j+1}$ and $\hat{m}_{j+1}$ shows that the update
formula for the Kalman mean can be written as
\begin{equation}
\label{eq:kmu}
m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}, \quad \hat{m}_{j+1}=Mm_j.
\end{equation}
This update formula has the very natural interpretation that the mean update
is formed as a linear combination of the evolution of the noise-free dynamics
and of the data. Equations \eqref{eq:kf update} and \eqref{eq:kmu} show that the Kalman gain $K_{j+1}$ determines the weight given to the new observation $y_{j+1}$ in the state estimation.
The update formula \eqref{eq:kmu} may be also derived from an optimization perspective,
the topic of the next subsection.

\subsubsection{Optimization Perspective: Mean of  Kalman Filter}
\label{sec:okf}
Since \(\pi_{j+1}\) is Gaussian, its mean agrees with its mode. Thus, formulae \eqref{eq:smoot} implies that
\begin{align*}
	m_{j+1} &=  {\rm argmax}_v \pi_{j+1}(v)  \\
	& = {\rm argmin}_v {\mathsf{J}}(v),
\end{align*}
where
$${\mathsf{J}}(v):= \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2.$$
In other words, \(m_{j+1}\) is chosen to fit both the observed data
\(y_{j+1}\) and the predictions \(\hat{m}_{j+1}\) as well as possible.
The covariances \(\Gamma\) and \(\hat{C}_{j+1}\)  determine
the relative weighting between the two quadratic terms.
The solution of the minimization problem is given by \eqref{eq:kmu}, as may be verified by direct differentiation of ${\mathsf{J}}.$

An alternative derivation which is helpful in more sophisticated contexts is to cast the problem in terms of constrained minimization. Write $v'=v-\hat{m}_{j+1}$, $y'=y_{j+1} - H \hat{m}_{j+1}$ and
$C'=\hat{C}_{j+1}.$ Then
 minimization of ${\mathsf{J}}$ may be reformulated as
\begin{align*}
        m_{j+1} = \hat{m}_{j+1}+{\rm argmin}_{v'} \left( \frac{1}{2} |y' - H v'|_{\Gamma}^2 + \frac{1}{2} \langle v', b \rangle \right),
\end{align*}
where the minimization is now subject to the constraint $C'b=v'.$ Using
Lagrange multipliers we write
\begin{equation}
\label{eq:LM}
{\mathsf{I}}(v')= \frac{1}{2} |y' - H v'|_{\Gamma}^2 + \frac{1}{2} \langle v', b \rangle
+\langle \lambda, C'b-v' \rangle;
\end{equation}
computing the dervative and setting to zero gives
\begin{subequations}
\label{eq:LM_EL}
\begin{align*}
-H^T\Gamma^{-1}(y'-Hv')+\frac12 b-\lambda & =0,\\
\frac12 v'+C'\lambda &=0,\\
v'-C'b &=0.
\end{align*}
\end{subequations}
The last two equations imply that $C'(2\lambda+b)=0.$ Thus we set
$\lambda=-\frac{1}{2}b$ and drop the second equation, replacing the first by
$$-H^T\Gamma^{-1}(y'-HC'b)+b=0.$$
Solving for $b$ gives
\begin{align*}
v&=\hat{m}_{j+1}+v'\\
&=\hat{m}_{j+1}+C'b\\
&=\hat{m}_{j+1}+C'(H^T\Gamma^{-1}HC'+I)^{-1}H^T\Gamma^{-1}y'\\
&=\hat{m}_{j+1}+C'(H^T\Gamma^{-1}HC'+I)^{-1}H^T\Gamma^{-1}(y_{j+1} - H \hat{m}_{j+1})\\
&=(I-K_{j+1}H)\hat{m}_{j+1}+K_{j+1}y_{j+1},
\end{align*}
where we have defined
$$K_{j+1}=C'(H^T\Gamma^{-1}HC'+I)^{-1}H^T\Gamma^{-1}.$$
It remains to show that $K_{j+1}$ agrees with the definition given
in Corollary \ref{cor:k2}. To see this we note that if we choose $S$ to
be any matrix satisfying
$K_{j+1}=C'H^T S^{-1}$ then
$$H^T S^{-1}=(H^T\Gamma^{-1}HC'+I)^{-1}H^T\Gamma^{-1}$$
so that
$$(H^T\Gamma^{-1}HC'+I)H^T=H^T\Gamma^{-1} S.$$
Thus
$$H^T\Gamma^{-1}HC'H^T+H^T=H^T\Gamma^{-1}S$$
which may be achieved by choosing any $S$ so that
$$\Gamma^{-1}(HC'H^T+\Gamma)=\Gamma^{-1}S$$
and multiplication by $\Gamma$ gives the desired formula for $S$.

\subsubsection{Optimality of Kalman Filter}

The following theorem states that the Kalman filter gives the best
estimator of the mean in an online setting.
In the following $\operatorname{\mathbb{E}}$ denotes expectation with respect to
all randomness present in the problem statement, through the initial
condition, the noisy dynamical evolution, and the noisy data. Furthermore
$\operatorname{\mathbb{E}} [\cdot|Y_j]$ denotes conditional expectation, given
the data $Y_j$ upto time $j$.

\begin{theorem}[Optimality of Kalman Filter]
	Let \(\{m_j\}\) be the sequence computed using the Kalman filter, and \(\{z_j\}\) be any sequence in \(\mathbbm{R}^\Du\) such that $z_j$ is $Y_j$
measurable.\footnote{For practical purposes this means $z_j$ is a fixed
non-random function of given observed $Y_j.$} Then, for all $j \in \mathbbm{N},$
	\[ \operatorname{\mathbb{E}} \Big[ |v_j - m_j |^2 \mid Y_j \Big] \leq \operatorname{\mathbb{E}} \Big[ |v_j - z_j |^2 \mid Y_j \Big].\]
\end{theorem}

\begin{proof}
Note that $m_j$ and $z_j$ are fixed and non-random, given $Y_j$. Thus we have:
\begin{align*}
	\operatorname{\mathbb{E}} \Big[ |v_j - z_j |^2 \mid Y_j \Big] =& \operatorname{\mathbb{E}} \Big[ |v_j - m_j + m_j - z_j |^2 \mid Y_j \Big] \\
	=& \operatorname{\mathbb{E}} \Big[ |v_j - m_j|^2 + 2 \Big\langle v_j - m_j, m_j - z_j
\Big \rangle + |m_j - z_j|^2 \mid Y_j \Big] \\
 =& \operatorname{\mathbb{E}} \Big[ |v_j - m_j|^2 \mid Y_j \Big] + 2 \Big \langle \operatorname{\mathbb{E}} \Big[ v_j - m_j \mid Y_j \Big] , m_j - z_j \Big \rangle + |m_j - z_j|^2 \\
	=& \operatorname{\mathbb{E}} \Big[ |v_j - m_j|^2 \mid Y_j \Big] + 2 \Big \langle \operatorname{\mathbb{E}} \Big[ v_j \mid Y_j \Big]-m_j , m_j - z_j \Big \rangle + |m_j - z_j|^2 \\
	=& \operatorname{\mathbb{E}} \Big[ |v_j - m_j|^2 \mid Y_j \Big] + 0 + |m_j - z_j|^2 \\
	\geq & \operatorname{\mathbb{E}} \Big[ |v_j - m_j|^2 \mid Y_j \Big].
\end{align*}
The fifth step follows since \(m_j = \operatorname{\mathbb{E}} \big[ v_j \mid Y_j \big].\)
\end{proof}

\subsection{Kalman Smoother}
We next discuss the Kalman smoother, which refers to the smoothing problem in the linear-Gaussian setting. As with the Kalman filter, it is possible
to solve the problem explicitly because the smoothing distribution is itself a Gaussian. The explicit formulae computed help to build
intuition about the smoothing distribution more generally. We recall remark \ref{rem:marginal}, which implies that the filtering distribution at time $j=J$ determines the marginal of the Kalman smoother on its last coordinate. However, the filtering distributions do not determine the
Kalman smoother in its entirety.

\subsubsection{Defining Linear System}
Let $V= \{v_0,\ldots,v_J\}$ and $Y =\{y_1,\ldots, y_J\}$. Using Bayes theorem
and the fact that $\{\xi_j\}$, $\{\eta_j\}$ are mutually independent
i.i.d. sequences, independent of $v_0$, we have
\begin{equation*}
\operatorname{\mathbbm{P}} (V|Y)\propto \operatorname{\mathbbm{P}}(Y|V) \operatorname{\mathbbm{P}}(V)= \prod\limits_{j=1}^J\operatorname{\mathbbm{P}}(y_j|v_j)\times \prod\limits_{j=0}^{J-1} \operatorname{\mathbbm{P}} (v_{j+1}|v_j)\times \operatorname{\mathbbm{P}} (v_0).
\end{equation*}
Noting that
$$v_{j+1}|v_j \sim \mathcal{N}(Mv_j,\Sigma), \quad y_j|v_j \sim \mathcal{N}(Hv_j,\Gamma)$$
the Kalman smoothing distribution can be expressed as
\begin{equation}
\operatorname{\mathbbm{P}} (V|Y) \propto \exp\bigl(-{\mathsf{J}}(V)\bigr),
\end{equation}
where
\begin{equation}\label{eq:jformula}
{\mathsf{J}}(V):=\frac{1}{2}|v_0-m_0|^{2}_{C_0}+\frac{1}{2} \sum\limits_{j=0}^{J-1}|v_{j+1}-Mv_j|^{2}_{\Sigma}+\frac{1}{2}\sum\limits_{j=0}^{J-1}|y_{j+1}-Hv_{j+1}|^{2}_{\Gamma}.
\end{equation}

\begin{theorem}[Characterization of the Kalman Smoother]
Suppose that Assumption \ref{a:kf} holds. Then $\operatorname{\mathbbm{P}}(V | Y)$ is Gaussian with a block tridiagonal precision matrix $\Omega>0$ and  mean $m$ solving $\Omega m=r$, where
\begin{align}
\Omega  =& \begin{bmatrix}
 \Omega_{0,0} & \Omega_{0,1} &  &  &  &  \\
 \Omega _{1,0} & \Omega_{1,1} & ... &  & 0  &  \\
 0 & ... & ... &  &  &  \\
  & 0 & ... & ... & ... &  \\
  &  &  & ... & \Omega_{J-1,J-1} & \Omega_{J-1,J} \\
  &  &  &  & \Omega_{J,J-1} & \Omega_{J,J}
 \end{bmatrix}
\end{align}
with
\begin{align*}
\Omega_{0,0}&=C_0^{-1} +M^T\Sigma^{-1}M,\\
\Omega_{j,j}&=\Sigma^{-1}+M^T\Sigma^{-1}M+H^T\Gamma^{-1}H, \quad  1\leq j \leq J-1,\\
\Omega_{J,J}&=\Sigma^{-1}+H^T\Gamma^{-1}H,\\
\Omega_{j,j+1}&=-M^T\Sigma^{-1}, \quad  1\leq j\leq J-1,\\
r_0&=C_0^{-1}m_0,\\
r_j&=H^T\Gamma^{-1}y_j, \quad  1\leq j\leq J.
\end{align*}
\end{theorem}

\begin{proof}
We may write ${\mathsf{J}}(V)=\frac{1}{2}\vert \Omega^{1/2}(V-m)\vert^2 +q$ with $q$ independent of $V$, by definition. Note that  $\Omega$ is then the Hessian of ${\mathsf{J}}(V)$, and differentiating in equation \eqref{eq:jformula} we obtain that
\begin{align*}
\Omega_{0,0}&= \partial_{v_0}^2{\mathsf{J}}(V)=C_0^{-1} + M^T\Sigma^{-1}M,\\
\Omega_{j,j}&= \partial_{v_j}^2{\mathsf{J}}(V)=\Sigma^{-1} + M^T\Sigma^{-1}M+H^T\Gamma^{-1}H,\\
\Omega_{J,J}&= \partial_{v_J}^2{\mathsf{J}}(V)=\Sigma^{-1} +H^T\Gamma^{-1}H,\\
\Omega_{j-1,j}&= \Omega_{j,j-1}=\partial_{v_j,v_{j-1}}^2{\mathsf{J}}(V)=-M^T\Sigma^{-1}.
\end{align*}
Otherwise, for all other values of indices $\{k,l\}$, $\Omega_{k,l}=0.$ This proves that the matrix $\Omega$ has a block tridiagonal structure.

Now we focus on finding $m$. We have that $\nabla_V{\mathsf{J}}(V)=\Omega(V-m)$, so that $-\nabla_V{\mathsf{J}}(V)\vert_{V=0}=\Omega m$. Thus, we find $r$ as
\begin{align*}
r_0=&-\nabla_{v_0}{\mathsf{J}}(V)\vert_{V=0}=-(-C_0^{-1}m_0)=C_0^{-1}m_0,\\
r_j=&-\nabla_{v_j}{\mathsf{J}}(V)\vert_{V=0}=-(-H^T\Gamma^{-1}y_j)=H^T\Gamma^{-1}y_j.
\end{align*}
We have shown that $\Omega$ is symmetric and that  $\Omega\ge 0$; to prove that $\Omega$ is a
precision matrix, we need to show that $\Omega>0.$ Take, for the sake of argument, $Y = 0$ and $m_0=0$ in equation \eqref{eq:jformula}, so that every term in the expansion of ${\mathsf{J}}(V)$ involves $V$. It is evident that in such case ${\mathsf{J}}(V) = V^T\Omega V$. Suppose that $V^T\Omega V = 0$ for some nonzero $V$. Then by positive-definiteness of $C_0, \Sigma$, and $\Gamma$, it must be that $v_0 = 0$ and $v_{j+1} = M v_j$ for $j = 0,1,\dots,J$. Thus we must have $V = 0$. This proves that $\Omega$ is positive-definite.
\end{proof}
\begin{remark}
We note once again that since the smoothing distribution in the linear-Gaussian setting is itself Gaussian, its mean agrees with its mode. Therefore, the posterior mean found above is the unique minimizer of ${\mathsf{J}}(V),$ that is, the MAP estimator.
\end{remark}
\subsubsection{Solution of the Linear System}

We may also obtain $m$ via Gaussian elimination, using
the block tridiagonal structure of $\Omega$, as follows.
First we form the matrix sequence $\{\Omega_j\}$:
\begin{align}
\Omega_0 &=\Omega_{0,0}, \nonumber\\
\Omega_{j+1} &=\Omega_{j+1,j+1}-\Sigma^{-1}M^T\Omega_j^{-1}M^T\Sigma^{-1}, \quad j=0,\dots, J-1;
\label{eqn:Ljm}
\end{align}
and we form the vector sequence $\{z_j\}$:
\begin{align*}
z_0&= C_0^{-1}m_0, \\
z_{j+1}&= H^T\Gamma^{-1}y_{j+1}-\Sigma^{-1}M^T\Omega_j^{-1}z_j.
\end{align*}
We may then read off $m_J$ from the equation $\Omega_Jm_J=z_J$ and
finally we perform back-substitution to obtain
$$\Omega_jm_j=z_j-\Omega_{j,j+1}m_{j+1}, \quad j=J-1,\ldots,1.$$
Note that $m_J$ found this way coincides with the mean of the Kalman
filter at $j=J.$

\begin{proposition}
The matrices $\{\Omega_j\}$ in \eqref{eqn:Ljm} are positive definite.
\end{proposition}
\begin{proof}
The proof of this theorem relies on the following two lemmas:
\begin{lemma}\label{lemm1}
If $$X:=\begin{bmatrix}
X_1 &  \times& \times & \times \\
\times & X_2 & \times & \times \\
\times & \times & ... &\times  \\
\times & \times & \times & X_d
\end{bmatrix}$$ is positive-definite symmetric then $X_i$ is
positive-definite symmetric for all $i \in \{1,\dots, d\}.$
\end{lemma}
\begin{lemma} \label{lemm2}
Let $B$ be a block  lower (or upper) triangular matrix with identity on the
diagonal. Then, $B$ is an invertible matrix.
\end{lemma}
Using Lemma \ref{lemm1}, we deduce that $\Omega_0=\Omega_{0,0}$ is positive-definite symmetric. Consider the matrix $B\in\mathbbm{R}^{d(J+1)\times d(J+1)}$ defined as:
$$B= \begin{bmatrix}
 I & 0 & & 0 \\
 -\Omega_{1,0}\Omega_0^{-1} & I & ... & ... \\
 0 &  & ... & 0 \\
  &  & 0 & I
 \end{bmatrix}.$$
We compute
$$B\Omega B^T= \begin{bmatrix}
\Omega_0 & 0 & & &0 \\
0 & \Omega_1 & \Omega_{1,2} & ... & 0\\
& \Omega_{2,1} & \Omega_{2,2} & ... &\\
& & & & \\
0& & & \Omega_{J-1,J-1} &\Omega_{J-1,J} \\
0& & & \Omega_{J,J-1}& \Omega_{J,J}\\ \end{bmatrix}.$$
By Lemma \ref{lemm1}, the matrix
 $$\tilde{\Omega}=\begin{bmatrix}
\Omega_1 & \Omega_{1,2} & ... & 0\\
 \Omega_{2,1} & \Omega_{2,2} & ... &\\
 & & & \\
 & & \Omega_{J-1,J-1} &\Omega_{J-1,J} \\
 & & \Omega_{J,J-1}& \Omega_{J,J}\\ \end{bmatrix}$$ is positive definite,
and so is $\Omega_0.$

Lemma \ref{lemm1} and the positive definiteness of $\tilde{\Omega}$ imply that $\Omega_1$ is positive definite. Therefore, by Lemma \ref{lemm2} the matrix
 $$B_2=\begin{bmatrix}
I& 0 & & 0\\
-\Omega_{2,1}\Omega_1^{-1}& I & ... & ...\\
& & ... &0\\
0& & & I\\
\end{bmatrix}$$
is invertible.
Thus we have $$B_2 \tilde{\Omega}B_2^T = \begin{bmatrix}
\Omega_1 & 0 & & & 0\\
0 & \Omega_2 & \Omega_{2,3} & ... & \\
& \Omega_{3,2} & \Omega_{3,3} & & 0\\
& & & & \\
& & & \Omega_{J-1,J-1} &  \Omega_{J-1,J} \\
0& & & \Omega_{J,J-1}& \Omega_{J,J} \\
\end{bmatrix},
$$
giving the positive definiteness of $\Omega_2$. Iterating the argument shows that all the $\Omega_j$ are  positive definite.
\end{proof}

\subsection{Discussion and Bibliography}
The original paper of Kalman, which is arguably the first systematic
presentation of a  methodology to combine models with data,
is \cite{kalman1960new}. The continuous time analogue of
that work may be found in \cite{kalman1961new}.
The book \cite{harvey1990forecasting} overviews the subject in the
context of time-series analysis and economics.
The proof of Corollary \ref{cor:k2} may be found in \cite{law2015data}.
The paper \cite{sanz2015long} contains an application of the
optimality property of the Kalman filter (which applies beyond
the linear Gaussian setting to the mean of the filtering distribution
in quite general settings).

The subject of Kalman smoothing is overviewed in the text
\cite{anderson1979optimal}; see also \cite{rauch1965maximum}.
A link between the standard implementation of the smoother
and Gauss-Newton methods for MAP estimation is made in
\cite{bell1994iterated}. Fur further details on the
Kalman smoother, in both discrete and continuous time,
see \cite{law2015data} and \cite{hairer2005analysis}.

\section{\Large{\sffamily{Optimization for
Filtering and Smoothing: 3DVAR and 4DVAR}}}

In the previous chapter we showed how the mean of the Kalman
filter could be derived through an optimization principle,
once the predictive covariance is known.
In this chapter we discuss two optimization based approaches to
filtering and smoothing, namely the 3DVAR and 4DVAR methodologies.
We emphasize that the methods we present in this chapter do not provide
approximations of the filtering and smoothing probability distributions;
they simply provide estimates of the signal, given data, in the
filtering (on-line) and smoothing (off-line) data scenarios.

\subsection{The Setting}
3DVAR borrows from the Kalman filter optimization principle,
but substitutes a fixed given covariance for the predictive covariance.
Here ``VAR'' refers to variational, and encodes the concept of
optimization. The 3D and 4D, respectively, refer to three Euclidean
spatial dimensions and to three Euclidean
spatial dimensions plus a time dimension; this nomenclature reflects
the historical derivation of these problems in the geophysical sciences,
but the specific structure of fields over three dimensional Euclidean
space plays no role in the generalized form of the methods described
here. The key distinction is that 3DVAR solves a sequence of optimization
problems at each point in time (hence is an on-line filtering
method); in contrast 4DVAR solves an optimization
problem which involves data distributed over time (and is an off-line
smoothing method).

Throughout we consider the set-up in which the dynamics model
is nonlinear, but the observation operator is linear, commonly
occurring in applications. We thus have
a discrete-time dynamical system with noisy state
transitions and noisy observations given by
\begin{align*}
        \text{Dynamics Model:}  \quad v_{j+1} &= \Psi(v_j) + \xi_j, \quad j \in \mathbbm{Z}^+. \\
        \text{Data Model:}  \quad y_{j+1} &= Hv_{j+1} + \eta_{j+1}, \quad j \in \mathbbm{Z}^+, \text{ for some} \,\, H \in \mathbbm{R}^{k \times d}.  \\
        \text{Probabilistic Structure:}  \quad v_0 &\sim \mathcal{N}(m_0, C_0), \quad \xi_j \sim \mathcal{N}(0, \Sigma), \quad \eta_j \sim \mathcal{N}(0, \Gamma).\\
 \text{Probabilistic Structure:} \quad v_0 &\perp \{\xi_j\} \perp \{\eta_j\} \text{ independent.}
\end{align*}

\subsection{3DVAR}
We introduce 3DVAR by analogy with the update formula \eqref{eq:kmu}
for the Kalman filter, and its derivation through optimization from
section \ref{sec:okf}. The primary differences between 3DVAR and the Kalman
filter mean update is that $\Psi(\cdot)$ can be nonlinear for 3DVAR, and
that for 3DVAR we have no closed update formula for the covariances.
To deal with this second issue 3DVAR uses a fixed predicted covariance,
independent of time $j$, and pre-specified.
The resulting minimization problem, and
its solution, is described in Table \ref{3DVAR:t1}, making the
analogy with the Kalman filter.  Note that the minimization
itself is of a quadratic functional, and so may be solved by means of
linear algebra. The constraint formulation used for the Kalman filter,
in section \ref{sec:okf}, may also be applied and used to derive the mean
update formula.
\FloatBarrier
\begin{table}
	\begin{tabular}{ | c | c |}
		\hline
		Kalman Filter & 3DVAR  \\ \hline
		$m_{j+1} = \arg \min_v {\mathsf{J}}(v)$ & $m_{j+1} = \arg \min_v {\mathsf{J}}(v)$  \\ \hline
		${\mathsf{J}}(v) = \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2$  &  ${\mathsf{J}}(v) = \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}}^2$ \\
		\hline
		$\hat{m}_{j+1} = M m_j$ & $\hat{m}_{j+1} = \Psi(m_j)$ \\ \hline
		$m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}$ & $m_{j+1} = (I-KH) \hat{m}_{j+1} + K y_{j+1}$\\ \hline
	\end{tabular}
	\caption{Comparison of Kalman Filter and 3DVAR update formulae.}
		\label{3DVAR:t1}
\end{table}
\FloatBarrier

The Kalman gain $K$ for 3DVAR is fixed, because the predicted covariance
$\hat{C}$ is fixed. By analogy with Corollary \ref{cor:k2} we
have the following formulae for the 3DVAR gain matrix $K$, and the
update formula for the estimator $m_j$:
\begin{align*}
                S &= H \hat{C} H^T + \Gamma, \\
                K &= \hat{C} H^T S^{-1}, \\
m_{j+1} &= (I-KH) \Psi({m}_{j}) + K y_{j+1}.
        \end{align*}
The method also delivers an implied analysis covariance $C= (I - K H) \hat{C}.$
Note that the resulting algorithm which maps $m_j$ to $m_{j+1}$
may be specified directly in terms of the gain $K$, without
need to introduce $\hat{C}, C$ and $S$. In the remainder of this
section we simply view $K$ as fixed and given.
In this setting we
show that the 3DVAR algorithm produces accurate state estimation
under vanishing noise assumptions in the dynamics/data model.
The governing assumptions concerning the dynamics/data model
are encapsulated in:

\begin{assumption}
\label{a:3dvar}
Consider the dynamics/data model under the assumptions that
$\xi_j \equiv 0, \Gamma = \gamma^2 \Gamma_0, |\Gamma_0| = 1$
and assume that the data $y_{j+1}$ used in the 3DVAR
algorithm is found from observing
a true signal $v_j^{\dagger}$ given by
\begin{align*}
        \emph{Dynamics Model:}  \quad v_{j+1}^{\dagger} &= \Psi(v_j^{\dagger}), \quad j \in \mathbbm{Z}^+. \\
        \emph{Data Model:}  \quad y_{j+1} &= Hv_{j+1}^{\dagger} + \gamma  \eta^{\dagger}_{j+1,0}, \quad j \in \mathbbm{Z}^+.
\end{align*}
\end{assumption}

With this assumption of noise-free dynamics ($\xi_j \equiv 0$)
we deduce that the 3DVAR filter produces output which,
asymptotically, has an error of the same size as the observational noise
error $\gamma.$ The key additional assumption in the theorem that
allows this deduction is
a relationship between the Kalman gain $K$ and
the derivative $D\Psi(\cdot)$ of the dynamics model.
Encoded in the assumption are two ingredients: that the observation
operator $H$ is rich enough in principle to learn enough components
of the system to synchronize the whole system; and that $K$ is designed
cleverly enough to effect this synchronization. The proof of the
theorem is simply using these two ingredients and then controlling
the small stochastic perturbations, arising from Assumption \ref{a:3dvar}.

\begin{theorem}[Accuracy of 3DVAR]
Let Assumption \ref{a:3dvar} hold with $\eta^{\dagger}_{j,0} \sim \mathcal{N}(0,\Gamma_0).$ an i.i.d. sequence. Assume
that, for the gain matrix $K$ appearing in the 3DVAR method,
there exists a norm $\lVert \cdot \rVert$ on $\mathbbm{R}^\Du$ and constant $\lambda \in (0,1)$ such that, for all $v \in \mathbbm{R}^\Du,$
$$\lVert (I-KH)D \Psi (v) \rVert \leq \lambda.$$
Then there is constant $c>0$ such that the 3DVAR algorithm satisfies the following large-time asymptotic
error bound:
$$\limsup_{j\to\infty}  \operatorname{\mathbb{E}}[\lVert m_j - v^{\dagger}_j \rVert] \leqslant \frac{c\gamma}{1-\lambda},$$
where the expectation is taken with respect to the sequence $\{\eta^{\dagger}_{j,0}\}.$
\end{theorem}

\begin{proof}
We have
	\begin{align*}
		v^{\dagger}_{j+1} &= \Psi(v^{\dagger}_{j}), \\
m_{j+1} &= (I-KH)\Psi(m_j) + Ky_{j+1},
\end{align*}
and hence that
\begin{align*}
	v^{\dagger}_{j+1} &= (I-KH)\Psi(v^{\dagger}_{j}) + KH\Psi(v^{\dagger}_{j}),\\
	m_{j+1}	&= (I-KH)\Psi(m_j) + KH\Psi(v^{\dagger}_{j}) + \gamma K\eta^{\dagger}_{j+1,0}.
	\end{align*}
Define $e_j=m_j-v_j^{\dagger}.$ By subtracting the evolution
equation for $v_j^{\dagger}$ from that for $m_j$ we obtain, using
the mean value theorem,
\begin{align*} \label{3DVAR:eq3}
	e_{j+1} &= m_{j+1} - v^{\dagger}_{j+1} & \\
	&= (I-KH)(\Psi(m_j)-\Psi(v^{\dagger}_j)) + \gamma K \eta^{\dagger}_{j+1,0} & \\
	&= \left( (I-KH) \int_0^1  {D} \Psi \left(s m_j + (1-s)v_j^\dagger \right) ds \right) e_j + \gamma K \eta_{j+1,0}^\dagger. &&
\end{align*}

\noindent As a result, by the triangle inequality,
\begin{align*}
	\lVert e_{j+1} \rVert  & \leq  \left\lVert \left(  \int_0^1 (I-KH)D \Psi \left(s m_j + (1-s)v_j^\dagger \right) ds \right) e_j \right\rVert + \lVert \gamma K \eta_{j+1,0}^\dagger \rVert \\
	& \leq   \left(  \int_0^1 \left\lVert (I-KH)D \Psi \left(s m_j + (1-s)v_j^\dagger \right)  \right\rVert ds \right)  \lVert e_j \rVert  + \lVert \gamma K \eta_{j+1,0}^\dagger \rVert &  \\
	& \leq \lambda \lVert e_j \rVert + \gamma \lVert K \eta_{j+1,0}^{\dagger} \rVert. &
\end{align*}
Taking expectations on both sides, we obtain, for $c:= \operatorname{\mathbb{E}}[\lVert K \eta_{0,j+1}^{\dagger} \rVert] > 0$,
\begin{equation} \label{3DVAR:eq5}
	\begin{split}
		\operatorname{\mathbb{E}} [\lVert e_{j+1} \rVert] & \leq \lambda \operatorname{\mathbb{E}} [\lVert e_j \rVert] + \gamma \operatorname{\mathbb{E}} [\lVert K \eta_{j+1,0}^{\dagger} \rVert] \\
		& \leq \lambda \operatorname{\mathbb{E}} [\lVert e_j \rVert] + \gamma c.
	\end{split}
\end{equation}
Using the  discrete Gronwall inequality we have that:
\begin{equation} \label{3DVAR:eq6}
	\begin{split}
		\operatorname{\mathbb{E}} [\lVert e_j \rVert] & \leq \lambda^j \operatorname{\mathbb{E}} [\lVert e_0 \rVert] + \sum_{i=0}^{j-1} c \lambda^i \gamma \\
		& \leq \lambda^j \operatorname{\mathbb{E}} [\lVert e_0 \rVert] +  c \gamma \frac{1-\lambda^j}{1-\lambda},
	\end{split}
\end{equation}
where $e_0 =  m_0 - v_0$. As $\lambda<1$, the desired statement follows.
\end{proof}

\subsection{4DVAR}

Recall that 3DVAR differs from 4DVAR because, whilst also based on an
optimization principle, 4DVAR is applied in a distributed fashion over
all data in the time interval $j=1,\ldots, J$; in contrast 3DVAR is
applied sequentially from time $j-1$ to time $j,$ for $j=1,\ldots, J$.
We consider two forms of the methodology: {\em weak constraint 4DVAR (w4DVAR)}, in which the fact that the dynamics model contains randomness is allowed
for in the optimization; and {\em 4DVAR} (sometimes known as {\em strong constraint 4DVAR}) which can be derived from w4DVAR in the limit of $\Sigma \to 0$ (no randomness in the dynamics).

The objective function minimized in w4DVAR is
\begin{equation}\label{eq:objectivew4dvar}
	{\mathsf{J}}(V) = \frac{1}{2}|v_0 - m_0|^2_{C_0} + \frac{1}{2}  \sum^{J-1}_{j=0}  |v_{j+1} - \Psi(v_j)|^2_{\Sigma} +  \frac{1}{2} \sum^{J-1}_{j=0} |y_{j+1} - H{v_{j+1}}|^2_{\Gamma},
\end{equation}
where $V=\{v_{j}\}^{J}_{j=0} \in \mathbbm{R}^{d(J+1)}$,
$Y=\{y_{j}\}^{J}_{j=1} \in \mathbbm{R}^{k J}$, $v_j \in \mathbbm{R}^\Du$, $y_j \in \mathbbm{R}^\Dy$, $H$
is the observation operator, $\Sigma$ is the random dynamical system covariance, $\Gamma$ is the data noise covariance and $m_0$ and $C_0$ are the mean and
covariance of the initial state.
The three terms in the objective function enforce, in turn, the initial
condition $v_0$, the dynamics model and the data model. Note that, because
$\Psi$ is nonlinear, the objective is not quadratic and cannot be optimized
in closed form. In contrast, each step of 3DVAR required solution of a
quadratic optimization, tractable in closed form.

\begin{theorem}[Minimizer Exists for w4DVAR]
Assume that $\Psi$ is bounded and continuous. Then ${\mathsf{J}}$ has a minimizer, which is a MAP estimator for the smoothing problem.
\end{theorem}

\begin{proof}
Recall Theorem \ref{thm:achievable-obj}, which shows that the MAP estimator based on the smoothing distribution $\operatorname{\mathbbm{P}}(V|Y) \propto \exp\bigl( -{\mathsf{J}}(V) \bigr)$ is attained provided that ${\mathsf{J}}$ is guaranteed to be non-negative, continuous and satisfy ${\mathsf{J}}(V) \to \infty$ as $|V| \to \infty.$ Now, the objective ${\mathsf{J}}$ defined by equation \eqref{eq:objectivew4dvar} is clearly non-negative, and it is continuous since $\Psi$ is assumed to be continuous. It remains to show that ${\mathsf{J}}(V) \to \infty$ as $|V| \to \infty.$
Let $R$ be a bound for $\Psi,$ so that $|\Psi(v_j)|_\Sigma \le R$ for all $v_j \in \mathbbm{R}^\Du.$
Then, since
$${\mathsf{J}}(V) \ge \frac{1}{2}|v_0|_{C_0}^2 - 2|v_0|_{C_0} |m_0|_{C_0}  + \frac{1}{2} \sum_{j=0}^{J-1} \Bigl( |v_{j+1} |^2_\Sigma
- 2R|v_{j+1}|_{\Sigma}  \Bigr)$$
it follows that ${\mathsf{J}}(V) \to \infty$ as $|V| \to \infty$ and the proof is complete.
\end{proof}

We now consider the vanishing dynamical noise limit of w4DVAR.
This is to minimize
\begin{equation*}
	{\mathsf{J}}_0(V) = \frac{1}{2}|v_0 - m_0|^2_{C_0} +  \frac{1}{2}  \sum^{J-1}_{j=0} |y_{j+1} - H{v_{j+1}}|^2_{\Gamma}
\end{equation*}
subject to the hard constraint that
$$v_{j+1}=\Psi(v_j), \quad j=0,\ldots, J-1.$$
This is 4DVAR.
Note that by using the constraint 4DVAR can be written as a minimization over
$v_0$, rather than over the entire sequence $\{v_j\}_{j=0}^{J}$ as is required
in w4DVAR.

We let ${\mathsf{J}}_{\sigma}$ denote the objective function ${\mathsf{J}}$ from w4DVAR
in the case where $\Sigma \to \sigma^2\Sigma_0$.
Roughly speaking, the following result shows that minimizers of ${\mathsf{J}}_{\sigma}$
converge as $\sigma \to 0$ to points in ${\mathbbm{R}}^{k(J+1)}$ which
satisfy the hard constraint associated with 4DVAR.

\begin{theorem}[Small Signal Noise Limit of w4DVAR]
Suppose that $\Psi$ is bounded and continuous and let $V^{\sigma}$ be a minimizer of ${\mathsf{J}}_{\sigma}.$
Then as $\sigma \to 0$ there is a convergent
subsequence of $V^{\sigma}$ with
limit $V^*$ satisfying $v^*_{j+1} = \Psi(v^*_j)$.
\end{theorem}

\begin{proof}
Throughout this proof $c$ is a constant which may change from
instance to instance, but is independent of $\sigma.$
Consider $V \in \mathbbm{R}^{d(J+1)}$ defined by  $v_0=m_0$
and  $v_{j+1}=\Psi(v_{j})$. Then $V$ is bounded, as $\Psi(\cdot)$ is bounded,
and the bound is independent of $\sigma$. Furthermore
\begin{equation*}
	{\mathsf{J}}_{\sigma}(V) =  \frac{1}{2} \sum^{J-1}_{j=0} |y_{j+1}-Hv_{j+1}|^2_\Gamma \leq c,
\end{equation*}
where $c$ is independent of $\sigma$.
It follows that
\begin{equation*}
	{\mathsf{J}}_{\sigma}(V^{\sigma}) \leq {\mathsf{J}}_{\sigma}(V) \leq c.
\end{equation*}
Thus
\begin{align*}
	\frac{1}{2}|v^{\sigma}_{j+1} - \Psi(v^{\sigma}_{j})|^2_{\Sigma_0} &= \frac{\sigma^2}{2}|v^{\sigma}_{j+1} - \Psi(v^{\sigma}_{j})|^2_{\Sigma} = \sigma^2 {\mathsf{J}}_\sigma(V^\sigma) \leq \sigma^2 c, \\
	 \frac{1}{2}|v^{\sigma}_{0} - m_0|^2_{C_0} &\leq  {\mathsf{J}}_\sigma(V^\sigma) \le  c .
\end{align*}
Since $\Psi$ is bounded these bounds imply that $|V^{\sigma}| \leq c$.
Therefore
there is a limit $V^{*} : V^{\sigma} \to V^*$ along
a subsequence. By continuity
\begin{equation*}
	0 \leq \frac{1}{2}|v^{*}_{j+1} - \Psi(v^{*}_{j})|^2_{\Sigma_0} \leftarrow \frac{1}{2}|v^{\sigma}_{j+1} - \Psi(v^{\sigma}_{j})|^2_{\Sigma_0} \leq \sigma^{2}c. \\
\end{equation*}
Letting $\sigma \to 0$ we obtain that $v^*_{j+1} = \Psi(v^*_j).$
\end{proof}

\subsection{Discussion and Bibliography}

The 3DVAR and 4DVAR methodologies, in the context of weather forecasting,
are discussed in \cite{lorenc1986analysis} and \cite{fisher2009data}
respectively.
The accuracy analysis presented here is similar to
that which first appeared in the papers
\cite{brett2013accuracy}, \cite{moodey2013nonlinear}
and was developed further in \cite{law2012analysis,law2016filter}.
It arises from considering stochastic perturbations of the
seminal work of Titi and co-workers, exemplified by the paper
\cite{hayden2011discrete}. For an overview of variational methods,
and their links to problems in physics and mechanics, see the book
\cite{abarbanel2013predicting}, and the references therein; see
also the paper \cite{broecker2017existence}.

\section{\Large{\sffamily{The Gaussian Approximation}}}

Recall the inverse problem of finding $u$ from $y$ given by \eqref{eq:jc0},
and the Bayesian formulation which follows from Assumption \ref{a:jc1}.
In the previous chapter we explored the idea of obtaining a point estimator using an optimization perspective arising
from maximizing the posterior pdf. We related this idea to finding the center of a ball of radius $\delta$ with maximal probability in the limit $\delta \to 0.$
Whilst the idea is intuitively appealing, and reduces the complexity of Bayesian inference from determination of an entire distribution to
determination of a single point, the approach has a number of limitations,
in particular for noisy, multi-peaked or high dimensional
posterior distributions; the examples at the end of the previous chapter
illustrated this.

In this chapter we again adopt an optimization approach to the problem of Bayesian inference, but instead seek a Gaussian distribution $p = \mathcal{N}(\vct{\mu}, \vct{\Sigma})$ that minimizes the Kullback-Leibler divergence from  the posterior $\pi^y(\vct{u})$; since the Kullback-Leibler divergence is not symmetric this leads to two distinct problems, both of which we will study.

\subsection{The Kullback-Leibler Divergence}
\begin{definition}
Let $\pi,  \pi' >0$ be two probability distributions on $\mathbbm{R}^\Du$. The
{\em Kullback-Leibler (K-L) divergence}, or {\em relative entropy},
of $\pi$ with respect to $\pi'$ is defined by
\begin{equation*}
\begin{split}
d_{\mbox {\tiny{\rm KL}}}(\pi \| \pi') &:=  \int_{\mathbbm{R}^\Du} \log{\left(\frac{\pi(u)}{\pi'(u)}\right) \pi(u)du  }\\
 &= \operatorname{\mathbb{E}}{^\pi\left[ \log{\left(\frac{\pi}{\pi'}\right) }\right]} \\
 &= \operatorname{\mathbb{E}}{^{\pi'}\left[ \log{\left(\frac{\pi}{\pi'}\right)\frac{\pi}{\pi'} }\right]}.
\end{split}
\end{equation*}
\end{definition}

Kullback-Leibler is a divergence in that $d_{\mbox {\tiny{\rm KL}}}(\pi, \pi') \ge 0,$ with equality if and only if $\pi = \pi'.$ However, unlike Hellinger and total variation it is not a distance. In particular,
the K-L divergence is not symmetric: in general
$$ d_{\mbox {\tiny{\rm KL}}}(\pi \| \pi') \neq d_{\mbox {\tiny{\rm KL}}}(\pi' \| \pi),$$
a fact that will be important in this chapter.
Nevertheless, it is useful for at least four reasons: (1) it provides an upper bound for many distances; (2) its logarithmic structure allows explicit computations
that are difficult using actual distances; (3) it satisfies many convenient analytical properties such as being convex in both arguments and lower-semicontinuous in the topology of weak convergence (4) it has an information theoretic and physical interpretation.

\begin{lemma} \label{l:kllemma}
The K-L divergence provides
the following upper bounds for Hellinger and total variation distance:
$$ d_{\mbox {\tiny{\rm H}}}(\pi, \pi')^2 \leq \frac{1}{2}d_{\mbox {\tiny{\rm KL}}}(\pi \| \pi'),
\quad d_{\mbox {\tiny{\rm TV}}}(\pi, \pi')^2 \leq d_{\mbox {\tiny{\rm KL}}}(\pi \| \pi').$$
\end{lemma}

\begin{proof}

The second inequality follows from the first one by Lemma \ref{l:dh2}; thus
we prove only the first inequality.
Consider the function $\phi:\mathbbm{R}^+ \mapsto \mathbbm{R}$
defined by
$$\phi(x) = x - 1 - \log{x}.$$
Note that
\begin{equation*}
\begin{split}
\phi'(x) &= 1 - \frac{1}{x}, \\
\phi''(x) &= \frac{1}{x^2}, \\
\phi(\infty) & = \phi(0) =\infty.
\end{split}
\end{equation*}
Thus the function is convex on its domain.
As the minimum of $\phi$ is attained at $x = 1$, and as $\phi(1) = 0$,
we deduce that $\phi(x) \geq 0$  for all $x \in (0, \infty).$
Hence,
\begin{equation*}
\begin{split}
x-1 &\geq \log{x} \hspace{1cm} \text{ for all } x \ge 0, \\
\sqrt{x} - 1&\geq \frac{1}{2}\log{x} \hspace{1cm} \text{for all } x  \ge 0.
\end{split}
\end{equation*}
We can use this last inequality to bound the Hellinger distance:
\begin{equation*}
\begin{split}
d_{\mbox {\tiny{\rm H}}}(\pi,\pi')^2 &= \frac{1}{2}\int \left( 1 - \sqrt{\frac{\pi'}{\pi}} \right)^2 \pi du \\
&=  \frac{1}{2}\int \left( 1 + \frac{\pi'}{\pi} -  2\sqrt{\frac{\pi'}{\pi}} \right) \pi du \\
&= \int \left( 1 - \sqrt{\frac{\pi'}{\pi}} \right) \pi du \leq -\frac{1}{2}\int \log\Bigl({\frac{\pi'}{\pi}}\Bigr)\pi du = \frac{1}{2}d_{\mbox {\tiny{\rm KL}}}(\pi \| \pi').
\end{split}
\end{equation*}
\end{proof}

\subsection{Best Gaussian Fit By Minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$}
\label{sec:existence}

In this section we prove the existence of an approximating Gaussian.
To streamline the exposition we work under the following assumption, which can be relaxed:

\begin{assumption}\label{asp:1}
The posterior distribution is constructed under the following assumptions:
\begin{itemize}
\item The loss function $\mathsf{L}(u):= -\log\nu\bigl(y-G(u)\bigr)$ is
non-negative and bounded above.
\item The prior is a centered isotropic Gaussian: $\rho \sim \mathcal{N}(0, \lambda^{-1}\Id).$
\end{itemize}
\end{assumption}
Let $\mathcal{A}$ be the set of Gaussian distributions on $\mathbbm{R}^\Du$ with positive definite covariance,
$$ \mathcal{A} = \{\mathcal{N}(\vct{\mu},\mtx{\Sigma}): \vct{\mu} \in \mathbbm{R}^\Du, \mtx{\Sigma} \in \mathbbm{R}^{d\times d} \text{ positive-definite symmetric}\}.$$

We have the following theorem:
\begin{theorem}[Best Gaussian Approximation]
\label{t:bi1}
Under Assumption \ref{asp:1}, there exists at least one probability distribution $p \in \mathcal{A}$ at which the infimum
$$\inf_{p \in \mathcal{A}} d_{\mbox {\tiny{\rm KL}}} (p \| \pi) $$
is attained.
\end{theorem}

\begin{proof}
The K-L divergence can be computed explicitly as
\begin{equation*}
\begin{split}
d_{\mbox {\tiny{\rm KL}}} (p \| \pi) = &\operatorname{\mathbb{E}}^p{\log{p}} - \operatorname{\mathbb{E}}^p \log \pi \\
 = &\operatorname{\mathbb{E}}^p
\left(
-\frac{1}{2}\vert \vct{u} - \vct{\mu} \vert^2_{\vct{\Sigma}}
- \frac{1}{2}\log \left( (2\pi)^d \text{det} \vct{\Sigma} \right) \right. + \left. \mathsf{L} (\vct{u}) + \frac{\lambda}{2} \vert \vct{u} \vert^2 +\log Z
\right).
\end{split}
\end{equation*}
Note that $Z$ is the normalization constant for $\pi$ and is independent
of $p$ and hence of $\vct{\mu}$ and $\vct{\Sigma}.$
We can represent a given random variable $\vct{u} \sim p$ by writing
 $\vct{u} = \vct{\mu} + \vct{\Sigma}^{1/2}\xi$, where $\xi \sim \mathcal{N}(\vct{0}, \vct{\Id})$,
and hence
$$ \vert \vct{u} \vert^2 = \vert \vct{\mu} \vert^2 + \vert \Sigma^{1/2}\xi|^2 + 2\langle \vct{\mu}, \vct{\Sigma}^{1/2}\xi \rangle$$
to obtain
\begin{equation*}
\begin{split}
d_{\mbox {\tiny{\rm KL}}} (p \| \pi) &=  - \frac{d}{2} - \frac{d}{2}\log(2\pi) - \frac{1}{2}\log \text{det}\vct{\Sigma}   +\operatorname{\mathbb{E}}^p \mathsf{L} (\vct{u}) +  \frac{\lambda}{2} \vert \vct{\mu} \vert^2+ \frac{\lambda}{2} \text{tr}(\vct{\Sigma})   + \log Z.\\
\end{split}
\end{equation*}
 Define ${\mathsf{I}}(\mu,\Sigma) = \operatorname{\mathbb{E}}^p \mathsf{L} (\vct{u}) + \frac{\lambda}{2} \vert \vct{\mu} \vert^2 + \frac{\lambda}{2} \text{tr}(\vct{\Sigma}) - \frac{1}{2}\log \text{det}\vct{\Sigma}.$
Clearly, there is a correspondence between minimizing $d_{\mbox {\tiny{\rm KL}}}(p \| \pi)$ over $p \in {\cal A}$ and minimizing ${\mathsf{I}}(\mu,\Sigma)$ over $\mu \in \mathbbm{R}^d$ and positive definite $\Sigma.$
Note that:
\begin{itemize}
\item ${\mathsf{I}}(0,\Id) <\infty.$
\item For any $\Sigma,$ ${\mathsf{I}}(\mu,\Sigma) \to \infty$ as $|\mu|\to \infty.$
\item For any $\mu,$ ${\mathsf{I}}(\mu,\Sigma) \to \infty$ as $\text{tr}(\Sigma)\to 0$ or $\text{tr}(\Sigma) \to \infty.$
\end{itemize}
Therefore, there are $M,r,R>0$ such that the infimum of ${\mathsf{I}}(\mu,\Sigma)$ over $\mu\in \mathbbm{R}^d$ and positive definite $\Sigma$  is equal to the infimum of ${\mathsf{I}}(\mu,\Sigma)$ over
$$\tilde{\mathcal{A}} := \{(\vct{\mu},\mtx{\Sigma}): \vct{\mu} \in \mathbbm{R}^\Du, \mtx{\Sigma} \in \mathbbm{R}^{d\times d} \text{ positive-definite symmetric}, |\mu|\le M, r\le \text{tr}(\Sigma) \le R  \}.$$
Since ${\mathsf{I}}$ is continuous in $\tilde{\mathcal{A}}$ it achieves its infimum and the proof is complete.
\end{proof}

We remark that the theorem establishes the existence of a  best Gaussian approximation. However, minimizers need not be unique.

\subsection{Best Gaussian Fit By Minimizing $d_{\mbox {\tiny{\rm KL}}}(\pi\|p)$}
In this section we show that the best Gaussian approximation in Kullback-Leibler with respect to its second argument is unique and given by moment matching.
\label{sec:existence}
\begin{theorem}[Best Gaussian by Moment Matching]
Assume that $\bar{\mu} := \operatorname{\mathbb{E}}^{\pi}[u]$ is finite and that
$\bar{\Sigma} := \operatorname{\mathbb{E}}^{\pi}[(u-\bar{\mu})\otimes(u-\bar{\mu})]$ is
positive-definite. Then the
infimum $$\inf_{p \in \mathcal{A}} d_{\mbox {\tiny{\rm KL}}} (\pi\|p)$$
is attained at the element in $\mathcal{A}$ with mean $\bar{\mu}$ and
covariance $\bar{\Sigma}.$
\end{theorem}

\begin{proof}
By definition
\begin{align}
d_{\mbox {\tiny{\rm KL}}}(\pi \|p) = -\operatorname{\mathbb{E}}^{\pi}{[\log{p}]} + \operatorname{\mathbb{E}}^{\pi}{[\log{\pi}]}.
\label{Dkl}
\end{align}
Since the second term does not involve $p$, we study minimization of
\begin{align*}
-\operatorname{\mathbb{E}}^{\pi}{[\log{p}]} &= -\operatorname{\mathbb{E}}^{\pi}{\left[ \log\left(\frac{1}{\sqrt{(2\pi)^d\text{det}\Sigma}}\text{exp}\left(-\frac{1}{2}\big|u-\mu\big|_{\Sigma}^2\right)\right)\right]}\\
 &= \frac{1}{2}\operatorname{\mathbb{E}}^{\pi}{\left[\big|u-\mu\big|_{\Sigma}^2\right]} +\frac{1}{2}\log{\text{det}\Sigma}+\frac{d}{2}\log{2\pi}.
\end{align*}
Let $\Omega = \Sigma^{-1}$. Then our task is equivalent to minimizing the
following function of $\mu$ and $\Omega$:
\begin{align*}
{\mathsf{I}}(\mu,\Omega) = \frac{1}{2}\operatorname{\mathbb{E}}^{\pi}{\left[ \langle u-\mu, \Omega(u-\mu)\rangle\right]-\frac{1}{2}\log{\text{det}\Omega}}.
\end{align*}
First we find the critical points of ${\mathsf{I}}$ by taking its first order partial derivative with respect to $\mu$ and $\Omega$ and setting both to zero:
\begin{align*}
\partial_\mu{\mathsf{I}} &= -\operatorname{\mathbb{E}}^{\pi}\left[\Omega(u-\mu)\right] = 0;\\
\partial _\Omega{\mathsf{I}} &= \frac{1}{2}\partial_\Omega \left( \operatorname{\mathbb{E}}^{\pi}\left[(u-\mu)\otimes(u-\mu):\Omega\right]\right)-\frac{1}{2\text{det}\Omega}\partial_\Omega \text{det}\Omega\\
& = \frac{1}{2}\operatorname{\mathbb{E}}^{\pi}\left[(u-\mu)\otimes(u-\mu)\right] - \frac{1}{2}\Omega^{-1} = 0; \quad\quad
\end{align*}
where we have used the relation $\partial_\Omega\text{det}\Omega = \text{det}{\Omega}\cdot \Omega^{-1}.$ Solving the above two equations gives us the critical point, expressed in terms
of mean and covariance,
$$(\bar{\mu},\bar{\Sigma}) = (\operatorname{\mathbb{E}}^{\pi}[u], \operatorname{\mathbb{E}}^{\pi}[(u-\bar{\mu})\otimes(u-\bar{\mu})]).$$

Next we will show $(\bar{\mu},\bar{\Sigma}^{-1})$ is a minimizer of ${\mathsf{I}}$ or,
equivalently, that $(\bar{\mu},\bar{\Sigma})$ is a minimizer of the expression
given in \eqref{Dkl}. To this end, if we deal with a vector $\theta$ parametrizing the distribution $p$, it will be sufficient to show that the Hessian of
equation~\eqref{Dkl} with respect to $\theta$ is positive definite. Recalling
the label $d$ of the dimension of the parameter $u$ of unknowns we then have:
\begin{align*}
p_{\theta}(u) &= \sqrt{\frac{\text{det}\Omega}{(2\pi)^d}}\text{exp}\left(-\frac{1}{2}(u-\mu)^{T}\Omega(u-\mu)\right)\\
&=\sqrt{\frac{\text{det}\Omega}{(2\pi)^d}}\text{exp}\left[-\left(\frac{1}{2}u^{T}\Omega u-\mu^{T}\Omega u+\frac{1}{2}\mu^{T}\Omega \mu\right)\right]\\
&= \sqrt{\frac{\text{det}\Omega }{(2\pi)^d}}\text{exp}\left(-\frac{1}{2}\mu^{T}\Omega \mu\right)\text{exp}\left(-\frac{1}{2}u^{T}\Omega u+\mu^{T}\Omega u\right).
\end{align*}
Utilizing vectorization of matrices we may write:
\begin{align*}
u^{T}\Omega u &= \Omega :uu^{T} = [\text{vec}(\Omega )]^{T}\text{vec}(uu^{T})\\
\mu^{T}\Omega u &= (\Omega \mu)^{T}u\\
\Rightarrow -\frac{1}{2}u^{T}\Omega u+\mu^{T}\Omega u & = \begin{bmatrix} \Omega \mu \\ -\frac{1}{2}\text{vec}(\Omega ) \end{bmatrix}^{T} \begin{bmatrix} u\\ \text{vec}(uu^{T}) \end{bmatrix}.
\end{align*}
Then we let:
\begin{align*}
\theta =  \begin{bmatrix} \Omega \mu \\ -\frac{1}{2}\text{vec}(\Omega ) \end{bmatrix},\quad\quad T(u) = \begin{bmatrix} u\\ \text{vec}(uu^{T}) \end{bmatrix}.
\end{align*}
The pdf $p_{\theta}(u)$ can then be written in the following form:
\begin{align}
\label{expo}
p_{\theta}(u) &= \frac{1}{Z(\theta)}\text{exp}\left(\theta^{T}T(u)\right)\\
\text{with}\quad Z(\theta)  &= \int_{\mathbbm{R}^\Du}\text{exp}\left(\theta^{T}T(u)\right)du.
\end{align}
Using equation~\eqref{expo} we can rewrite equation \eqref{Dkl} as:
\begin{align}
H(\theta) = d_{\mbox {\tiny{\rm KL}}}(\pi\|p_{\theta}) = -\theta^{T}\operatorname{\mathbb{E}}^{\pi}\left[T(u)\right]+\log\left(Z(\theta)\right)+\operatorname{\mathbb{E}}^{\pi}\left[\pi(u)\right].
\end{align}
Notice that
\begin{align*}
\nabla_\theta \log\left(Z(\theta)\right) &= \frac{1}{Z(\theta)}\int_{\mathbbm{R}^\Du}\nabla_\theta \left[\text{exp}\left(\theta^{T}T(u)\right)\right]du\\
&= \frac{1}{Z(\theta)}\int_{\mathbbm{R}^\Du}T(u)\text{exp}\left(\theta^{T}T(u)\right)du\\
&= \operatorname{\mathbb{E}}^{p_\theta}\left[T(u)\right].
\end{align*}
Therefore we can calculate the gradient and Hessian of $H(\theta)$ as follows:
\begin{align*}
\nabla_\theta H(\theta) &= -\operatorname{\mathbb{E}}^{\pi}\left[T(u)\right]+\operatorname{\mathbb{E}}^{p_\theta}\left[T(u)\right],\\
\left[\nabla^{2}_\theta H(\theta) \right]_{ij}&= \frac{\partial \log\left(Z(\theta)\right)}{\partial \theta_i \theta_j}= \frac{\partial}{\partial\theta_j}\left( \frac{1}{Z(\theta)}\int_{\mathbbm{R}^\Du}T_i(u) e^{\theta^{T}T(u)}du\right)\\
&= \frac{1}{Z(\theta)}\frac{\partial}{\partial \theta_j}\int_{\mathbbm{R}^\Du} T_i(u)e^{\theta^{T}T(u)}dx +\int_{\mathbbm{R}^\Du} T_i(u)e^{\theta^{T}T(u)}du \cdot \frac{\partial}{\partial \theta_j}Z(\theta)^{-1}\\
&= \frac{1}{Z(\theta)}\int_{\mathbbm{R}^\Du}T_i(u)T_j(u)e^{\theta^{T}T(u)}du\\
&\quad\quad\quad -\frac{1}{Z(\theta)^2}\int_{\mathbbm{R}^\Du}T_i(u)e^{\theta^{T}T(u)}du \cdot \int_{\mathbbm{R}^\Du}T_j(u)e^{\theta^{T}T(u)}du\\
&= \operatorname{\mathbb{E}}^{p_\theta}\left[T_iT_j\right] - \operatorname{\mathbb{E}}^{p_\theta}\left[T_i\right]\operatorname{\mathbb{E}}^{p_\theta}\left[T_j\right]\\
&= \left[\text{Cov}^{p_\theta}(T)\right]_{ij}.
\end{align*}
Therefore the Hessian of the objective function is
the covariance matrix of $T(u)$ under the multivariate normal distribution $p_\theta$ and by construction, is positive semi-definite. Therefore $d_{\mbox {\tiny{\rm KL}}}(\pi\|p_\theta)$ is convex in $\theta$ and the critical point $(\bar{\mu},\bar{\Sigma})$ is a corresponding minimizer.
\end{proof}

\begin{remark}
Notice that the preceding proof of convexity
holds for any distribution $p$ that can alternatively be parametrized by a vector $\theta$ with the general form of equation \eqref{expo}. In particular
for such problems the convexity of $d_{\mbox {\tiny{\rm KL}}}(\pi\|p_\theta)$ follows,
ensuring the existence of a minimizer. In fact, equation \eqref{expo}
is a reduced form of the following more general expression:
\begin{align}
p_\theta(u) &= h(u)\text{exp}\left(\theta^{T}T(u)-A(\theta)\right)
\label{general_expo}\\
\text{with}\quad\quad A(\theta) &= \log\left[\int_{\mathbbm{R}^\Du}h(u)\text{exp}\left(\theta^{T}T(u)\right)du\right].
\end{align}
Since $h(u)$ is independent of $\theta$, the conclusion of the
previous theorem carries over to distributions with the form of
\eqref{general_expo}. Such distributions are said to termed
the {\em exponential family} in the statistics literature, and
$\theta$ is called the natural parameter, $T(u)$ the  sufficient statistic, $h(u)$ the base measure, and $A(\theta)$ the log-partition. The Gaussian
distribution is a special case in which $h(u)$ is constant with respect to $u$.
In what follows we show some examples of other exponential family members.
\end{remark}

\begin{example}\textbf{The Bernoulli distribution}\\[0.05in]
Let $u$ be a random variable following the Bernoulli distribution with $\operatorname{\mathbbm{P}}(u=1) = q$ and $\operatorname{\mathbbm{P}}(u=0) = 1-q$. Then the point mass function describing
the distribution of $u$ is:
\begin{align*}
p(u;q) &= q^u(1-q)^{1-u}\\
&= \text{exp}\left[u\log\left(\frac{q}{1-q}\right)+\log(1-q)\right]\\
&= (1-q)\text{exp}\left[u\log\left(\frac{q}{1-q}\right)\right].
\end{align*}
Therefore $\theta = \log\left(\frac{q}{1-q}\right)$, $T(u) = u$, $A(\theta) = \log(1+e^{\theta})$, and $h(u) = 1$.
\end{example}

\begin{example}\textbf{The Poisson distribution}\\[0.05in]
The point mass function describing
the distribution of $u$ is, in this case,
\begin{align*}
p(u;\lambda) &= \frac{\lambda^{u}e^{-\lambda}}{u!}\\
& = \frac{1}{u!}e^{u\log\lambda-\lambda}.
\end{align*}
Therefore $\theta = \log{\lambda}$, $T(u) = u$, $A(\theta) = e^{\theta}$ and $h(u) = \frac{1}{u!}$.\\
\end{example}

\subsection{Comparison between $d_{\mbox {\tiny{\rm KL}}}(\pi \|p)$ and $d_{\mbox {\tiny{\rm KL}}}(p \|\pi)$}
It is instructive to compare the two different minimization
problems, both leading to a ``best Gaussian'', that we described
in the preceding two sections.
We write the two relevant divergences as follows and then explain the
nomenclature:
\begin{align*}
&d_{\mbox {\tiny{\rm KL}}}(p\|\pi) = \operatorname{\mathbb{E}}^{p}{\left[\log\left(\frac{p}{\pi}\right)\right]}= \operatorname{\mathbb{E}}^{p}{[\log{p}]} - \operatorname{\mathbb{E}}^{p}{[\log{{\pi}}]} \quad \text{"Mode-seeking"}\\
&d_{\mbox {\tiny{\rm KL}}}(\pi\|p) = \operatorname{\mathbb{E}}^{\pi}{\left[\log\left(\frac{\pi}{p}\right)\right]}= \operatorname{\mathbb{E}}^{\pi}{[\log{\pi}]} - \operatorname{\mathbb{E}}^{\pi}{[\log{{p}}]} \quad \text{"Mean-seeking"}
\end{align*}

\begin{figure}[!tbp]
\hspace*{\fill}%
  \centering
  \subfloat[Minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$]{\includegraphics[width=0.3\textwidth]{figures/lec5_mode.png}}
  \hfill
  \subfloat[Minimizing $d_{\mbox {\tiny{\rm KL}}}(\pi\|p)$]{\includegraphics[width=0.3\textwidth]{figures/lec5_mean.png}}
  \hspace*{\fill}
  \caption{(a) Minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ can lead to serious information loss while (b) minimizing $d_{\mbox {\tiny{\rm KL}}}(\pi\|p)$ ensures a comprehensive consideration of all components of $\pi$.}
\label{f:mvm}
\end{figure}

Note that when minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ we want $\log{\frac{p}{\pi}}$ to be small, which can happen when $p \simeq \pi$ or $p \ll \pi$. This illustrates
the fact that minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ may miss out components of $\pi$.
For example, in Figure \ref{f:mvm}(a)  $\pi$ is a bi-modal like distribution
but minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ over Gaussians $p$ can only give a single mode
approximation which is achieved by matching one of the modes;
we may think of this as  ``mode-seeking''. In contrast, when minimizing
$d_{\mbox {\tiny{\rm KL}}}(\pi\|p)$ over Gaussians $p$ we want $\log{\frac{\pi}{p}}$ to be small where $p$ appears as the denominator. This implies that wherever $\pi$ has some mass we must let $p$ also have some mass there in order to keep $\frac{\pi}{p}$ as close as possible to one. Therefore the minimization is carried out by allocating the mass of $p$ in a way such that on average the divergence between $p$ and $\pi$ attains its minimum as shown in Figure ~\ref{f:mvm}(b); hence
the label ``mean-seeking.''  Different applications will favor different
choices between the mean and mode seeking approaches to Gaussian approximation.
\label{sec:difference}

\subsection{Variational Formulation of Bayes Theorem}
\label{sec:posterior}
This chapter has been concerned with finding the best Gaussian approximation
to a measure with respect to KL  divergences. Bayes Theorem \ref{t:bayes}
itself can be formulated through a closely related minimization
principle.  Consider a posterior $\pi^y(u)$ in the following form:
\begin{align*}
\pi^y(u) = \frac{1}{Z}\text{exp}\bigl(-\mathsf{L}(u)\bigr)\rho(u),
\end{align*}
where $\rho(u)$ is the prior, $\mathsf{L}(u)$ is the negative log-likelihood,
and $Z$ the normalization constant. We assume here for exposition that all densities are positive. Then we can express $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ in terms of the prior as follows:
\begin{align*}
d_{\mbox {\tiny{\rm KL}}}(p\|\pi) &= \int_{\mathbbm{R}^\Du} \text{log}\left( \frac{p}{\pi}\right)p du\\
&= \int_{\mathbbm{R}^\Du} \text{log}\left( \frac{p}{\rho}\frac{\rho}{\pi}\right)p du\\
&=  \int_{\mathbbm{R}^\Du} \text{log}\left( \frac{p}{\rho}\text{exp}\bigl(\mathsf{L}(u)\bigr)Z\right)p du\\
&= d_{\mbox {\tiny{\rm KL}}}(p\|\rho) + \operatorname{\mathbb{E}}^{p}[\mathsf{L}(u)] +\log{Z}.
\end{align*}
If we define
$${\mathcal J}(p) =  d_{\mbox {\tiny{\rm KL}}}(p\|\rho) + \operatorname{\mathbb{E}}^{p}[\mathsf{L}(u)]$$
then we have the following:
\begin{theorem}[Bayes Theorem as an Optimization Principle]
The posterior distribution $\pi$ is given by the following minimization principle:
$$\pi = \text{\rm argmin}_{p\in {\mathscr{P}}} {\mathcal J}(p),$$
where ${\mathscr{P}}$ contains all probability densities on $\mathbbm{R}^\Du$.
\end{theorem}
\begin{proof}
Since $Z$ is the normalization constant for $\pi$ and is independent of $p$,
the minimizer of $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ will also be the minimizer of ${\mathcal J}(p).$ Since the global minimizer of $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ is
attained at $p=\pi$ the result follows.
\end{proof}

Why is it useful to view the posterior as the minimizer of an energy? There are at least three advantages of this viewpoint. First, the variational formulation provides a natural way to approximate the posterior by restricting the minimization problem to distributions satisfying some computationally desirable property. For instance, variational Bayes methods often restrict the minimization to densities with product structure and in this chapter we have studied restriction to the class of Gaussian distributions. Second, variational formulations allow to show convergence of posterior distributions indexed by some parameter of interest by studying the Gamma-convergence of the associated objective functionals. Third, variational formulations provide natural paths, defined by a gradient flow, towards the posterior. Understanding these flows and their rates of convergence is helpful in the choice of sampling algorithms.

\subsection{Discussion and Bibliography}
The definition of the K-L divergence, and upper-bounds
in terms of probability metrics, can be found in \cite{gibbs2002choosing}.
For a basic introduction to variational methods,
and the moment-matching version of Gaussian
approximation, see \cite{bishop}.
The problem of finding a Gaussian approximation of a general
finite dimensional probability distribution is studied in \cite{lu2016gaussian}, and
infinite dimensional formulations are considered in \cite{pinski2015kullback} and the
companion paper \cite{pinski2015algorithms}. The approximation in Theorem \ref{t:bi1}
consists of a single Gaussian distribution.
If the posterior has
more than one mode, a single Gaussian may not be appropriate.
For an approximation composed of Gaussian mixtures,
the reader is referred to \cite{lu2016gaussian}.
The paper \cite{trillos2019variational} highlights how minimization of K-L divergence arises naturally in the optimization of local entropy and heat regularized costs in deep learning.
The formulation of Bayes Theorem as an optimization principle is
well-known and widely used in the machine learning community
where it goes under the name ``variational Bayes''; see
the book \cite{mackay2003information} and the
paper \cite{bissiri2016general} for clear expositions of this subject. The variational formulation has been used to establish convergence of Bayesian procedures in \cite{trillos2017consistency} and \cite{garcia2018continuum} and to define gradient flows that converge to the posterior as a way to develop and analyze Markov chain Monte Carlo proposals in \cite{trillos2018bayesian}.
 For more information about the properties and examples of exponential family distributions, we refer to \cite{nielsen2009statistical}. and for background on matrix calculations that were used in this chapter we refer to  \cite{petersen2008matrix}.

\section{\Large{\sffamily{Filtering Approach to the Inverse Problem}}}

In this final chapter we demonstrate how the two separate themes that
underpin this course, inverse problems and data assimilation, may be
linked. This opens up the possibility of transferring ideas from filtering
into the setting of quite general inverse problems.
In the first section we describe the general, abstract, connection
and introduce the revolutionary idea of sequential Monte Carlo
methods (SMC).
In the second section we analyze the concrete case of applying the EnKF
to solve an inverse problem, leading to ensemble Kalman inversion (EKI).
In the final section we link the EKI methodology to SMC.

\subsection{General Formulation}

Recall the inverse problem of finding $u \in \mathbbm{R}^\Du$ from $y \in \mathbbm{R}^\Dy$ where
\begin{align}
\label{eq:need99}
y=G(u)+\eta,\; \eta\sim \mathcal{N}(0,\Gamma_0)
\end{align}
and the related loss function
\begin{equation*}
\phi_0(u)=\frac{1}{2}|y-G(u)|^2_{\Gamma_0}.
\end{equation*}
The reason for writing $\Gamma_0$, rather than $\Gamma$, will become apparent
below and will also be exploited in the third section of this chapter where
we study EKI.

If we put a prior $\rho$ on the unknown $u$ then the posterior takes
the form
$$\pi(u)=\frac{1}{Z}\exp(-\phi_0(u))\rho(u).$$
Let $J\in\mathbbm{N}$ and choose $h$ so that $Jh=1$. Then
define the family of pdfs $\{\pi_j\}_{j=0}^J$ by
$$\pi_j(u)=\frac{1}{Z_j}\exp(-jh\phi_0(u))\rho(u).$$
It follows that $\rho_0 = \rho$ and $\rho_J=\pi$ and, furthermore, we may update the
sequence $\{\pi_j\}_{j=0}^J$ sequentially using the formula
$$\pi_{j+1}(u)=\frac{Z_j}{Z_{j+1}}\exp\bigl(-h\phi_0(u)\bigr)\pi_j(u).$$
This simply corresponds to application of Bayes Theorem to the
inverse problem
\begin{align}
\label{eq:need00}
y=G(u)+\eta,\; \eta\sim \mathcal{N}(0,\Gamma),
\end{align}
with $\Gamma=\frac{1}{h}\Gamma_0$
and with prior $\pi_j.$ With this choice of $\Gamma$ the identity \eqref{eq:need00}
gives the likelihood proportional to $\exp(-h\phi_0(u))$. The update
may be written
$$\pi_{j+1}(u)={\mathcal{A}}\pi_j,$$
noting that the analysis operator ${\mathcal{A}}$ is nonlinear, corresponding
to multiplication by $\exp(-h\phi_0(u))$ and then normalization to
a pdf.

If we let ${\mathcal{P}}_j$ denote any Markov kernel for which $\pi_j$ is
invariant then we obtain
$$\pi_{j+1}={\mathcal{A}}{\mathcal{P}}_j \pi_j.$$
This update formula should be compared with the filtering update formula
\begin{align*}
\pi_{j+1} = {\mathcal{A}}_j {\mathcal{P}}\pi_j
\end{align*}
introduced and used in earlier chapters.

Whilst the $j-$dependence in Bayes rule and the Markov kernel
is interchanged between the inverse problem and the filtering problem,
this makes little material difference to implementation.
Firstly, once a Markov kernel $Pred$ is identified under which $\pi=\pi_J$
is invariant, it can be easily adapted to find a family of kernels
${\mathcal{P}}_j$ under which $\pi_j$ is invariant, simply by rescaling the
observation covariance. Secondly the fact that the data $y$ is fixed,
rather than changing at each step, makes little difference to
the implementation.
This perspective opens up an entire field, known as {\em sequential
Monte Carlo}, in which ideas from filtering
may be transferred to other quite different problems, including Bayesian
inversion, as explained here.

Note however (see the discussion at the end of section \ref{sec:overview})
that of the filtering methods we have introduced so far only the bootstrap filter
applies directly to the case of {\em nonlinear} observation operators; since
$G$ is in general nonlinear this means that extra ideas are required to
implement the optimal particle filter, 3DVAR, ExKF and EnKF. One approach is
to replace the sequential optimization principles by the non-quadratic
optimization problem required for nonlinear $G$; we do not discuss this idea
in any detail but it is a viable option. Another approach is to use the linearization
technique which we now outline in the context of EKI.

\subsection{Ensemble Kalman Inversion}

We now make a detour into the subject of how to use
filters to estimate parameters $u$ from data $y$ satisfying
\eqref{eq:need00}. The approach we study,
and which we will relate to sequential Monte Carlo in
the next section, is to introduce an artificial
time dynamic. This can be done quite simply as follows: we write
\begin{align*}
u_{j+1} = &  u_j,\\
y_{j+1} = &  G(u_{j+1}) + \eta_{j+1},
\end{align*}
and we can think of finding the filtering distribution on $u_j|Y_j.$
We discuss how to relate the $y_j$ to the one data point $y$ below.
For now we take $\eta_j \sim \mathcal{N}(0,\Gamma)$ but we will revisit this
choice in the next section, linking the problem to solution of \eqref{eq:need99}.
Because the observation operator $G$ is, in general, nonlinear, this
does not render our system in a form where we can readily apply the
EnKF. To this end we introduce a new variable $w_{j}$ and rewrite the
filter as:
\begin{align*}
u_{j+1} = &  u_j,\\
w_{j+1} = &  G(u_{j}),\\
y_{j+1} = &  w_{j+1} + \eta_{j+1}.
\end{align*}

We introduce the new variables $v=(u,w)^T$, nonlinear map $\Psi(v)=(u, G(u))^T$
and linear operators $H =  [0, I], H^{\perp} = [I,0].$  Then if we
write $v_j=(u_j,w_j)^T$ we may write the dynamical system in the form
\begin{subequations}
\label{eq:new}
\begin{align}
v_{j+1} = & {} \Psi(v_j)\\
y_{j+1} = & {} Hv_{j+1} + \eta_{j+1}.
\end{align}
\end{subequations}
We note that
$$Hv= w, \quad H^{\perp}v=u.$$

\begin{remark}
Typically either of the following  are used to construct artificial
data $\{y_j\}$ for the filtering algorithm, given a single instance of data $y$:
 $$y_{j+1}=
  \begin{cases}
  & y \text{ (unperturbed obervations)}\\
  & y+\bar{\eta}_{j+1}, \bar{\eta}_{j+1}\sim \mathcal{N}(0,\Gamma)\text{ (perturbed observations)}
  \end{cases}$$
The first choice is natural if viewing the algorithm as a sequential
optimizer; the latter, in the linear case, is natural when seeking to draw
samples from the posterior.\qed
\end{remark}

We now apply the EnKF to the dynamics/data model \eqref{eq:new}.
We obtain, for $n = 1,\dots,N$,
\begin{align}
\label{eq:eq1}
\hat{v}_{j+1}^{(n)}&=\Psi(v_j^{(n)}),\\
\bar{v}_{j+1}&= \frac{1}{N}\sum_{n=1}^N\hat{v}_{j+1}^{(n)},\\
\widehat{C}_{j+1}&=\frac{1}{N}\sum_{n=1}^N(\hat{v}_{j+1}^{(n)}-\bar{v}_{j+1})\otimes(\hat{v}_{j+1}^{(n)}-\bar{v}_{j+1}),\\
v_{j+1}^{(n)}&=
(I-K_{j+1}H)\hat{v}_{j+1}^{(n)}+ K_{j+1}y_{j+1}^{(n)}
\end{align}
with the Kalman gain
\begin{align*}
K_{j+1}& =\widehat{C}_{j+1}H^T S_{j+1},\\
S_{j+1}& =(HC_{j+1}H^T+\Gamma)^{-1}.
\end{align*}
Now we may simplify these expressions by using the specific $\Psi$, $v$, $H$ arising in the inverse problem:
\begin{align*}
\widehat{C}_{j+1}=
\begin{bmatrix}
C_{j+1}^{uu} & C_{j+1}^{uw}\\
(C_{j+1}^{uw})^T & C_{j+1}^{ww}
\end{bmatrix}
\;\;
\bar{v}_{j+1}=
\begin{pmatrix}
\bar{u}_{j+1}\\
\bar{w}_{j+1}
\end{pmatrix}
\end{align*}
Here $$\bar{u}_{j+1} = \frac{1}{N}\sum_{n=1}^N u_{j}^{(n)},\;\bar{w}_{j+1} = \frac{1}{N}\sum_{n=1}^N G(u_{j}^{(n)}):=\bar{G}_j$$
and $$C_{j+1}^{uw} = \frac{1}{N}\sum_{n=1}^N(u_{j}^{(n)}-\bar{u}_{j+1})\otimes(G(u_{j}^{(n)})-\bar{G}_{j}), \; C_{j+1}^{ww} = \frac{1}{N}\sum_{n=1}^N(G(u_{j}^{(n)})-\bar{G}_{j})\otimes(G(u_{j}^{(n)})-\bar{G}_{j}).$$
There is a similar expression for $C_{j+1}^{uu}$, but as we will show in what
follows it is not needed for the unknown parameter $u$ update formula.
Noting that, because of the structure of $H$, $S_{j+1}=(C_{j+1}^{ww}+\Gamma)^{-1}$
we obtain
\begin{align}
K_{j+1}=
\begin{pmatrix}\label{eq:2}
C_{j+1}^{uw}(C_{j+1}^{ww}+\Gamma)^{-1}\\
(C_{j+1}^{ww})(C_{j+1}^{ww}+\Gamma)^{-1}
\end{pmatrix}.
\end{align}
Combining equation \eqref{eq:2} with the update equation within \eqref{eq:eq1}
it follows that
\begin{align*}
\{v_j^{(n)}\}_{n\in \{1,N\}} \rightarrow \{v_{j+1}^{(n)}\}_{n\in \{1,N\}}
\end{align*}
and
\begin{align*}
\{H^{\perp}v_j^{(n)}\}_{n\in \{1,N\}} \rightarrow \{H^{\perp}v_{j+1}^{(n)}\}_{n\in
\{1,N\}}
\end{align*}
and hence that
\begin{align*}
u_{j+1}^{(n)} = H^{\perp}v_{j+1}^{(n)} = u_j^{(n)}+ C_{j+1}^{uw}(C_{j+1}^{ww}+\Gamma)^{-1}(y_{j+1}^{(n)}-G(u_j^{(n)})).
\end{align*}
Thus we have derived the EKI step \begin{equation}
\label{eq:EnKF}
u_{j+1}^{(n)}  = u_j^{(n)}+ C_{j+1}^{uw}\bigl(C_{j+1}^{ww}+\Gamma)^{-1}(y_{j+1}^{(n)}-G(u_j^{(n)})\bigr).
\end{equation}
The full algorithm is described below
\FloatBarrier
\begin{algorithm}
\caption{Algorithm for Ensemble Kalman Inversion}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Initial distribution $\rho_0 = \rho$, observations $Y_{J}$, number of particles $N$
\STATE {\bf Initial Sampling}: Draw $N$ particles $u_0^{(n)} \sim \rho$ so that $\rho^N=S^N\rho$
\vspace{0.04in}
\STATE{\bf Subsequent Sampling} For $j = 0,1,\dots,J-1$, perform
\begin{enumerate}
\item Set $\bar{u}_{j+1} = \frac{1}{N}\sum_{n=1}^N u_{j}^{(n)}.$
\item Set $\bar{G}_j= \frac{1}{N}\sum_{n=1}^N G(u_{j}^{(n)}).$
\item Set $C_{j+1}^{uw} = \frac{1}{N}\sum_{n=1}^N(u_{j}^{(n)}-\bar{u}_{j+1})\otimes(G(u_{j}^{(n)})-\bar{G}_{j}).$
\item Set $C_{j+1}^{uw} = \frac{1}{N}\sum_{n=1}^N(u_{j}^{(n)}-\bar{u}_{j+1})\otimes(G(u_{j}^{(n)})-\bar{G}_{j}), \; C_{j+1}^{ww} = \frac{1}{N}\sum_{n=1}^N(G(u_{j}^{(n)})-\bar{G}_{j})\otimes(G(u_{j}^{(n)})-\bar{G}_{j}).$
\item $u_{j+1}^{(n)}  = u_j^{(n)}+ C_{j+1}^{uw}\bigl(C_{j+1}^{ww}+\Gamma)^{-1}(y_{j+1}^{(n)}-G(u_j^{(n)})\bigr).$
\end{enumerate}
\vspace{0.04in}
\STATE {\bf Output}: $N$ particles $u_{J}^{(1)},u_{J}^{(2)}, \dots, u_{J}^{(N)}$
\end{algorithmic}
\label{alg:EKI}
\end{algorithm}
\FloatBarrier

\begin{remark}
This algorithm may be viewed as a derivative-free optimization method,
within the broad class that contains genetic, swarm or ant colony
optimization. \qed
\end{remark}

The algorithm has the following invariant subspace property.

  \begin{theorem}(Space of Ensemble)
\label{t:cite}
  Define $\mathcal{A} = span(u^{(n)}_{0})_{n \in \{ 1,\cdots, N \}}$, then
for all $j$ in $\{0,\dots,J\}$ and for all $n$ in $\{1,\cdots,N\}$,
$u^{(n)}_{j}$ defined by the iteration \eqref{eq:EnKF} lie
in $\mathcal{A}$.
  \end{theorem}
  \begin{proof}
  The proof proceeds by induction. We let $\mathfrak{u}_j \in {\mathbbm{R}^\Du}^N$
denote the collection of $\{u^{(n)}_{j}\}_{n \in \{1,\dots,N\}}.$
  At first, let $j$ be in $\{0,\dots,J\}$, $n$ and $m$ in $\{1,\dots,N\}$
and define the following two quantities:
  \begin{align*}
  d^{(n)}_{j}&:=(C^{ww}_{j+1}+\Gamma)^{-1}(y^{(n)}_{j+1}-G(u^{(n)}_{j})) \\
  D_{mn}(\mathfrak{u}_j)&:=-\langle d^{(n)}_{j},G(u^{(m)}_{j})-\bar{G_j} \rangle
  \end{align*}
  Notice that the $d^{(n)}_{j}$ are elements of the data space ${\mathbb R}^J$,
whereas the $D_{mn}(\mathfrak{u}_j)$ are scalar quantities. Furthermore notice that
$$u^{(n)}_{j+1}=u^{(n)}_{j}-\frac{1}{N} \sum\limits_{n=1}^N D_{mn}(\mathfrak{u}_j)(u^{(m)}_{j}-\bar{u_j}).$$
From the definition of $\bar{G_j}$ and bilinearity of the inner product
it follows that $\sum\limits_{n=1}^N D_{mn}(\mathfrak{u}_j) = 0$. Therefore the
update expression may be rewritten as,
for all $j \in \{0,\dots,J\}$ and $n \in \{1,\dots,N\}$,
$$u^{(n)}_{j+1}=u^{(n)}_{j}-\frac{1}{N} \sum\limits_{n=1}^N D_{mn}(\mathfrak{u}_j)u^{(m)}_{j}.$$
Hence if the property holds for all the particles of the time step $j$, it will clearly be the case for all the particles at time step $j+1$.
  \end{proof}

\begin{remark}
The choice of the initial ensemble of particles
$\{u^{(n)}_{0}\}_{n \in \{1,\dots,N\}}$ is thus crucial to the
performance of the EnKF,
since the algorithm remains in the initial
space of the initial ensemble.
In the setting of Bayesian inverse problems the initial ensemble
is frequently created by drawing from the prior.
Alternatively, if the prior is Gaussian, the first $N$ Karhunen-Loeve
eigenfunctions may be used, ordered by decreasing variance contribution.
More generally any truncated basis for the space $\mathbbm{R}^\Du$ is
a natural initial ensemble. However the question of how to adaptively
learn a good choice of ensemble subspace, in
response to observed data, is unexplored and potentially fruitful. \qed
\end{remark}

\begin{remark}
We describe an alternative way to approach the derivation of the EKI update
formulae. We apply Theorem \ref{eq:extra1} with the specific structure
arising from the dynamical system used in EKI. To this end we define
\begin{equation}
\label{eq:z2}
{\mathsf{I}}_{n}(b) := \frac{1}{2} \mid y_{j+1}^{(n)} - G({u}_{j}^{(n)})
-\frac{1}{N}\sum_{n=1}^N b_n \bigl(G({u}_{j}^{(n)})-\bar{G}_j\bigr) \mid ^{2}_{\Gamma}+\frac{1}{2N}\sum_{n=1}^N b_n^2.
\end{equation}
Once this quadratic form has been minimized with respect to $b$ then the
upate formula  \eqref{eq:deenz3} gives
\begin{subequations}
\label{eq:deenz33}
\begin{align*}
u_{j+1}^{(n)}&={u}_{j}^{(n)}+\frac{1}{N}\sum_{n=1}^N b_n\bigl({u}^{(n)}_{j}-\bar{u}_{j}\bigr),\\
w_{j+1}^{(n)}&=G({u}_{j}^{(n)})+\frac{1}{N}\sum_{n=1}^N b_n\bigl(G({u}^{(n)}_{j})-\bar{G}_{j}\bigr),\\
\end{align*}
\end{subequations}
(Note that the vector $b$ depends on $n$; we have supressed this dependence
for notational convenience). \color{red} Revise \normalcolor
Theorem \ref{t:cite} is an immediate consequence of this structure.
\qed
\end{remark}

\subsection{Linking Ensemble Kalman Inversion and SMC}

He we link the two preceding sections.
In the first subsection
we describe how ensemble Kalman inversion may be linked with
SMC through a particular scaling of $\Gamma$ with $h$. We then
take the limit $h \to 0$ and obtain a system or ordinary
differential equations.
In the second subsection we study the resulting algorithm in
the case in which $G$ is linear, leading to insight into the
EKI algorithm in the iterated context.

\subsubsection{Continuous Time Limit}

Although we have emphasized the optimization perspective on ensemble methods,
we may think of one-step of ensemble Kalman inversion as approximating
the filtering mapping $\pi_j=\operatorname{\mathbbm{P}}(u_j|Y_j) \mapsto  \pi_{j+1}=
\operatorname{\mathbbm{P}}(u_{j+1}|Y_{j+1});$
this will be a good approximation with measures are close to Gaussian and
when the number of ensemble members $N$ is large.
In order to link this to the mapping $\pi_{j+1}={\mathcal{A}}\pi_j$ in the
presentation of SMC in the first section of this chapter,
we set $\Gamma=\frac{1}{h}\Gamma_0.$
Let $\mathfrak{u} \in {\mathcal {X}}^N$ denote the
collection of $\{u^{(n)}\}_{n \in \{1,\dots,N\}}.$
Now define
$$D^0_{mn}(\mathfrak{u}):=\langle G(u^{(n)})-y,G(u^{(m)})-\bar{G} \rangle_{\Gamma_0},$$
where
$$\bar{G}:=\frac{1}{N}\sum_{m=1}^N G(u^{(m)}).$$
Note that then, to leading order in $h \ll 1$,
$$D_{mn}(\mathfrak{u})\approx h\langle G(u^{(n)})-y,G(u^{(m)})-\bar{G} \rangle_{\Gamma_0} = hD^0_{mn}(u).$$ It follows that, also to leading order in $h$,
\begin{equation*}
u^{(n)}_{j+1}\approx u^{(n)}_{j}-\frac{h}{N} \sum\limits_{m=1}^N D^0_{mn}(\mathfrak{u}_j)u^{(m)}_{j}
\end{equation*}
where $\mathfrak{u}_j$ is as defined analogously to the previous section
to denote the collection of particles at discrete time $j$.
Then letting $h\longrightarrow 0$ yields:
\begin{equation}
\label{eq:has}
\frac{du^{(n)}}{dt}=-\frac{1}{N}\sum\limits_{m=1}^N D^0_{mn}(\mathfrak{u})u^{(m)}
\end{equation}

\begin{remark}
Notice that equation \eqref{eq:has}
has families of fixed points where: (i) either the particles fit
the data exactly (which corresponds to the left hand side in the inner product
defining $D^0_{mn}(u)$ being zero);  or (ii) all particles collapse on their
mean value (which corresponds to the right hand side of the same inner product).
This suggests that the system of ordinary differential equations which
describes the behaviour of ensemble Kalman inversion is driven by
two desirable attributes: matching the data and achieving consensus. \qed
\end{remark}

\subsubsection{Linear Setting}
Now suppose $G(\cdot)$ is a linear map denoted $A\cdot$,
and define
$$\bar{u}=\frac{1}{N}\sum\limits_{m=1}^N u^{(m)}.$$
In this setting we have
\begin{align*}
\frac{du^{(n)}}{dt}&=-\frac{1}{N}\sum\limits_{m=1}^N\langle Au^{(n)}-y,A(u^{(m)}-\bar{u}) \rangle_{\Gamma_0}u^{(m)} \\
&=-\frac{1}{N}\sum\limits_{m=1}^N\langle Au^{(n)}-y,A(u^{(m)}-\bar{u}) \rangle_{\Gamma_0}(u^{(m)}-\bar{u}) \\
&= -\frac{1}{N}\sum\limits_{m=1}^N\langle A^T\Gamma_0^{-1}(Au^{(n)}-y),u^{(m)}-\bar{u} \rangle (u^{(m)}-\bar{u}) \\
&= -\frac{1}{N}\sum\limits_{m=1}^N (u^{(m)}-\bar{u})\otimes (u^{(m)}-\bar{u})(A^T\Gamma_0^{-1}(Au^{(n)}-y)).
\end{align*}
If we define
\begin{equation}
C(\mathfrak{u})=\frac{1}{N}\sum\limits_{m=1}^N (u^{(m)}-\bar{u})\otimes (u^{(m)}-\bar{u})
\end{equation}
then we find that, for $n=1, \dots, N$,
\begin{align*}
\frac{du^{(n)}}{dt}&=-C(\mathfrak{u}) (A^T\Gamma_0^{-1}(Au^{(n)}-y)\\
&=-C(\mathfrak{u})\nabla \Phi (u^{(n)}),
\end{align*}
where
$$\Phi(u)=\frac12|y-Au|_{\Gamma_{0}}^2.$$
This corresponds to a gradient descent in the subspace defined by the
initial ensemble. In particular
\begin{align*}
\frac{d}{dt}\Phi (u^{(n)}(t))&=\langle \nabla \Phi (u^{(n)}(t)),\frac{du^{(n)}}{dt}(t) \rangle \\
&= -|C(u)^{1/2}\nabla \Phi (u^{(n)}(t))|^{2} \leq 0
\end{align*}
demonstrating that the loss function $\Phi$ is decreasing along each trajectory
associated to each ensemble member.

We note also that, if we define $e^{(n)}=u^{(n)}-\bar{u}$, then
$$\frac{de^{(n)}}{dt}=-C(\mathfrak{u}) A^T\Gamma_0^{-1}Ae^{(n)}.$$
Because
$$C:=C(\mathfrak{u})=\frac{1}{N}\sum_{n=1}^N e^{(n)} \otimes e^{(n)}$$
it follows that
$$\frac{dC}{dt}=-2CA^T\Gamma_0^{-1}AC.$$
Note further that if $C$ is invertible (requires $N$ at least as big as the dimension
$d$) then the inverse $P$ satisfies
$$\frac{dP}{dt}=2A^T\Gamma_0^{-1}A$$
so that $\|P\| \to 0$ as $t \to \infty.$ This fact, suitably interpreted,
even in the case where $C$ is only invertible on a subspace,
drives the covariance to zero and causes ensemble collapse -- consensus -- and
at the same time $\Phi$ is minimized over an appropriate subspace.

\subsection{Discussion and Bibliography}

The idea of using particle filters to sample general distributions,
including those arising in Bayesian inversion, maybe be found in
\cite{del2006sequential}; a recent application to a Bayesian
inverse problem, which demonstrated the potential of the
methodology in that context, is \cite{kantas2014sequential}.
A simple proof of convergence of the method may be found in
\cite{beskos2015sequential}; it is based on the proof described in
\cite{rebeschini2015can} for the standard bootstrap particle filter.

The use of the ensemble Kalman filter for parameter
estimation was introduced in the papers \cite{lorentzen2001underbalanced,anderson2014kalman} in which a physical dynamical model was
appended with trivial dynamics for the parameters in order to estimate them;
the idea was extended to learn an entire field of parameter values
in \cite{naevdal2002near}.
The paper \cite{skjervheim2011ensemble} was the first to do what we do
here, namely to consider all the data at once and a single mapping
of unknowns parameters to data. In that paper only one iteration is
used; the papers \cite{chen2012ensemble,emerick2013ensemble}
demonstrated how iteration could be useful.
See also the book \cite{oliver2008inverse} for the use of ensemble
Kalman methods in oil reservoir simulation.

Development of the method
for general inverse problems was undertaken in \cite{iglesias2013ensemble},
and further development of iterative methods is described in
\cite{iglesias2015iterative,iglesias2016regularizing}.
Analysis of ensemble inversion was undertaken in
\cite{schillings2017analysis}, including the continuous-time limit
and gradient flow structure described here. The potential for
ensemble inversion in the context of hierarchical Bayesian
methods is demonstrated in \cite{chada2017parameterizations}.
The idea that we explain here, of obtaining an evolution equation for
the covariance which is satisfied exactly by ensemble methods, appears in
the remarkable papers
\cite{reich2011dynamical, bergemann2012ensemble}, and in the other
papers of Bergemann and Reich referenced therein. Their work is at the heart of
the analysis in \cite{schillings2017analysis} which demonstrates
the ensemble collapse and approximation properties of the ensemble
method when applied to linear inverse problems.

\section{\Large{\sffamily{Monte Carlo and Importance Sampling }}}
In this chapter we introduce Monte Carlo sampling and importance sampling.
They are two general techniques for estimating expectations computed
with respect to a particular distribution by generating samples from
(a possibly different) distribution; they may also be viewed as
approximations of a given measure via sums of Dirac measures.

Throughout this chapter we focus interest on
computing expectations with respect to
a probability distribution with pdf $\pi$ given in the form
\begin{equation}
\label{eq:MC1}
\pi(u) = \frac{1}{Z}g(u)\rho(u),
\end{equation}
where $Z$ is a normalizing constant.
Monte Carlo sampling will use samples from $\pi$ itself; importance
sampling will use samples from $\rho$. A particular application of
this setting is where $\rho$ is the prior, $\pi$ the posterior
and $g$ the likelihood.
Let $f:\mathbbm{R}^\Du\longrightarrow\mathbbm{R}$ denote the
function whose expectation we are interested in computing.
For any pdf $p$ on $\mathbbm{R}^\Du$
we write
\begin{equation}
\label{eq:meas2}
p(f)=\operatorname{\mathbb{E}}^{p}\left[f(u)\right]=\int_{\mathbbm{R}^\Du} f(u)p(u)du.
\end{equation}
In this chapter we will generalize the concept of pdf to include
Dirac mass distributions. A Dirac mass at $v$ will be viewed as
having pdf $\delta(\cdot-v)$ where $\delta(\cdot)$ integrates to one
and takes the value zero everywhere except at the origin.

\subsection{Monte Carlo Sampling}
\label{sec:MCdefn&thm}
If we have $N$ random samples $u^{(1)},\ldots,u^{(N)}$, generated i.i.d. according to $\pi$, then we can estimate $\pi$ by the Monte Carlo estimator $\pi_{MC}^N$, which is defined as
\begin{equation}
\label{eq:indeed}
\pi^{N}_{MC} := \frac{1}{N} \sum_{n=1}^{N}\delta(u-u^{(n)}).
\end{equation}
This gives rise to the following estimator of $\pi(f):$
\begin{equation*}
\pi^{N}_{MC}(f) = \frac{1}{N}\sum_{n=1}^{N}f(u^{(n)}),\,\, u^{(n)} \sim \pi \quad {\rm i.i.d.}
\end{equation*}

\newcommand*\diff{\mathop{}\!\mathrm{d}}
      \begin{theorem}[Monte Carlo Error]
\label{t:MC}
       For $f:\mathbbm{R}^\Du\longrightarrow\mathbb{R}$ denote $|f|_\infty := \sup_{u\in \mathbbm{R}^\Du} |f(u)|.$ We have
       \begin{equation*}
        \begin{split}
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\pi^{N}_{MC}(f)-\pi(f)\right]}\right|  = 0,\\
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{MC}(f)-\pi(f)\right)^2\right]}\right|  \leq \frac{1}{N}.
        \end{split}
        \end{equation*}
      \end{theorem}

      \begin{proof}
Define
$$\bar{f}(u) = f(u) - \pi(f).$$
To prove the first result, namely that the estimator is unbiased, note that
        \begin{equation*}
        \begin{split}
        &\operatorname{\mathbb{E}}{\left[\pi^{N}_{MC}(f)-\pi(f)\right]}\\
        & = \frac{1}{N} \sum_{n=1}^{N} \operatorname{\mathbb{E}}{\left[f\left(u^{(n)}\right) - \pi(f)\right]}\\
         &=\frac{1}{N} \sum_{n=1}^{N} \bigl(\pi\left(f\right)-\pi\left(f\right)\bigr)\\
         &= \frac{1}{N}\cdot 0 = 0.
        \end{split}
        \end{equation*}
Therefore the supremum of its absolute value is also zero.
For the second result, which bounds the variance of the estimator, we observe that $\operatorname{\mathbb{E}}{[\bar{f}]} = 0$ and, then,
        \begin{equation*}
        \begin{split}
        \operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{MC}(f)-\pi(f)\right)^2\right]} &= \frac{1}{N^2}\sum_{n=1}^{N}\sum_{n=1}^{N}\operatorname{\mathbb{E}}{\left[\bar{f}\left(u^{(n)}\right)\bar{f}\left(u^{(n)}\right)\right]}\\
        &= \frac{1}{N^2}\sum_{n=1}^{N} \operatorname{\mathbb{E}}{\left[\bar{f}\left(u^{(n)}\right)^2\right]} \\
        &= \frac{1}{N}\operatorname{\mathbb{E}}{\left[\bar{f}\left(u^{(1)}\right)^2\right]} = \frac{1}{N}\text{Var}_{\pi}[f]
        \end{split}
        \end{equation*}
        since $u^{(n)}$ are i.i.d.
In particular we have
\begin{equation}
\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{MC}(f)-\pi(f)\right)^2\right]}
= \frac{1}{N}  \text{Var}_{\pi}[f] \le \frac{1}{N}\pi(f^2)
\label{eq:refer}
\end{equation}
since
        \begin{equation*}
        \text{Var}_{\pi}[f] = \pi(f^2) -\pi(f)^2 \leq \pi(f^2).
        \end{equation*}
        Therefore
        \begin{equation*}
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{MC}(f)-\pi(f)\right)^2\right]}\right| = \sup_{|f|_\infty\leq1} \left| \frac{1}{N}\text{Var}_{\pi}[f]\right| \leq \frac{1}{N}.
        \end{equation*}
      \end{proof}

The theorem shows that the Monte Carlo estimator  $\pi^{N}_{MC}$ is
an unbiased approximation for the posterior $\pi$ and that, by choosing
$N$ large enough,
expectation of any bounded function $f$
can in principle be approximated by Monte Carlo
sampling to arbitrary accuracy. Furthermore, although the
convergence is slow with respect to $N$ there is no dependence on the
dimension of the problem or on the properties of $f$, other than its
supremum.
~

      \begin{example}[Approximation of an Integral]
\label{ex:q}
       Let $f:\mathbb{R}\longrightarrow\mathbb{R}$ be a sigmoid function defined on $\mathbb{R}$ and shown in Figure \ref{F:MC1}(a) below as the blue solid curve.
Let $u\sim \pi=0.1~\mathcal{N}(-5,1) +0.9~\mathcal{N}(5,1)$ be a Gaussian mixture consisting of two Gaussian distributions. We wish to approximate the expected value of $f(u) \times \mathbb{I}_{[a,b]}(u)$ and $\mathbb{I}_{[a,b]}(u) = \begin{cases} 1 & \text{ if } u \in [a,b] \\ 0 & \text{ otherwise} \end{cases}$. We use Monte Carlo sampling to generate $N$ random samples $u^{(1)},\ldots,u^{(N)}$ and compute the error between the actual integral and Monte Carlo estimator. The integral and estimator are in the form:
        \begin{equation*}
        \begin{split}
        &\pi(f) = \int_{a}^{b} f(u)\pi(u) \diff u,\\
        &\pi^{N}_{MC}(f) = \frac{1}{N} \sum_{n=1}^{N}f(u^{(n)})\mathbb{I}_{[a,b]}(u^{(n)}).
        \end{split}
        \end{equation*}
The results of a set of numerical experiments with $a=-5, b=5$ and varying $N$ are shown
in Figure \ref{F:MC1}(b).
A randomly chosen subset of the samples used when $N=100$ is displayed in Figure \ref{F:MC1}(a).

       \begin{figure}[thbp]
        \centering
        \begin{tabular}{@{}c@{}}
            \includegraphics[width=0.5\textwidth]{figures/lec6_pic2.png}
        \end{tabular}
        \hspace{0.0in}
        \begin{tabular}{@{}c@{}}
            \includegraphics[width=0.55\textwidth]{figures/lec6_pic3.png}
        \end{tabular}
        \caption{Large sampling number reduces the estimation error by Monte Carlo method.}
        \label{F:MC1}
       \end{figure}

      \end{example}

\subsection{Importance Sampling}
\label{sec:ISdefn&thm}
Standard Monte Carlo sampling can only be used when it is possible to sample from the desired
{\em target} distribution $\pi$. When it is not possible to sample from $\pi$,
we can draw samples from another {\em proposal} distribution $\rho$ instead.
We then need to evaluate $g$ in \eqref{eq:MC1} at each sample and use
it as an importance weight in the approximation of the desired distribution.
This is the idea of importance sampling.

Consider $\pi$ as in equation \eqref{eq:MC1}. Given a test function $f: \mathbbm{R}^d \to \mathbbm{R}$ we can rewrite its expectation with respect to $\pi$ in terms of expected values with respect to $\rho$ as follows:
\begin{equation*}
\pi(f) = \frac{\rho(fg)}{\rho(g)},
\end{equation*}
where we used that $Z = \rho(g).$
Approximating with standard Monte Carlo the numerator and the denominator gives
\begin{align*}
\pi(f) & \approx  \sum_{n=1}^{N }w^{(n)} f\left(u^{(n)}\right),\,\, u^{(n)} \sim \rho \,\, {\rm i.i.d.}  \\
&= \pi^{N}_{IS}(f),
\end{align*}
where
\begin{equation*}
w^{(n)} := \frac{g\left(u^{(n)}\right)}{\sum_{m=1}^{N}g\left(u^{(m)}\right)}, \quad \quad \pi^{N}_{IS} := \sum_{n=1}^{N }w^{(n)}\delta(u-u^{(n)}).
\end{equation*}
Thus, given $N$ samples $u^{(1)}, \ldots, u^{(N)}$ generated i.i.d. according to $\rho$ we can estimate $\pi$ with the \emph{particle approximation measure} $\pi^{N}_{IS}.$
The quality of this estimator may be assessed by considering the worst-case bias and mean squared error when using the particle approximation measure to estimate expectations over the class of bounded test functions $\{f: \mathbbm{R}^\Du \to \mathbbm{R}: |f|_\infty =1\}.$  We will show that worst-case error upper-bounds can be obtained in terms of the $\chi^2$ divergence between the target and the proposal, suggesting that the performance of importance sampling depends on the closeness between target and proposal.

\begin{definition}
Let $\pi>0$ and $\pi'>0$ be two probability distributions on $\mathbbm{R}^\Du.$ The \emph{$\chi^2$ divergence} of $\rho$ with respect to $\rho'$ is
\begin{equation}\label{eq:chi2}
d_{\mbox {\tiny{$ \chi^2$}}}(\pi \| \pi') := \int_{\mathbbm{R}^\Du} \Bigl( \frac{\pi(u)}{\pi'(u)} -1  \Bigr)^2 \, \pi'(u) du.
\end{equation}
\end{definition}
The $\chi^2$  is a divergence but not a distance as it is, in general, not symmetric. The next result shows that,  similarly as for standard Monte Carlo, the mean squared error of $\pi^{N}_{IS}(f)$ as an estimator of $\pi(f)$ is order $N^{-1}.$ However, there are two main differences: the estimator is now biased, and the constant in the mean squared error depends on the $\chi^2$ divergence between the target and the proposal.

    \begin{theorem}[Importance Sampling Error]
      \label{IS}
       We have
       \begin{equation*}
        \begin{split}
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\pi^{N}_{IS}(f)-\pi(f)\right]}\right|  \leq 2 \frac{1 + d_{\mbox {\tiny{$ \chi^2$}}}(\pi\|\rho)}{N},\\
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f)-\pi(f)\right)^2\right]}\right|  \leq 4 \frac{1 + d_{\mbox {\tiny{$ \chi^2$}}}(\pi\|\rho)}{N}.
        \end{split}
       \end{equation*}
      \end{theorem}

      \begin{proof}
        Given
        \begin{equation*}
        \pi(u) = \frac{1}{Z}g(u)\rho(u) = \frac{1}{\rho(g)}g(u)\rho(u),
        \end{equation*}
        we have that $d_{\mbox {\tiny{$ \chi^2$}}}(\pi\|\rho) = \frac{\rho(g^2)}{\rho(g)^2} -1.$ To ease the notation we denote $\zeta: = \frac{\rho(g^2)}{\rho(g)^2}.$
        We rewrite
        \begin{equation*}
        \begin{split}
        \pi(f) &= \frac{\rho(gf)}{\rho(g)} \simeq \frac{\rho^{N}_{MC}(gf)}{\rho^{N}_{MC}(g)} =\pi^{N}_{IS}(f).\\
        \end{split}
        \end{equation*}
       Then we have
       \begin{equation}\label{eq:splitting}
        \begin{split}
        \pi^{N}_{IS}(f) -\pi(f) &= \pi^{N}_{IS}(f) -  \frac{\rho(gf)}{\rho(g)}\\
        &=  \frac{\pi^{N}_{IS}(f)\left(\rho(g)-\rho^{N}_{MC}(g)\right)}{\rho(g)} - \frac{\left(\rho(gf)-\rho^{N}_{MC}(gf)\right)}{\rho(g)}.
        \end{split}
        \end{equation}
The expectation of the second term is zero and hence
       \begin{equation*}
       \begin{split}
\left| \operatorname{\mathbb{E}}{\left[\pi^{N}_{IS}(f) -\pi(f)\right]}\right|
&= \frac{1}{\rho(g)}\left|\operatorname{\mathbb{E}}{\left[\pi^{N}_{IS}(f)\left(\rho(g)-\rho^{N}_{MC}(g)\right)\right]}\right|\\
        &\leq \frac{1}{\rho(g)}\left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f)-\pi(f)\right)\left(\rho(g)-\rho^{N}_{MC}(g)\right)\right]}\right|,\\
        \end{split}
        \end{equation*}
       since $\operatorname{\mathbb{E}}{\left[\rho(g)-\rho^{N}_{MC}(g)\right]}=0$.
       Using the Cauchy-Schwarz inequality, the second result
from this theorem (whose proof follows) and \eqref{eq:refer} from the proof of
Theorem \ref{t:MC} we have, for all $|f|_\infty \leq 1$,

       \begin{equation*}
       \begin{split}
\left| \operatorname{\mathbb{E}}{\left[\pi^{N}_{IS}(f) -\pi(f)\right]}\right|
&\leq \frac{1}{\rho(g)} \left(\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f)-\pi(f)\right)^2\right]}\right)^{1/2}\left(\operatorname{\mathbb{E}}{\left[\left(\rho(g)-\rho^{N}_{MC}(g)\right)^2\right]}\right)^{1/2}\\
        &\leq \frac{1}{\rho(g)}\left(\frac{4\zeta}{N}\right)^{1/2}\left(\frac{\rho(g^2)}{N}\right)^{1/2} = \frac{2\zeta}{N}.
       \end{split}
       \end{equation*}

\begin{comment}
       The expectation of the second term is derived as
       \begin{equation*}
       \begin{split}
        \left| \operatorname{\mathbb{E}}{\left[\frac{\rho(g)-\rho^{N}_{MC}(g)}{\rho(g)}\right]}\right| &= \frac{1}{\rho(g)}\left|\operatorname{\mathbb{E}}{\left[\rho(g)-\rho^{N}_{MC}(g)\right]}\right| =0.
       \end{split}
       \end{equation*}

       Therefore,
       \begin{equation*}
       \begin{split}
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\pi^{N}_{IS}(f)-\pi(f)\right]}\right| &=  \sup_{|f|_\infty\leq1}\left| \operatorname{\mathbb{E}}{\left[\frac{\pi^{N}_{IS}(f)\left(\rho(g)-\rho^{N}_{MC}(g)\right)}{\rho(g)}\right]}-0\right|
        \leq \frac{2r}{N}.
       \end{split}
       \end{equation*}
\end{comment}

We now prove the second result. We use the splitting of
$\pi^{N}_{IS}(f) -\pi(f)$ into the sum of two terms
as derived in equation \ref{eq:splitting}. Using \eqref{eq:refer}, the basic inequality $(a-b)^2 \le 2(a^2 + b^2)$ and that for all $|f|_\infty \leq 1, |\pi^{N}_{IS}(f)| \leq 1$, we have, for all $|f|_\infty \leq 1$,
        \begin{equation*}
        \begin{split}
        &\left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f) -\pi(f)\right)^2\right]}\right|\\
        &\leq \frac{2}{\rho(g)^2} \left( \operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f)\right)^2\left(\rho(g)-\rho^{N}_{MC}(g)\right)^2\right]}+ \operatorname{\mathbb{E}}{\left[\left(\rho(gf)-\rho^{N}_{MC}(gf)\right)^2\right]} \right)\\
        &\leq \frac{2}{\rho(g)^2} \left( \operatorname{\mathbb{E}}{\left[\left(\rho(g)-\rho^{N}_{MC}(g)\right)^2\right]}+ \operatorname{\mathbb{E}}{\left[\left(\rho(gf)-\rho^{N}_{MC}(gf)\right)^2\right]} \right)\\
        &= \frac{2}{\rho(g)^2N}\bigl(\text{Var}_{\rho}\left[g\right] +\text{Var}_{\rho}\left[gf\right] \bigr)\\
        &\leq \frac{2}{\rho(g)^2N} \left( \rho(g^2)+\rho(g^2f^2)\right)\\
        &\leq  \frac{4\rho(g^2)}{\rho(g)^2N} = \frac{4\zeta}{N}.
        \end{split}
        \end{equation*}
        Therefore,
        \begin{equation*}
        \begin{split}
         \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f)-\pi(f)\right)^2\right]}\right|  \leq \frac{4\zeta}{N}.
        \end{split}
        \end{equation*}

      \end{proof}

This theorem shows that, unlike Monte Carlo, the Importance Sampling estimator $\pi^{N}_{IS}$ is biased for $\pi$.
The rate of convergence of the bias is twice that of the standard deviation,
however.
As for Monte Carlo the rate of convergence of the variance of the error
is governed by the inverse of $N$, but is independent of the dimension $d$
of the state space. However
for Importance Sampling to be accurate (with a limited number of samples $N$)
it is important that the $\chi^2$ divergence between the target and the proposal is not too large.

      \begin{example}[Change of Measurement]
\label{ex:q2}
We consider a similar set-up as in Example \ref{ex:q}, integrating
a sigmoid function, shown in blue in Figure \ref{Fig. 3}, with respect to
a probability measure $\pi$ which is bimodal, shown in red in
Figure \ref{Fig. 3}; we again restrict the support of the desired integral.
We estimate the integral using Importance Sampling based on $N$ random
samples $u^{(1)},\ldots,u^{(N)}$ from the measure
$\rho=\mathcal{N}(\mu,\,\sigma^{2})$, shown in green in Figure \ref{Fig. 3}.
The estimator of the integral is given by
        \begin{align*}
        \begin{split}
        \pi^{N}_{IS}(f) &= \sum_{n=1}^{N}w^{(n)} f(u^{(n)}){\mathbbm{1}}_{[a,b]}(u^{(n)}),\\
        w^{(n)}&=\frac{g(u^{(n)})}{\sum_{m=1}^{N}g(u^{(m)})}.
        \end{split}
        \end{align*}
Here $g$ is a function proportional to the ratio of the densities of
$\pi$ and $\rho.$
If $\pi(u^{(n)})>\rho(u^{(n)})$, the samples should have been denser,
so we raise the weight on $f(u^{(n)})$ in proportion to
$\frac{\pi(u^{(n)})}{\rho(u^{(n)})}> 1$. If $\pi(u^{(n)})<\rho(u^{(n)})$, the samples should have been less dense, so we lower the weight on $f(u^{(n)})$ in proportion to $\frac{\pi(u^{(n)})}{\rho(u^{(n)})}< 1$.

        \FloatBarrier
        \begin{figure}[thbp]
        \centering
        \begin{tabular}{@{}c@{}}
            \includegraphics[width=0.5\textwidth]{figures/lec6_pic4a.png}
        \end{tabular}
        \hspace{0.0in}
        \begin{tabular}{@{}c@{}}
            \includegraphics[width=0.5\textwidth]{figures/lec6_pic4b.png}
        \end{tabular}
        \caption{Importance Sampling is a change of measurement via the importance weights. The red curve shows a bimodal distribution $\pi$ and the green curve shows a Gaussian distribution $\rho$. The blue curve is the function to be integrated, on its support $[-5,5].$ The upper figure shows samples
from the posterior $\pi$ itself; these would be used for Monte Carlo
sampling; the lower curve shows samples from the prior $\rho$, as used
for Importance Sampling. The importance weights capture and compensate for the difference of sampling from these two distributions.}
          \label{Fig. 3}
		\end{figure}
		\FloatBarrier
      \end{example}

\subsection{Discussion and Bibliography}
A classic reference on the Monte Carlo method
which includes discussion of several variance-reduction techniques is \cite{hammersley1964percolation}. The
chapter notes \cite{anderson2014monte} give a comparison of Monte Carlo and Importance Sampling with examples. The paper \cite{kawai2017adaptive} further explores advanced Importance Sampling via adaptive algorithms.
When $N$ is large enough, $\pi^{N}_{MC}$ arising from a Monte Carlo simulation should be close to $\pi$ \cite{evans1995methods}.
In practice, all probabilities, integrals and summations can be approximated by the Monte Carlo method \cite{tokdar2010importance}.
A review of importance sampling, from the perspective of filtering
and sequential importance resampling, may be found in
\cite{agapiou2017importance}; the proofs in this chapter closely follow the presentation in that paper. Necessary sample size results for importance sampling in terms of several divergences between target and proposal were established in \cite{sanz2018importance}.
The subject of multi level Monte Carlo (MLMC) has made the use of
Monte Carlo methods practical in new areas of application;
see \cite{giles2015multilevel} for an overview. The methodology applies
when approximating expectations over infinite dimensional spaces,
and distributes the computational budget over different levels of
approximation, with the goal of optimizing the cost per unit
error, noting that the latter balances sampling and approximation based
sources.

\section{\Large{\sffamily{Filtering and Smoothing Problems and Well-Posedness}}}

In this chapter we introduce data assimilation problems in which
the model of interest, and the data associated with it, have
a time-ordered nature. We distinguish between the filtering problem
(on-line) in which the data is incorporated sequentially as it
comes in, and the smoothing problem (off-line) which is a specific instance
of the inverse problems that have been the subject of the preceding
chapters.

\subsection{Formulation of Filtering and Smoothing Problems}
\label{sec:filtering}
Consider the {\em stochastic dynamics model} given by
\begin{align*}
v_{j+1} &= \Psi(v_j) + \xi_j, \: j \in \mathbbm{Z}^+, \notag \\
v_0 &\sim \mathcal{N}(m_0, C_0), \: \xi_j \sim \mathcal{N}(0, \Sigma) \: \text{i.i.d.} \notag,
\end{align*}
where we assume that $v_0$ is independent of the sequence $\{ \xi_j \}$;
this is often written as $v_0 \perp \{ \xi_j \}.$
Now we add the {\em data model} given by
\begin{align*}
y_{j+1} &= h(v_{j+1}) + \eta_{j+1} , \: j \in \mathbbm{Z}^+, \notag \\
\eta_j &\sim \mathcal{N}(0, \Gamma) \: \text{i.i.d.} \notag,
\end{align*}
where we assume that $\{\eta_j\} \perp v_0$ for all $j$ and
$\eta_k \perp \xi_j $ for all $j,k$.
The following will be assumed in the remainder of this book.
\begin{assumption}
\label{a:fas}
The matrices $C_0,$ $\Sigma$ and $\Gamma$ are positive-definite.
Further, we have $\Psi \in C(\mathbbm{R}^\Du, \mathbbm{R}^\Du)$ and $h \in C(\mathbbm{R}^\Du, \mathbbm{R}^\Dy)$.
\end{assumption}

\noindent
We define
\begin{equation}
V := \{v_0, \ldots, v_J\}, \: Y := \{y_1, \ldots, y_J\},
\: \text{and} \: Y_j := \{y_1, \ldots, y_j\}.  \notag
\end{equation}
The sequence $V$ is often termed the {\em signal} and the sequence
$Y$ the {\em data}.

\begin{definition}[The Smoothing Problem]
The {\em smoothing problem} is to find the probability density
$\Pi(V):=\operatorname{\mathbbm{P}}(V|Y) = \operatorname{\mathbbm{P}}(\{v_0, \ldots, v_J\}|\{y_1, \ldots, y_J\})$ on $\mathbbm{R}^{d(J+1)}$
for some fixed integer $J.$ We refer to $\Pi$ as the {\em  smoothing distribution}.
\end{definition}
\begin{definition}[The Filtering Problem]
The {\em filtering problem} is to find the probability densities
$\pi_j(v_j):=\operatorname{\mathbbm{P}}(v_j|Y_j)$ on $\mathbbm{R}^\Du$ for $j=1, \dots, J.$
We refer to $\pi_j$ as the {\em filtering distribution at time $j.$}
\end{definition}

The key conceptual issue to appreciate concerning the filtering problem, in
comparison with the smoothing problem, is that interest is focused
on characterizing, or approximating, a sequence of probably distributions,
defined  in an iterative fashion as the data is acquired sequentially.

\begin{remark}\label{rem:marginal}
We note the following identity:
\begin{align*}
\int \Pi(v_0, \ldots, v_J)dv_0dv_1 \ldots dv_{J-1} = \pi_J(v_J).
\end{align*}
This expresses the fact that the marginal of the smoothing distribution at time $J$ corresponds to the
filtering distribution at time $J$. Note also that, in general, for $j<J$
\begin{equation}
\int \Pi(v_0, \ldots , v_J) dv_0\ldots dv_{j-1}\hat{dv_{j}}dv_{j+1} \ldots dv_{J} \neq \pi_j(v_j), \notag
\end{equation}
since the expression on the left-hand side of the equation depends on data
$Y_J$, whereas that on the right-hand side depends only on $Y_j$, and $j < J$.
\end{remark}

\subsection{The Smoothing Problem}
\label{sec:smooth}

\subsubsection{Formula for pdf of the Smoothing Problem}
The smoothing distribution can be found by combining a prior on $v$ and a likelihood function using Bayes formula.
The prior is the probability distribution on $v$ implied by the distribution of $v_0$ and the stochastics dynamics model; the likelihood function is defined by the data model. We now derive the prior and the likelihood separately.

The prior distribution can be derived as follows:
\begin{subequations}
\begin{align}
\operatorname{\mathbbm{P}}(V) &= \operatorname{\mathbbm{P}}(v_J, v_{J-1}, \ldots , v_0) \notag \\
&= \operatorname{\mathbbm{P}}(v_J|v_{J-1}, \ldots , v_0)\operatorname{\mathbbm{P}}(v_{J-1}, \ldots , v_0) \notag \\
&= \operatorname{\mathbbm{P}}(v_J|v_{J-1})\operatorname{\mathbbm{P}}(v_{J-1}, \ldots , v_0). \notag
\end{align}
\end{subequations}
The third equality comes from the Markov, or memoryless, property which
follows from the independence of the elements of the sequence $\{\xi_j\}.$
By induction, we have:
\begin{subequations}
\begin{align}
\operatorname{\mathbbm{P}}(V) &= \prod_{j=0}^{J-1}\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\operatorname{\mathbbm{P}}(v_0) \notag \\
&= \frac{1}{Z_\rho}\exp\bigl(-\mathsf{R}(V)\bigr) \notag \\
&=: \rho(V), \notag
\end{align}
\end{subequations}
where $Z_\rho>0$ is a normalizing constant and
\begin{equation}
\mathsf{R}(V) := \frac{1}{2}|v_0-m_0|^2_{C_0} + \frac{1}{2}  \sum_{j=0}^{J-1} |v_{j+1} - \Psi(v_j)|^2_{\Sigma}. \notag
\end{equation}

The likelihood function, which incorporates the measurements gathered from observing the system, depends only on the measurement model and may be derived as follows:
\begin{align}
\operatorname{\mathbbm{P}}(Y|V) &= \prod_{j=0}^{J-1} \operatorname{\mathbbm{P}}(y_{j+1}|v_0,\ldots , v_{J}) \notag \\
&= \prod_{j=0}^{J-1}\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1}) \notag \\
& \propto \exp\bigl(-\mathsf{L}(V;Y)\bigr), \notag
\end{align}
where
\begin{equation}
\mathsf{L}(V;Y) :=  \frac{1}{2} \displaystyle{\sum_{j=0}^{J-1}}|y_{j+1} - h(v_{j+1})|_{\Gamma}^2. \notag
\end{equation}
The factorization of $\operatorname{\mathbbm{P}}(Y|V)$ in terms of the product of the
$\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})$ follows from the independence of the
elements of $\{\eta_j\}$ and the fact that the observation at time $j+1$
depends only on the state at time $j+1.$

Using Bayes Theorem \ref{t:bayes} we find the smoothing distribution by combining the likelihood and the prior
\begin{align*}
\Pi(V) & \propto \operatorname{\mathbbm{P}}(Y|V)\operatorname{\mathbbm{P}}(V)\\
&  =\frac{1}{Z}\exp\bigl(-\mathsf{R}(V)-\mathsf{L}(V;Y)\bigr).  \notag
\end{align*}
Note that $V \in \mathbbm{R}^{d(J+1)}$, and $Y \in \mathbbm{R}^{k J}$.

\subsubsection{Well-Posedness of the Smoothing Problem}
Now we study the well-posedness of the smoothing problem with respect
to perturbations in the data. To this end we consider two smoothing
distributions corresponding to different observed data sequences $Y,Y'$:
\begin{align*}
\Pi(V)&:=\operatorname{\mathbbm{P}}(V|Y) = \frac{1}{Z}\exp\bigl(-\mathsf{R}(V)-\mathsf{L}(V;Y)\bigr), \notag \\
\Pi'(V)&:=\operatorname{\mathbbm{P}}(V|Y')= \frac{1}{Z'}\exp\bigl(-\mathsf{R}(V)-\mathsf{L}(V;Y')\bigr). \notag
\end{align*}
We make the following assumptions:
\begin{assumption}
There is finite non-negative constant $R$ such that the
data $Y, Y'$ and the observation function $h$ satisfy:
\label{a:swp}
\begin{itemize}
\item $|Y|, |Y'| \le R;$
\item Letting $\phi(V) :=\Bigl( \sum_{j=1}^J (|h(v_j)|^2) \Bigr)^{1/2}$, it holds that $\operatorname{\mathbb{E}}^{\rho}[\phi(V)]<\infty$.
\end{itemize}
\end{assumption}

The following theorem shows well-posedness of the smoothing problem.

\begin{theorem}[Well-posedness of Smoothing]
\label{t:wps}
Under Assumption \ref{a:swp}, there is $\kappa \in [0, \infty),$ independent of $Y$ and $Y'$
such that $$d_{\mbox {\tiny{\rm H}}}(\Pi, \Pi') \leq \kappa|Y-Y'|.$$
\end{theorem}

\begin{proof}
We show that the proof of Theorem \ref{thm1}, which established well-posedness for Bayesian inverse problems under Assumption \ref{a:dh1}, applies in the smoothing context as well. To do so we rewrite the problem in the same notation used in Chapter 1, and show that Assumption \ref{a:swp} above implies Assumption \ref{a:dh1}.
Write
\begin{align*}
\Pi(V)&= \frac{1}{Z}\exp\bigl(-\mathsf{L}(V;Y)\bigr) \rho(V)= \frac{1}{Z} g(V;Y) \rho(V),\\
\Pi'(V)& =  \frac{1}{Z'}\exp\bigl(-\mathsf{L}(V;Y')\bigr)  \rho(V) = \frac{1}{Z'} g(V;Y') \rho(V), \notag
\end{align*}
where $Z,Z'>0$ are normalization constants.
Here $ |Y-Y'|$ plays the role of $\delta$ in Theorem \ref{thm1}.
Since $g(V;Y) = \exp\bigl(-\mathsf{L}(V;Y)\bigr)$ and $\mathsf{L}(V;Y)$ is positive we have that
$$\sup_{v} \Bigl|\sqrt{g(V;Y)}\Bigr| + \Bigl|\sqrt{g(V;Y')}\Bigr|  \le 2,  $$
and so Assumption \ref{a:dh1} (ii) is satisfied. To see that Assumption \ref{a:dh1} (i) is also satisfied, note that $e^{-x}$ is Lipschitz-1. Therefore, using the Cauchy-Schwarz inequality and some algebraic manipulations, there is  $\kappa$ independent of $Y$ and $Y'$ such that
\begin{align*}
\Bigl|\sqrt{g(V,Y)} - \sqrt{g(V;Y')} \Bigr|&\le  \frac{1}{2} \Bigl| \mathsf{L}(V;Y) - \mathsf{L}(V;Y')  \Bigr|  \\
&=\frac{1}{2} \biggl| \sum_{j=0}^{J-1}\frac12 \Bigl( |y_{j+1} - h(v_{j+1})|_\Gamma^2 - |y'_{j+1} - h(v_{j+1})|_\Gamma^2   \Bigr)  \biggr|  \\
&\le \kappa \sum_{j=0}^{J-1} |y_{j+1} - y'_{j+1}|_\Gamma  |y_{j+1} + y'_{j+1} -2 h(v_{j+1})|_\Gamma
   \\
   &\le \kappa |Y-Y'| \phi(V),
\end{align*}
where $\phi$ is defined in Assumption \ref{a:swp}.
This shows that under Assumption \ref{a:swp} the likelihood function of the smoothing problem satisfies Assumption \ref{a:dh1} with $\delta =|Y-Y'|$; Theorem \ref{t:wps} follows from Theorem \ref{thm1}.
\end{proof}

\subsection{The Filtering Problem}
\subsubsection{Formula for pdf of the Filtering Problem}
Filtering consists on the iterative updating of a sequence of probability distributions, as new data arrives.
This rests on the formula
\begin{equation}
\pi_{j+1} = {\mathcal{A}}_j {\mathcal{P}}\pi_j, \notag
\end{equation}
where ${\mathcal{P}}$ is a {\em linear} Markov map and ${\mathcal{A}}_j$ is a {\em nonlinear}
likelihood map (Bayes theorem).
We introduce $\hat{\pi}_{j+1}=\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)$ and recall
that ${\pi}_{j}=\operatorname{\mathbbm{P}}(v_{j}|Y_j).$
Then the maps ${\mathcal{P}}$ and ${\mathcal{A}}_j$ are defined by
\begin{eqnarray}
\begin{aligned}
{ \text{\bf Prediction Step:}} ~~~~~  \hat{\pi}_{j+1} = {\mathcal{P}} \pi_j. \\
{\text{ \bf Analysis Step:}} ~~~~~  \pi_{j+1} = {\mathcal{A}}_j \ \hat{\pi}_{j+1}.
\end{aligned}
\label{eq:pna}
\end{eqnarray}

The map ${\mathcal{P}}$ is sometimes termed {\em prediction}; the map ${\mathcal{A}}_j$ is
termed {\em analysis}. Throughout this section, all integrals are over $\mathbbm{R}^\Du.$
First we derive the linear Markov map ${\mathcal{P}}$. By the Markov property of the
stochastic dynamics model we have
\begin{align}
\hat{\pi}_{j+1}(v_{j+1}) &= \operatorname{\mathbbm{P}}(v_{j+1}|Y_j)\\
&= \int \operatorname{\mathbbm{P}}(v_{j+1}|Y_j, v_j)\operatorname{\mathbbm{P}}(v_j|Y_j)dv_j \notag \\
&= \int\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\operatorname{\mathbbm{P}}(v_j|Y_j)dv_j \notag \\
&= \int\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\pi_j(v_j)dv_j \notag \\
&= \frac{1}{(2\pi)^{d/2} ({\rm det}\Sigma)^{1/2}}\int\exp\Bigl(-\frac{1}{2}|v_{j+1}-\Psi(v_j)|^2_\Sigma \Bigr)\pi_j(v_j)dv_j. \notag
\end{align}
This defines the linear integral operator ${\mathcal{P}}$.

Now we derive the nonlinear likelihood map ${\mathcal{A}}_j$ by using Bayes theorem:
\begin{equation}
 \pi_{j+1} = {\mathcal{A}}_j\hat{\pi}_{j+1}. \notag
 \end{equation}
We note that
\begin{align}
\pi_{j+1}(v_{j+1}) &= \operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1})\\
&= \operatorname{\mathbbm{P}}(v_{j+1}|Y_j, y_{j+1}) \notag \\
&= \frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1}, Y_j)\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)}{\operatorname{\mathbbm{P}}(y_{j+1}|Y_j)} \notag \\
&= \frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)}{\operatorname{\mathbbm{P}}(y_{j+1}|Y_j)} \notag \\
&= \frac{\exp(-\frac{1}{2}|y_{j+1}-h(v_{j+1})|^2_{\Gamma})\hat{\pi}_{j+1}(v_{j+1})}{\int \exp(-\frac{1}{2}|y_{j+1}-h(v_{j+1})|^2_{\Gamma})\hat{\pi}_{j+1}(v_{j+1})dv_{j+1}}.\notag
\end{align}
This defines the nonlinear map ${\mathcal{A}}_{j+1}$ through multiplication
by the likelihood, and then normalization to a probability measure.

\subsubsection{Well-Posedness of the Filtering Problem}
Now we establish the well-posedness of the filtering problem. We let
\begin{equation*}
\pi_J = \operatorname{\mathbbm{P}}(v_J|Y), \quad \quad \pi_J' = \operatorname{\mathbbm{P}}(v_J|Y')
\end{equation*}
be two filtering distributions arising from observed data $Y = Y_J$ and $Y' = Y_J'.$ As noted in Remark \ref{rem:marginal}, the filtering distribution at time $J$ is the $J$-th marginal of the smoothing distribution; using this observation, the well-posedness of the filtering problem is a direct consequence of the well-posedness of the smoothing problem.
\begin{corollary}[Well-posedness of Filtering]
Under Assumption \ref{a:swp},
there exists $\kappa = \kappa(R)$, such that $d_{\mbox {\tiny{\rm TV}}}(\pi_J, \pi_J') \leq \kappa |Y-Y'|$.
\end{corollary}

\begin{proof}
Let $\Pi, \Pi'$ be the posterior distributions from the smoothing problems $\Pi = \operatorname{\mathbbm{P}}(V|Y)$ and $\Pi' = \operatorname{\mathbbm{P}}(V|Y')$. We
note that there exists $\kappa$ such that $d_{\mbox {\tiny{\rm TV}}}(\Pi,\Pi') \leq \kappa|Y-Y'|$
by Theorem \ref{t:wps} and by the fact that the Hellinger metric bounds
the total variation metric (Lemma \ref{l:dh2}).
Let $f: \mathbbm{R}^\Du \to \mathbbm{R}$ and
$F: \mathbbm{R}^{d(J+1)} \mapsto \mathbbm{R}.$ Then
\begin{align}
d_{\mbox {\tiny{\rm TV}}} (\pi_J, \pi_{J}') &= \frac{1}{2} \sup_{|f|_\infty \leq 1 }\Bigl|\operatorname{\mathbb{E}}^{\pi_J}[f(v_J)] - \operatorname{\mathbb{E}}^{\pi_{J'}}[f(v_J)] \Bigr| \notag \\
&= \frac{1}{2} \sup_{|f|_{\infty} \leq 1} \Bigl|\operatorname{\mathbb{E}}^\Pi[f(v_J)] - \operatorname{\mathbb{E}}^{\Pi'}[f(v_J)] \Bigr| \notag \\
&\leq \frac{1}{2} \sup_{|F|_{\infty} \leq 1} \Bigl| \operatorname{\mathbb{E}}^\Pi[F(V)] - \operatorname{\mathbb{E}}^{\Pi'}[F(V)]  \Bigr|\notag \\
&= d_{\mbox {\tiny{\rm TV}}}(\Pi, \Pi') \notag \\
&\leq \kappa|Y-Y'|. \notag
\end{align}
Here the first inequality follows from the fact that $\{|f|\leq 1\}$ can be viewed as a subset of $\{|F| \leq 1\}$.
\end{proof}

\subsection{Discussion and Bibliography}

The book \cite{law2015data} gives a mathematical introduction to
data assimilation;
for further information on the smoothing problem as presented here,
see section 2.3 in that book;
for further information on the filtering problem as presented here,
see section 2.4.
The books \cite{abarbanel2013predicting,reich2015probabilistic} give
alternative foundational presentations of the subject.
The books \cite{kalnay2003atmospheric,oliver2008inverse,majda2012filtering,carrassi2018data}
study data assimilation in the context of weather forecasting, oil reservoir
simulation, turbulence modeling and geophysical sciences, respectively.

\section{\Large{\sffamily{Markov Chain Monte Carlo}}}
In this chapter we study Markov chain Monte Carlo (MCMC),
a methodology that allows to obtain approximate samples from a given {\em target} distribution $\pi.$ As with Monte Carlo and importance sampling, MCMC may be viewed as approximating the target distribution by a sum of Dirac measures, thus allowing to approximate expectations with respect to the target. Implementation of Monte Carlo pressuposes that independent samples from the target can be obtained. Importance sampling and MCMC bypass this restrictive assumption: importance sampling by appropriately weighting independent samples from a proposal distribution, and MCMC by drawing correlated samples from a Markov kernel that has the target as invariant distribution. After some discussion of the general MCMC methodology, we will specify to the case where $\pi$ is a posterior distribution given via Bayes theorem from the product of the likelihood function and the prior distribution. In this context, we will analyze the convergence of the pCN algorithm, a popular MCMC method for computing posterior expectations in high dimensional inverse problems which uses these two ingredients, the prior and the likelihood, as part of its design.

\subsection{The Idea Behind MCMC}

The idea of MCMC is that, given a target distribution $\pi,$ it is possible to construct a Markov kernel that can be sampled from and has $\pi$ as its invariant distribution (a formal definition of the invariance property will be given below). Samples  $\{ u^{(n)} \}_{n=1}^{N}$  drawn iteratively from the kernel may be used to approximate posterior expectations
$$\pi(f) \approx \frac{1}{N} \sum_{n=1}^N f(u^{(n)}).$$
The samples are given uniform weights $1/N$ but, in contrast to standard Monte Carlo, they are not independent and they are not drawn from the target $\pi.$ However,  if the chain is guaranteed to satisfy certain {\em sample path ergodicity} then the estimator in the right hand-side is asymptotically unbiased for $\pi(f)$ and satisfies a central limit theorem for suitable test functions $f.$

Addressing the design and analysis of MCMC methods in generality and depth is beyond the scope of a single chapter;  entire books are devoted to this subject. We will restrict our discussion to a particular class of MCMC methods, known as Metropolis-Hastings algorithms. We will prove that the Metropolis-Hastings kernel is invariant with respect to the desired target distribution, and we will show \emph{geometric ergodicity} of the pCN Metropolis-Hastings algorithm, meaning that the distribution $\pi_n$ of the $n$-th sample approaches the invariant distribution exponentially fast in total variation distance. The idea is illustrated in Figure \ref{fig:MCMC_chain}: after
an initial number of {\em burn-in} steps the samples from
the chain start to concentrate in regions where the target distribution has greatest mass. We will not discuss sample path ergodicity, noting simply that a general abstract theory exists to deduce it from geometric ergodicity.

  \FloatBarrier
    \begin{figure}[htp]
      \centering
      \includegraphics[width=0.75\columnwidth]{figures/lec7_MCMC_chain1.png}
      \caption{The Markov chain samples points from distribution $\pi_n$ at step $n$, and the sampling distribution converges towards the target distribution
$\pi$ whose high density regions are represented by the dashed circles.}
      \label{fig:MCMC_chain}
    \end{figure}
    \FloatBarrier

\subsection{The Metropolis-Hastings Algorithm}

Here we outline the Metropolis-Hastings algorithm. The
algorithm has two ingredients: a {\em proposal distribution} $q(u,v)$,
which is a Markov transition kernel; and an acceptance probability
$a(u,v)$ that will be used to convert the proposal kernel into a kernel $p_{\mbox {\tiny{\rm MH}}}(u,v)$ that is invariant with respect to the given target $\pi.$ Given the $n^{\rm th}$ sample $u^{(n)},$ we generate
$u^{(n+1)}$ by drawing $v^\star$ from the distribution $q(u^{(n)}, \cdot)$ and
accepting the result, which means setting $u^{(n+1)}=v^\star$,
with probability $a(u^{(n)}, v^{\star})$, or instead setting $u^{(n+1)} = u^{(n)}$ with
the remaining probability $1 - a(u^{(n)}, v^{\star})$.
The acceptance probability is given by
\begin{eqnarray}
\label{eq:MHA}
a(u,v) = \text{min} \Bigg( \frac{\pi(v) q(v, u)}{\pi(u) q(u, v)},\, 1\Bigg).\end{eqnarray}
Algorithm \ref{algMH} makes precise the steps just described:

\FloatBarrier
\begin{algorithm}
\caption{\label{algMH} Metropolis-Hastings Algorithm}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Target distribution $\pi$, initial distribution $\pi_0,$ Markov kernel $q(u,v)$, number of samples $N.$  \\
\vspace{.04in}
\STATE {\bf Initial Draw}: Draw initial sample $u^{(0)} \sim \pi_0.$
 \\
\vspace{.04in}
\STATE{{{\bf Subsequent Samples}}}: For $n = 0,1,\dots,N -1$, perform the following steps:
\begin{enumerate}
\item Sample $v^{\star} \sim q(u^{(n)},\cdot).$
\item Calculate the acceptance probability $a_n:=a(u^{(n)},v^{\star}).$
\item Update
\begin{align*}
u^{(n+1)} =
\begin{cases}
 v^{\star}, & \text{w.p.} ~~~ a_n, \\ u^{(n)}, & \text{otherwise}.
\end{cases}
 \end{align*}
\end{enumerate}
\STATE{\bf Output} Samples $u^{(1)},u^{(2)},\dots,u^{(N)}.$
\label{alg_1}
\end{algorithmic}
\end{algorithm}
\FloatBarrier

The Metropolis-Hastings algorithm implicitly defines a Markov kernel $p_{\mbox {\tiny{\rm MH}}}(u,\cdot)$ which specifies the density of the $(n+1)^{\rm th}$ sample given that the $n^{\rm th}$ sample is located at $u.$ For $u\neq v,$  the Metropolis-Hastings kernel has the following simple expression in terms of the proposal kernel and the acceptance probability
\begin{equation}\label{eq:pmh}
p_{\mbox {\tiny{\rm MH}}}(u,v) =  a(u,v) q(u,v);
\end{equation}
this expression may be deduced noting that in order to move to a new location, the proposal needs to be accepted.

\begin{remark}
\begin{itemize}
\item In order to implement the Metropolis-Hastings algorithm one needs to be able to sample from the proposal kernel $q(u,\cdot)$ and evaluate the acceptance probability $a(u,v).$ Importantly, the target distribution only appears in the acceptance probability $a(u,v),$ and only the ratio $\pi(v)/\pi(u)$ is involved. Therefore the Metropolis-Hastings algorithm may be implemented for target distributions that are only specified up to an unknown normalizing constant.
\item If $q(u,v) = q(v,u)$ the acceptance probability simplifies to $\pi(v)/\pi(u).$ In such a case, moves to regions of higher target density are always accepted, while moves to regions of smaller but non-zero target density are accepted with positive probability in order to ensure exploration of the target space. If $q$ is not symmetric, the method favors moves that are easier to be reversed, namely moves for which  $q(v,u) >q(u,v).$
\item The Metropolis-Hastings algorithm is extremely flexible due to the freedom in the choice of proposal kernel $q(u,v)$. The ergodic behavior of the algorithm is heavily dependent on the choice of proposal.
\item The accept-reject step may be implemented by drawing, independently
from the proposal, a uniformly distributed random variable $\theta_n$ in the interval $[0,1].$ If $\theta_n \in [0,a_n)$ then the proposal is accepted ($u^{(n+1)}=v^\star$); it is rejected ($u^{(n+1)}=u^{(n)})$ otherwise.
\end{itemize}
\end{remark}

\subsection{Invariance of the Target Distribution \texorpdfstring{$\pi$}{}}
In this section we show that the Metropolis-Hastings kernel is invariant with respect to the target $\pi.$ We start by introducing formal definitions of detailed balance and invariance of a Markov kernel, and showing that the former implies the latter. We then prove that the Metropolis-Hastings kernel satisfies detailed balance with respect to $\pi$, and hence it is invariant.

\subsubsection{Detailed Balance and its Implication}
The following definition makes precise the notion of detailed balance and invariant distribution of a Markov kernel.

\begin{definition}
A Markov kernel $p(u,v)$ satisfies detailed balance with respect to $\pi$ if, for any $u, v \in \mathbbm{R}^\Du,$
\begin{equation*}
\pi(u) p(u,v) = \pi(v) p(v,u).
\end{equation*}
We say that $\pi$ is an invariant distribution of the Markov kernel $p(u,v)$ if, for any $v\in \mathbbm{R}^\Du,$
\begin{equation}
\int_{\mathbbm{R}^\Du} \pi(u) p(u,v)\, du = \pi(v).
\end{equation}
\end{definition}

Detailed balance of $p(u,v)$ with respect to $\pi$ implies that $\pi$ is invariant for $p(u,v)$. To see this, note that if $p(u,v)$ satisfies detailed balance with respect to $\pi$ then
$$\int_{\mathbbm{R}^\Du} \pi(u) p(u,v)\, du = \pi(v) \int_{\mathbbm{R}^\Du} p(v,u)\, du = \pi(v).$$
Invariance guarantees that, if the chain is distributed according to $\pi$ at a given  step, then it will also be distributed according to $\pi$ in the following step. Detailed balance guarantees that the in/out flow between any two states is preserved, which is a stronger condition. The concept of detailed balance is illustrated through the flow diagram shown in Figure \ref{fig:DB4}. The flow to/from any node must be the same under detailed balance, so the density at each node must remain constant at each step, as shown in Figure \ref{fig:DB4}(b); Figure \ref{fig:DB4}(a) fails
to satisfy this.

\subsubsection{Detailed Balance and the Metropolis-Hastings Algorithm}

Consider a generic Markov kernel $q(u,v)$ as shown in Figure \ref{fig:DB4}(a),
which does not satisfy detailed balance, which we accept-reject
according to the Metropolis algorithm.
Intuitively, this Metropolis-Hastings kernel, limits the ``flow'' between nodes such that detailed balance is satisfied with respect to the distribution $\pi$, as shown in Figure \ref{fig:DB4}(b). There the red-highlighted numbers represent where the transition probabilities of the kernel have been reduced. The following theorem establishes the detailed balance of the Metropolis-Hastings kernel with respect to the target $\pi$; it implies, as a consequence, that the Metropolis-Hastings kernel is invariant with respect to the target.

      \begin{theorem}[Metropolis-Hastings and Detailed Balance]
       The Metropolis-Hastings kernel satisfies detailed balance with
respect to the distribution $\pi$.
      \end{theorem}

      \begin{proof}
      We need to show that, for any $u,v \in \mathbbm{R}^d,$
      $$\pi(u) p_{\mbox {\tiny{\rm MH}}}(u,v) = \pi(v) p_{\mbox {\tiny{\rm MH}}}(u,v).$$
If $u=v$ the equality is trivial. Let us assume henceforth that  $u\neq v,$ so that by equation \eqref{eq:pmh} we have
$$p_{\mbox {\tiny{\rm MH}}} (u,v) = a(u,v) q(u,v).$$
We rewrite
\begin{align*}
	p_{\mbox {\tiny{\rm MH}}}(u, v) &= \text{min }\biggl( \frac{\pi(v) q(v,u)}{\pi(u) q(u,v)}  ,1   \biggr) ~ q(u, v)\\
	&= \frac{1}{\pi(u)} \times \text{min }\Bigl( \pi(u) q(u,v), \pi(v) q(v,u)\Bigr).
\end{align*}
Thus, invoking symmetry,
	$$ \pi(u) p_{\mbox {\tiny{\rm MH}}}(u, v) = \text{min }\bigl( \pi(u) q(u,v), \pi(v) q(v,u)\bigr) = \pi(v) p_{\mbox {\tiny{\rm MH}}}(v, u).$$
	 \end{proof}

The invariance of the Metropolis-Hastings $p_{\mbox {\tiny{\rm MH}}}$ with respect to $\pi$ implies that if the initial sample is drawn from the target ($\pi_0 = \pi)$, then all subsequent samples are also distributed according to the target $(\pi_n = \pi).$

    \begin{figure}
      \centering
      \includegraphics[width=0.85\columnwidth]{figures/lec7_DB4.png}
      \caption{Toy representation of MCMC with 3 states ($u, v, w$). The numbers associated with each term represent an example case. (a) The top chain represents a Markov chain with Markov kernel $q$ which does not satisfy detailed balance with respect to $\pi$.  (b) The bottom chain utilizes Markov kernel $p$, which satisfies detailed balance by modifying the transition probabilities of $q$.}
      \label{fig:DB4}
    \end{figure}

\subsection{Convergence to the Target Distribution}
In the previous section we showed that if we initialize the Metropolis-Hastings algorithm with distribution $\pi$, all samples produced by the algorithm will be distributed according to $\pi.$
But our original problem was exactly that we were not able to sample from $\pi$.
Our aim in this section is to show that, for certain Metropolis-Hastings methods, the law $\pi_n$ of the $n$-th sample converges to $\pi$ regardless of the initial distribution $\pi_0.$
This is a strong form of  {\em ergodic} behavior which does not hold in general, as illustrated by the chain depicted in Figure~\ref{fig-convergence}.

\begin{figure}
  \centering
  \includegraphics[scale=0.70]{figures/lec7_period.jpg}
  \caption{The arrows represent transitions with probability one in a four
state Markov chain. The invariant distribution is the uniform distribution but for $\pi_{0} = \delta_{A}$, we do not have ergodicity.}\label{fig-convergence}
\end{figure}

In order to understand the mechanisms behid ergodicity we will
first consider Markov chains with finite state space. We then study a specific Metropolis-Hastings
algorithm, known as the {\em preconditioned Crank-Nicolson  (pCN)} method,
which applies to targets $\pi$ defined by their density with respect to
a Gaussian distribution.

\subsubsection{Finite State Space}

We consider a Markov chain on the finite state space
$S = \left\{ 1 , \cdots , d \right\}.$
We illustrate a {\em coupling} approach to proving ergodicity and then,
in the next subsection, use it to study the pCN method on a continuous
state space.

\begin{theorem}[Ergodicity in Finite State Spaces]
Let $u_{n}$ be a Markov chain with state space $S$, transition kernel $p$,
and initial distribution $\pi_{0}$. Assume that
  \begin{equation}
    \label{eqn-unif}
    \varepsilon \coloneqq \min_{i,j \in S} p\left( i,j \right) > 0.
  \end{equation}
  Then, $p$ has a unique invariant distribution $\pi$, and the following convergence result holds:
  \begin{equation}\label{eq:tvbound}
    d_{\mbox {\tiny{\rm TV}}}( \pi_{n}, \pi) \leq \left( 1 - \varepsilon \right)^{n},
  \end{equation}
  where $\pi_n$ is the law of $u_n.$
\end{theorem}
\begin{proof}
  First note that the Markov kernel $p$ maps a probability distribution on $S$
into another probability distribution on $S$; it thus maps a compact, convex set into
itself. By Brouwer's fixed point theorem it follows that $p$ has a fixed point
in this space, ensuring that an invariant distribution exists. We will now show that for any invariant distribution $\pi$ equation \eqref{eq:tvbound} holds, which then implies the uniqueness of the invariant distribution.

Let $\pi$ be an invariant distribution, a probability vector on $S$. Proving
convergence to equilibrium amounts to ``forgetting the past'', to show that the long time behavior of the Markov chain does not depend on the initial distribution $\pi_0$ and in fact converges to $\pi$. In general, $u_{n+1}$ will be strongly dependent on $u_{n}$, but
the condition given in \eqref{eqn-unif} implies that there is always some residual chance that the chain jumps to any new state, at each step, independently
of where it is currently located, $u_{n}$. We will show that this residual
probability of the chain to make a ``totally random'' move will diminish the
stochastic dependence on $u_{0}$ as $n$ increases.

To formalize this idea, let $b_n$ be i.i.d. Bernoulli random variables with $\operatorname{\mathbbm{P}}\left( b_n = 1 \right) = \varepsilon$ and $\operatorname{\mathbbm{P}}\left( b_n = 0 \right) = 1-\varepsilon;$ furthermore assume that the sequence $\{b_n\}$ is independent of the
randomness defining draws from $\{p(u_n,\cdot)\}$.
Because of the lower bound on $p$ we may define a new Markov chain as follows:
  \begin{equation} \label{eq:kernel}
    w_{n + 1} \sim
    \begin{cases}
      &s\left(w_{n}, \cdot \right), \hfill \text{ for } b_n = 0,\\
      &r\left(w_{n}, \cdot \right), \hfill \text{ for } b_n = 1,
    \end{cases}
  \end{equation}
  where
  \begin{equation*}
    s\left( i, j \right) := \frac{p\left(i, j \right) - \varepsilon r\left(i, j \right)}{1- \varepsilon}
  \end{equation*}
and  $r$ is the uniform transition kernel with equal probability of
transitioning to each state in $S: r(i,j)=d^{-1}$ for all $(i,j) \in S \times S.$
We may now compute
  \begin{align*}
    \operatorname{\mathbbm{P}}\left( w_{n + 1} = j | w_{n} = i \right) &= \varepsilon \operatorname{\mathbbm{P}}\left( w_{n + 1} = j | w_{n} = i, b_n = 1 \right) + \left( 1 - \varepsilon \right) \operatorname{\mathbbm{P}}\left( w_{n + 1} = j | w_{n} = i, b_n = 0 \right)\\
    &= \varepsilon r\left( i,j \right) + p\left( i,j \right) - \varepsilon r\left( i,j \right) \\
    &= p\left( i,j \right).
  \end{align*}
  Thus the kernel defined by \eqref{eq:kernel} is equivalent in law to $p$.
  However, by introducing the ancillary random variables $b_n$, we have
made explicit the concept of ``forgetting the past entirely, with a small probability'' at every step. We may now use this to complete the proof.
Let $f:S \mapsto \mathbbm{R}$ be an arbitrary test function with $|f|_\infty \leq 1$ and $\tau \coloneqq \min\left( n \in \mathbbm{N}: b_n = 1 \right).$
Then, regardless of how $w_n$ is initialized,
  \begin{align*}
    \operatorname{\mathbb{E}}\left[ f\left( w_{n} \right) \right]
    &= \operatorname{\mathbb{E}}\left[ f\left( w_{n} \right) | \tau \geq n \right]\operatorname{\mathbbm{P}}\left( \tau \geq n \right)  + \sum \limits_{l = 0 }^{n-1} \operatorname{\mathbb{E}}\left[f \left(w_{n}\right) | \tau = l  \right] \operatorname{\mathbbm{P}}\left( \tau =l \right)  \\
    &=\underbrace{\operatorname{\mathbb{E}}\left[ f\left( w_{n} \right) | \tau \geq n \right]\operatorname{\mathbbm{P}}\left( \tau \geq n \right)}_{|\cdot| \leq \left( 1-\varepsilon \right)^{n}}  + \underbrace{\sum \limits_{l = 0 }^{n-1} \operatorname{\mathbb{E}}_{u_{0} \sim \mathfrak{u}\left( \cdot \right)}\left[f\left(w_{n-l}\right) \right] \operatorname{\mathbbm{P}}\left( \tau =l \right)}_{\text{independent of original initial distribution}},
  \end{align*}
where $\mathfrak{u}$ denotes the uniform distribution on $S.$

Now consider two Markov chains $w_n$ and $w_n'$ with kernel \eqref{eq:kernel}, the first initialized from $\pi_0$ and the second from an invariant distribution $\pi.$ The law of $w_n$ agrees with the law $\pi_n$ of the original chain $u_n,$ while for the second chain it follows from invariance that $\pi_n' = \pi.$ We will use the variational characterization of the total variation distance established in Lemma \ref{lemmatv}. Employing the preceding identity and noting that the contribution which is independent of the initial distribution will cancel in the two different Markov chains, we obtain
  \begin{align*}
    d_{\mbox {\tiny{\rm TV}}}( \pi_{n}, \pi_n') = \frac{1}{2} \sup \limits_{|f|_\infty \leq 1} \left| \operatorname{\mathbb{E}}^{\pi_{n}}\left[ f\left( u \right) \right] - \operatorname{\mathbb{E}}^{\pi_n'}\left[ f\left( u \right) \right]\right| \leq \left( 1 - \varepsilon \right)^{n}.
  \end{align*}
Since  $\pi_n' = \pi$ the desired result follows.
\end{proof}

Before extending the above argument to a setting with continuous state
space, we make two remarks:
\begin{remark}
The coupling proof we have just exhibited may be generalized in a number of
ways; in particular:

  \begin{itemize}
    \item The distribution $r$ need not be uniform; it  was only chosen so
for convenience. What is important is that $r_{i,j}$ is lower bounded,
independently of $i$, for all $j$. Adapting $r$ to the $p$ at hand, might
in some cases greatly improve the above bound -- a larger $\epsilon$ might
be identified.
    \item Convergence to equilibrium can also be shown if condition~\eqref{eqn-unif} holds with $p$ replaced by the $n$-step transition kernel $p^{n}$.
Again, for some chains this may yield faster bounds on the convergence
to equilibrium.
  \end{itemize}
  \end{remark}

\subsubsection{The pCN Method}

The coupling argument used in the previous subsection for Markov chains with finite state space may also be employed to study ergodicity of Markov chains on a continuous state space.
To illustrate this we consider a particular Metropolis-Hastings algorithm,
the pCN method, applied to a specific Bayesian inverse problem.
Before we get into the details of the inverse problem we describe the
idea behind the pCN method at a high level.
The idea is this. If the desired target distribution has the form
\begin{equation}\label{eq:formoftarget}
\pi(u)= \frac{1}{Z}  \tilde{g}(u)  \tilde{\rho}(u)
\end{equation}
and if the Metropolis-Hastings proposal kernel $q$ satisfies detailed balance with respect to $\tilde{\rho}$, then \eqref{eq:MHA} simplifies to give
\begin{eqnarray}
\label{eq:MHA2}
a(u,v) = \text{min} \Bigg( \frac{\tilde{\rho}(v)\tilde{g}(v) q(v, u)}{\tilde{\rho}(u)\ \tilde{g}(u) q(u, v)}, 1\Bigg)= \text{min} \Bigg( \frac{  \tilde{g}(v)}{\tilde{g}(u)}, 1\Bigg).\end{eqnarray}
We will apply and study this idea in the case where $\tilde{\rho}$ is a Gaussian distribution, in which case it is straightforward to construct a proposal kernel that satisfies detailed balance with respect to $\tilde{\rho}$. This scenario arises naturally in Bayesian inverse problems where the the prior is defined in terms of a Gaussian. We now formalize the inverse problem setting that we consider by imposing certain assumptions on the likelihood and the prior, and then relate both to the functions $\tilde{g}$ and $\tilde{\rho}$ in Equation \ref{eq:formoftarget}.

\begin{assumption}
\label{a:erg}
We make the following assumptions on the Bayesian inverse problem:
\begin{itemize}
    \item Bounded likelihood: there are $g^-, g^+ >0$ such that, for all $u \in \mathbbm{R}^\Du,$
    $$0 <g^- <g(u) < g^+.$$
    \item Truncated Gaussian prior: there is a compact set $B \subset \mathbbm{R}^\Du$ with positive Lebesgue measure such that   $\rho(u) =\mathbb{1}_{B}(u)   z(u),$ where $z=\mathcal{N}\left( 0,\hat{C} \right)$.
\end{itemize}
\end{assumption}

Under Assumption \ref{a:erg} we obtain for the posterior density
\begin{equation*}
 \pi\left( u \right)  \propto g(u) \mathbb{1}_{B}\left( u \right) z(u),
\end{equation*}
which is of the form in Equation \eqref{eq:formoftarget} with $\tilde{g}(u) = g(u) \mathbb{1}_B(u)$ and $\tilde{\rho}(u) = z(u).$

The pCN method has proposal kernel which is invariant with respect to
the Gaussian $z =\mathcal{N}\left( 0,\hat{C} \right)$ defined by
\begin{equation}\label{eq:pcnp}
q(u,\cdot) \sim \mathcal{N}\Bigl( (1-\beta^2)^{1/2} u, \beta^2 \hat{C}  \Bigr).
\end{equation}
Thus, given the sample $u^{(n)},$ the pCN proposes a new sample
\begin{equation*}
  v^\star \sim \left( 1 - \beta^{2} \right)^{1/2}u^{(n)} + \beta \xi^{(n)},
\quad \xi^{(n)}\sim \mathcal{N}\left( 0, \hat{C} \right),
\end{equation*}
which only requires to sample a Gaussian.

\begin{theorem}[Ergodicity for pCN Method]
\label{t:pCN}
Assume that we apply the pCN method to sample from a
posterior density $\pi$ arising from Assumptions \ref{a:erg} with initial
condition drawn from any density supported on $B$. Then
there exists a constant $\varepsilon \in \left( 0,1 \right)$ such that
  \begin{equation*}
    d_{\mbox {\tiny{\rm TV}}} ( \pi_{n} , \pi) \leq \left( 1 - \varepsilon \right)^{n},
  \end{equation*}
where $\pi_{n}$ is the law of the $n$-th sample from the pCN Metropolis-Hastings algorithm.
\end{theorem}
Recall the notation for the covariance weighted inner-product and
resulting norm described in the introduction to these notes.
Before proving the above theorem, we will prove an auxiliary lemma.

\begin{lemma}
Let the assumptions of Theorem \ref{t:pCN} hold.  The proposal kernel \eqref{eq:pcnp} satisfies detailed balance with respect to the Gaussian $z =\mathcal{N}\left( 0,\hat{C} \right)$  and we have the following expression for the pCN accept-reject probability:
  \begin{align*}
    a\left( u,v \right)
         =\min\left( \frac{g(v)}{g(u)}  \mathbb{1}_{B} \left( v \right) ,1  \right).
  \end{align*}
\end{lemma}
\begin{proof}
We need to show that $z(v) q(v,u)$ is symmetric in $u$ and $v.$ By direct calculation,
\begin{align*}
-\log\bigl( z(v) q(v,u) \bigr)  &=\frac{1}{2}|v|^{2}_{\hat{C}} + \frac{1}{2 \beta^{2}} \left| u - \left( 1 - \beta^{2} \right)^{1/2} v\right|_{\hat{C}}^{2}\\
  &=\left( \frac{1}{2} + \frac{\left( 1 - \beta^{2} \right)}{2 \beta^{2}}  \right) |v|^{2}_{\hat{C}} + \frac{1}{2\beta^{2}} |u|^{2}_{\hat{C}} - \frac{\left( 1 - \beta^{2} \right)^{1/2}}{\beta^{2}} \langle u , v \rangle_{\hat{C}}\\
  &= \frac{1}{2 \beta^{2}}\left( |v|^{2}_{\hat{C}} + |u|^{2}_{\hat{C}} \right) - \frac{\left( 1 - \beta^{2} \right)^{1/2}}{\beta^{2}} \langle u , v \rangle_{\hat{C}}.
\end{align*}
For the expression for the acceptance probability note that since $u_{0} \in B$, we have $u_{n} \in B$ for all $n$ as any proposed move out of $B$ will be rejected. Thus $\mathbb{1}_{B}\left( u \right)$ may be dropped from the formula for the acceptance probability.
\end{proof}

Using this lemma, we can prove ergodicity much as we did in the previous
subsection in the finite state space setting.
The main idea is that, restricted to the bounded set $B$, the probability density of the transition kernel will be bounded away from zero by some $\varepsilon$.
Splitting off a ``forgetful part'' that is triggered with probability $\varepsilon$ will then yield the result.

\begin{proof}[Proof of Theorem \ref{t:pCN}]
Note again that since $u_0 \in B$ we have $u_n \in B$ for all $n \ge 1.$
Note further that since $B$ is compact and $q$ is continuous in both of its arguments, there is $q^- >0$ such that, for any $u, v \in B,$
$$q(u,v) \ge q^-.$$
Let $p$ be the Markov kernel defined by the pCN Metropolis-Hastings algorithm. It follows that, for $u, v \in B,$
\begin{align*}
p(u,v) & \ge q(u,v) a(u,v) \\
& \ge q^- \frac{g^-}{g^+} =: \varepsilon \text{Leb} (B),
\end{align*}
where the last equation defines $\varepsilon$ and $\text{Leb}(B)$ denotes the Lebesgue measure of $B$ (which is assumed to be positive).
Analogously to the discrete proof, we now define $b_n$ to be i.i.d.
Bernoulli random variables with $\operatorname{\mathbbm{P}}\left( b_n = 1 \right) = \varepsilon$, independently of all other randomness, and consider the transition rule
  \begin{equation*}
    u_{n +1} \sim
    \begin{cases}
      &s\left( u_{n} , \cdot \right), \hfill \text{ for } b_n = 0,\\
      &r\left( \cdot \right), \hfill \text{ for } b_n = 1,
    \end{cases}
  \end{equation*}
  where $r$ denotes the uniform distribution on $B$ and, for $A\subset B$ and $u \in B,$
  \begin{equation*}
    s\left( u, A \right) \coloneqq \frac{p\left(u, A \right) - \varepsilon r\left( A \right)}{1- \varepsilon}.
  \end{equation*}
Just as in the discrete case, one can check that the resulting Markov kernel is equal to the pCN Metropolis-Hastings kernel $p(\cdot,\cdot)$. Exponential convergence is then concluded in exactly the
same way as in the discrete case.
\end{proof}

\subsection{Discussion and Bibliography}

The book \cite{gamerman2006markov} is a useful
basic introduction to  MCMC and the book
\cite{brooks2011handbook} presents state of the art as of 2010.
The paper \cite{cotter2013} overviews the pCN method
and related MCMC algorithms specifically designed for inverse
problems and other sampling problems in high dimensional
state spaces.
The book \cite{lindvall2002chapters} describes the coupling method
in a general setting.
The book \cite{meyn2012markov} contains a wide-ranging presentation
of Markov chains, and their long-time behavior, including ergodicity,
and coupling. Furthermore the book describes the general methodology
for going from convergence of expectations in (possibly weighted)
total variation metrics to sample path ergodicity and almost sure convergence
of time averages, a topic we did not cover in this chapter.
The paper \cite{mattingly2002ergodicity} describes the
coupling methodology in the context of stochastic differential
equations and their approximations.

\section{\Large{\sffamily{The Linear-Gaussian Setting }}}
\label{sec: posteriordistribution}

Recall the inverse problem of estimating an unknown parameter $\vct{u} \in \mathbbm{R}^\Du$ from data $y\in \mathbbm{R}^\Dy$ under the model assumption
\begin{equation}\label{eq:yurelation}
y = G(u) + \eta.
\end{equation}
In this chapter we study the linear-Gaussian setting, where the forward map $G(\cdot)$ is linear and both the prior on $u$ and the distribution of the observation noise $\eta$ are Gaussian. This setting is highly amenable to analysis and arises frequently in applications. Moreover, as we will see throughout this book, many methods employed in nonlinear or non-Gaussian settings build on ideas from the linear-Gaussian case by performing linearization or invoking Gaussian approximations.
Having established a formula for the posterior, we investigate the effect that the choice of prior has on our solution. We do this by quantifying the spread of the posterior distribution in the small noise (approaching zero) limit. This provides intuitive understanding concerning the impact of the prior for overdetermined, determined, and underdetermined regimes, corresponding to $d <k,d =k,$ and $d>k,$ respectively.

The following will be assumed throughout this chapter.
\begin{assumption}
The relationship between unknown $u\in \mathbbm{R}^\Du$ and data $y\in \mathbbm{R}^\Dy$ defined by equation \eqref{eq:yurelation} holds. Moreover,
\label{a:sz1}
\begin{description}
	\item[1] Linearity of the forward model: $G(\vct{u})=\mtx{A}\vct{u}$, for some $\mtx{A}\in\mathbbm{R}^{k \times d}$;
    \item[2] Gaussian prior: $\vct{u}\sim \rho(\vct{u})=\mathcal{N}(\hat{m},\hat{C})$, where $\hat{C}$ is positive definite;
    \item[3] Gaussian noise: $\vct{\eta}\sim \nu(\vct{\eta})=\mathcal{N}(\zerovct,\mtx{\Gamma})$, where $\mtx{\Gamma}$ is positive definite;
    \item[4] $u$ and $\eta$ are independent.
\end{description}
\end{assumption}

\subsection{Derivation of the Posterior Distribution}

Under Assumption  \ref{a:sz1} the likelihood on $\vct{y}$ given $\vct{u}$ is a Gaussian,
\begin{align}
	\vct{y}|\vct{u}\sim \mathcal{N}(\mtx{A}\vct{u},\mtx{\Gamma}).
\end{align}
Therefore, using Bayes formula \eqref{eq:bayesformula} we see that the posterior $\pi^y(u)$ is given by
\begin{align*}
	\pi^{\vct{y}}(\vct{u})&=\frac{1}{Z}\nu(\vct{y}-\mtx{A}\vct{u})\rho(\vct{u})\\
	&=\frac{1}{Z}\exp\left(-\frac{1}{2}|\vct{y}-\mtx{A}\vct{u}|_{\mtx{\Gamma}}^2\right)\exp\left(-\frac{1}{2}|\vct{u}-\hat{m} |_{\hat{C}}^2\right)\\
	&=\frac{1}{Z}\exp\left(-\frac{1}{2}|\vct{y}-\mtx{A}\vct{u}|_{\mtx{\Gamma}}^2-\frac{1}{2}|\vct{u}-\hat{m}|_{\hat{C}}^2\right)\\
	&=\frac{1}{Z}\exp\bigl(-{\mathsf{J}}(\vct{u})\bigr),
\end{align*}
with
\begin{equation}
\label{eq:ju}
{\mathsf{J}}(\vct{u})=\frac{1}{2}|\vct{y}-\mtx{A}\vct{u}|_{\mtx{\Gamma}}^2+\frac{1}{2}|\vct{u} - \hat{m}|_{\hat{C}}^2.
\end{equation}
Since the posterior distribution can be written as the exponential of a quadratic in $u$ it follows that the posterior is Gaussian. Its mean and covariance are given in the following result.

\begin{theorem}[Posterior is Gaussian]
\label{t:sz1}
   	Under Assumption \ref{a:sz1}
the posterior distribution is  Gaussian,
   		\begin{align}
			\vct{u}|\vct{y} \sim \pi^{\vct{y}}(\vct{u}) =\mathcal{N}(m,C).\label{eq: posterior}
		\end{align}
The posterior mean $m$ and covariance $C$ are given by the
following formulae:
		\begin{align}
			m&=(\mtx{A}^{T}\mtx{\Gamma}^{-1}\mtx{A}+\hat{C}^{-1})^{-1}(\mtx{A}^{T}\mtx{\Gamma}^{-1}\vct{y} + \hat{C}^{-1}\hat{m}),\label{eq: postmean} \\
			C&=(\mtx{A}^{T}\mtx{\Gamma}^{-1}\mtx{A}+\hat{C}^{-1})^{-1}.
			\label{eq: postvar}
		\end{align}

	\label{thm: posteriorgaussian}
\end{theorem}
\begin{proof}
	Since $\pi^{\vct{y}}(\vct{u})=\frac{1}{Z}\exp\bigl(-{\mathsf{J}}(\vct{u})\bigr)$
with ${\mathsf{J}}(u)$ given by \eqref{eq:ju},
a quadratic function of $\vct{u}$, it follows that
the posterior distribution $\pi^{\vct{y}}(\vct{u})$ is Gaussian.
Denoting the mean and variance of $\pi^{\vct{y}}(\vct{u})$ by
$m$ and $C$, we can write ${\mathsf{J}}(\vct{u})$ in the following form
	\begin{align}
		{\mathsf{J}}(\vct{u})&=\frac{1}{2}|\vct{u}-m|_{C}^2+ q ,
		\label{eq: ju2}
	\end{align}
where the term $q$ does not depend on $\vct{u}$.
Now matching the coefficients of the quadratic and linear terms
in equations \eqref{eq:ju} and \eqref{eq: ju2}, we get
	\begin{align*}
		C^{-1}&=\mtx{A}^{T}\mtx{\Gamma}^{-1}\mtx{A}+\hat{C}^{-1}, \\
		C^{-1}m&=\mtx{A}^{T}\mtx{\Gamma}^{-1}\vct{y} + \hat{C}^{-1}\hat{m}.
	\end{align*}
Therefore equations \eqref{eq: postmean} and \eqref{eq: postvar} follow.
\end{proof}

Equation \eqref{eq: ju2} shows that the posterior mean $m$ minimizes ${\mathsf{J}}(u)$ given by \eqref{eq:ju}. This demonstrates that the posterior mean is found by compromising between maximizing the likelihood (by making small the \emph{loss} term $\frac{1}{2}|\vct{y}-\mtx{A}\vct{u}|_{\mtx{\Gamma}}^2$) and minimizing deviations from the prior mean (by making small the \emph{regularization} term $\frac{1}{2}|\vct{u} -\hat{m}|_{\hat{C}}^2$). The relative importance given to both objectives is determined by the relative size of the prior covariance $\hat{C}$ and the noise covariance $\Gamma.$
An important feature of the linear-Gaussian setting is that the posterior covariance $C$ does not depend on the data $y;$ this is not true in general.

As we saw in the previous chapter, the posterior mean estimator and the MAP estimator are typically different. However, in the linear-Gaussian setting the posterior is Gaussian, and therefore both estimators agree.

\begin{corollary}[Characterization of  Bayes Estimators]
The posterior mean and MAP estimators under Assumptions \ref{a:sz1} agree, and are given by $u_{\mbox {\tiny{\rm MAP}}} = u_{\mbox {\tiny{\rm PM}}} = m$ defined in equation \eqref{eq: postmean}.
\end{corollary}

\begin{example}	\label{exp: map}
	Let $\mtx{\Gamma}=\gamma^2\Id$, $\hat{C}=\sigma^2\Id,$ $\hat{m} =0,$ and
set $\lambda=\frac{\sigma^2}{\gamma^2}$. Then
	$${\mathsf{J}}_\lambda(\vct{u})\triangleq\gamma^2{\mathsf{J}}(\vct{u})=\frac{1}{2}|\vct{y}-\mtx{A}\vct{u}|^2+\frac{\lambda}{2}|\vct{u}|^2.$$
Since $m$ minimizes ${\mathsf{J}}_\lambda(\cdot)$ it follows that
	\begin{align}
		(\mtx{A}^{T}\mtx{A}+\lambda\Id)m=\mtx{A}^{T}\vct{y}.
		\label{eq: expmean}
	\end{align}
\end{example}

Example \ref{exp: map} provides a link between Bayesian inversion and
optimization approaches to inversion: ${\mathsf{J}}_\lambda(\vct{u})$ can be seen as
the objective functional in a linear regression model with a regularizer
$\frac{\lambda}{2}|\vct{u}|^2$, as used in ridge regression. Equation \eqref{eq: expmean} for $m$ is exactly the normal equation
with regularizer in the least square problem.
In fact, in the general case, equation \eqref{eq: postmean} can be also viewed as a generalized normal equation. This point of view helps us
understand the structure of Bayesian regularization by linking it to the deep understanding of optimization approaches to inverse problems. A more extensive account of the optimization perspective and its interplay with Bayesian formulations will be given in the following chapter.

 \subsection{Small Noise Limit of the Posterior Distribution}
 In this section we study the small observational noise limit of the posterior distribution in the linear-Gaussian setting. While most of the ideas can be extended beyond this setting, the explicit calculations that the linear-Gaussian setting allows for provide helpful intuition. Throughout this section we assume the following.

 \begin{assumption}
\label{a:sz2}
In addition to the linear-Gaussian setting described in Assumption \ref{a:sz1},
$\vct{\eta}:=\gamma \vct{\eta}_0$ where $\vct{\eta}_0 \sim \mathcal{N}(\zerovct,\mtx{\Gamma}_0)$ so that $\Gamma=\gamma^2\Gamma_0.$
\end{assumption}

Note that substituting $\Gamma = \gamma^2 \Gamma_0$  into \eqref{eq: postmean} and \eqref{eq: postvar} we obtain that
 \begin{align}
m =& (A^{T} \Gamma_0^{-1} A + \gamma^2 \hat{C}^{-1})^{-1} (A^{T}\Gamma_0^{-1}y +\gamma^2 \hat{C}^{-1} \hat{m}), \label{eq:meansmallnoise}\\
C =&  \gamma^2 (A^{T} \Gamma_0^{-1} A + \gamma^2 \hat{C}^{-1})^{-1}. \label{eq:covsmallnoise}
 \end{align}

In the next three subsections we study the small noise limiting behavior as $\gamma \to 0$ of the posterior mean and covariance in the overdetermined, determined, and underdetermined regimes.  We recall that $\Rightarrow$ denotes weak convergence of probability measures. We will use repeatedly that weak convergence of Gaussian distributions is equivalent to the convergence of their means and covariances. In particular, the weak limit of a sequence of Gaussians with means converging to $m^+$ and covariance matrices converging to zero is a Dirac mass $\delta_{m^+}.$

 \subsubsection{Overdetermined Case}
 We start with the overdetermined case $d<k$.
\begin{theorem}[Small Noise Limit of Posterior Distribution - Overdetermined] \label{lec2:thm1}
Suppose that Assumption \ref{a:sz2} holds, that ${\rm Null}(A)= {0}$ and that $d<k.$
Then in the limit $\gamma^2 \to 0,$
$$\pi^y \Rightarrow \delta_{m^+},$$
where $m^+$ is the solution of
the least-squares problem
\begin{equation}\label{lec1:thm1_1}
m^+  =\arg\min_{u \in \mathbbm{R}^\Du} |\Gamma_0^{-1/2} (y- Au)   |^2.
\end{equation}
\end{theorem}

\begin{proof}

Since Null$(A)= {0}$ and $\Gamma_0$ is invertible
we deduce that there is $\alpha>0$ such that, for all $u \in \mathbbm{R}^\Du,$
 \[
  \langle u, A^{T} \Gamma^{-1}_0 Au \rangle = | \Gamma_0^{-1/2} Au |^2 \geq \alpha |u|^2.
 \]
 Thus $A^{T} \Gamma_0^{-1} A$ is positive definite and invertible. It follows that as $\gamma \to 0$, the posterior covariance converges to the zero matrix, $C \to 0,$ and the posterior mean satisfies the limit
 \[
 m \to  m^{*} =   (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1} y.
 \]
This proves the weak convergence of $\pi^y$ to $\delta_{m^{*}}$. It remains to characterize $m^{*}$.  Since Null$(A)= {0}$, the minimizers of
 \[
 \mathsf{L}(u) :=\frac{1}{2} | \Gamma_0^{-1/2}(y - Au)|^2
  \]

  are unique and satisfy the normal equations $A^{T}\Gamma_0^{-1}Au = A^{T} \Gamma_0^{-1}y$. Hence $m^{*}$ solves the desired least-squares problem and coincides with $m^+$ given in \eqref{lec1:thm1_1}.
 \end{proof}

\begin{remark}
In the overdetermined case where $A^{T}\Gamma_0 A$ is invertible, the small observational noise limit leads to a posterior which is a Dirac, centered at the solution of the least-square problem \eqref{lec1:thm1_1}.   Therefore, in this limit the prior plays no role in the Bayesian inference.
\end{remark}

\begin{theorem}[Posterior Consistency]
Suppose that the assumptions of Theorem \ref{lec1:thm2_1} hold and that the data satisfies
\begin{equation} \label{eq:datapostcons}
y=Au^\dagger+\gamma \eta_0^\dagger, \quad\quad {\rm for \,\, fixed} \quad u^\dagger \in \mathbbm{R}^\Du, \eta_0^\dagger \in \mathbbm{R}^\Dy.
\end{equation}
Then, for any sequence $M(\gamma)\rightarrow\infty$ as $\gamma\rightarrow0$,
	\begin{align}
		\operatorname{\mathbbm{P}} ^{\pi^y} \bigl\{|\vct{u}-\vct{u}^\dagger|^2>M(\gamma)\gamma^2\bigr\}\rightarrow0,
		\label{eq: maincovergence}
	\end{align}
	where $\operatorname{\mathbbm{P}}^{\pi^y}$ denotes probability under the posterior distribution.
	\end{theorem}

\begin{remark}
	For any $\varepsilon>0$, set $M(\gamma)=\frac{\varepsilon^2}{\gamma^2}$ in Theorem \ref{thm: maintheorem} to obtain
	$$\operatorname{\mathbbm{P}}^{\pi_y} \bigl\{|\vct{u}-\vct{u}^\dagger|>\varepsilon \bigr\}\rightarrow0.$$
\end{remark}

\begin{proof}(Theorem \ref{lec2:thm1})
Throughout this proof we let $c$ be a constant independent of $\gamma$ that may change from line to line, and we denote by $\operatorname{\mathbb{E}}$ expectation with respect to the posterior distribution, which is Gaussian with mean $m$ and covariance $C$ given by equations \eqref{eq:meansmallnoise} and \eqref{eq:covsmallnoise}. Denote
$$m^{*} =   (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1} y$$
as in the proof of the previous theorem.
We have that
\begin{equation}
\operatorname{\mathbb{E}}|u-u^\dagger|^2 \le c\Bigl( \operatorname{\mathbb{E}}|u-m|^2 + |m - m^{*}|^2 + |m^{*}-u^\dagger|^2  \Bigr).
\end{equation}
We now bound each of the three terms in the right-hand side.

For the first one,
\begin{align*}
\operatorname{\mathbb{E}}|u-m|^2 &= \operatorname{\mathbb{E}}\bigl[(u-m)^{T} (u-m)  \bigr]  = \operatorname{\mathbb{E}}\bigl[ {\rm Tr}[(u-m)\otimes(u-m)]   \bigr] \\
 &={\rm Tr} \operatorname{\mathbb{E}}\bigl[ (u-m)\otimes(u-m)   \bigr]  \\
&=  {\rm Tr}(C) \le \gamma^2 {\rm Tr}(A^{T} \Gamma_0^{-1} A).
\end{align*}
For the second term, note that
\begin{align*}
(A^{T} \Gamma_0^{-1} A) m^{*} &=  A^{T}\Gamma_0^{-1}y, \\
(A^{T} \Gamma_0^{-1} A + \gamma^2 \hat{C}^{-1}) m &= A^{T}\Gamma_0^{-1}y +\gamma^2 \hat{C}^{-1} \hat{m}.
\end{align*}
Therefore
\[ m-m^{*} = \gamma^2 (A^{T}\Gamma_0 A)^{-1} (\hat{C}^{-1} \hat{m}   -\hat{C}^{-1}m).\]
Since $m$ converges it is bounded, and so there is $c>0$ such that
\[ |m-m^{*}|^2 \le c \gamma^4. \]
Finally, for the third term we write
\begin{align*}
m^{*} &=   (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1} Au^\dagger  + \gamma (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1}  \eta_0^{\dagger} \\
 &= u^\dagger  + \gamma (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1}  \eta_0^{\dagger},
\end{align*}
which gives
\[ |m^{*} - u^\dagger|^2 \le c \gamma^2. \]
Using Markov's inequality and the three bounds above,
 $$\operatorname{\mathbbm{P}}^{\pi^y} \bigl\{|\vct{u}-\vct{u}^\dagger|^2>M(\gamma)\gamma^2 \bigr\}\leq\frac{\operatorname{\mathbb{E}}(|\vct{u}-\vct{u}^\dagger|^2)}{M(\gamma)\gamma^2}\leq\frac{c}{M(\gamma)}\rightarrow0,~{\rm as}~\gamma\rightarrow0.$$
\end{proof}

\subsubsection{Determined Case}
As a byproduct of the proof of Theorem \ref{lec2:thm1}, we can determine the limiting behavior of $\pi^y$ in the boundary case $d = k$.
\begin{theorem}[Small Noise Limit of Posterior Distribution - Determined]\label{lec1:thm1.5}
Suppose that Assumption \ref{a:sz2} holds, ${\rm Null}(A)= 0,$ and $d=k$. Then in the small noise limit $\gamma^2 \to 0$, $$\pi^y \Rightarrow \delta_{A^{-1 } y}.$$
\end{theorem}
\begin{proof}
In the proof of Theorem \ref{lec2:thm1}, the assumption $d < k$ is used only in that $A$ is not a square matrix and thus $A, A^{T}$ are not invertible. Denote by $(m, C)$ the mean and variance of the posterior $u|y$. Using the same argument, we have $C \to 0$ and
\[
m \to m^{*} =  (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1} y.
\]
Using that $A, A^{T}$ are square invertible matrices we obtain
\[
m^{*} = (A^{-1} \Gamma_0(A^{T})^{-1} ) A^{T} \Gamma_0^{-1 } y = A^{-1 }y.
\]
Therefore, $\pi^y(u) \Rightarrow \delta_{m^{*}} = \delta_{A^{-1}y}$.
\end{proof}

Note that here, as in the overdetermined case, the prior plays no role in the small noise limit. Moreover, it can be shown as above that posterior consistency holds. The proof is identical and therefore omitted.

\begin{theorem}[Posterior Consistency]
	Suppose that the assumptions of Theorem \ref{lec1:thm1.5} hold, and that the data satisfies
\begin{equation} \label{eq:datapostcons}
y=Au^\dagger+\gamma \eta_0^\dagger, \quad\quad {\rm for \,\, fixed} \quad u^\dagger, \eta_0^\dagger \in \mathbbm{R}^\Du.
\end{equation}
	Then for any sequence $M(\gamma)\rightarrow\infty$ as $\gamma\rightarrow0$,
	\begin{align}
		\operatorname{\mathbbm{P}} ^{\pi^y} \bigl\{|\vct{u}-\vct{u}^\dagger|^2>M(\gamma)\gamma^2 \bigr\}\rightarrow0.
		\label{eq: maincovergence}
	\end{align}
	\label{thm: maintheorem}
\end{theorem}

 \subsubsection{Underdetermined Case}
Finally we consider the underdetermined case $d > k$. We assume that $A \in \mathbbm{R}^{k \times d}$ with ${\rm Rank}(A) = k$ and write
 \begin{equation}\label{lec1:thm2_0}
 A = (A_0\  0) Q^{T} =(A_0 \ 0 )(Q_1\  Q_2)^{T}= A_0Q_1^{T},
 \end{equation}
 with $A_0 \in \mathbbm{R}^{k \times k} $ an invertible matrix, $Q = (Q_1 \ Q_2) \in \mathbbm{R}^{d \times d}$ an orthogonal matrix so that $Q^{T} Q= I$, $Q_1 \in \mathbbm{R}^{d \times k}, Q_2 \in \mathbbm{R}^{d \times (d-k)}$. We have the following result:
  \begin{theorem}[Small Noise Limit of Posterior Distribution - Underdetermined]\label{lec1:thm2}
Suppose that Assumption \ref{a:sz2} holds, that ${\rm Rank}(A) = k,$ and $d>k$. In the small noise limit $\gamma^2\to 0$,
$$\pi^y \Rightarrow \mathcal{N}(m^+, C^+),$$ where
\[
\begin{aligned}
m^+ &=\hat{C} Q_1 (Q_1^{T} \hat{C} Q_1)^{-1}  A_0^{-1}y + Q_2(Q_2^{T}\hat{C}^{-1}Q_2)^{-1}Q_2^{T} \hat{C}^{-1}\hat{m}, \\
C^+ &=Q_2(Q_2^{T}\hat{C}^{-1}Q_2)^{-1} Q_2^{T}. \\
\end{aligned}
\]
  \end{theorem}

Since ${\rm Rank}(C^+) = {\rm Rank}(Q_2) =d-k<d$ this theorem
demonstrates that, in the small observational noise limit, the posterior
retains uncertainty in a subspace of dimension $d-k$,
and has no uncertainty in a subspace of dimension $k$. As a consequence there is no posterior consistency in the underdetermined case.

\begin{example}  To help understand the result in Theorem \ref{lec1:thm2},
we consider a simple explicit example. Assume that $A = (A_0 \ 0) \in \mathbbm{R}^{k \times d}, \Gamma = \gamma^2 \Gamma_0 =\gamma^2  I_k, \hat{C} =I_d$, $\hat{m} = 0$. Let $u = \left(
\begin{array}{c}
u_1 \\
u_2 \\
\end{array}
\right) \sim \mathcal{N}(0, I_d), u_1 \in \mathbbm{R}^\Dy, u_2 \in \mathbbm{R}^{d - k}$.
The data then satisfies
\[
y = Au + \eta = A_0 u_1 +\eta, \  \eta \sim \mathcal{N}(0, \gamma^2 I_k).
\]
The posterior $u| y$ is $\pi_{\gamma}^y(u) = \frac{1}{Z_{\gamma}} \exp(-  {\mathsf{J}}_{\gamma}(u) )$, where
\begin{align}
 {\mathsf{J}}_{\gamma}(u) &= \frac{ 1 }{2\gamma^2} | y - A_0 u_1 |^2 + \frac{1}{2} | u |^2  \notag \\
 &=\left(  \frac{ 1 }{2\gamma^2} | y - A_0 u_1 |^2 + \frac{1}{2} | u_1 |^2\right)
 +\frac{1}{2} |u_2|^2.  \label{lec1:thm2_1}
\end{align}
It is clear that
\[
\pi_{\gamma}^{y}(u_1) \Rightarrow \delta_{A_0^{-1} y} (u_1).
\]
Once $u_1$ is fixed as $A_0^{-1}y$, the first term in \eqref{lec1:thm2_1} is a constant $\frac{1}{2} | A_0^{-1}y |^2$. Since $u_1$ and $u_2$ are independent we can derive, formally, the limiting posterior as follows
\[
 \pi_{\gamma}^y(u) \Rightarrow \delta_{A_0^{-1} y} (u_1) \otimes \frac{1}{Z} \exp(- \frac{1}{2} |u_2|^2) = \delta_{A_0^{-1} y} (u_1) \otimes \mathcal{N}(0, I_{d-k}),
\]
where $Z = \int_{\mathbbm{R}^{d-k}} \exp(-\frac{1}{2}|u_2|^2 ) du_2$. In fact, this is exactly the limiting posterior measure given in Theorem \ref{lec1:thm2}.
\end{example}

To prove Theorem \ref{lec1:thm2}, we use the following decomposition of the identity $I_d.$
\begin{lemma}
Let $\hat{C} \in \mathbbm{R}^{d \times d}$ be invertible and $Q = (Q_1 \ Q_2)$ be an orthonormal matrix with $Q_1 \in \mathbbm{R}^{d \times k}, Q_2 \in \mathbbm{R}^{d \times (d-k)}$. We have the following decomposition of $I_d$
\begin{equation}\label{lec1:lem}
I_d =  \hat{C} Q_1 (Q_1^{T} \hat{C} Q_1)^{-1}   Q_1^{T} +Q_2(Q_2^{T}\hat{C}^{-1}Q_2)^{-1} Q_2^{T}\hat{C}^{-1}.
\end{equation}
\end{lemma}
\begin{proof}
Denote by $R$ the right-hand side of \eqref{lec1:lem}. Since $Q$ is orthonormal, we have $Q_1^{T} Q_2 = 0, Q_2^{T}Q_1 = 0$ and thus
\[
Q_1^{T}(R- I) = 0, \quad Q^{T}_2\hat{C}^{-1}  (R-I) = 0 .
\]
If $B :=(Q_1 \ \hat{C}^{-1} Q_2)$ is full rank, the above identities imply that $B^{T}(R-I) = 0$ and thus $R= I$. Note that
\[
Q^{T} B = \left(
\begin{array}{c}
Q_1^{T} \\
Q_2^{T} \\
\end{array}
\right)
( Q_1 \ \hat{C}^{-1} Q_2)  =
 \left(
\begin{array}{cc}
I_k  & Q_1^{T}\hat{C}^{-1} Q_2 \\
0 &  Q_2^{T}\hat{C}^{-1} Q_2 \\
\end{array}
\right).
\]
Since the last matrix is invertible, $B$ is invertible and the proof is complete.
\end{proof}

\begin{proof}[Proof of Theorem \ref{lec1:thm2}]
Using \eqref{lec1:lem}, we can decompose $u$ as follows
\[
\begin{aligned}
u &=\underbrace{  \hat{C} Q_1 (Q_1^{T} \hat{C} Q_1)^{-1} }_{S} \underbrace{Q_1^{T} u}_{u_1} + \underbrace{Q_2(Q_2^{T}\hat{C}^{-1}Q_2)^{-1}}_{T} \underbrace{Q_2^{T}\hat{C}^{-1}u}_{u_2}  = Su_1 + Tu_2 . \\
\end{aligned}
\]
Here $u_1$ and $u_2$ are Gaussian with $u_2 \sim \mathcal{N}(Q_2^{T}\hat{C}^{-1} \hat{m} , Q_2^{T}\hat{C}^{-1} Q_2  )$. The identity
\[
{\rm Cov}(u_1,u_2) = Q_1^{T} {\rm Cov}(u,u) \hat{C}^{-1} Q_2 = Q_1^{T} Q_2 = 0
\]
shows that $u_1$ and $u_2$ are independent, written $u_1 \perp u_2$. From \eqref{lec1:thm2_0}, we have
\begin{equation}\label{lec1:thm2_2}
y = Au + \eta  = A_0 Q_1^{T} u+ \eta= A_0 u_1 + \eta.
\end{equation}
Since $u\perp \eta$ and $u_1 \perp u_2$, we have that $u_2 \perp y, u_1$. We apply conditional probability to yield
\[
\pi^y(u_1, u_2)  :=\operatorname{\mathbbm{P}}(u_1, u_2 | y) =\operatorname{\mathbbm{P}}(u_2) \operatorname{\mathbbm{P}}(u_1 | y)  .
\]
Equation \eqref{lec1:thm2_2} suggests that $(y, u_1,\eta)$ with $A_0 \in \mathbbm{R}^{k \times k}$ invertible is Gaussian distributed and its posterior is exactly $\operatorname{\mathbbm{P}}(u_1 | y)$. Theorem \ref{lec1:thm1.5} shows that $\operatorname{\mathbbm{P}}(u_1 | y ) \Rightarrow \delta_{A_0^{-1}y} (u_1)$ as the noise vanishes, that is, as  $\gamma^2 \to 0$. Note that $u_2 \perp u_1$ and $u_2 \perp y$. The limiting posterior measure  $(u_1,u_2) | y$ is
\begin{equation}\label{lec1:thm2_3}
\pi^y(u_1, u_2) \Rightarrow \operatorname{\mathbbm{P}}(u_2) \otimes \delta_{A_0^{-1}y}(u_1)
\end{equation}
as $\gamma^2 \to 0$. Recall $u = Su_1 + Tu_2$ and $u_2 \sim \mathcal{N}(Q_2^{T}\hat{C}^{-1} \hat{m} , Q_2^{T}\hat{C}^{-1} Q_2  )$. The mean and variance of the limiting posterior measure $u|y$ is
\[
\begin{aligned}
m^+ &= \operatorname{\mathbb{E}}( Su_1 + Tu_2 | y  ) =  SA_0^{-1}y +T\operatorname{\mathbb{E}}( u_2 ) =SA_0^{-1}y + TQ_2^{T}\hat{C}^{-1}\hat{m}, \\
C^+ & = {\rm Var}(Su_1 + Tu_2 | y) = {\rm Var}( Tu_2) = T   Q_2^{T}\hat{C}^{-1} Q_2 T^{T}  =Q_2(Q_2^{T}\hat{C}^{-1}Q_2)^{-1} Q_2^{T}. \\
\end{aligned}
\]
We have thus completed the proof.
\end{proof}

\begin{remark}
Equation \eqref{lec1:thm2_3} shows that in the limit of zero  observational noise, the uncertainty is only in the variable $u_2$. Since ${\rm Span}(T) = {\rm Span}(Q_2)$ and $u = SA_0^{-1} y + Tu_2$, the uncertainty we observed is in ${\rm Span}(Q_2)$. The prior plays a role in the posterior measure,
in the limit of zero observational noise, but only in the
variables $u_2.$
\end{remark}

\subsection{Discussion and Bibliography}

The linear Gaussian setting plays, for several reasons, a central role in the study of
inverse problems. One is that it allows explicit
solutions which can be used to give insight into the subject area more
generally. The second is that in the large data limit many Bayesian
posteriors are approximately Gaussian. The paper \cite{franklin1970well},
which is in the linear-Gaussian setting, plays an important role in the
history of Bayesian inversion as it was arguably the first to formulate
Bayesian inversion in function space.

We have also employed the Gaussian setting to present a basic form of posterior consistency. For a treatment in infinite dimensions see
\cite{knapik2011bayesian,agapiou2013posterior,nickl2017bernstein}.
For the consistency problem in the classical statistical setting,
see the books \cite{gine2015mathematical,van1998asymptotic}.
The book \cite{van1998asymptotic} also contains definition and properties of
convergence in probability as used here.
For the non-statistical approach to inverse
problems, and consistency results, see \cite{engl1996regularization} and
the references therein.

\section{\Large{\sffamily{Optimal Particle Filter }}}
\label{sec:Lecture12}

This chapter is devoted to the optimal particle filter. Like the
bootstrap filter from the previous chapter, the optimal particle filter
approximates the filtering distribution by a sum of Dirac measures.
The setting will initially be the same
(nonlinear stochastic dynamics and nonlinear observations), but we will see that
the optimal particle filter cannot be implemented, in general, in the fully nonlinear case. For this reason, we will specify to the case of  linear observation operator, where the optimal particle filter can be  implemented in a straightforward fashion and may be characterized as a set of interacting
3DVAR filters. We conclude this chapter by
summarizing and comparing all of the filtering methods introduced in this and
preceding chapters, highlighting the setting in which they may be practically applied.

\subsection{Introduction}
The bootstrap particle filter from the previous chapter is based on
approximating the two components of a specific factorization of the
 filtering step. The factorization is
\begin{align*}
                                \operatorname{\mathbbm{P}}(v_{j+1}|Y_{j}) &= {\mathcal{P}} \operatorname{\mathbbm{P}}(v_{j}|Y_{j}),\\
                                \operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1}) &= {\mathcal{A}}_j \operatorname{\mathbbm{P}}(v_{j+1}|Y_{j}).
                        \end{align*}
This gives the factorization
		\[		\operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1})={\mathcal{A}}_j {\mathcal{P}} \operatorname{\mathbbm{P}}(v_{j}|Y_{j}),  \]
which is the basis for the bootstrap particle filter.
It is natural to ask if there are other factorizations of the
filtering update and whether they might lead to improved
particle filters. In this lecture we derive the Optimal Particle Filter (OPF)
which does just that.
We demonstrate a connection with 3DVAR, and we discuss the sense in which the
OPF has desirable properties in comparison with the BPF.
As usual we denote by $\pi_j$ the filtering distribution $\operatorname{\mathbbm{P}}(v_{j}|Y_{j}).$

\subsection{The Bootstrap and Optimal Particle Filters Compared}
The fundamental filtering problem that we are interested in is
determination of $\operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1})$ from $\operatorname{\mathbbm{P}}(v_{j}|Y_{j}).$
In the BPF, we are approximating the following manipulation:
\begin{align*}
	\operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1}) & = \operatorname{\mathbbm{P}}(v_{j+1}|y_{j+1}, Y_j)\\
	& = \frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1}, Y_j)\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)}{\operatorname{\mathbbm{P}}(y_{j+1}|Y_j)}\\
	& =\frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1}, Y_j)}{\operatorname{\mathbbm{P}}(y_{j+1}|Y_j)}     \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j,Y_j)\operatorname{\mathbbm{P}}(v_j|Y_j)dv_j\\
	&=\frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})}{\operatorname{\mathbbm{P}}(y_{j+1}|Y_j)}       \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\operatorname{\mathbbm{P}}(v_j|Y_j)dv_j\\
	&= {\mathcal{A}}_j {\mathcal{P}} \operatorname{\mathbbm{P}}(v_j|Y_j).
\end{align*}
The Markov kernel ${\mathcal{P}} ^{BPF}$ acts on arbitrary density $\pi$ by
\begin{equation*}
	{\mathcal{P}} ^{BPF}\pi(v_{j+1}) = \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\pi(v_j)dv_j,
\end{equation*}
and ${\mathcal{A}}_j$ acts on an arbitrary density $\pi$ by application of Bayes theorem, taking into account the likelihood of the data
\begin{equation*}
	{\mathcal{A}}_j\pi(v_{j+1}) = \frac{1}{Z}\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\pi(v_{j+1}),
\end{equation*}
with $Z$ normalization to a probability density.
The above manipulations are summarized by the relationship
\begin{equation}
\label{eq:pf}
\pi_{j+1}={\mathcal{A}}_j {\mathcal{P}}\pi_j.
\end{equation}
Note that in this factorization we apply a Markov kernel and then
Bayes theorem.  In contrast, to derive the OPF we perform the following manipulation:
\begin{align*}
	\operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1}) & = \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1},v_j|Y_{j+1})  \, dv_j \\
	& = \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j,Y_{j+1})\operatorname{\mathbbm{P}}(v_j|Y_{j+1})dv_j  \, \\
& = \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1},Y_j)\operatorname{\mathbbm{P}}(v_j|y_{j+1},Y_j) \, dv_j\\
	& = \int_{\mathbbm{R}^\Du} \operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1})\operatorname{\mathbbm{P}}(v_j|y_{j+1},Y_{j}) \, dv_j\\
	& = \int_{\mathbbm{R}^\Du}  \operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1})  \frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_j,Y_{j}) }{\operatorname{\mathbbm{P}}(y_{j+1}|Y_{j})}  \operatorname{\mathbbm{P}}(v_j|Y_{j}) \,dv_j\\
& = \int_{\mathbbm{R}^\Du} \operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1})  \frac{  \operatorname{\mathbbm{P}}(y_{j+1}|v_j) }{\operatorname{\mathbbm{P}}(y_{j+1}|Y_{j})}  \operatorname{\mathbbm{P}}(v_j|Y_{j}) \, dv_j \\
	& = {\mathcal{P}}^{OPF}_j {\mathcal{A}}_j^{OPF}\operatorname{\mathbbm{P}}(v_j|Y_{j}),
\end{align*}
with Markov kernel for particle update
\begin{equation*}
	{\mathcal{P}}_j^{OPF}\pi(v_{j+1}) = \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1})\pi(v_j)dv_j
\end{equation*}
and application of Bayes theorem to include the likelihood
\begin{equation*}
	{\mathcal{A}}_j^{OPF}\pi(v_j) = \frac{1}{Z}\operatorname{\mathbbm{P}}(y_{j+1}|v_j)\pi(v_j).
\end{equation*}
Thus we have
\begin{equation}
\label{eq:opf}
\pi_{j+1}={\mathcal{P}}_j^{OPF} {\mathcal{A}}_j^{OPF}\pi_j.
\end{equation}
Note that in the factorization given by OPF we apply Bayes theorem and then a Markov kernel, the opposite order to the BPF. Moreover the propagation mechanism is different--it sees the data through
the Markov kernel ${\mathcal{P}}_j^{OPF}$-- and hence the weighting
of the particles is also different:  the OPF weights are proportional to the likelihood $\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})$ and the BPF weights are proportional to  $\operatorname{\mathbbm{P}}(y_{j+1}|v_{j})$ which may be, in general, not available in closed form.  We will see that the particle updates use a 3DVAR procedure.
In the BPF, the evolution of the particles and the observation of the data are kept separate from each other -- the Markov kernel ${\mathcal{P}}$ depends only on the dynamics and not the observed data and is thus independent of $j$.

\subsection{Implementation of Optimal Particle Filter: Linear Observation Setting}
In general it is not possible to implement the OPF in the fully nonlinear setting because of two computational bottlenecks:
\begin{itemize}
\item There may not be a closed formula for evaluating the likelihood $\operatorname{\mathbbm{P}}(y_{j+1}|v_{j}),$ making unfeasible the computation of the particle weights.
\item It may not be possible to sample from the Markov kernel $\operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1}),$ making unfeasible the propagation of particles.
\end{itemize}
However, when the observation function $h(\cdot)$ is linear, i.e. $h(\cdot) = H\cdot$ for some $H \in R^{k \times d}$ both bottlenecks are overcome.
We thus consider the following setting, which arises in many applications:
\begin{align*}
	v_{j+1} & = \Psi(v_j) + \xi_j, && \xi_j \sim \mathcal{N}(0,\Sigma)\text{ i.i.d.}, \\
	y_{j+1} & = Hv_{j+1} +\eta_{j+1}, && \eta_{j+1} \sim \mathcal{N}(0,\Gamma)\text{ i.i.d.},
\end{align*}
with $v_0 \sim \mathcal{N}(m_0,C_0)$ and $v_0, \{\xi_j\}, \{\eta_j\}$ independent. We will now show that in this linear observation both bottlenecks are overcome.
First, note that combining the dynamics and data models we may write
\begin{equation*}
	y_{j+1} = H\Psi(v_j) + H\xi_j + \eta_{j+1},
\end{equation*}
which shows that the conditional distribution for $y_{j+1}$ given $v_j$ is
\begin{equation*}
	\operatorname{\mathbbm{P}}(y_{j+1} | v_j) = \mathcal{N}(H\Psi(v_j),S),
\end{equation*}
where $S= H\Sigma H^T + \Gamma.$
We will use this formula to compute the weights, thus overcoming the first computational bottleneck.

We now show that under the linear observation assumption ${\mathcal{P}}^{OPF}_j$ is a Gaussian kernel, and hence the second computational bottleneck is overcome too.
We have
\begin{align*}
	\operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1}) &\propto \operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1}, v_j)\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\\
	&= \operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\\
	&\propto \exp \left(-\frac{1}{2}|y_{j+1}-Hv_{j+1}|^2_\Gamma -\frac{1}{2}|v_{j+1}-\Psi(v_{j})|^2_\Sigma \right)\\
	&=\exp\bigl(-{\mathsf{J}}_{\textsc{opt}}(v_{j+1})\bigr).
\end{align*}
This is a Gaussian distribution for $v_{j+1}$ as ${\mathsf{J}}_{\textsc{opt}}(v_{j+1})$ is quadratic with respect to $v_{j+1}$.\footnote{${\mathsf{J}}_{\textsc{opt}}$ is identical to
${\mathsf{J}}$ on the right-hand side of Table \ref{3DVAR:t1}, with $\hat{C}$ replaced by $\Sigma.$} Consequently, we can compute the mean $m_{j+1}$ and covariance $C$
(which, note, is independent of $j$) of this Gaussian by matching the mean
and quadratic terms in the relevant quadratic forms:
\begin{align*}
	C^{-1} &= H^T\Gamma^{-1}H + \Sigma^{-1},\\
	C^{-1}m_{j+1} &= \Sigma^{-1}\Psi(v_j) + H^T\Gamma^{-1}y_{j+1}.
\end{align*}
Then $\operatorname{\mathbbm{P}}(v_{j+1}|y_{j+1}, v_j)=\mathcal{N}(m_{j+1},C).$
This is hence a special case of 3DVAR in which the
analysis covariance is fixed at $C$; note that when we derived 3DVAR we
fixed the predictive covariance $\hat{C}$ which, here, is fixed at $\Sigma.$
As with the Kalman filter, and with 3DVAR, it is possible to implement
the prediction step through the following mean and covariance formulae
which avoid inversion in state space, and require inversion only in data
space:
\begin{align*}
	m_{j+1} & = (I-KH)\Psi(v_j) + Ky_{j+1},\\
	C & = (I-KH)\Sigma,\\
	K & = \Sigma H^T S^{-1},\\
	S & = H \Sigma H^T + \Gamma.
\end{align*}
Furthermore, as for 3DVAR, the inversion of $S$ need only be performed once
in a pre-processing step before the algorithm is run.
Since the expression for $\operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1})$ is Gaussian we now
have the ability to sample directly from ${\mathcal{P}}_j^{OPF}$.
The OPF is thus given by the following update algorithm for approximations
$\pi_j^N \approx \pi_j$ in which we generalize the notational conventions
used in the previous chapter to formulate particle filters as random
dynamical systems:

\FloatBarrier
\begin{algorithm}
\caption{Algorithm for the Optimal Particle Filter with linear observation map}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Initial distribution $\operatorname{\mathbbm{P}}(v_0) = \pi_0$, observations $Y_{J}$, number of particles $N.$
\STATE {\bf Initial Sampling}: Draw $N$ particles $v_0^{(n)} \sim \pi_0$ so that $\pi_0^N=S^N\pi_0.$
\vspace{0.04in}
\STATE{\bf Subsequent Sampling} For $j = 0,1,\dots,J-1$, perform:
\begin{enumerate}
\item Set $\hat{v}_{j+1}^{(n)} = (I-KH)\Psi(v_{j}^{(n)}) + Ky_{j+1} + 		\zeta_{j+1}^{(n)}$ with $\zeta_{j+1}^{(n)}$ i.i.d.  $\mathcal{N}(0,C).$
\item Set $\bar{w}_{j+1}^{(n)} = \exp\left(-\frac{1}{2}|y_{j+1}-H\Psi(v_j^{(n)})|_S^2\right).$
\item Set $w_{j+1}^{(n)} = \bar{w}_{j+1}^{(n)} / \sum_{n=1}^N\bar{w}_{j+1}^{(n)}.$
\item Set $v_{j+1}^{(n)}=\sum_{m=1}^{N} {\mathbbm{1}}_{I_{j+1}^{(m)}}(r_{j+1}^{(n)})\hat{v}_{j+1}^{(m)}.$
\item Set $\pi_{j+1}^{N}(v_{j+1}) = \frac{1}{N}\sum_{n=1}^N \delta\left(v_{j+1}-{v}_{j+1}^{(n)}\right).$
\end{enumerate}
\vspace{0.04in}
\STATE {\bf Output}: $N$ particles $v_{J}^{(1)},v_{J}^{(2)}, \dots, v_{J}^{(N)}.$
\end{algorithmic}
\label{alg:opf}
\end{algorithm}
\FloatBarrier

It would be desirable to interpret this algorithm as an approximation of the
filter update \eqref{eq:opf} in the form
$$\pi_{j+1}^N =S^N {\mathcal{P}}_j^{OPF}{\mathcal{A}}_j^{OPF} \pi_j^N,  \quad \pi_j^0=S^N \pi_0.$$
However the order in which the resampling and the particle propagation
occurs implies that this is not possible. The following slight modification
of the OPF, however, may indeed be thought of as an approximation of
this form; we simply reorder the resampling and the propagation.
We refer to the resulting algorithm as the Gaussianized Optimal
Particle filter (GOPF).
We may write the resulting algorithm as follows:

\FloatBarrier
\begin{algorithm}
\caption{Algorithm for the Gaussianized Optimal Particle Filter}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Initial distribution $\operatorname{\mathbbm{P}}(v_0) = \pi_0$, observations $Y_{J}$, number of particles $N.$
\STATE {\bf Initial Sampling}: Draw $N$ particles $v_0^{(n)} \sim \pi_0$ so that $\pi_0^N=S^N\pi_0.$
\vspace{0.04in}
\STATE{\bf Subsequent Sampling} For $j = 0,1,\dots,J-1$, perform:
\begin{enumerate}
\item Set $\bar{w}_{j+1}^{(n)} = \exp\left(-\frac{1}{2}|y_{j+1}-H\Psi(v_j^{(n)})|_S^2\right).$
\item Set $w_{j+1}^{(n)} = \bar{w}_{j+1}^{(n)} / \sum_{n=1}^N\bar{w}_{j+1}^{(n)}.$
\item Set $\hat{v}_{j}^{(n)}=\sum_{m=1}^{N} {\mathbbm{1}}_{I_{j+1}^{(m)}}(r_{j+1}^{(n)}){v}_{j}^{(m)}.$
\item Set ${v}_{j+1}^{(n)} = (I-KH)\Psi(\hat{v}_{j}^{(n)}) + Ky_{j+1} +          \zeta_{j+1}^{(n)}$ with $\zeta_{j+1}^{(n)}$ i.i.d  $\mathcal{N}(0,C).$
\item Set $\pi_{j+1}^{N}(v_{j+1}) = \frac{1}{N}\sum_{n=1}^N \delta\left(v_{j+1}-{v}_{j+1}^{(n)}\right).$
\end{enumerate}
\vspace{0.04in}
\STATE {\bf Output}: $N$ particles $v_{J}^{(1)},v_{J}^{(2)}, \dots, v_{J}^{(N)}.$
\end{algorithmic}
\label{alg:opf}
\end{algorithm}
\FloatBarrier

\subsection{``Optimality'' of the Optimal Particle Filter}
Particle filter methods rely on approximating the distribution for the model by a swarm of point Dirac functions; it is clear that the distribution will not be well approximated by only a small number of particles in most cases. Consequently, a performance requirement for particle filter methods is that they do not lead to degeneracy of the particles. Resampling leads to degeneracy if
a small number of particles have all the weights. Conversely, non-degeneracy
may be promoted by ensuring that the weights $w_j^{(n)}$ are similar in magnitude, so that a small number of particles are not overly favoured during the resampling step. This condition can be formulated as a requirement that the variance of the weights be minimized; doing this results in the OPF.

To understand this perspective
we consider an arbitrary particle update kernel of the form
$\pi(v_{j+1}|v_j^{(n)},Y_{j+1})$
and we study the resulting particle filter without resampling.
It is then the case that the particle weights are updated according
to the formula
			\begin{equation*}
				\bar{w}^{(n)}_{j+1} = \bar{w}^{(n)}_j \frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v^{(n)}_{j})}{\pi(v_{j+1}|v_j^{(n)},Y_{j+1})}.
			\end{equation*}

\begin{theorem}[Meaning of Optimality]
	The choice of $\operatorname{\mathbbm{P}}(v_{j+1}|v_j^{(n)}, y_{j+1})$ as the particle update kernel $\pi(v_{j+1}|v_j^{(n)},Y_{j+1})$ results in the minimal
variance of the weight $w^{(n)}_{j+1}$ with respect to all possible
choices of the particle
update kernel $\pi(v_{j+1}|v_j^{(n)},Y_{j+1}).$
\end{theorem}

\begin{proof}
We calculate the variance of the unnormalized weights (treated as random variables) $\bar{w}^{(n)}_{j+1}$ with respect to the transition density $\pi(v_{j+1}|v_j^{(n)},Y_{j+1})$ and obtain
\begin{align*}
	{\rm Var}_{\pi(v_{j+1}|v_j^{(n)},Y_{j+1})} &[\bar{w}^{(n)}_{j+1}]  = \int_{\mathbbm{R}^\Du} \left(\bar{w}^{(n)}_{j+1}\right)^2\pi(v_{j+1}|v_j^{(n)},Y_{j+1}) dv_{j+1}\\
	 &- \left[\int_{\mathbbm{R}^\Du} \bar{w}^{(n)}_{j+1}\pi(v_{j+1}|v_j^{(n)},Y_{j+1}) dv_{j+1} \right]^2\\
	& = \left(\bar{w}^{(n)}_{j}\right)^2\int_{\mathbbm{R}^\Du} \frac{\left(\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v^{(n)}_{j})\right)^2}{\pi(v_{j+1}|v_j^{(n)},Y_{j+1})} dv_{j+1}\\
	& \quad\quad\quad\quad\quad\quad\quad - \left(\bar{w}^{(n)}_{j}\right)^2\left[\int_{\mathbbm{R}^\Du} \operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v^{(n)}_{j}) dv_{j+1}\right]^2\\
	& = \left(\bar{w}^{(n)}_{j}\right)^2\left[\int_{\mathbbm{R}^\Du}\frac{\left(\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v^{(n)}_{j})\right)^2}{\pi(v_{j+1}|v_j^{(n)},Y_{j+1})} dv_{j+1} - \operatorname{\mathbbm{P}}(y_{j+1}|v^{(n)}_{j})^2\right].
\end{align*}
Choosing $\pi(v_{j+1}|v_j^{(n)},Y_{j+1}) = \operatorname{\mathbbm{P}}(v_{j+1}|v_j^{(n)}, y_{j+1})$, as in the OPF, we obtain
\begin{align*}
	{\rm Var}_{\operatorname{\mathbbm{P}}(v_{j+1}|v_j^{(n)}, Y_{j+1})}[\bar{w}^{(n)}_{j+1}]
	& = \left(\bar{w}^{(n)}_{j}\right)^2\left[\int_{\mathbbm{R}^\Du}\frac{\left(\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v^{(n)}_{j})\right)^2}{\operatorname{\mathbbm{P}}(v_{j+1}|v_j^{(n)}, y_{j+1})} dv_{j+1} - \operatorname{\mathbbm{P}}(y_{j+1}|v^{(n)}_{j})^2\right]\\
	& = \left(\bar{w}^{(n)}_{j}\right)^2\left[\operatorname{\mathbbm{P}}(y_{j+1}|v^{(n)}_{j})^2 - \operatorname{\mathbbm{P}}(y_{j+1}|v^{(n)}_{j})^2\right]\\
	& = 0.
\end{align*}
\end{proof}

\begin{remark}
The optimal particle filter is optimal in the very precise sense of the
theorem. Note in particular that no optimality criterion is asserted by
this theorem with respect to iterating the particle updates, and in particular
when resampling is included. The nomenclature ``optimal'' should thus be
treated with caution.

\end{remark}

\subsection{Particle Filters for High Dimensions}
Particle filters often perform poorly for high-dimensional systems due to a collapse of the particle weights: only a few particles carry most of the weight; the optimal particle filter can ameliorate this issue because the proposal uses the data, meaning that particle predictions are more likely to be weighted highly. There have been attempts to formulate update steps that help to mitigate this weight collapse. Essentially, these methods aim to push the particles towards the region of high likelihood, such that all the particles will be representative of the distribution.
As will show in the next lecture, there are interesting particle based
methods which, whilst not statistically consistent, do perform well as
signal estimators. This perspective on particle filter type methods, namely
to use them for smart signal estimation rather than as estimators of the
filtering distribution, may become increasingly useful in high dimensional
systems.

\subsection{Filtering Overview}
\label{sec:overview}

Having introduced a range of filtering methods in this and the preceding
four chapters, it is helpful to summarize what these methods achieve,
in a comparitive fashion.

\subsubsection{Dynamical Model}
The stochastic dynamics model is defined by
\begin{equation*}
v_{j+1} = \Psi(v_j) + \xi_j, \quad j \ge 0,
\end{equation*}
where $\xi_j \sim \mathcal{N}(0,\Sigma)$ are independent and identically distributed random variables, also independent of the initial condition $v_0 \sim \mathcal{N}(m_0,C_0)$.
\FloatBarrier
\begin{figure}[htp]
  \begin{center}
    \includegraphics[width=0.5\textwidth]{figures/model2.png}
  \end{center}
  \caption{Prediction step.} \label{fig:model}
\end{figure}
\FloatBarrier
The filtering distribution of $v_j$ is denoted by
$\pi_j(v_{j}) = \operatorname{\mathbbm{P}}(v_j|Y_j).$
This is propagated by the stochastic dynamics model according to the
formula $\hat{\pi}_{j+1}={\mathcal{P}}\pi_j$ where ${\mathcal{P}}$ is defined in \eqref{eq:markov}.
This is shown schematically in Figure \ref{fig:model}. Note that ${\mathcal{P}}$ is independent of step $j$ because the Markov chain defined by the stochastic
dynamics model is time-homogeneous. In the absence of data the probability
distribution simply evolves through repeated application of ${\mathcal{P}}$.

\subsubsection{Data Model}
The data model is given by
\begin{equation*}
y_{j+1} = h(v_{j+1}) + \eta_{j+1},
\end{equation*}
where $\eta_{j+1} \sim {N}(0,\Gamma)$ are independent and identically distributed random variables, independent of both $v_0$ and the i.i.d. sequence
$\{\xi_j\}.$
Given data, we now view the prediction from the dynamical model
as a prior which we condition on the data. We write this prior at
time $j+1$ as $\hat{\pi}_{j+1}$, and prediction using the
stochastic dynamics model gives $\hat{\pi}_{j+1}={\mathcal{P}}\pi_j.$

Using the data to update this prior by Bayes theorem gives an improved
estimate of the distribution $\pi_{j+1}(v_{j+1})$ via the formula
\begin{equation*}
\pi_{j+1} = {\mathcal{A}}_j \widehat{\pi}_{j+1}
\end{equation*}
where ${\mathcal{A}}_j$ is given by \eqref{eq:bayes-again}.
This is shown schematically in Figure \ref{fig:data}. Note that ${\mathcal{A}}_j$ is nonlinear because $\widehat{\pi}_{j+1}$ appears in both the numerator and the denominator. It depends on $j$ because the data $y_{j+1}$ appears in the equation and this will change with each set of measurements.

\begin{figure}[h]
  \begin{center}
    \includegraphics[width=0.5\textwidth]{figures/data.png}
  \end{center}
  \caption{Update step.} \label{fig:data}
\end{figure}

\subsubsection{Interaction of the Dynamics and Data}

Recall the notation $Y_{j}=\{y_1, \dots, y_j\}$
for the collection of all data until time $j+1$.
The dynamics and data models described in the two previous subsections
can be combined as follows. Using the dynamics,
the probability distribution is propagated forward in time by the prediction step. The data is then used to update the probability distribution at that time in the analysis step.
The two can be combined to obtain
\begin{equation*}
{\pi}_{j+1} = {\mathcal{A}}_j {\mathcal{P}} \pi_j.
\end{equation*}
The combination of the two steps is shown
schematically in Figure \ref{fig:interaction}.

\begin{figure}[h]
  \begin{center}
    \includegraphics[width=0.8\textwidth]{figures/interaction.png}
  \end{center}
  \caption{Prediction and update step combined.} \label{fig:interaction}
\end{figure}

\subsubsection{Summary of Applicability of Discrete Filtering Methods}

There are several filtering methods for performing the prediction and analysis steps. Some methods, such as the bootstrap particle filter, can be applied generally to nonlinear problems. However, others require a linear model $( \Psi(\cdot) = M\cdot)$ and/or linear observations $(h(\cdot) = H\cdot)$.
Some of the methods we have described provably approximate the
probability distribution updates. Some just estimate the state and
simply use covariance information to weight the relative importance
of the predictions from the model and of the data.

The applicability of the methods introduced is summarized in the
table below, with respect to linearity/nonlinearity of the dynamics
and the observation model. Furthermore  ${\bf P}$ is used to denote
 methods which provably approximate the filtering distributions
$\pi_j$ in the large particle limit; ${\bf S}$ denotes methods
which only attempt to estimate the state, using the data.

\begin{description}[leftmargin=!,labelwidth=\widthof{\bfseries }]\setlength{\itemsep}{4pt}
\item[Kalman Filter] $ \Psi(\cdot) = M\cdot, h(\cdot) = H\cdot$. ${\bf P}.$
\item[3DVAR] $\text{general}\, \Psi, h(\cdot) = H\cdot $. ${\bf S}.$
\item[Bootstrap Particle Filter] $ \text{general}\, \Psi, h$. ${\bf P}.$
\item[Optimal Particle Filter] $\text{general}\, \Psi, h(\cdot) = H\cdot$. ${\bf P}$.
\item[Extended Kalman Filter] $\text{general}\, \Psi, h(\cdot) = H\cdot $. ${\bf S}.$
\item[Ensemble Kalman Filter] $\text{general}\, \Psi, h(\cdot) = H\cdot $. ${\bf S}.$
\end{description}

Some of these constraints can be relaxed on the setting in which they apply
can be relaxed, but the list above describes
the methods as we present them in these notes.
Furthermore the last two methods can accurately predict probability distributions
in situations where approximate Gaussianity holds; this may be induced
by small noise and by large data. However it is important to appreciate
that the raison d'etre of the EnKF is to facilitate the solution of
problems with high dimensional state space; typically small ensemble sizes are
used and the algorithm is employed far from the regime in which it is
able to provably approximate the distributions, even if they are close
to Gaussian. It is for this reason that we prefer to think of the EnKF
as an ensemble state estimator.

\subsection{Discussion and Bibliography}

Particle filters often perform poorly for high-dimensional systems due to
the fact that the particle weight typically concentrates on one, or
a small number, of particles --see the work of Bickel and Snyder in
\cite{bickel2008sharp, snyder2008obstacles, Snyder2011}. This is the
issue that the optimal particle filter tries to ameliorate; the
paper \cite{agapiou2017importance} shows calculations which demonstrate
the extent to which this amelioration is manifest in theory.
The optimal particle filter is discussed, and further references given,
in the very clear paper \cite{doucet2000sequential}; see section IID.
Throughout much of this chapter we considered the case of Gaussian additive noise
and linear observation operator
in which case the prediction step is tractable; the paper \cite{doucet2000sequential} discusses the more general setting.
The order in which the prediction and resampling is performed can
be commuted in this case and a discussion of this fact may be found in
\cite{pitt1999filtering}; this leads to the distinction between
what we term the GOPF and the OPF.
The convergence of the optimal particle
filter is studied in \cite{johansen2008note}.
The formulation of the bootstrap and optimal particle filters as random
dyamical systems may be found in \cite{kelly2016ergodicity}. An attempt to alleviate weight collapse by introducing alternative update steps can be found in \cite{Snyder2011}.

\section{\Large{\sffamily{Filtering Approach to the Inverse Problem}}}

In this final chapter we demonstrate how the two separate themes that
underpin this course, inverse problems and data assimilation, may be
linked. This opens up the possibility of transferring ideas from filtering
into the setting of quite general inverse problems.
In the first section we describe the general, abstract, connection
and introduce the revolutionary idea of sequential Monte Carlo
methods (SMC).
In the second section we analyze the concrete case of applying the EnKF
to solve an inverse problem, leading to ensemble Kalman inversion (EKI).
In the final section we link the EKI methodology to SMC.

\subsection{General Formulation}

Recall the inverse problem of finding $u \in \mathbbm{R}^\Du$ from $y \in \mathbbm{R}^\Dy$ where
\begin{align}
\label{eq:need99}
y=G(u)+\eta,\; \eta\sim \mathcal{N}(0,\Gamma_0)
\end{align}
and the related loss function
\begin{equation*}
\mathsf{L}_0(u)=\frac{1}{2}|y-G(u)|^2_{\Gamma_0}.
\end{equation*}
The reason for writing $\Gamma_0$, rather than $\Gamma$, will become apparent
below and will also be exploited in the third section of this chapter where
we study EKI.

If we put a prior $\rho$ on the unknown $u$ then the posterior takes
the form
$$\pi(u)=\frac{1}{Z}\exp\bigl(-\mathsf{L}_0(u)\bigr)\rho(u).$$
Let $J\in\mathbbm{N}$ and choose $h$ so that $Jh=1$. Then
define the family of pdfs $\{\pi_j\}_{j=0}^J$ by
$$\pi_j(u)=\frac{1}{Z_j}\exp\bigl(-jh\mathsf{L}_0(u)\bigr)\rho(u).$$
It follows that $\pi_0 = \rho$ and $\pi_J=\pi$ and, furthermore, we may update the
sequence $\{\pi_j\}_{j=0}^J$ sequentially using the formula
$$\pi_{j+1}(u)=\frac{Z_j}{Z_{j+1}}\exp\bigl(-h\mathsf{L}_0(u)\bigr)\pi_j(u).$$
This simply corresponds to application of Bayes theorem to the
inverse problem
\begin{align}
\label{eq:need00}
y=G(u)+\eta,\; \eta\sim \mathcal{N}(0,\Gamma),
\end{align}
with $\Gamma=\frac{1}{h}\Gamma_0$
and with prior $\pi_j.$ With this choice of $\Gamma$ the identity \eqref{eq:need00}
gives the likelihood proportional to $\exp(-h\mathsf{L}_0(u))$. The update
may be written
$$\pi_{j+1}(u)={\mathcal{A}}\pi_j,$$
noting that the analysis operator ${\mathcal{A}}$ is nonlinear, corresponding
to multiplication by $\exp(-h\mathsf{L}_0(u))$ and then normalization to
a pdf.

If we let ${\mathcal{P}}_j$ denote any Markov kernel for which $\pi_j$ is
invariant then we obtain
$$\pi_{j+1}={\mathcal{A}}{\mathcal{P}}_j \pi_j.$$
This update formula should be compared with the filtering update formula
\begin{align*}
\pi_{j+1} = {\mathcal{A}}_j {\mathcal{P}}\pi_j
\end{align*}
introduced and used in earlier chapters.

Whilst the $j-$dependence in Bayes rule and the Markov kernel
is interchanged between the inverse problem and the filtering problem,
this makes little material difference to implementation.
Firstly, once a Markov kernel ${\mathcal{P}}$ is identified under which $\pi=\pi_J$
is invariant, it can be easily adapted to find a family of kernels
${\mathcal{P}}_j$ under which $\pi_j$ is invariant, simply by rescaling the
observation covariance. Secondly the fact that the data $y$ is fixed,
rather than changing at each step, makes little difference to
the implementation.
This perspective opens up an entire field, known as {\em sequential
Monte Carlo}, in which ideas from filtering
may be transferred to other quite different problems, including Bayesian
inversion, as explained here.

Note however (see the discussion at the end of section \ref{sec:overview})
that of the filtering methods we have introduced so far only the bootstrap filter
applies directly to the case of {\em nonlinear} observation operators; since
$G$ is in general nonlinear this means that extra ideas are required to
implement the optimal particle filter, 3DVAR, ExKF and EnKF. One approach is
to replace the sequential optimization principles by the non-quadratic
optimization problem required for nonlinear $G$; we do not discuss this idea
in any detail but it is a viable option. Another approach is to use the linearization
technique which we now outline in the context of EKI.

\subsection{Ensemble Kalman Inversion}

We now make a detour into the subject of how to use
filters to estimate parameters $u$ from data $y$ satisfying
\eqref{eq:need00}. The approach we study,
and which we will relate to sequential Monte Carlo in
the next section, is to introduce an artificial
time dynamic. This can be done quite simply as follows: we write
\begin{align*}
u_{j+1} &=   u_j,\\
y_{j+1} & =   G(u_{j+1}) + \eta_{j+1},
\end{align*}
and we can think of finding the filtering distribution on $u_j|Y_j.$
We discuss how to relate the $y_j$ to the one data point $y$ below.
For now we take $\eta_j \sim \mathcal{N}(0,\Gamma)$ but we will revisit this
choice in the next section, linking the problem to solution of \eqref{eq:need99}.
Because the observation operator $G$ is, in general, nonlinear, this
does not render our system in a form where we can readily apply the
EnKF. To this end we introduce a new variable $w_{j}$ and rewrite the
filter as:
\begin{align*}
u_{j+1} & =   u_j,\\
w_{j+1} & =   G(u_{j}),\\
y_{j+1} & =   w_{j+1} + \eta_{j+1}.
\end{align*}

We introduce the new variables $v=(u,w)^T$, nonlinear map $\Psi(v)=\bigl(u, G(u)\bigr)^T$
and linear operators $H =  [0, I], H^{\perp} = [I,0].$  Then if we
write $v_j=(u_j,w_j)^T$ we may write the dynamical system in the form
\begin{subequations}
\label{eq:new}
\begin{align}
v_{j+1} &=  \Psi(v_j),\\
y_{j+1} &= Hv_{j+1} + \eta_{j+1}.
\end{align}
\end{subequations}
We note that
$$Hv= w, \quad H^{\perp}v=u.$$

\begin{remark}
Typically either of the following  are used to construct artificial
data $\{y_j\}$ for the filtering algorithm, given a single instance of data $y$:
 $$y_{j+1}=
  \begin{cases}
  & y \text{ (unperturbed obervations).}\\
  & y+\bar{\eta}_{j+1}, \bar{\eta}_{j+1}\sim \mathcal{N}(0,\Gamma)\text{ (perturbed observations).}
  \end{cases}$$
The first choice is natural if viewing the algorithm as a sequential
optimizer; the latter, in the linear case, is natural when seeking to draw
samples from the posterior.\qed
\end{remark}

We now apply the EnKF to the dynamics/data model \eqref{eq:new}.
We obtain, for $n = 1,\dots,N$,
\begin{align}
\label{eq:eq1}
\hat{v}_{j+1}^{(n)}&=\Psi(v_j^{(n)}),\\
\bar{v}_{j+1}&= \frac{1}{N}\sum_{n=1}^N\hat{v}_{j+1}^{(n)},\\
\widehat{C}_{j+1}&=\frac{1}{N}\sum_{n=1}^N \bigl(\hat{v}_{j+1}^{(n)}-\bar{v}_{j+1}\bigr)\otimes\bigl(\hat{v}_{j+1}^{(n)}-\bar{v}_{j+1} \bigr),\\
v_{j+1}^{(n)}&=
(I-K_{j+1}H)\hat{v}_{j+1}^{(n)}+ K_{j+1}y_{j+1}^{(n)},
\end{align}
with the Kalman gain
\begin{align*}
K_{j+1}& =\widehat{C}_{j+1}H^T S_{j+1},\\
S_{j+1}& =(HC_{j+1}H^T+\Gamma)^{-1}.
\end{align*}
Now we may simplify these expressions by using the specific $\Psi$, $v$, $H$ arising in the inverse problem. Writing
\begin{align*}
\widehat{C}_{j+1}=
\begin{bmatrix}
C_{j+1}^{uu} & C_{j+1}^{uw}\\
(C_{j+1}^{uw})^T & C_{j+1}^{ww}
\end{bmatrix}, \quad \quad
\;\;
\bar{v}_{j+1}=
\begin{pmatrix}
\bar{u}_{j+1}\\
\bar{w}_{j+1}
\end{pmatrix},
\end{align*}
we have
$$\bar{u}_{j+1} = \frac{1}{N}\sum_{n=1}^N u_{j}^{(n)},\; \quad \quad \bar{w}_{j+1} = \frac{1}{N}\sum_{n=1}^N G(u_{j}^{(n)}):=\bar{G}_j,$$
and
\begin{align*}
C_{j+1}^{uw} &= \frac{1}{N}\sum_{n=1}^N \bigl(u_{j}^{(n)}-\bar{u}_{j+1}\bigr)\otimes \bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr),  \\
 C_{j+1}^{ww} &= \frac{1}{N}\sum_{n=1}^N \bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr)\otimes\bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr).
\end{align*}
There is a similar expression for $C_{j+1}^{uu}$, but as we will show in what
follows it is not needed for the unknown parameter $u$ update formula.
Noting that, because of the structure of $H$, $S_{j+1}=(C_{j+1}^{ww}+\Gamma)^{-1}$
we obtain
\begin{align}
K_{j+1}=
\begin{pmatrix}\label{eq:2}
C_{j+1}^{uw}(C_{j+1}^{ww}+\Gamma)^{-1}\\
(C_{j+1}^{ww})(C_{j+1}^{ww}+\Gamma)^{-1}
\end{pmatrix}.
\end{align}
Combining equation \eqref{eq:2} with the update equation within \eqref{eq:eq1}
it follows that
\begin{align*}
\bigl\{v_j^{(n)}\bigr\}_{n=1}^N \rightarrow \bigl\{v_{j+1}^{(n)} \bigr\}_{n= 1}^N
\end{align*}
and
\begin{align*}
\bigl\{H^{\perp}v_j^{(n)} \bigr\}_{n= 1}^N \rightarrow \bigl\{H^{\perp}v_{j+1}^{(n)} \bigr\}_{n= 1}^N,
\end{align*}
and hence that
\begin{align*}
u_{j+1}^{(n)} = H^{\perp}v_{j+1}^{(n)} = u_j^{(n)}+ C_{j+1}^{uw}(C_{j+1}^{ww}+\Gamma)^{-1} \Bigl(y_{j+1}^{(n)}-G(u_j^{(n)})\Bigr).
\end{align*}
Thus we have derived the EKI step \begin{equation}
\label{eq:EnKF}
u_{j+1}^{(n)}  = u_j^{(n)}+ C_{j+1}^{uw} (C_{j+1}^{ww}+\Gamma)^{-1}\Bigl(y_{j+1}^{(n)}-G(u_j^{(n)})\Bigr).
\end{equation}
The full algorithm is described below:
\FloatBarrier
\begin{algorithm}
\caption{Algorithm for Ensemble Kalman Inversion}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Initial distribution $\pi_0 = \rho$, observations $Y_{J}$, number of particles $N.$
\STATE {\bf Initial Sampling}: Draw $N$ particles $u_0^{(n)} \sim \rho$ so that $\pi_0^N=S^N\rho.$
\vspace{0.04in}
\STATE{\bf Subsequent Sampling} For $j = 0,1,\dots,J-1$, perform
\begin{enumerate}
\item Set $\bar{u}_{j+1} = \frac{1}{N}\sum_{n=1}^N u_{j}^{(n)}.$
\item Set $\bar{G}_j= \frac{1}{N}\sum_{n=1}^N G(u_{j}^{(n)}).$
\item Set $C_{j+1}^{uw} = \frac{1}{N}\sum_{n=1}^N \bigl(u_{j}^{(n)}-\bar{u}_{j+1}\bigr) \otimes \bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr).$
\item Set $C_{j+1}^{ww} = \frac{1}{N}\sum_{n=1}^N \bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr)\otimes\bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr).$
\item $u_{j+1}^{(n)}  = u_j^{(n)}+ C_{j+1}^{uw}\bigl(C_{j+1}^{ww}+\Gamma)^{-1}(y_{j+1}^{(n)}-G(u_j^{(n)})\bigr).$
\end{enumerate}
\vspace{0.04in}
\STATE {\bf Output}: $N$ particles $u_{J}^{(1)},u_{J}^{(2)}, \dots, u_{J}^{(N)}.$
\end{algorithmic}
\label{alg:EKI}
\end{algorithm}
\FloatBarrier

\begin{remark}
This algorithm may be viewed as a derivative-free optimization method,
within the broad class that contains genetic, swarm or ant colony
optimization. \qed
\end{remark}

The algorithm has the following invariant subspace property.

  \begin{theorem}(Space of Ensemble)
\label{t:cite}
  Define $\mathcal{A} = {\text{Span}} \bigl\{u^{(n)}_{0}: n = 1,\ldots, N \bigr\}$, then
for all $0\le j \le J$ and all  $1\le n \le N$,
$u^{(n)}_{j}$ defined by the iteration \eqref{eq:EnKF} lie
in $\mathcal{A}$.
  \end{theorem}
  \begin{proof}
  The proof proceeds by induction. We let $\mathfrak{u}_j \in ({\mathbbm{R}^\Du})^N$
denote the collection of $\{u^{(n)}_{j}\}_{n= 1}^N.$
  At first, let $0\le j \le J$ and let  $1\le m \le N$
and define the following two quantities:
  \begin{align*}
  d^{(n)}_{j}&:=(C^{ww}_{j+1}+\Gamma)^{-1} \Bigl(y^{(n)}_{j+1}-G(u^{(n)}_{j})\Bigr), \\
  D_{mn}(\mathfrak{u}_j)&:=-\langle d^{(n)}_{j},G(u^{(m)}_{j})-\bar{G}_j \rangle.
  \end{align*}
  Notice that the $d^{(n)}_{j}$ are elements of the data space ${\mathbb R}^J$,
whereas the $D_{mn}(\mathfrak{u}_j)$ are scalar quantities. Furthermore notice that
\begin{align*}
u_{j+1}^{(n)}  & = u_j^{(n)}+ C_{j+1}^{uw} (C_{j+1}^{ww}+\Gamma)^{-1}\Bigl(y_{j+1}^{(n)}-G(u_j^{(n)})\Bigr)\\
& = u_j^{(n)} + C_{j+1}^{uw} d^{(n)}_{j} \\
& = u_j^{(n)} + \frac{1}{N}\sum_{m=1}^N \bigl(u_{j}^{(m)}-\bar{u}_{j+1}\bigr) \otimes \bigl(G(u_{j}^{(m)})-\bar{G}_{j}\bigr) d^{(n)}_{j} \\
& = u^{(n)}_{j} - \frac{1}{N} \sum\limits_{m=1}^N D_{mn}(\mathfrak{u}_j)(u^{(m)}_{j}-\bar{u}_{j+1}).
\end{align*}
From the definition of $\bar{G}_j$ and the bilinearity of the inner product
it follows that $\sum\limits_{m=1}^N D_{mn}(\mathfrak{u}_j) = 0$. Therefore the
update expression may be rewritten, for $0\le j \le J$ and $1\le n \le N,$ as
$$u^{(n)}_{j+1}=u^{(n)}_{j}-\frac{1}{N} \sum\limits_{m=1}^N D_{mn}(\mathfrak{u}_j)u^{(m)}_{j}.$$
Hence if the property holds for all the particles of the time step $j$, it will clearly be the case for all the particles at time step $j+1$.
  \end{proof}

\begin{remark}
The choice of the initial ensemble of particles
$\{u^{(n)}_{0}\}_{n= 1}^N$ is thus crucial to the
performance of the EnKF,
since the algorithm remains in the initial
space of the initial ensemble.
In the setting of Bayesian inverse problems the initial ensemble
is frequently created by drawing from the prior.
Alternatively, if the prior is Gaussian, the first $N$ covariance
eigenvectors may be used, ordered by decreasing variance contribution.
More generally any truncated basis for the space $\mathbbm{R}^\Du$ is
a natural initial ensemble. However the question of how to adaptively
learn a good choice of ensemble subspace, in
response to observed data, is unexplored and potentially fruitful. \qed
\end{remark}

\begin{remark}
We describe an alternative way to approach the derivation of the EKI update
formulae. We apply Theorem \ref{eq:extra1} with the specific structure
arising from the dynamical system used in EKI. To this end we define
\begin{equation}
\label{eq:z2}
{\mathsf{I}}_{n}(b) := \frac{1}{2} \Big\| y_{j+1}^{(n)} - G({u}_{j}^{(n)})
-\frac{1}{N}\sum_{n=1}^N b_n \bigl(G({u}_{j}^{(n)})-\bar{G}_j\bigr)\Big\| ^{2}_{\Gamma}+\frac{1}{2N}\sum_{n=1}^N b_n^2.
\end{equation}
Once this quadratic form has been minimized with respect to $b$ then the
upate formula  \eqref{eq:deenz3} gives
\begin{align*}
u_{j+1}^{(n)}&={u}_{j}^{(n)}+\frac{1}{N}\sum_{n=1}^N b_n\bigl({u}^{(n)}_{j}-\bar{u}_{j}\bigr),\\
w_{j+1}^{(n)}&=G({u}_{j}^{(n)})+\frac{1}{N}\sum_{n=1}^N b_n\bigl(G({u}^{(n)}_{j})-\bar{G}_{j}\bigr).
\end{align*}
(Note that the vector $b$ depends on $n$; we have supressed this dependence
for notational convenience.)
Theorem \ref{t:cite} is an immediate consequence of this structure.
\qed
\end{remark}

\subsection{Linking Ensemble Kalman Inversion and SMC}
Here we link the two preceding sections.
In the first subsection
we describe how EKI may be linked with
SMC through a particular scaling of $\Gamma$ with $h$. We then
take the limit $h \to 0$ and obtain a system or ordinary
differential equations.
In the second subsection we study the resulting algorithm in
the case in which $G$ is linear, leading to insight into the
EKI algorithm in the iterated context.

\subsubsection{Continuous Time Limit}

Although we have emphasized the optimization perspective on ensemble methods,
we may think of one-step of ensemble Kalman inversion as approximating
the filtering mapping $\pi_j=\operatorname{\mathbbm{P}}(u_j|Y_j) \mapsto  \pi_{j+1}=
\operatorname{\mathbbm{P}}(u_{j+1}|Y_{j+1});$
this will be a good approximation when the measures are close to Gaussian and
 the number of ensemble members $N$ is large.
In order to link this to the mapping $\pi_{j+1}={\mathcal{A}}\pi_j$ in the
presentation of SMC in the first section of this chapter,
we set $\Gamma=\frac{1}{h}\Gamma_0,$ and we assume unperturbed observations $y_j^{(n)} = y$ for all $j.$
Let ${\mathfrak{u}}_j \in (\mathbbm{R}^\Du)^N$ denote, as in the previous section, the
collection of particles  $\{u^{(n)}_j\}_{n= 1}^N$ at discrete time $j.$
Now define
$$D^0_{mn}(\mathfrak{u}_j):=\Bigl\langle G(u^{(n)}_j)-y,G(u^{(m)}_j)-\bar{G}_j \Bigr\rangle_{\Gamma_0}.$$
Note that then, to leading order in $h \ll 1$,
\begin{align*}
D_{mn}(\mathfrak{u}_j) &= -\Bigl\langle (C^{ww}_{j+1}+\Gamma)^{-1}(y -G(u^{(n)}_{j})),G(u^{(m)}_{j})-\bar{G}_j \Bigr\rangle \\
&\approx h\langle G(u_j^{(n)})-y,G(u^{(m)})-\bar{G}_j \rangle_{\Gamma_0} \\
&= hD^0_{mn}(\mathfrak{u}_j).
\end{align*}
It follows that, also to leading order in $h$,
\begin{equation*}
u^{(n)}_{j+1}\approx u^{(n)}_{j}-\frac{h}{N} \sum\limits_{m=1}^N D^0_{mn}(\mathfrak{u}_j)u^{(m)}_{j}.
\end{equation*}
Then letting $h\to 0$ yields
\begin{equation}
\label{eq:has}
\frac{du^{(n)}}{dt}=-\frac{1}{N}\sum\limits_{m=1}^N D^0_{mn}(\mathfrak{u})u^{(m)},
\end{equation}
where, similarly as above, $\mathfrak{u}$ represents the collection of $N$ particles.

\begin{remark}
Notice that equation \eqref{eq:has}
has families of fixed points where: (i) either the particles fit
the data exactly (which corresponds to the left-hand side in the inner product
defining $D^0_{mn}(u)$ being zero);  or (ii) all particles collapse on their
mean value (which corresponds to the right-hand side of the same inner product).
This suggests that the system of ordinary differential equations which
describes the behaviour of ensemble Kalman inversion is driven by
two desirable attributes: matching the data and achieving consensus. \qed
\end{remark}

\subsubsection{Linear Setting}
Now suppose $G(\cdot)$ is a linear map denoted $A\cdot$,
and define
$$\bar{u}=\frac{1}{N}\sum\limits_{n=1}^N u^{(n)}.$$
In this setting we have
\begin{align*}
\frac{du^{(n)}}{dt}&=-\frac{1}{N}\sum\limits_{m=1}^N \bigl\langle Au^{(n)}-y,A(u^{(m)}-\bar{u}) \bigr\rangle_{\Gamma_0}u^{(m)} \\
&=-\frac{1}{N}\sum\limits_{m=1}^N \bigl\langle Au^{(n)}-y,A(u^{(m)}-\bar{u}) \bigr\rangle_{\Gamma_0}(u^{(m)}-\bar{u}) \\
&= -\frac{1}{N}\sum\limits_{m=1}^N \bigl\langle A^T\Gamma_0^{-1}(Au^{(n)}-y),u^{(m)}-\bar{u} \bigr\rangle (u^{(m)}-\bar{u}) \\
&= -\frac{1}{N}\sum\limits_{m=1}^N \bigl(u^{(m)}-\bar{u} \bigr)\otimes \bigl(u^{(m)}-\bar{u}\bigr) \bigl(A^T\Gamma_0^{-1}(Au^{(n)}-y)\bigr).
\end{align*}
If we define
\begin{equation}
C(\mathfrak{u})=\frac{1}{N}\sum\limits_{n=1}^N \bigl(u^{(n)}-\bar{u}\bigr) \otimes \bigl(u^{(n)}-\bar{u}\bigr)
\end{equation}
then we find that, for $n=1, \dots, N$,
\begin{align*}
\frac{du^{(n)}}{dt}&=-C(\mathfrak{u}) A^T\Gamma_0^{-1}(Au^{(n)}-y)\\
&=-C(\mathfrak{u})\nabla \mathsf{L}_0 (u^{(n)}),
\end{align*}
where
$$\mathsf{L}_0(u)=\frac12|y-Au|_{\Gamma_{0}}^2.$$
This corresponds to a gradient descent in the subspace defined by the
initial ensemble. In particular
\begin{align*}
\frac{d}{dt}\mathsf{L} \bigl(u^{(n)}(t)\bigr)&=\Bigl\langle \nabla \mathsf{L} \bigl(u^{(n)}(t)\bigr),\frac{du^{(n)}}{dt}(t) \Bigr\rangle \\
&= - \Bigl|C(u)^{1/2}\nabla \mathsf{L} \bigl(u^{(n)}(t)\bigr) \Bigr|^{2} \leq 0,
\end{align*}
demonstrating that the loss function $\mathsf{L}$ is decreasing along the trajectory
associated to each ensemble member.

We note also that, if we define $e^{(n)}=u^{(n)}-\bar{u}$, then
$$\frac{de^{(n)}}{dt}=-C(\mathfrak{u}) A^T\Gamma_0^{-1}Ae^{(n)}.$$
Because
$$C:=C(\mathfrak{u})=\frac{1}{N}\sum_{n=1}^N e^{(n)} \otimes e^{(n)}$$
it follows that
$$\frac{dC}{dt}=-2CA^T\Gamma_0^{-1}AC.$$
Note further that if $C$ is invertible (requires $N  \ge d$) then the inverse $P$ satisfies
$$\frac{dP}{dt}=2A^T\Gamma_0^{-1}A$$
so that $\|P\| \to \infty$ as $t \to \infty.$ This fact, which can be suitably interpreted
when $C$ is only invertible on a subspace, suggests that the covariance shrinks with time, causing ensemble collapse -- consensus -- while at the same time driving minimization of the loss $\mathsf{L}$ over an appropriate subspace.

\subsection{Discussion and Bibliography}

The idea of using particle filters to sample general distributions,
including those arising in Bayesian inversion, maybe be found in
\cite{del2006sequential}; a recent application to a Bayesian
inverse problem, which demonstrated the potential of the
methodology in that context, is \cite{kantas2014sequential}.
A simple proof of convergence of the method may be found in
\cite{beskos2015sequential}; it is based on the proof described in
\cite{rebeschini2015can} for the standard bootstrap particle filter.

The use of the ensemble Kalman filter for parameter
estimation was introduced in the papers \cite{lorentzen2001underbalanced,anderson2014kalman} in which a physical dynamical model was
appended with trivial dynamics for the parameters in order to estimate them;
the idea was extended to learn an entire field of parameter values
in \cite{naevdal2002near}.
The paper \cite{skjervheim2011ensemble} was the first to do what we do
here, namely to consider all the data at once and a single mapping
of unknown parameters to data. In that paper only one iteration is
used; the papers \cite{chen2012ensemble,emerick2013ensemble}
demonstrated how iteration could be useful.
See also the book \cite{oliver2008inverse} for the use of ensemble
Kalman methods in oil reservoir simulation.

Development of the method
for general inverse problems was undertaken in \cite{iglesias2013ensemble},
and further development of iterative methods is described in
\cite{iglesias2015iterative,iglesias2016regularizing}.
Ensemble inversion was analyzed in
\cite{schillings2017analysis}, including the continuous-time limit
and gradient flow structure described here. The potential for
ensemble inversion in the context of hierarchical Bayesian
methods is demonstrated in \cite{chada2017parameterizations}.
The idea that we explain here, of obtaining an evolution equation for
the covariance which is satisfied exactly by ensemble methods, appears in
the remarkable papers
\cite{reich2011dynamical, bergemann2012ensemble}, and in the other
papers of Bergemann and Reich referenced therein. Their work is at the heart of
the analysis in \cite{schillings2017analysis} which demonstrates
the ensemble collapse and approximation properties of the ensemble
method when applied to linear inverse problems.

\section{\Large{\sffamily{Optimization Perspective }}}

In this chapter we explore the properties of Bayesian inversion
from the perspective of an optimization problem which corresponds
to maximizing the posterior probability, in a sense which we will make
precise. We demonstrate the properties of the point estimator resulting
from this optimization problem, showing its positive and negative
attributes, the latter motivating our work in the following chapter.

\subsection{The Setting}

Once again we work in the inverse problem setting of finding $u\in \mathbbm{R}^\Du$
from $y\in \mathbbm{R}^\Dy$ given by
$$ y = G(u) + \eta$$
 with noise $\eta\sim\nu(\cdot)$ and prior $u\sim\rho,$ as in Assumption \ref{a:jc1}. The posterior $\pi^y(u)$ on $u|y$ is given by Theorem \ref{t:bayes} and has the form
\[
\pi^y (\vct u) = \frac{1}{Z} \nu (\vct y- \vct G(\vct u)) \rho (\vct u).
\]
We may define a loss:
\[
\mathsf{L} (\vct u) = - \log \nu \bigl(\vct y-\vct G(\vct u)\bigr),
\]
and a regularizer
\[
\mathsf{R}(\vct u) = - \log \rho(\vct u).
\]
When added together these two functions of $u$ comprise
an objective function of the form
\[
{\mathsf{J}}(\vct u) = \mathsf{L}(\vct u) + \mathsf{R}(\vct u).
\]
Furthermore
\[
\pi^y (\vct u) = \frac{1}{Z} \nu \bigl(\vct y-\vct G(\vct u)\bigr) \rho (\vct u) \propto e^{-{\mathsf{J}}(\vct u)}.
\]
We see that minimizing the objective function ${\mathsf{J}}(\cdot)$
is equivalent to maximizing the posterior $\pi^y(\cdot)$.
Therefore, recalling Definition \ref{def:map}, the MAP estimator can be rewritten in terms of ${\mathsf{J}}$ as follows:
\begin{align*}
u_{\mbox {\tiny{\rm MAP}}} &= \arg\max_{\vct{u}\in\mathbbm{R}^\Du}\pi^{\vct{y}}(\vct{u}) \\
&= \arg \min_{u \in \mathbbm{R}^\Du} {\mathsf{J}}(u).
\end{align*}

We will provide conditions under which the MAP estimator is attained in Theorem \ref{thm:achievable-obj} and we will give an interpretation of MAP estimators in terms of maximizing the probability of infinitesimal balls in Theorem
\ref{thm:ball-density}. This interpretation allows to generalize the definition of MAP estimators to measures that do not possess a Lebesgue density.

\begin{example}
Consider the Gaussian setting of Assumption \ref{a:sz1}.
Then, since the posterior is Gaussian, its mode agrees with its mean, which is given by $m$ as defined in Theorem \ref{t:sz1}.
\end{example}

\begin{example}
	\label{ex:loss-l2}
	If $\eta = \mathcal{N}(\zerovct ,\mtx \Gamma)$, then
	\(\nu\bigl(\vct y-\vct G(\vct u)\bigr) \propto \exp(- \frac{1}{2} |\vct y-\vct G(\vct u)|_{\mtx \Gamma} ^2)\). So the loss in this case is \(\mathsf{L}(\vct u) = \frac{1}{2} |\vct y-\vct G(\vct u)|_{\mtx \Gamma}^2\), a \(\mtx \Gamma\)-weighted \(\mathsf{L}_2\) loss.
\end{example}
\begin{example}
	\label{ex:reg-l2}
	If we have prior \(\rho(\vct u) = \mathcal{N}(\zerovct ,\mtx \hat{C})\), then
ignoring $u-$independent normalization factors, which appear as constant
shifts in ${\mathsf{J}}(\cdot)$, we may take the regularizer as \(\mathsf{R}(\vct u) = \frac{1}{2} |\vct u|_{\mtx \hat{C}}^2\). In particular, if \(\mtx \hat{C} = \lambda^{-1} \Id \), then \(\mathsf{R}(\vct u) = \frac{\lambda}{2} |\vct u|^2\), an \(\mathsf{L}_2\) regularizer.
\end{example}

If we combine Example \ref{ex:loss-l2} and Example \ref{ex:reg-l2}, we obtain a canonical objective function
$${\mathsf{J}}(\vct u) = \frac{1}{2} |\vct y-\vct G(\vct u)|_{\mtx \Gamma}^2 + \frac{\lambda}{2} |\vct  u|^2.$$
To connect with future discussions, here \(\lambda\) corresponds to prior precision, and may be learned from data, such as in hierarchical methods.

\begin{example}
	\label{ex:reg-l1}
	As an alternative to the \(\mathsf{L}_2\) regularizer,
consider \(\vct u=(u_1,\ldots,u_d)\) with \(u_i\) having prior
distribution i.i.d. Laplace. Then \(\rho(\vct u) \propto \exp(-\lambda \sum_{i=1}^d |u_i|) = \exp(-\lambda |\vct u|_1)\). In this case
\(\mathsf{R}( \vct u ) = \lambda |\vct u|_1\), an \(\mathsf{L}_1\) regularizer.
If we combine this prior with the weighted \(\mathsf{L}_2\) loss above
then we have objective function $${\mathsf{J}}(\vct u) = \frac{1}{2} |\vct y - \vct G ( \vct u ) |_{\mtx \Gamma}^2 + \lambda |\vct u |_1.$$
Even though this objective function promotes sparse solutions, samples
from the underlying posterior distribution are not typically sparse.
\end{example}

\subsection{Theory}

For any optimization problem for an objective function with a finite
infimum, it is of interest to determine whether the infinimum is attained. We have the following result.

		\begin{theorem}[Attainable MAP Estimator]
			\label{thm:achievable-obj}
			Assume that ${\mathsf{J}}$ is non-negative, continuous and that ${\mathsf{J}}(u) \to \infty$ as $|u| \to \infty. $
Then ${\mathsf{J}}$ attains it infimum. Therefore,
the MAP estimator of $u$ based on the posterior $\pi^y(u) \propto \exp\bigl(-{\mathsf{J}}(u)\bigr)$ is attained.
		\end{theorem}
		\begin{proof}
			By the assumed growth and non-negativity of ${\mathsf{J}}$, there is $R$ such that $\inf_{u\in \mathbbm{R}^\Du} {\mathsf{J}}(u) = \inf_{u\in B(0,R)}$ where $B(0,R)$ denotes the closed ball of radius $R$ around the origin. Since ${\mathsf{J}}$ is assumed to be continuous, its infimum over $B(0,R)$ is attained and the proof is complete.
		\end{proof}

\begin{remark}
Suppose that
\begin{enumerate}
				\item \(\vct G \in C(\mathbbm{R}^\Du, \mathbbm{R}^\Dy)\), i.e. \(\vct G\) is a continuous function;
				\item the objective function \({\mathsf{J}}(\vct u)\) has \(\mathsf{L}_2\) loss as defined in Example \ref{ex:loss-l2} and \(\mathsf{L}_p\) regularizer \(\mathsf{R}(\vct u) = \frac{\lambda}{p} |\vct u|_p^p\), \(p \in (0,\infty)\).
			\end{enumerate}
Then the assumptions on ${\mathsf{J}}$ in Theorem \ref{thm:achievable-obj} are satisfied.  This shows that if \(\vct G\) is continuous, the infinimum of ${\mathsf{J}}$ defined with \(\mathsf{L}_2\) loss and \(\mathsf{L}_p\) regularizer is attained at the MAP estimator of the  corresponding Bayesian problem with posterior pdf proportional to $\exp\bigl(-{\mathsf{J}}(u)\bigr)$.
\end{remark}

\begin{remark}
Notice that the assumption that ${\mathsf{J}}(u) \to \infty$ is not restrictive: this condition needs to hold in order to be able to normalize $\pi^y (u)\propto \exp\bigl(-{\mathsf{J}}(u)\bigr)$ into a pdf, which is implicitly assumed in the second part of the theorem statement.
\end{remark}

Intuitively the MAP estimator maximizes posterior probability. We make this
precise in the following theorem which links the objective function ${\mathsf{J}}(\cdot)$
to small ball probabilities.

		\begin{theorem}[Objective Function and Posterior Probability]
			\label{thm:ball-density}
			Making the same assumptions as in Theorem \ref{thm:achievable-obj}, let
			\[
			\alpha(\vct u,\delta) := \int_{\vct v \in B(\vct u, \delta)} \pi^y(\vct v) d\vct v = \operatorname{\mathbbm{P}}^{\pi^y}\bigl(B(\vct u,\delta)\bigr),
			\]
			be the posterior probability of a ball with radius \(\delta\)
centered at \(u\). Then, for all \(\vct u_1, \vct u_2 \in \mathbbm{R}^\Du\), we have
			\[
			\lim_{\delta \rightarrow 0} \frac{\alpha(\vct u_1, \delta)}{\alpha(\vct u_2, \delta)} = e^{{\mathsf{J}}(\vct u_2) - {\mathsf{J}}(\vct u_1)}.
			\]
		\end{theorem}
		\begin{proof}
Let $u_1, u_2 \in \mathbbm{R}^\Du$ and let $\epsilon>0.$ By continuity of ${\mathsf{J}}$ we have that, for all $\delta$ sufficiently small,
\begin{align*}
e^{-{\mathsf{J}}(u_1) - \epsilon} \le e^{-{\mathsf{J}}(v)} &\le e^{-{\mathsf{J}}(u_1) + \epsilon} \quad \text{for all} \,\, v \in B(u_1, \delta), \\
e^{-{\mathsf{J}}(u_2) - \epsilon} \le e^{-{\mathsf{J}}(v)} &\le e^{-{\mathsf{J}}(u_2) + \epsilon} \quad \text{for all}  \,\, v \in B(u_2, \delta).
\end{align*}
Therefore, for all $\delta$ sufficiently small,
\begin{align*}
&B_\delta e^{-{\mathsf{J}}(\vct u_1) - \epsilon}
			\leq
			\int_{v \in B(u_1,\delta)} e^{-{\mathsf{J}}(\vct v)} d\vct v \leq
			 B_\delta e^{-{\mathsf{J}}(\vct u_1) + \epsilon}, \\
&B_\delta e^{-{\mathsf{J}}(\vct u_2) - \epsilon}
			\leq
			\int_{v \in B(u_2,\delta)} e^{-{\mathsf{J}}(\vct v)} d\vct v \leq
			 B_\delta e^{-{\mathsf{J}}(\vct u_2) + \epsilon},
\end{align*}
		where \(B_\delta \) is the Lebesgue measure of a ball with radius \(\delta\). Taking the ratio of $\alpha$'s and using the above bounds we obtain that, for all $\delta$ sufficiently small,
		\[
			e^{{\mathsf{J}}(\vct u_2) - {\mathsf{J}}(\vct u_1) -2\epsilon} \leq \frac{\alpha(\vct u_1, \delta) }{\alpha(\vct u_2, \delta)} \leq  e^{{\mathsf{J}}(\vct u_2) - {\mathsf{J}} ( \vct u_1) + 2\epsilon}.
			\]
			Since $\epsilon$ was arbitrary the desired result follows.

		\end{proof}

\begin{remark}
This theorem shows that maximizing the probability of an
infinitesimally small ball is the same as minimizing the objective
function ${\mathsf{J}}(\cdot).$ This is obvious in finite dimensions, but
the proof above generalizes beyond measures
which possess a Lebesgue density, and may be used in infinite
dimensions.
\end{remark}

\subsection{Examples}

By means of examples we now probe the question of whether or not the MAP
estimator captures useful information about the posterior distribution.

\begin{example}
	If the posterior is single-peaked, such as a Gaussian or a Laplace distribution, as shown in Figure \ref{fig:gaussian-laplace}, the MAP estimator, i.e. minimizer of the objective functions, reasonably summarizes the most likely value of the unknown parameter.
\end{example}

\begin{figure}
  \centering
  \includegraphics[width=0.7\columnwidth]{figures/lecture4_gaussian-laplace.png}
  \caption{Posterior (left) and objective function (right) for \(\mathcal{N}(0,1)\) posterior (orange) and \(\mathrm{Laplace}(0,1)\) posterior (blue). }
  \label{fig:gaussian-laplace}
\end{figure}

We next consider several examples where a point estimator ---or a \(\delta\)-radius ball with \(\delta \rightarrow 0\)--- fails to summarize adequately the posterior distribution.
\begin{example}
	If the posterior is rather unevenly distributed, such as a slab-and-spike distribution, as shown in Figure~\ref{fig:slab-n-spike}, then it is
less clear that the MAP estimator usefully summarizes the posterior. For example, for the case in Figure \ref{fig:slab-n-spike} we may want the solution output of our Bayesian problem to be a weighted average of two Gaussian distributions, or two point estimators each with a separate
mean located at one of the two minima of the objective functions,
and weight describing the probability mass associated with each of
those two points.
\end{example}

\begin{figure}
	\centering
	\includegraphics[width=0.9\columnwidth]{figures/lecture4_slab-n-spike.png}
	\caption{Posterior (left) and objective function (right) for a posterior that is a sum of two Gaussian distributions, \(\mathcal{N}(0,0.1^2)\) with probability \(0.1\) and \(\mathcal{N}(10,3^2)\) with probability \(0.9\).}
	\label{fig:slab-n-spike}
\end{figure}

\begin{example}
	In addition to a multiple-peak posterior, there are cases where the objective function and the associated posterior density are simply very rough. In these cases, the small-scale roughness should be ignored, while the large-scale variation should be captured. For example in Figure \ref{fig:rough}, the objective function is very rough, and has a unique minimizer at a point far from \(0\). However, it also has a larger pattern: it tends to be smaller around \(0\), while larger away from \(0\). The MAP estimator cannot capture this large scale pattern, as it is found by minimizing the objective function. It is arguably the
case that \(u=0\) is a better point estimate. An alternative way to interpret this phenomena is that there is a natural ``temperature'' to this problem, in the sense that variations lower than this temperature could be viewed as random noise that do not capture meaningful information.
\end{example}
\begin{figure}
	\centering
	\includegraphics[width=0.9\columnwidth]{figures/lecture4_rough.png}
	\caption{Posterior (left) and objective function (right) from an objective function that is very rough in the small scale, but contains a regular patter on the larger scale. This specific example is generated by white noise summed with a quadratic function for the objective function, and the posterior is computed from the objective function.}
	\label{fig:rough}
\end{figure}

The preceding examples suggest that multi-peak distributions, or multi-minimum
objective functions, can cause problems for MAP estimation as a point estimator.
Next we illustrate that if the dimension \(d\) of the parameter \(\vct u \in \mathbbm{R}^\Du\) is high, then a single point estimator, even if a MAP estimator, is typically not a good summary of the posterior.
\begin{figure}[ht!]
	\centering
	\includegraphics[width=0.7\columnwidth]{figures/lecture4_highdim.png}
	\caption{Empirical density of \(\ell_2\) norm of \(\mathcal{N}(\zerovct,\Id)\) random vectors for various dimension: \(d=1\) (blue), \(d=5\) (orange), \(d=10\) (green), \(d=50\) (red), and \(d=100\) (purple). The empirical density is obtained from \(10000\) samples for each distribution.}
	\label{fig:highdim}
\end{figure}
\begin{example}
	We consider what is the ``typical size'' of a vector \(\vct u\) drawn
from the standard Gaussian distribution \(\mathcal{N}(\zerovct, \Id)\), as the dimension
increases. In Figure \ref{fig:highdim} we display the empirical density of the norm of such random vectors. We can see that at low dimensions, such as when \(d=1\), obtaining a value close to the mode \(\vct u=0\) is highly likely. In higher dimensions, however,
the probability for a vector from this distribution to have a small $\ell_2-$norm becomes increasingly small as $d$ grows.
 For example, let us consider the probability for the norm to be less than \(5\). Then \(\operatorname{\mathbbm{P}} (|\vct u| <5)\) is \(0.99999943\) when \(d=1\), \(0.99986\) when \(d=5\), \(0.99465\) when \(d=10\), \(0.001192\) when \(d=50\), and \(1.135 \times 10^{-15}\) when \(d=100\). So we see that, as the dimension increases, with probability close to \(1\) a sample from the posterior would
have a norm far from \(0\). Indeed, for \(d=1000\), the \(5\)th and \(95\)th percentiles are respectively \(30.3464\) and \(32.7823\). This means when \(d=1000\), we most likely will find a vector with size around \(31\), not \(0\).
Another way to see this is that, since the components $u_i$ of $u$
are i.i.d. standard unit Gaussians we have that, by the strong law of large
numbers,
$$\frac{1}{d}\sum_{i=1}^d u_i^2 \to 1$$
as $d \to \infty$ almost surely. Thus, with high probability, the
$\ell_2-$norm is of size $\sqrt{d}.$
This example suggests that in high dimension, a point estimatormay not capture enough information about the density.
\end{example}

The preceding examples demonstrate that MAP estimators should be treated
with caution as they may not capture the desired posterior information in
many cases. This motivates the study of alternative ways ---beyond MAP estimators--- to capture information from the posterior distribution. One such approach is to fit one or several Gaussian distributions to the posterior by minimizing an appropriate distance-like measure between distributions. This will be discussed in the next chapter.

\subsection{Discussion and Bibliography}

The optimization perspective on inversion predates the development of
the Bayesian approach as a computational tool, because it is typically
far cheaper to implement. The subject of classical regularization
techniques for inversion is discussed in \cite{engl1996regularization}.
The concept of MAP estimators, which links probability to
optimization, is discussed in the books
\cite{kaipio2006statistical,tarantola2005inverse}
in the finite dimensional setting.
The paper \cite{dashti2013map} studies this connection precisely: it
defines the MAP estimator for infinite dimensional Bayesian inverse problems, and the corresponding variational formulation, in a Gaussian setting.
The paper \cite{helin2015maximum} studies related ideas, but in the
non-Gaussian setting, and \cite{agapiou2017sparsity} generalizes
the variational formulation of MAP estimators to non-Gaussian priors that
are sparsity promoting.
The paper \cite{worthen2014towards} shows an application of optimization
based inversion to a large-scale geophysical application.

\section{\Large{\sffamily{Bayesian Inverse Problems and Well-Posedness}}}
In this chapter we introduce the Bayesian approach to inverse problems.  We will show that the Bayesian formulation leads to a form of well-posedness: small perturbations of the forward model or the observed data translate into small perturbations of the Bayesian solution. Well-posedness will be established in Hellinger distance. The total variation and Hellinger distances between probability densities play an important role in the analysis of Bayesian methodology. We introduce both distances in this chapter, as well as some characterizations and bounds between them that will be used throughout this book.

\subsection{Formulation of Bayesian Inverse Problems}
We consider the following setting. We let $G:  \mathbbm{R}^\Du \to \mathbbm{R}^\Dy$ be a forward model and aim to recover an unknown parameter $u \in \mathbbm{R}^\Du$ from data $y \in \mathbbm{R}^\Dy$ given by
\begin{equation}
 y = G(u) + \eta,
\label{eq:jc0}
\end{equation}
where $\eta \in \mathbbm{R}^\Dy$ represents observation noise.
We view $(u, y ) \in \mathbbm{R}^\Du \times \mathbbm{R}^\Dy$ as a random variable, whose distribution is specified by means of the following assumption.
\begin{assumption}\label{a:jc1}
We make the following probabilistic assumptions:
 \begin{itemize}
 \item $u \sim \rho(u), u \in \mathbbm{R}^\Du $.
 \item $\eta \sim \nu(\eta), \eta \in \mathbbm{R}^\Dy$.
 \item $ u$ and $\eta$ are independent, written $u \perp \eta.$
 \end{itemize}
\end{assumption}
Here $\rho$ and $\nu$ describe the (Lebesgue) probability density
functions (pdfs) of the random variables $u$ and $\eta$ respectively.
Then $\rho(u)$ is called the \emph{prior} pdf, $y |  u \sim \nu\bigl(y - G(u)\bigr)$ for each fixed $u\in \mathbbm{R}^\Du$ determines the \emph{likelihood} function.  In this
probabilistic perspective, the solution to the inverse problem is the conditional distribution of $u$ given $y$, which is called the \emph{posterior} distribution and will be denoted by $u | y\sim \pi^y(u).$  From the posterior pdf one can infer parameter values that are consistent with both the data and the prior pdf. The posterior also contains information on the uncertainty remaining in the parameter recovery: for instance, large posterior covariance may indicate that the data contains insufficient information to recover the input parameter.

 \subsection{Formula for Posterior pdf: Bayes Theorem}
Bayes theorem is a bridge connecting the prior, the likelihood and the posterior.
 \begin{theorem}[Bayes theorem]
\label{t:bayes}
 Let Assumption \ref{a:jc1} hold and assume that
\[
 Z = Z(y) :=\int_{\mathbbm{R}^\Du} \nu\bigl( y - G(u)\bigr)\rho(u) du  >0.\]
 Then $u | y\sim \pi^y(u),$ where
\begin{equation}\label{eq:bayesformula}
 \pi^y(u) = \frac{1}{Z} \nu\bigl(y - G(u)\bigr)\rho(u).
\end{equation}
 \end{theorem}
 \begin{proof}
 Denote by $\operatorname{\mathbbm{P}} (\cdot)$ the pdf of a random variable and by $\operatorname{\mathbbm{P}}(\cdot | \cdot )$ its conditional conditional pdf.
We have
 \[
 \begin{aligned}
 \operatorname{\mathbbm{P}}(u , y) &=  \operatorname{\mathbbm{P}} (u | y) \operatorname{\mathbbm{P}}(y),  \textrm{ if } \operatorname{\mathbbm{P}}(y) > 0 ,\\
 \operatorname{\mathbbm{P}}(u, y) &=  \operatorname{\mathbbm{P}} (y | u ) \operatorname{\mathbbm{P}}(u),  \textrm{ if } \operatorname{\mathbbm{P}}(u) > 0 .\\
\end{aligned}
 \]
 Note that the marginal pdf on $y$ is given by
 \[
 \operatorname{\mathbbm{P}}(y) = \int_{\mathbbm{R}^\Du} \operatorname{\mathbbm{P}}( u, y) du,
 \]
and similarly for $\operatorname{\mathbbm{P}}(u).$
 Assume $\operatorname{\mathbbm{P}}(y) >0 $. Then
 \begin{equation}
\label{eq:jc1}
 \operatorname{\mathbbm{P}}(u | y) = \frac{1}{\operatorname{\mathbbm{P}}(y) } \operatorname{\mathbbm{P}}(y | u) \operatorname{\mathbbm{P}}(u)   = \frac{1}{Z} \nu\bigl(y - G(u )\bigr) \rho(u)
 \end{equation}
 for both $\operatorname{\mathbbm{P}}(u) > 0$ and $\operatorname{\mathbbm{P}}(u) = 0$. Here we remark that
 \[
\operatorname{\mathbbm{P}}(y)=Z=\int_{\mathbbm{R}^\Du} Z \operatorname{\mathbbm{P}}(u | y) du = \int_{\mathbbm{R}^\Du} \nu\bigl(y - G(u)\bigr) \rho(u) du>0.
 \]
Thus the assumption that $\operatorname{\mathbbm{P}}(y) >0 $ is justified and the desired result
follows from equation \eqref{eq:jc1}.
\end{proof}

We will often denote the likelihood function by $g(u) := \nu \bigl(y - G(u)\bigr)$; we then write
$$ \pi(u) = \frac{1}{Z} g(u) \rho(u),$$
omitting the data $y$ in both the likelihood function and the posterior pdf.

\begin{remark}
The proof of Theorem \ref{t:bayes} shows that in order to apply Bayes formula \eqref{eq:bayesformula} one needs to guarantee that the normalizing constant $\operatorname{\mathbbm{P}}(y) = Z$ is positive. In other words, the marginal density of the observed data $y$ needs to be positive, i.e. the observed data needs to be consistent with the probabilistic assumptions \eqref{a:jc1}. From now on it will be assumed that $\operatorname{\mathbbm{P}}(y) = Z >0$ without further notice.
\end{remark}

The posterior distribution $\pi^y(u)$ contains all the knowledge on the parameter $u$ available in the prior and the data. In applications it is often useful, however, to summarize the posterior distribution through a few numerical values or through parameter regions of prescribed posterior probability (known as credible intervals). Summarizing the posterior is particularly important if the parameter is high dimensional, since then visualizing the posterior or detecting regions of high posterior probability is nontrivial. Two natural numerical summaries are the posterior mean and the posterior mode.
\begin{definition}\label{def:map}
The {\em posterior mean estimator} of $u$ given data $y$ is the mean of the posterior distribution:
$$u_{\mbox {\tiny{\rm PM}}} = \int_{\mathbbm{R}^\Du} u \pi^y(u) \, du.$$
	The {\em maximum a posteriori (MAP) estimator} of $u$ given data $y$ is the mode of the
posterior distribution $\pi^{\vct{y}}(\vct{u})$,  defined as
	$$u_{\mbox {\tiny{\rm MAP}}} =\arg\max_{\vct{u}\in\mathbbm{R}^\Du}\pi^{\vct{y}}(\vct{u}).$$
\end{definition}

The posterior mean and the MAP estimators already suggest the importance of computing maxima and integrals in the practical implementation of Bayesian formulations of inverse problems and data assimilation. For this reason, optimization and sampling will play an important role in this book.  An alternative way to make Bayesian formulations tractable is to approximate the posterior by a simple distribution, often a Gaussian or a combination of Dirac masses. An optimization perspective for inverse problems and data assimilation will be studied in Chapters 3 and 7, respectively, and the use of Gaussian approximations will be discussed in Chapters 4 and 10; Dirac approximations constructed via sampling will be studied in Chapters 5 and 6 (inverse problems) and in Chapters 11 and 12 (data assimilation).

We next consider two simple examples of a direct application of Bayes formula.

\begin{example}
Let $d=k=1$, $\eta \sim \nu = \mathcal{N}(0, \gamma^2)$ and
\[
\rho(u) = \begin{cases}
\frac{1}{2} ,  &  u\in (-1,1), \\
0 ,  &  u\in (-1,1 )^c. \\
\end{cases}
\]
Suppose that the observation is generated by $y = u + \eta$. Using Bayes Theorem \ref{t:bayes}, we derive the posterior
\[
\pi^y(u) =
 \begin{cases}
 \frac{1}{2Z} \exp(  -\frac{1}{2\gamma^2}|y- u |^2   ),  &  u \in (-1,1) , \\
 0 ,  & u \in (-1,1)^c , \\
 \end{cases}
\]
where $Z$ is a normalizing constant ensuring $\int_{\mathbbm{R}} \pi^y(u) du = 1$. The support of $\pi^y$, i.e. $(-1,1)$,  agrees with the support of the prior $\rho$. Now we find
the MAP estimator.
From the explicit formula for $\pi^y$, we have
\[
u_{\mbox {\tiny{\rm MAP}}}  = \arg\max_{u \in \mathbbm{R}} \pi^y(u) = \begin{cases}
y &  \textrm{if } y \in (-1, 1),  \\
-1&  \textrm{if } y  \leq - 1 , \\
1 & \textrm{if } y\geq 1 . \\
\end{cases}
\]
In this example, the prior on $u$ is supported
on $(-1,1)$ and the posterior on $u|y$ is supported on $(-1,1)$.
If the data lies in $(-1,1)$ then the MAP estimator is the data itself; otherwise it is the extremal point of
the prior support which matches the sign of the data. The posterior mean is
$$u_{\mbox {\tiny{\rm PM}}} = \frac{1}{2Z} \int_{-1}^1 u \exp\Bigl( -\frac{1}{2\gamma^2} |y - u|^2 \Bigr) \, du,$$
which may be  approximated for instance by using the sampling methods described in Chapters 5 and 6.

\end{example}

The following example illustrates once again the application of Bayes formula, and shows that the posterior may be concentrated near a low-dimensional manifold of the input parameter space $\mathbbm{R}^d$. In such a case it is important to understand the geometry of the support of the posterior density, which cannot be captured by point estimation or Gaussian approximations.

\begin{example}
Let $d =2, k=1, \rho \in C(\mathbbm{R}^2, \mathbbm{R}^+), 0 < \rho(u) \leq \rho_{\max} < +\infty$, for all  $u \in \mathbbm{R}^2$ and
 \[
 y = G(u) + \eta = u_1^2 + u_2^2 + \eta , \quad \eta \sim \nu = \mathcal{N}(0, \gamma^2),
\quad 0< \gamma \ll 1.
 \]
Assume that $y > 0$. Using Bayes theorem, we obtain
the posterior distribution
\[
\pi^y(u) = \frac{1}{Z} \exp\left(  - \frac{1}{2 \gamma^2}| u_1^2 + u_2^2 - y|^2   \right) \rho(u).
\]
We now show that the posterior concentrates on a manifold: the circumference  $\{ u \in \mathbbm{R}^2: u_1^2 + u_2^2=y\}.$
Denote by $A^{\pm}:=\{ u \in \mathbbm{R}^2 :  | u_1^2 + u_2^2 - y |^2 \leq \gamma^{2 \pm \delta}   \}$, for some $\delta \in (0,2)$, and let $\rho_{\min} = \inf_{u\in B} \rho(u)$, where $B$ is the closed ball of radius $2\sqrt{y}$ centered at the origin. Since $\rho(u)$ is positive and continuous and $B$ is compact, $\rho_{\min} > 0$. Let $u^+ \in A^+ \subset B, u^- \in (A^-)^c$. Taking the small noise limit yields
 \[
 \frac{\pi^y(u^+)}{ \pi^y(u^-) } \geq \exp\left( -\frac{1}{2} \gamma^{\delta} + \frac{1}{2} \gamma^{-\delta} \right)  \frac{\rho_{\min}}{\rho_{\max}}\to \infty, \ \textrm{as } \gamma \to 0^+.
 \]
Therefore, conditional on $y>0$, the posterior $\pi^y$ concentrates, as $\gamma \to 0,$ on the circumference with radius $\sqrt{y}$.
  \begin{figure}[h]
   \centering
   \includegraphics[width =0.45\textwidth, height = 0.4\textwidth]{figures/lecture1_cartoon.png}
   \caption{The posterior measure concentrates on a circumference with radius $\sqrt{y}$. Here, the blue shadow area is $A^+$ and the green shadow area is $(A^-)^c$. }
 \end{figure}
\end{example}

\subsection{Well-posedness of Bayesian Inverse Problems}

In this section we show that the Bayesian formulation of inverse problems
leads to a form of well-posedness.
More precisely, we study the sensitivity of the posterior pdf to perturbations of the forward model $G.$  In many inverse problems the ideal forward model $G$ is not accessible but can be approximated by some computable $G_\delta;$ consequently $\pi^y$ is replaced by $\pi_\delta^y$. An example that is often found in applications, to which the theory contained herein may be generalized,
is when  $G$ is an operator acting on an infinite-dimensional space
which is approximated, for the purposes of computation, by some finite-dimensional operator $G_\delta$. We seek to prove that, under certain assumptions, the small difference between $G$ and $G_\delta$ (forward error) leads to
similarly small difference between $\pi^y$ and $\pi^y_\delta$
(inverse error):

\vspace{0.1in}
\noindent{\bf Meta Theorem: Well-posedness}\\
       \[|G-G_\delta|=O(\delta)\ \Longrightarrow \ d(\pi^y,\pi_\delta^y)=O(\delta),\]
{\em for small enough $\delta>0$ and some metric $d(\cdot,\cdot)$ on probability densities.}
\vspace{0.1in}

This result will be formalized in Theorem \ref{thm1} below, which  shows that the $O(\delta)$-convergence of $\pi_\delta^y$ with respect to some distance $d(\cdot,\cdot)$ can be guaranteed under certain assumptions on the likelihood. We will conclude the chapter by showing an example where these assumptions hold true. In order to discuss these issues we will need to
introduce metrics on probability densities.

\subsubsection{Metrics on Probability Densities}
Here we introduce the total variation and the Hellinger distance, both of which have been used to show well-posedness results. In this chapter we will use the Hellinger distance to establish well-posedness of Bayesian inverse problems, and in Chapter 7 we employ the total variation distance to establish well-posedness of Bayesian formulations of filtering and smoothing in data assimilation.

\begin{definition}
The \emph{total variation distance} between two probability densities $\pi,\pi'$ is defined by
\[d_{\mbox {\tiny{\rm TV}}} (\pi,\pi')\coloneqq\frac{1}{2}\int|\pi(u)-\pi'(u)|du=\frac{1}{2}\|\pi-\pi'\|_{L^1}.\]
The \emph{Hellinger distance} between two probability densities $\pi,\pi'$ is defined by
\[d_{\mbox {\tiny{\rm H}}}(\pi,\pi')\coloneqq\Big(\frac{1}{2}\int|\sqrt{\pi(u)}-\sqrt{\pi'(u)}|^2du\Big)^{1/2}=\frac{1}{\sqrt{2}}\|\sqrt{\pi}-\sqrt{\pi'}\|_{L^2}.\]
\end{definition}

In the rest of this subsection we will establish bounds between the Hellinger and total variation distance, and show how both distances can be used to bound the difference of expected values computed with two different densities; these results will be used in subsequent chapters.  Before doing so, the next lemma motivates our choice of normalization constant $1/2$ for total variation distance and $1/\sqrt{2}$ for Hellinger distance: they are chosen so that the maximum possible distance between two densities is one.

      \begin{lemma} \label{l:dh1}
For any probability densities $\pi,\pi'$,
       \[0\leq d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')\leq 1,\quad 0\leq d_{\mbox {\tiny{\rm H}}}(\pi,\pi')\leq 1.\]
      \end{lemma}
      \begin{proof} The lower bounds follow immediately from the
definitions. We only need to prove the upper bounds:
      \[d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')=\frac{1}{2}\int|\pi(u)-\pi'(u)|du\leq \frac{1}{2}\int\pi(u)du+\frac{1}{2}\int\pi'(u)du=1, \]
      \begin{align*}
      d_{\mbox {\tiny{\rm H}}}(\pi,\pi')=&\ \biggl(\frac{1}{2}\int \Bigl|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Bigr|^2du\biggr)^{1/2}\\
      =&\ \bigg(\frac{1}{2}\int\Big(\pi(u)+\pi'(u)-2\sqrt{\pi(u)\pi'(u)}\,\, \Big)du \bigg)^{1/2}\\
      \leq&\ \Big(\frac{1}{2}\int\big(\pi(u)+\pi'(u)\big)du\Big)^{1/2}\\
      =&\ 1.\end{align*}
      \end{proof}
   The proofs also show that $\pi$ and $\pi'$ have total variation and Hellinger distance equal to one if and only if they have disjoint supports, that is, if $\int \pi(u)\pi'(u) du=0.$

The following result gives bounds between total variation and Hellinger distance.

      \begin{lemma} \label{l:dh2} For any probability densities $\pi,\pi'$,
       \[\frac{1}{\sqrt{2}}d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')\leq d_{\mbox {\tiny{\rm H}}}(\pi,\pi')\leq \sqrt{d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')}.\]
      \end{lemma}
      \begin{proof} Using the Cauchy–Schwartz inequality
      \begin{align*}
      d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')=&\ \frac{1}{2}\int\Big|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Big|\Big|\sqrt{\pi(u)}+\sqrt{\pi'(u)}\Big|du\\
      \leq&\ \biggl(\frac{1}{2}\int\Big|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Big|^2du\biggr)^{1/2}\biggl(\frac{1}{2}\int\Big|\sqrt{\pi(u)}+\sqrt{\pi'(u)}\Big|^2du\biggr)^{1/2}\\
      \leq&\ d_{\mbox {\tiny{\rm H}}}(\pi,\pi')\biggl(\frac{1}{2}\int\big(2\pi(u)+2\pi'(u)\big)du\biggr)^{1/2}\\
      =&\ \sqrt{2}d_{\mbox {\tiny{\rm H}}}(\pi,\pi').
      \end{align*}
      Notice that $|\sqrt{\pi(u)}-\sqrt{\pi'(u)}|\leq |\sqrt{\pi(u)}+\sqrt{\pi'(u)}|$ since $\sqrt{\pi(u)},\sqrt{\pi'(u)}\geq0$. Thus we have
      \begin{align*}
      d_{\mbox {\tiny{\rm H}}}(\pi,\pi')=&\ \biggl(\frac{1}{2}\int\Big|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Big|^2du\biggr)^{1/2}\\
      \leq&\ \biggl(\frac{1}{2}\int\Big|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Big|\Big|\sqrt{\pi(u)}+\sqrt{\pi'(u)}\Big|du\biggr)^{1/2}\\
      \leq&\ \Big(\frac{1}{2}\int\big|\pi(u)-\pi'(u)\big|du\Big)^{1/2}\\
      =&\ \sqrt{d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')}.
      \end{align*}
      \end{proof}

The following two lemmas show that if two densities are close in total variation or in Hellinger distance, expectations computed with respect to both densities are also close. In addition, the following lemma also provides a useful characterization of the total variation distance which will be used repeatedly throughout this book.

      \begin{lemma} \label{lemmatv} Let $f$ be a function such that $\sup_{u\in \mathbbm{R}^\Du}|f(u)| =: |f|_\infty<\infty$, then
      \[\big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\big|\leq 2|f|_\infty d_{\mbox {\tiny{\rm TV}}}(\pi,\pi').\]
Moreover, the following variational characterization of the total variation distance holds:
\begin{equation}
d_{\mbox {\tiny{\rm TV}}}(\pi, \pi') = \frac12 \sup_{|f|_\infty<1} \big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\big|.
\end{equation}
      \end{lemma}
      \begin{proof}
      For the first part of the lemma, note that
     	\begin{align*}
     	\Big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\Big|=&\ \Big|\int_{\mathbbm{R}^\Du}f(u)\bigl(\pi(u)-\pi'(u)\bigr)du\Big|\\
     	\leq&\ 2|f|_\infty\cdot\frac{1}{2}\int_{\mathbbm{R}^\Du}|\pi(u)-\pi'(u)|du\\
     	=&\ 2|f|_\inftyd_{\mbox {\tiny{\rm TV}}}(\pi,\pi').
     	\end{align*}
     	This in particular shows that, for any $f$ with $|f|_\infty =1,$
     	 $$d_{\mbox {\tiny{\rm TV}}}(\pi, \pi') \ge \frac12 \big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\big|.$$
     Our goal now is to show a choice of $f$ with $|f|_\infty =1$ that achieves equality.	Define $f(u):= \text{sign} \Bigl( \pi(u)- \pi'(u) \Bigr)$, so that $f(u) \Bigl( \pi(u)- \pi'(u) \Bigr) = |\pi(u) - \pi'(u)|.$ Then, $|f|_\infty =1$ and
     	\begin{align*}
     	d_{\mbox {\tiny{\rm TV}}}(\pi, \pi') &= \frac12 \int_{\mathbbm{R}^\Du} |\pi(u) - \pi'(u)| \, du \\
     	&= \frac12 \int_{\mathbbm{R}^\Du} f(u) \Bigl( \pi(u) - \pi'(u) \Bigr) \, du \\
     	& = \frac12 \Big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\Big|.
     	\end{align*}
     	This completes the proof of the variational characterization.
      \end{proof}

      \begin{lemma} \label{Lemma: hellinger} Let $f$ be a function such that $\operatorname{\mathbb{E}}^\pi[|f|^2]+\operatorname{\mathbb{E}}^{\pi'}[|f|^2]=: f_2^2< \infty$, then
      \[\big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\big|\leq 2 f_2 d_{\mbox {\tiny{\rm H}}}(\pi,\pi').\]
      \end{lemma}
      \begin{proof}
     	\begin{align*}
     	\Big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\Big|=&\ \biggl|\int_{\mathbbm{R}^\Du}f(u)\Bigl(\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Bigr)\Bigl(\sqrt{\pi(u)}+\sqrt{\pi'(u)} \, \Bigr) \,du\biggr|\\
     	\leq&\ \biggl(\frac{1}{2}\int \Big|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Big|^2du\biggr)^{1/2} \biggl(2\int|f(u)|^2\Big|\sqrt{\pi(u)}+\sqrt{\pi'(u)}\Big|^2 \,du\biggr)^{1/2}\\
     	\le &\ d_{\mbox {\tiny{\rm H}}}(\pi,\pi')\Big(4\int|f(u)|^2\bigl(\pi(u)+\pi'(u)\bigr)du\Big)^{1/2}\\
     	= &\ 2f_2 \,d_{\mbox {\tiny{\rm H}}}(\pi,\pi').
     	\end{align*}
      \end{proof}

     Note  that the result for Hellinger only assumes that $f$ is square integrable with respect to $\pi$ and $\pi'$. In contrast, the result for total variation distance assumes that $f$ is bounded, which is a stronger condition.

\subsubsection{Approximation Theorem}
We denote by
\[g(u)=\nu\bigl(y-G(u)\bigr)\quad \text{and}\quad g_\delta(u)=\nu\bigl(y-G_\delta(u)\bigr)\]
the likelihoods associated with $G(u)$ and $G_\delta(u),$ so that
\[\pi^y(u)=\frac{1}{Z}g(u)\rho(u)\quad \text{and}\quad \pi_\delta^y(u)=\frac{1}{Z_\delta}g_\delta(u)\rho(u),\]
where $Z, Z_\delta>0$ are the corresponding normalizing constants. Before we proceed to our main result we make some assumptions: \\
\begin{assumption}\label{a:dh1}
There exist $\delta^+>0$ and $K_1,K_2<\infty$ such that, for all $\delta\in (0,\delta^+),$
	\begin{itemize}
	\item[(i)] $|\sqrt{g(u)} - \sqrt{g_\delta(u)}|\leq \phi(u)\delta$, for some $\phi(u)$
such that $\operatorname{\mathbb{E}}^{\rho} [\phi^2(u)] \le K_1$;
	\item[(ii)] $\sup_{u\in\mathbbm{R}^\Du}(|\sqrt{g(u)}|+|\sqrt{g_\delta(u)}|) \le K_2.$
	\end{itemize}
\end{assumption}

\begin{remark}
Assumption \ref{a:dh1} only involves conditions on the likelihood $g$ and the approximate likelihood $g_\delta.$ While our presentation in this chapter emphasizes that this approximation may arise due to the need of approximating the forward model $G,$ another important scenario that the theory covers is approximation due to perturbations of the data $y.$ Well-posedness results that guarantee the stability under data perturbations of Bayesian data assimilation will be established in Chapter 7.
\end{remark}

Now we state the main result of this section:
\begin{theorem}[Well-posedness of Posterior] Under Assumption \ref{a:dh1} we have
      \[d_{\mbox {\tiny{\rm H}}}(\pi^y,\pi_\delta^y)\leq c \delta,\quad \delta\in(0,\tilde{\delta}^+),\]
      for some $\tilde{\delta}^+>0$ and some $c\in(0,+\infty)$ independent of $\delta$.
      \label{thm1}\\
\end{theorem}

Notice that this theorem together with Lemma \ref{Lemma: hellinger} guarantee that expectations computed with respect to $\pi^y$ and $\pi_\delta^y$ are order $\delta$ apart.
To prove Theorem \ref{thm1}, we first show a lemma which characterizes
the normalization factor $Z_\delta$ in the small $\delta$ limit.

     \begin{lemma}\label{lem1} Under Assumption \ref{a:dh1} there exist $ \tilde{\delta}^+>0$, $c_1,c_2\in(0,+\infty)$ such that
      \[|Z-Z_\delta|\leq c_1\delta\quad \text{and}\quad Z,Z_\delta>c_2,\quad \text{for}\ \delta\in(0,\tilde{\delta}^+).\]
      \end{lemma}
      \begin{proof}
Since $Z=\int g(u)\rho(u)du$ and
$Z_\delta=\int g_\delta(u)\rho(u)du$
we have
      \begin{align*}
      |Z-Z_\delta|=&\ \int \bigl(g(u)-g_\delta(u)\bigr)\rho(u)du\\
      \leq&\ \Big(\int \Bigl|\sqrt{g(u)}- \sqrt{g_\delta(u)}\Bigr|^2\rho(u)du\Big)^{1/2}\Big(\int \Bigl|\sqrt{g(u)}+\sqrt{g_\delta(u)}\Bigr|^2\rho(u)du\Big)^{1/2}\\
      \leq&\ \Big(\int \delta^2 \phi(u)^2\rho(u)du\Big)^{1/2}\Big(\int K_2^2\rho(u)du\Big)^{1/2}\\
      \leq&\ \sqrt{K_1}K_2\delta,\quad \delta\in(0,\delta^+).
      \end{align*}
      And when $\delta\leq \tilde{\delta}^+\coloneqq\min\{\frac{Z}{2\sqrt{K_1}K_2},\delta^+\}$, we have
      \[Z_\delta\geq Z-|Z-Z_\delta|\geq \frac{1}{2}Z.\]
      The lemma follows by taking $c_1=\sqrt{K_1}K_2$ and $c_2=\frac{1}{2}Z$.\\
      \end{proof}

      \begin{proof}[Proof of Theorem \ref{thm1}]
      We break the distance into two error parts, one caused by the difference between $Z$ and $Z_\delta$, the other caused by the difference between $g$ and $g_\delta$:
      \begin{align*}
      d_{\mbox {\tiny{\rm H}}}(\pi^y,\pi_\delta^y)=&\ \frac{1}{\sqrt{2}}\Bigl\|\sqrt{\pi^y}-\sqrt{\pi_\delta^y}\Bigr\|_{L^2}\\
      =&\  \frac{1}{\sqrt{2}}\Big\|\sqrt{\frac{g\rho}{Z}}-\sqrt{\frac{g\rho}{Z_\delta}}+\sqrt{\frac{g\rho}{Z_\delta}}-\sqrt{\frac{g_\delta\rho}{Z_\delta}}\Big\|_{L^2}\\
      \leq&\ \frac{1}{\sqrt{2}}\Big\|\sqrt{\frac{g\rho}{Z}}-\sqrt{\frac{g\rho}{Z_\delta}}\Big\|_{L^2}+\frac{1}{\sqrt{2}}\Big\|\sqrt\frac{g\rho}{Z_\delta}-\sqrt{\frac{g_\delta\rho}{Z_\delta}}\Big\|_{L^2}.
      \end{align*}
      Using Lemma \ref{lem1}, for $\delta\in(0,\tilde{\delta}^+)$, we have
      \begin{align*}
      \Big\|\sqrt{\frac{g\rho}{Z}}-\sqrt{\frac{g\rho}{Z_\delta}}\Big\|_{L^2}=&\ \Big|\frac{1}{\sqrt{Z}}-\frac{1}{\sqrt{Z_\delta}}\Big|\Big(\int g(u)\rho(u)du\Big)^{1/2}\\
      =&\ \frac{|Z-Z_\delta|}{(\sqrt{Z}+\sqrt{Z_\delta})\sqrt{Z_\delta}}\\
      \leq&\ \frac{c_1}{2c_2}\delta,
      \end{align*}
      and
      \[\Big\|\sqrt{\frac{g\rho}{Z_\delta}} - \sqrt{\frac{g\rho}{Z_\delta}}\Big\|_{L^2}=\frac{1}{\sqrt{Z_\delta}}\Big(\int\Bigl|\sqrt{g(u)}- \sqrt{g_\delta(u)}\Bigr|^2\rho(u) du\Big)^{1/2} \leq \sqrt{\frac{K_1}{c_2}}\delta.\]
      Therefore
      \[d_{\mbox {\tiny{\rm H}}}(\pi^y,\pi_\delta^y)\leq \frac{1}{\sqrt{2}}\frac{c_1}{2c_2}\delta+\frac{1}{\sqrt{2}}\sqrt{\frac{K_1}{c_2}}\delta=c \delta,\]
      with $c=\frac{1}{\sqrt{2}}\frac{c_1}{2c_2}+\frac{1}{\sqrt{2}}\sqrt{\frac{K_1}{c_2}}$ independent of $\delta$.
      \end{proof}

\subsubsection{Example}
Many inverse problems arise from differential equations with unknown input parameters. Here we consider a simple but typical example where $G(u)$ comes from the solution of an ODE, which needs to be solved numerically. Let $x(t)$ be the solution to the initial value problem
\begin{equation}
\label{eq:dh1}
\frac{d x}{dt}=F(x;u),\quad x(0)=0,
\end{equation}
where $F:\mathbbm{R}^\Dy\times \mathbbm{R}^\Du\rightarrow \mathbbm{R}^\Dy$ is a function such that $F(x;u)$ and the partial Jacobian $D_xF(x;u)$ are uniformly bounded with respect to $(x,u)$, i.e.
\[|F(x;u)|,|D_xF(x;u)|< F_{{\rm max}}  ,\quad \text{ for all } (x,u)\in \mathbbm{R}^\Dy\times \mathbbm{R}^\Du,\]
for some constant $F_{{\rm max}}$, and thus $F(x;u)$ is Lipschitz in $x$ in that
\[|F(x;u)-F(x';u)|\leq F_{{\rm max}} |x-x'|,\quad \text{ for all } x,x'\in \mathbbm{R}^\Dy.\]

Now consider the inverse problem setting
$$y=G(u)+\eta,$$
where
$$G(u):=x(1)=x(t)|_{t=1},$$
and $\eta\sim \mathcal{N}(0,\gamma^2 I_k)$. We assume that the exact mapping $G(u)$ is replaced by some numerical approximation $G_\delta(u)$. In particular, $G_\delta(u)$ is given by using the forward Euler method to solve the ODE \eqref{eq:dh1}. Define $X_0=0$, and
\[X_{\ell+1}=X_\ell+\delta F(X_\ell;u),\quad \ell\geq0,\]
where $\delta=\frac{1}{L}$ for some large integer $L$. Finally define $G_\delta(u):=X_L$. \\

In what follows, we will prove that $G_\delta(u)$ is uniformly bounded and close to $G(u)$ when $\delta$ is small, and then we will use these results to show
that Assumption \ref{a:dh1} is satisfied. Therefore, we can apply Theorem~\ref{thm1} to this example and claim that the approximate posterior $\pi_\delta^y$ is close to the unperturbed one $\pi^y$.\\

Define $t_\ell=\ell\delta$, $x_\ell=x(t_\ell)$. The following lemma gives an estimate on the error generated from using the forward Euler method.

\begin{lemma} \label{l:maruyama} Let $E_\ell:=x_\ell-X_\ell$. Then there is $c<\infty$ independent of $\delta$ such that
\[|E_\ell|\leq c\delta,\quad 0\leq \ell\leq L. \]
In particular
\[|G(u)-G_\delta(u)|=|E_L|\leq c\delta.\]
      \end{lemma}
      \begin{proof}
For simplicity of exposition we consider the case $k=1$; the case
$k>1$ is almost identical, simply requiring the integral form for
the remainder term in the Taylor expansion.  Using Taylor expansion
in the case $k=1,$ there is $\xi_\ell\in[t_\ell,t_{\ell+1}]$ such that
     	\begin{align*}
     	x_{\ell+1}=&\ x_\ell+\delta \frac{dx}{dt}(t_\ell)+\frac{\delta^2}{2}\frac{d^2x}{dt^2}(\xi_\ell)\\
     	=&\ x_\ell+\delta F(x_\ell;u)+\frac{\delta^2}{2}D_xF\bigl(x(\xi_\ell);u\bigr)F\bigl(x(\xi_\ell);u\bigr).
     	\end{align*}
     	Thus we have
     	\begin{align*}
     	|E_{\ell+1}|=&\ |x_{\ell+1}-X_{\ell+1}|\\
     	=&\ \Big|x_\ell-X_\ell+\delta\Bigl(F(x_\ell;u)-F(X_\ell;u)\Bigr)+\frac{\delta^2}{2}D_xF\bigl(x(\xi_\ell);u\bigr)F\bigl(x(\xi_\ell);u\bigr)\Big|\\
     	\leq&\ |x_\ell-X_\ell|+\delta \bigl|F(x_\ell;u)-F(X_\ell;u)\bigr|+\frac{\delta^2}{2} \bigl|D_xF\bigl(x(\xi_\ell);u\bigr) \bigr| \bigl|F\bigl(x(\xi_\ell);u\bigr)\bigr|\\
     	\leq&\ |E_\ell|+\delta F_{{\rm max}}|E_\ell|+ \frac{\delta^2}{2}F_{{\rm max}}^2.
     	\end{align*}
     	Noticing that $|E_0|=0$, the discrete Gronwall inequality gives
     	\begin{align*}
     	|E_\ell|\leq&\ (1+\delta F_{{\rm max}})^\ell|E_0|+\frac{(1+\delta F_{{\rm max}})^\ell-1}{\delta F_{{\rm max}}}\cdot\frac{\delta^2}{2}F_{{\rm max}}^2\\
     	\leq&\ \bigg( \Big(1+\frac{F_{{\rm max}}}{L}\Big)^L-1\bigg)\cdot\frac{F_{{\rm max}}\delta}{2}\\
     	\leq&\ \frac{(e^{F_{{\rm max}}}-1)F_{{\rm max}}}{2} \delta.
     	\end{align*}
     	The lemma follows by taking $c=\frac{(e^{F_{{\rm max}}}-1)F_{{\rm max}}}{2}$.
      \end{proof}

\begin{lemma} For any $u \in \mathbbm{R}^\Du,$
\[|G(u)|,|G_\delta(u)|<F_{{\rm max}}.\]
      \end{lemma}
      \begin{proof} For $G(u)$ we use that $F(x;u)$ is uniformly bounded, so that
     	\[|G(u)|=|x(1)|=\Big|\int_0^1F(x(t);u)dt\Big|\leq \int_0^1\bigl|F(x(t);u)\bigr|dt\leq F_{{\rm max}}.\]
     	As for $G_\delta(u)$, we first notice that
     	$$|X_{\ell+1}| = |X_\ell +\delta F(X_\ell;u)| \le |X_\ell| + \delta |F(X_\ell;u)| \le |X_\ell| + \delta F_{{\rm max}},$$
     	and by induction
     	$$|X_{\ell}| \le |X_0| +  \ell \delta F_{{\rm max}}  = \ell \delta F_{{\rm max}}.$$
     	In particular,
     	$$|G_\delta(u) | = |X_L| \le L \delta F_{{\rm max}} = F_{{\rm max}}.$$
      \end{proof}

To conclude this chapter we show that in this example Assumption \ref{a:dh1} is satisfied.
Recall that $\eta\sim \mathcal{N}(0,\gamma^2I_k)$, and thus
\[\sqrt{g(u)}=\sqrt{\nu\bigl(y-G(u)\bigr)}=\frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\exp\Big(-\frac{1}{4\gamma^2}|y-G(u)|^2\Big),\]
\[\sqrt{g_\delta(u)}=\sqrt{\nu\bigl(y-G_\delta(u)\bigr)}=\frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\exp\Big(-\frac{1}{4\gamma^2}|y-G_\delta(u)|^2\Big).\]
\begin{itemize}
\item For Assumption \ref{a:dh1}(i) notice that the function $e^{-w}$  is Lipschitz for $w>0$, with Lipschitz constant $1$. Therefore we have
\begin{align*}
\Big|\sqrt{g(u)}-\sqrt{g_\delta(u)}\Big|\leq&\ \frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\cdot \frac{1}{4\gamma^2}\cdot \big||y-G(u)|^2-|y-G_\delta(u)|^2\big|\\
= &\ \frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\cdot \frac{1}{4\gamma^2} \cdot |2y-G(u)-G_\delta(u)||G(u)-G_\delta(u)|\\
\leq &\ \frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\cdot \frac{1}{4\gamma^2} \cdot (2|y|+2F_{{\rm max}}) c\delta\\
=&\ \tilde{c}\delta.
\end{align*}
That is to say, Assumption \ref{a:dh1}(i) is satisfied with $\phi(u)=\tilde{c}$ and $\int_{\mathbbm{R}^\Du} \phi^2(u)\rho(u)du=\tilde{c}^2<\infty$.
\item Assumption \ref{a:dh1}(ii) is satisfied since
\[\sqrt{g(u)}=\frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\exp\Big(-\frac{1}{4\gamma^2}|y-G(u)|^2\Big)\leq \frac{1}{(2\pi)^{k/4}\gamma^{k/2}},\]
\[\sqrt{g_\delta(u)}=\frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\exp\Big(-\frac{1}{4\gamma^2}|y-G_\delta(u)|^2\Big)\leq \frac{1}{(2\pi)^{k/4}\gamma^{k/2}}.\]
\end{itemize}

\subsection{Discussion and Bibliography}

The book by Kaipio and Somersalo \cite{kaipio2006statistical} provides an introduction to the Bayesian approach to inverse problems, especially in the context of differential equations. An overview of the
subject of Bayesian inverse problems in differential equations, with
a perspective informed by the geophysical sciences,
is the book by Tarantola  \cite{tarantola2005inverse} (see, especially, Chapter 5).

In the paper \cite{stuart2010inverse} the Bayesian approach to regularization is reviewed,
developing a function space viewpoint on the subject. A well-posedness theory
and some algorithmic approaches which are used when adopting the Bayesian
approach to inverse problems are introduced.
The function space viewpoint on the subject is developed in more detail in the
chapter notes of Dashti and Stuart \cite{dashti2013bayesian}.
An application of this function space methodology to a large-scale
geophysical inverse problem is considered in
\cite{martin2012stochastic}.
The paper \cite{lieberman2010parameter} demonstrates the potential
for the use of dimension reduction techniques from control theory
within statistical inverse problems.

See \cite{gibbs2002choosing} for more detail on the subject of
metrics, and other distance-like functions, on probability measures.
See \cite{stuart2010inverse,dashti2013bayesian} for  more detailed
discussions on the well-posedness of Bayesian inverse problems with
respect to perturbations in the data; and see \cite{cotter2010approximation}
for applications concerning numerical approximation of
partial differential equations appearing in the forward model.
Related results, but using divergences rather than the Hellinger
metric, may be found in \cite{marzouk2009stochastic}.
The paper \cite{iglesias2014well} contains an interesting set of examples
where the Meta Theorem stated in this chapter fails
in the sense that, whilst well-posedness holds, the posterior is
H\"older with exponent less than one, rather than Lipschitz, with respect
to perturbations.

\section{\Large{\sffamily{Particle Filter}}}
\label{sec:Lecture11}

This chapter is devoted to the particle filter, a method that
approximates the filtering distribution by a sum of Dirac measures.
Particle filters provably converge to the filtering distribution as the number of Dirac measures
approaches infinity. We focus on the bootstrap particle filter, also known as sequential importance resampling;
it is linked to the material on Monte Carlo and importance sampling described in chapter 5.
We note that the Kalman filter completely characterizes the filtering
distribution, but only in the linear Gaussian setting; the Kalman-based methods introduced in the two previous chapters outside the linear-Gaussian assumption are built by approximating the predictive distribution via a Gaussian ansatz, and then using the Kalman formulae for the analysis step. Similarly, particle filters approximate the predictive distribution by a particle approximation measure and then solve exactly the analysis step. In this light both Kalman-based methods (with linear observations) and the bootstrap particle filter do exact application of Bayes formula with wrong priors.  However, particle filters have the potential of recovering an accurate approximation to the filtering distribution in nonlinear, non-Gaussian
settings provided that the number of particles is large enough; their caveat is that they tend to struggle in high dimensional problems.

\subsection{The Setting}
Let us return to the setting in which we introduced filtering
and smoothing, with nonlinear stochastic dynamical system and nonlinear
observation operator, namely the model
\begin{align*}
	v_{j+1} &= \Psi(v_j) + \xi_j, && \xi_j \sim \mathcal{N}(0, \Sigma) \text{ i.i.d.}, \\
	y_{j+1} &= h(v_{j+1}) + \eta_j, && \eta_j \sim \mathcal{N}(0, \Gamma) \text{ i.i.d.},
\end{align*}
with \(v_0 \sim \pi_0 := \mathcal{N}(m_0, C_0)\) independent of the i.i.d.
sequences \(\{\xi_j\}\) and  \(\{\eta_j\}\). Here \(\Psi(\cdot)\) drives the dynamical system and \(h(\cdot)\) is the observation operator.
Recall that we denote by \(Y_j = \{y_1,\hdots,y_j\}\) all the data up to time
\(j\) and by \(\pi_j\) the pdf of \(v_j | Y_j\), that is, \(\pi_j = \operatorname{\mathbbm{P}}(v_j|Y_j)\). The filtering problem is to determine \(\pi_{j+1}\) from \(\pi_j\). We may do so in two steps: first, we run forward the Markov chain generated by the stochastic dynamical system
(prediction), and second, we incorporate the data by an application of Bayes
Theorem (analysis).

For the prediction step, we define the operator \({\mathcal{P}}\) acting on a pdf \(\pi\) as an application of a Markov kernel defined by
\begin{equation}
\label{eq:markov}
	({\mathcal{P}} \pi)(v) = \int_{\mathbbm{R}^d} p(u,v) \pi(u) du,
\end{equation}
where \(p(u,v)\) is the associated pdf of the stochastic dynamics, so that
\begin{equation*}
	p(u,v) = \frac{1}{\sqrt{(2\pi)^d {\rm det}\Sigma}} \text{exp}\left( -\frac{1}{2}|v - \Psi(u)|_{\Sigma}^2 \right).
\end{equation*}
Thus we obtain
\begin{equation*}
	\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)=\hat{\pi}_{j+1} = {\mathcal{P}} \pi_j.
\end{equation*}
We then define the analysis operator \({\mathcal{A}}_j\) acting on a pdf \(\pi\) to correspond to an application of Bayes theorem, namely
\begin{equation}
\label{eq:bayes-again}
	({\mathcal{A}}_j \pi)(v) = \frac{\text{exp}(-\frac{1}{2}|y_{j+1} - h(v)|_{\Gamma}^2)\pi(v)}{\int_{\mathbbm{R}^d} \text{exp}(-\frac{1}{2}|y_{j+1} - h(v)|_{\Gamma}^2)\pi(v) dv}
\end{equation}
and so finally we obtain
\begin{equation*}
	\pi_{j+1} = {\mathcal{A}}_j \hat{\pi}_{j+1} = {\mathcal{A}}_j {\mathcal{P}} \pi_j.
\end{equation*}
We now describe a way to numerically approximate, and update,
the pdfs $\pi_j.$

\subsection{The Bootstrap Particle Filter}
The bootstrap particle filter (BPF) can be thought of as
performing sequential importance resampling. Let \(S^N\) be an operator acting on a pdf \(\pi\) by producing an $N-$samples Dirac approximation of \(\pi\), that is
\begin{equation*}
	(S^N \pi)(u) = \sum_{n=1}^N w_n \delta(u - u^{(n)}),
\end{equation*}
where \(u^{(1)},\hdots,u^{(N)}\) are i.i.d samples from \(\pi\) that are weighted uniformly i.e. \(w_n = \frac{1}{N}\). Note that $S^N \pi=\pi^{N}_{MC}$, as introduced in Chapter 5, equation \eqref{eq:indeed}. We will use the operator $S^N$ to approximate
the measure produced by the Markov kernel step ${\mathcal{P}}$ within the overall
filtering map ${\mathcal{A}}_j {\mathcal{P}}$.
Note that $S^N$ is a {\em random} map taking pdfs into pdfs
if we interpret weighted sums of Diracs as a pdf.

Let \(\pi_0^N = \pi_0 = \mathcal{N}(m_0, C_0)\) and let $\pi^N_j$ denote
a particle approximation of the pdf $\pi_j$
that we will determine in what
follows. We define
\begin{equation*}
	\hat{\pi}^N_{j+1} = S^N {\mathcal{P}} \pi^N_j;
\end{equation*}
this is an approximation of \(\hat{\pi}_{j+1}\) from the previous section. We then apply the operator \({\mathcal{A}}_j\) to act on \(\hat{\pi}^N_{j+1}\) by appropriately reconfiguring the weights \(w_j\) according to the data.

To understand this reconfiguration of the weights
we use the fact that, if
\begin{equation*}
	({\mathcal{A}} \pi)(v) = \frac{g(v) \pi(v)}{\int_{\mathbbm{R}^\Du} g(v) \pi(v) dv}
\end{equation*}
and if
$$\pi(v)=\frac{1}{N} \sum_{n=1}^N \delta(v-v^{(n)}),$$
then
$$({\mathcal{A}} \pi)(v)= \sum_{n=1}^N w^{(n)} \delta(v-v^{(n)}),$$
where
$$\bar{w}^{(n)}=g(v^{(n)})$$
and the ${w}^{(n)}$ are found from the $\bar{w}^{(n)}$ by renormalizing
them to sum to one.
We use this calculation concerning the application of Bayes formula
to sums of Diracs within the following desired approximation of the
filtering update formula:
\begin{equation*}
	\pi_{j+1} \approx \pi_{j+1}^N = {\mathcal{A}}_j \hat{\pi}_{j+1}^N = {\mathcal{A}}_j S^N {\mathcal{P}} \pi_j^N.
\end{equation*}
The steps for the method are summarized in Algorithm \ref{alg:bpf}.

\FloatBarrier
\begin{algorithm}
\caption{Bootstrap Particle Filter}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Initial distribution $\pi_0^N = \pi_0$, observations $Y_J$, number of particles $N.$  \\
\vspace{.04in}
\STATE {\bf Particle Generation}: For $j = 0,1,\dots,J-1$, perform
\begin{enumerate}
\item Draw $v_j^{(n)} \sim \pi_j^N$ for $n=1,\ldots,N$ i.i.d.
\item Set $\hat{v}^{(n)}_{j+1} = \Psi(v_j^{(n)}) + \xi_j^{(n)}$ with $\xi_{j}^{(n)}$ i.i.d.  $\mathcal{N}(0,\Sigma).$
\item Set $\bar{w}_{j+1}^{(n)} = \text{exp}(-\frac{1}{2}|y_{j+1} - h(\hat{v}^{(n)}_{j+1})|_{\Gamma}^2).$
\item Set $w_{j+1}^{(n)} = \bar{w}_{j+1}^{(n)} / \sum_{n=1}^N \bar{w}_{j+1}^{(n)}.$
\item Set $\pi_{j+1}^N (u) = \sum_{n=1}^N w_{j+1}^{(n)} \delta(u - \hat{v}^{(n)}_{j+1}).$
\end{enumerate}
\vspace{.04in}
\STATE{\bf Output:} pdf $\pi_{J}^N$ that approximates the distribution $\operatorname{\mathbbm{P}}(v_J|Y_J).$
\end{algorithmic}
\label{alg:bpf}
\end{algorithm}
\FloatBarrier

\subsection{Bootstrap Particle Filter Convergence}

We will now show that under certain conditions, the BPF
converges to the true particle filter distribution in
the limit \(N \rightarrow \infty\). The proof is similar to that of the
Lax-Equivalence Theorem from the numerical approximation of
evolution equations, part of which is the statement that
consistency and stability together imply convergence.
For the BPF consistency refers to a Monte Carlo error estimate, similar to that
derived in the chapter on importance sampling, and stability
manifests in bounds on the
Lipschitz constants for the operators ${\mathcal{P}}$ and ${\mathcal{A}}_j.$

Our first step is to define what we mean by convergence, that is, we need
a metric on probability measures. Notice that the operators \({\mathcal{P}}\) and \({\mathcal{A}}_j\) are deterministic, but the operator \(S^N\) is random since it requires sampling.
As a consequence the approximate pdfs \(\pi_j\) are also random.
Thus, in fact, we need a metric on random probability measures. To this
end, for random probability measures $\pi$ and $\pi'$, we define
\begin{equation*}
	d(\pi, \pi') = \sup_{|f|_\infty \leq 1} \Bigl( \operatorname{\mathbb{E}}\Bigl[\bigl(\pi(f) - \pi'(f)\bigr)^2\Bigr] \Bigr)^{1/2},
\end{equation*}
where the expectation is taken over the random variable, in our case,
the randomness from sampling with \(S^N\), and the supremum is taken over
all functions $f: \mathbbm{R}^d \to [-1,1].$
Here we have used the notation defined in equation \eqref{eq:meas2}.
The following lemma is straightforward to prove, and provides some useful
intuition about the metric.

\begin{lemma}
$d(\cdot,\cdot)$ as defined above does indeed define a metric on random
probability measures. Furthermore, when \(\pi,\pi'\) are deterministic,
then we have \(d(\pi,\pi') = 2d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')\).
\end{lemma}

We now prove three lemmas which together will enable us to prove convergence
of the BPF.  The first shows consistency; the second
and third show stability estimates for ${\mathcal{P}}$ and ${\mathcal{A}}_j$ respectively.

\begin{lemma}
	Let \({\mathscr{P}}\) be the set of probability densities on \(\mathbbm{R}^\Du\) then
	\[\sup_{\pi \in {\mathscr{P}}} d(\pi, S^N \pi) \leq \frac{1}{\sqrt{N}}.\]
\end{lemma}
\begin{proof}
Note that $S^N\pi$ agrees with $\pi_{MC}^N$ as defined in Chapter 5. Therefore Theorem \ref{t:MC} shows that for any $f$ with $|f|_\infty \le 1$ and for any probability density $\pi$ it holds that
$$\operatorname{\mathbb{E}}\left[ \bigl(S^N \pi(f) - \pi(f)\bigr)^2\right] \le \frac{1}{N}.$$
Taking the square root and the supremum over $f$ and $\pi$ gives the desired result.
\end{proof}

Now we prove a stability bound for the operator ${\mathcal{P}}$ defined in equation \eqref{eq:markov}.

\begin{lemma}
	\[d({\mathcal{P}} \pi, {\mathcal{P}}\pi') \leq d(\pi,\pi').\]
\end{lemma}

\begin{proof}  For $|f|_\infty \le 1$  define a function $q$ on $\mathbbm{R}^d$ by
\[q(v') = \int_{\mathbbm{R}^d}p(v',v)f(v)dv,  \]
where, recall, $p$ denotes the transition pdf associated to the stochastic dynamics.
Note that
\[|q(v')| \leq \int_{\mathbbm{R}^d}p(v',v)dv = 1 ,\]
and so
\begin{align*}
	\pi(q) &= \int_{\mathbbm{R}^d}q(v')\pi(v')dv' = \int_{\mathbbm{R}^d} \Bigg[ \int_{\mathbbm{R}^d}p(v',v)f(v)dv \Bigg] \pi(v')dv' \\
	&= \int_{\mathbbm{R}^d} \Bigg[ \int_{\mathbbm{R}^d}p(v',v)\pi(v')dv' \Bigg] f(v)dv \\
	&= \int_{\mathbbm{R}^d} ({\mathcal{P}}\pi)(v)f(v)dv
\end{align*}
by exchanging the order of integration. Consequently, we have
\[\pi(q) = ({\mathcal{P}}\pi)(f).\]
Finally,
\begin{align*}
	d({\mathcal{P}}\pi,{\mathcal{P}}\pi') &= \sup_{|f|_\infty \leq 1} \Bigl(\operatorname{\mathbb{E}} \Bigl[\bigl(({\mathcal{P}}\pi)(f) - ({\mathcal{P}}\pi')(f)\bigr)^2\Bigr]\Bigr)^{1/2} \\
	&\leq \sup_{|q|_\infty \leq 1} \Bigr( \operatorname{\mathbb{E}}  \Bigl[\bigl(\pi(q) - \pi'(q)\bigr)^2\Bigr]\Bigr)^{1/2} \\
	&= d(\pi,\pi').
\end{align*}
\end{proof}

To prove the next lemma and the main convergence
theorem of the BPF below,
we will make the following assumption which encodes the idea of a bound
on the observation operator:

\begin{assumption}
\label{a:LE}
There exists \(\kappa \in (0,1)\) such that, for all  $v\in\mathbbm{R}^d,$ and $j \in \{1,\hdots,J\},$
\begin{equation*}
	\kappa \leq g_j(v) \leq \kappa^{-1}.
\end{equation*}
\end{assumption}

It may initially appear strange to use the same constant $\kappa$ in the
upper and lower bounds, but recall that $g$ is undefined up to a multiplicative
constant. Consequently, given any upper and lower bounds, $g$ can be
scaled to achieve the bound as stated. Relatedly it is $\kappa^{-2}$
which appears in the stability constant in the next lemma;
if $g$ is not scaled to produce the same constant $\kappa$ in the
upper and lower bounds in
Assumption \ref{a:LE}, then it is the ratio of the upper and lower
bounds which would appear in the stability bound.

\begin{lemma} \label{lemma:analysis}
Let Assumption \ref{a:LE} hold. Then
	\[d({\mathcal{A}}_j \pi,{\mathcal{A}}_j \pi') \leq \frac{2}{\kappa^2} d(\pi,\pi').\]
\end{lemma}

\begin{proof}
\begin{align*}
	({\mathcal{A}}\pi)(f) - ({\mathcal{A}}\pi')(f) &= \frac{\pi(fg)}{\pi(g)} - \frac{\pi'(fg)}{\pi'(g)} \\
	&= \frac{\pi(fg)}{\pi(g)} - \frac{\pi'(fg)}{\pi(g)} + \frac{\pi'(fg)}{\pi(g)} - \frac{\pi'(fg)}{\pi'(g)} \\
	&= \frac{1}{\kappa} \bigg( \frac{\pi(\kappa fg) - \pi'(\kappa fg)}{\pi(g)} + \frac{\pi'(fg)}{\pi'(g)} \frac{\pi'(\kappa g) - \pi(\kappa g)}{\pi(g)} \bigg).
\end{align*}
Applying Bayes Theorem we obtain
\[\Bigl|\frac{\pi'(fg)}{\pi'(g)}\Bigr| = |({\mathcal{A}}\pi')(f)| \leq 1. \]
Therefore,
\[\bigl|({\mathcal{A}}\pi)(f) - ({\mathcal{A}}\pi')(f)\bigr| \leq \frac{1}{\kappa^2}\Bigl( \bigl|\pi(\kappa fg) - \pi'(\kappa fg)\bigr| + \bigl|\pi'(\kappa g) - \pi(\kappa g)\bigr|   \Bigr).  \]
It follows that
\[\operatorname{\mathbb{E}} \Bigl[\bigl( ({\mathcal{A}}\pi)(f) - ({\mathcal{A}}\pi')(f)\bigr)^2\Bigr] \leq \frac{2}{\kappa^2}\Bigl(\operatorname{\mathbb{E}} \Bigl[(\pi(\kappa fg) - \pi'(\kappa fg))^2\Bigr] + \operatorname{\mathbb{E}}\Bigl[\bigl(\pi'(\kappa g) - \pi(\kappa g)\bigr)^2\Bigr] \Bigr).  \]
Since $|\kappa g| \le 1$ we find that
\[\sup_{|f|_\infty \leq 1} \operatorname{\mathbb{E}}\Bigl[\bigl(({\mathcal{A}}\pi)(f) - ({\mathcal{A}}\pi')(f)\bigr)^2\Bigr] \leq \frac{4}{\kappa^4} \sup_{|f|_\infty \leq 1} \operatorname{\mathbb{E}}\Bigl[\bigl(\pi(f) - \pi'(f)\bigr)^2\Bigr],  \]
and hence
\[d({\mathcal{A}}_j\pi,{\mathcal{A}}_j\pi') \leq \frac{2}{\kappa^2} d(\pi,\pi'). \]
\end{proof}

\begin{theorem}[Convergence of the BPF]
Let Assumption \ref{a:LE} hold.
Then there exists a $c = c(J,\kappa)$ such that, for all $j = 1, \ldots, J,$
	\[d(\pi_j,\pi_j^N) \leq \frac{c}{\sqrt{N}}. \]
\end{theorem}

\begin{proof}
Let $e_j = d(\pi_j,\pi_j^N)$, then
\begin{align*}
	e_{j+1} &= d(\pi_{j+1},\pi_{j+1}^N) = d({\mathcal{A}}_j{\mathcal{P}}\pi_j,{\mathcal{A}}_jS^N {\mathcal{P}}\pi_j^N) \\
	&\leq d({\mathcal{A}}_j{\mathcal{P}}\pi_j  , {\mathcal{A}}_j{\mathcal{P}}\pi_j^N) + d({\mathcal{A}}_j{\mathcal{P}}\pi_j^N,{\mathcal{A}}_jS^N {\mathcal{P}}\pi_j^N)
\end{align*}
by the triangle inequality. Applying the stability bound for ${\mathcal{A}}_j$,
we have
\[e_{j+1} \leq \frac{2}{\kappa^2}\Bigl[d({\mathcal{P}}\pi_j   , {\mathcal{P}}\pi_j^N) + d(\hat{\pi}_j^N,S^N\hat{\pi}_j^N)\Bigr],\]
where $\hat{\pi}_j^N = {\mathcal{P}}\pi_j^N$. By the stability bound for ${\mathcal{P}}$,
\[d({\mathcal{P}}\pi_j  , {\mathcal{P}}\pi_j^N) \leq d(\pi_j, \pi_j^N) \]
and by the consistency bound for $S^N$
\[d(\hat{\pi}_j^N,S^N \hat{\pi}_j^N) \leq \frac{1}{\sqrt{N}}.\]
Therefore,
\begin{align*}
	e_{j+1} &\leq \frac{2}{\kappa^2}\Bigl(d(\pi_j,\pi_j^N) + \frac{1}{\sqrt{N}}\Bigr) \\
	&\leq \frac{2}{\kappa^2}\Bigl(e_j + \frac{1}{\sqrt{N}}\Bigr).
\end{align*}
We let $\lambda = 2/\kappa^2$ and note that \(\lambda \geq 2\) since \(\kappa \in (0,1]\). Then the discrete Gronwall inequality gives
\begin{equation*}
	e_{j} \leq \lambda^je_0 + \frac{\lambda}{\sqrt{N}}\frac{1-\lambda^j}{1-\lambda}.
\end{equation*}
Recall that \(\pi_0^N = \pi_0\) hence \(e_0 = 0\). Thus letting
\[c = \frac{\lambda(1 - \lambda^J)}{1 - \lambda}\]
completes the proof since \(\lambda(1 - \lambda^j)/(1 - \lambda)\) is increasing in \(j\).
\end{proof}

\subsection{The Bootstrap Particle Filter as a Random Dynamical System}

A nice interpretation of the BPF is to view it as a random dynamical system for a set of interacting
particles $\{v_j^{(n)}\}_{n=1}^N.$
To this end, a measure
\begin{equation*}
	\bar{\pi}_j^{N}(u) = \frac{1}{N}\sum_{n=1}^N \delta(u-v_j^{(n)}) \approx \pi_j^N(u) \approx \pi_j(u)
\end{equation*}
with equally weighted particles may be naturally defined after the resampling step from $\pi_j^N.$
It can then be seen that the BPF updates the particle positions
\begin{equation*}
\{v_j^{(n)} \}_{n = 1}^N  \mapsto \{v_{j+1}^{(n)} \}_{n = 1}^N
\end{equation*}
via the random map
\begin{align*}
	\hat{v}_{j+1}^{(n)} & = \Psi(v_j^{(n)}) + \xi_j^{(n)}, && \xi_j^{(n)} \sim \mathcal{N}(0,\Sigma) \text{ i.i.d.,}\\
	v_{j+1}^{(n)} & = \sum_{m=1}^N \mathbbm{1}_{I_{j+1}^{(m)}}\left(r_{j+1}^{(n)}\right) \hat{v}_{j+1}^{(m)}, && r_{j+1}^{(n)} \sim \text{Uniform}(0,1)\text{ i.i.d.}
\end{align*}
Here the supports $I_{j}^{(m)}$ of the indicator functions have widths
given by the weights appearing in  $ {\pi}_j^{N}(u).$ Specifically we have
\begin{align*}
	I_{j+1}^{(m)} & = \left[\alpha_{j+1}^{(m-1)},\alpha_{j+1}^{(m)}\right), &  \alpha_{j+1}^{(m+1)} = \alpha_{j+1}^{(m)}+w_{j+1}^{(m)}, \quad \alpha_{j+1}^{(0)} = 0.
\end{align*}
Note that, by construction, $\alpha_j^{(N)} = 1.$

Thus the underlying dynamical system on particles comprises $N$ particles
governed by two steps: (i) the underlying stochastic dynamics model, in
which the particles do not interact; (ii) a resampling of the resulting collection of particles, to reflect the different weights associated with them, in
which the particles do then interact. The interaction is driven by the
weights which see all the particle positions, and measure their goodness
of fit to the data.

\subsection{Discussion and Bibliography}

Particle filters are overviewed from an algorithmic viewpoint
in \cite{doucet2001introduction,doucet2000sequential},
and from a more mathematical perspective in \cite{del2004feynman}.
The convergence of particle filters was addressed
in \cite{crisan1998discrete}; the clean proof presented here
originates in \cite{rebeschini2015can} and may also
be found in \cite{law2015data}.
For problems in which the dynamics evolve in relatively low dimensional
spaces they have been an enormously successful. Generalizing
them so that they work for the high dimensional problems that arise,
for example, in geophysical applications, provides a major challenge
\cite{van2015nonlinear}.

\section{\Large{\sffamily{The Extended and Ensemble Kalman Filters }}}

In this chapter we describe the Extended Kalman Filter  (ExKF)\footnote{The extended
Kalman filter is often termed the EKF in the literature, a terminology
introduced before the existence of the EnKF;
we find it useful to write ExKF to uneqivocally distinguish it from the EnKF.}
and the Ensemble Kalman Filter (EnKF). The ExKF approximates the predictive covariance by linearization, while the EnKF approximates it by the empirical covariance of a collection of particles. The status of the two methods in relation to the true filtering distribution
is as follows. The ExKF is a provably accurate approximation of the
filtering distribution in situations in which small noise is present in both
signal and data, and the filtering distribution is well-approximated by a
Gaussian. The EnKF is also in principle a good approximation of the
filtering distribution in this situation, if a large number of particles
is used. However the EnKF is typically deployed for problems where the
use of a sufficiently large number of particles is impractical; it
is then better viewed as an online optimizer, in the spirit of 3DVAR,
but using multiple particles to better estimate the covariances
appearing in the quadratic objective function which is minimized
to find particle updates.

\subsection{The Setting}

Throughout this chapter we consider the setting in which 3DVAR was introduced
and may be applied: the dynamics model
is nonlinear, but the observation operator is linear.
For purposes of exposition we summarize it again here:
\begin{align*}
        v_{j+1} &= \Psi(v_j) + \xi_j, && \xi_j \sim \mathcal{N}(0, \Sigma) \text{ i.i.d.}, \\
        y_{j+1} &= Hv_{j+1} + \eta_{j+1}, && \eta_j \sim \mathcal{N}(0, \Gamma) \text{ i.i.d.},
\end{align*}
with \(v_0 \sim \mathcal{N}(m_0, C_0)\) independent of the independent i.i.d.
sequences \(\{\xi_j\}\) and  \(\{\eta_j\}\).
Throughout this chapter we assume that $v_j \in \mathbbm{R}^d, \,
y_j \in \mathbbm{R}^k.$

\subsection{The Extended Kalman Filter}

This method is derived by applying the Kalman methodology, using
linearization to propagate the covariance $C_j$ to the predictive
covariance $\hat{C}_{j+1}.$ Table \ref{3DVAR:t2} summarizes the idea,
after which we calculate the formulae required in full detail.

\FloatBarrier
\begin{table}
        \begin{tabular}{ | c | c |}
                \hline
                Kalman Filter & ExKF \\ \hline
                $m_{j+1} = \arg \min_v {\mathsf{J}}(v)$ & $m_{j+1} = \arg \min_v {\mathsf{J}}(v)$  \\ \hline
                ${\mathsf{J}}(v) = \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2$  &  ${\mathsf{J}}(v) = \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2$ \\
                \hline
                $\hat{m}_{j+1} = M m_j$ & $\hat{m}_{j+1} = \Psi(m_j)$ \\ \hline
$\hat{C}_{j+1}$ update exact & $\hat{C}_{j+1}$ update by linearization \\ \hline
                $m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}$ & $m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}$\\ \hline
        \end{tabular}
        \caption{Comparison of Kalman Filter and ExKF update formulae}
                \label{3DVAR:t2}
\end{table}
\FloatBarrier

We first recall the Kalman filter update formulae and their derivation. We
have
\begin{align}
\widehat{v}_{j+1} = Mv_{j}+\xi_{j}, \quad v_{j}  \sim \mathcal{N}(m_{j},C_{j}), \quad \xi_{j} \sim \mathcal{N}(0,\Sigma).
\end{align}
From this we deduce, by taking expectations, that
\begin{equation}
\widehat{m}_{j+1} = \operatorname{\mathbb{E}}[\widehat{v}_{j+1} \mid   Y_{j}]=\operatorname{\mathbb{E}}[Mv_{j}+\xi_{j} \mid  Y_{j}]= \operatorname{\mathbb{E}}[Mv_{j}\mid  Y_{j}] + \operatorname{\mathbb{E}}[\xi_{j}\mid  Y_{j}] =Mm_{j} .
\end{equation}
The covariance update is derived as follows.
\begin{align}
\widehat{C}_{j+1} &= \operatorname{\mathbb{E}}\Bigl[(\widehat{v}_{j+1}-\widehat{m}_{j+1})\otimes(\widehat{v}_{j+1}-\widehat{m}_{j+1})\mid  Y_{j} \Bigr] \nonumber \\
				  &= \operatorname{\mathbb{E}}\Bigl[(M(v_{j}-m_{j})+\xi_{j})\otimes(M(v_{j}-m_{j})+\xi_{j})\mid  Y_{j}\Bigr] \nonumber \\
                  &= \operatorname{\mathbb{E}} \Bigl[(M(v_{j}-m_{j}))\otimes(M(v_{j}-m_{j}))\mid  Y_{j} \Bigr]+\operatorname{\mathbb{E}}\Bigl[\xi_{j}\otimes\xi_{j}\mid  Y_{j}\Bigr]  \\
&\quad  +\operatorname{\mathbb{E}}\Bigl[(M(v_{j}-m_{j}))\otimes \xi_{j} \mid Y_{j}\Bigr]
+\operatorname{\mathbb{E}}\Bigl[\xi_{j}\otimes (M(v_{j}-m_{j})) \mid Y_{j}\Bigr] \nonumber \\
                  &=M\operatorname{\mathbb{E}}\Bigl[(v_{j}-m_{j})\otimes(v_{j}-m_{j}) \mid Y_{j}\Bigr]M^{T} + \Sigma \nonumber \\
                 &=MC_{j}M^{T}+\Sigma. \nonumber
\end{align}

For the ExKF, the prediction map $\Psi$ is no longer linear.
But since
$\xi_{j}$ is independent of $Y_{j}$ and $v_{j}$,
we obtain
\begin{equation*}
\widehat{m}_{j+1} = \operatorname{\mathbb{E}}\Bigl[\Psi(v_{j})+\xi_{j} \mid Y_{j}\Bigr] = \operatorname{\mathbb{E}}\Bigl[\Psi(v_{j})\mid Y_{j}\Bigr]+\operatorname{\mathbb{E}}\Bigl[\xi_{j} \mid Y_{j}\Bigr] = \operatorname{\mathbb{E}} \Bigl[\Psi(v_{j})\mid Y_{j}\Bigr].
\end{equation*}
If we assume that the fluctuations of $v_j$ around its mean $m_j$ (conditional
on data) are small then a reasonable approximation is to take
$\Psi(v_{j}) \approx \Psi(m_{j})$ so that
\begin{equation}
\label{eq:1a}
\widehat{m}_{j+1} = \Psi(m_{j}).
\end{equation}
For the predictive covariance we use linearization; we have
\begin{align*}
\widehat{C}_{j+1} \nonumber &= \operatorname{\mathbb{E}}\Bigl[(\widehat{v}_{j+1}-\widehat{m}_{j+1})
\otimes(\widehat{v}_{j+1}-\widehat{m}_{j+1})\mid  Y_{j}\Bigr]\nonumber \\
                 &= \operatorname{\mathbb{E}}\Bigl[(\Psi(v_{j})-\Psi(m_{j})+\xi_{j})\otimes(\Psi(v_{j})-\Psi(m_{j})+\xi_{j})\mid  Y_{j}\Bigr]\\
&=\operatorname{\mathbb{E}} \Bigl[(\Psi(v_{j})-\Psi(m_{j}))\otimes(\Psi(v_{j})-\Psi(m_{j}))\mid  Y_{j}\Bigr]+\Sigma\\
&\approx D\Psi(m_{j})\operatorname{\mathbb{E}}\Bigl[(v_{j}-m_{j})\otimes(v_{j}-m_{j})\mid  Y_{j}\Bigr]D\Psi(m_{j})^T+\Sigma,
\end{align*}
and so, again assuming that fluctuations of $v_j$ around its mean $m_j$
(conditional on data) are small, we invoke the approximation
\begin{equation}
\label{eq:1b}
\widehat{C}_{j+1}=D\Psi(m_{j})C_{j}D\Psi(m_{j})^{T}+\Sigma.
\end{equation}
To be self-consistent $\Sigma$ itself should be small.

The analysis step is the same as for the Kalman filter:

\begin{align}\label{eq:1c}
\begin{split}
S_{j+1}&=H\widehat{C}_{j+1}H^{T}+\Gamma,\\
K_{j+1}&= \widehat{C}_{j+1}H^{T}S_{j+1}^{-1}, \\
m_{j+1}&=(I-K_{j+1}H)\widehat{m}_{j+1}+K_{j+1}y_{j+1},\\
C_{j+1}&=(I-K_{j+1}H)\widehat{C}_{j+1}.\\
\end{split}
\end{align}

Thus the overall ExKF comprises equations \eqref{eq:1a}, \eqref{eq:1b}
and \eqref{eq:1c}.
Unlike the Kalman filter, for the extended Kalman filter
the maps $C_{j} \mapsto\widehat{C}_{j+1}\mapsto C_{j+1}$ depend
on the observed data, through the dependence of the predictive
covariance on the filter mean.
To be self-consistent with the ``small fluctuations around the mean''
assumptions made in the derivation of the ExKF, $\Sigma$ and $\Gamma$
should both be small.

The analysis step can also be defined by
\begin{align*}
C_{j+1}^{-1} &= \widehat{C}_{j+1}^{-1}+H^{T}\Gamma^{-1}H, \\
m_{j+1}&=\arg \min_v {\mathsf{J}}(v),
\end{align*}
where
\begin{equation}\label{eq:Ifilter}
{\mathsf{J}}(v) = \frac{1}{2} |y_{j+1}-Hv |_{\Gamma}^{2} + \frac{1}{2}|v-\widehat{m}_{j+1} |^{2}_{\widehat{C}_{j+1}}
\end{equation}
and $\widehat{m}_{j+1}, \widehat{C}_{j+1}$ are calculated as above
in the prediction steps \eqref{eq:1a}, \eqref{eq:1b}.
The constraint formulation of the minimization
problem, derived for the Kalman filter in section \ref{sec:okf}, may
also be used to derive the update formulae above.

\subsection{Ensemble Kalman Filter}

When the dynamical system is in high dimension, evaluation and storage of
the predicted covariance, and in particular the Jacobian required for
the update formula \eqref{eq:1b},
becomes computationally inefficient and expensive for the
ExKF. The EnKF was developed to overcome
this issue. The basic idea is to maintain an ensemble of particles,
and to use their empirical covariance
within a Kalman-type update. The method is summarized in
Table \ref{3DVAR:t3}. It may be thought of as an ensemble 3DVAR technique
in which a collection of particles are generated similarly to 3DVAR, but
interact through an ensemble estimate of their covariance.

\FloatBarrier
\begin{table}
        \begin{tabular}{ | c | c |}
                \hline
                Kalman Filter & EnKF \\ \hline
                $m_{j+1} = \arg \min_v {\mathsf{J}}(v)$ & $m_{j+1} = \arg \min_v {\mathsf{J}}(v)$  \\ \hline
                ${\mathsf{J}}(v) = \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2$  &  ${\mathsf{J}}_n(v) = \frac{1}{2} |y_{j+1}^{(n)} - H m|_{\Gamma}^2 + \frac{1}{2} |v - \hat{v}_{j+1}^{(n)}|_{\hat{C}_{j+1}}^2$ \\
                \hline
                $\hat{m}_{j+1} = M m_j$ & $\hat{v}_{j+1}^{(n)} = \Psi(v_j^{(n)})+\xi_j^{(n)}$ \\ \hline
$\hat{C}_{j+1}$ update exact & $\hat{C}_{j+1}$ update by ensemble estimate\\ \hline
                $m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}$ & $m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}$\\ \hline
        \end{tabular}
        \caption{Comparison of Kalman Filter and EnKF update formulae}
                \label{3DVAR:t3}
\end{table}
\FloatBarrier

\subsection{Derivation of EnKF}

In the basic form which we present here, the
EnKF is applied  when $\Psi$ is nonlinear, while the observation
operator $H$ is linear.
The $N$ particles used at step $j$ are denoted $\{v_{j}^{(n)}\}^{N}_{n=1}.$
They are all given equal weight so it is possible, in principle, to
make an approximation to the filtering distribution of the form
$$\pi_j^N (v_j) \approx \frac{1}{N}\sum_{n=1}^N \delta(v_j-v_{j}^{(n)}).$$
However the EnKF is typically used with a relatively small number $N$ of
particles and may be far from approximating the desired distribution in
$\mathbbm{R}^\Du.$ It is then better understood as a sequential optimization method,
similar in spirit to 3DVAR, as described above; this is our perspective.

The state of all the particles at time $j+1$ are predicted
to give $\{\widehat{v}_{j+1}^{(n)}\}^{N}_{n=1}$ using the dynamical
model. The resulting empirical covariance is then used
to define the objective function (\ref{eq:Ifilter}c) which is minimized
in order to perform the analysis step and obtain $\{{v}_{j+1}^{(n)}\}^{N}_{n=1}.$
The updates are denoted schematically by
\[\{v_{j}^{(n)}\}^{N}_{n=1} \xmapsto{\textcircled{p}} \{\widehat{v}_{j+1}^{(n)}\}^{N}_{n=1}\xmapsto{\textcircled{a}} \{v_{j+1}^{(n)}\}^{N}_{n=1}.\]
We now detail these two steps.

\textcircled{p} prediction:

\begin{subequations}
\label{eq:deenz}
\begin{align}
\widehat{v}_{j+1}^{(n)} &= \Psi(v_{j}^{(n)})+\xi^{(n)}_{j}, \quad n=1,\ldots,N, \\
\widehat{m}_{j+1} &= \frac{1}{N}\sum^{N}_{n=1} \widehat{v}_{j+1}^{(n)}, \\
\widehat{C}_{j+1} &= \frac{1}{N}\sum^{N}_{n=1}\bigl(\widehat{v}^{(n)}_{j+1}-\widehat{m}_{j+1}\bigr)\otimes \bigl(\widehat{v}^{(n)}_{j+1}-\widehat{m}_{j+1}\bigr).
\end{align}
\end{subequations}
Here we have
\begin{equation*}
\xi_{j}^{(n)} \sim \mathcal{N}(0,\Sigma), \quad  \text{i.i.d.}
\end{equation*}

\textcircled{a} analysis:
\begin{subequations}
\begin{align}
S_{j+1} &= H\widehat{C}_{j+1}H^{T} + \Gamma, \\
K_{j+1} &=\widehat{C}_{j+1}H^{T}S_{j+1}^{-1}, \\
y_{j+1}^{(n)}&=y_{j+1}+s\eta_{j+1}^{(n)},\quad n=1,\ldots,N,\\
v_{j+1}^{(n)} &= (I-K_{j+1}H)\widehat{v}_{j+1}^{(n)}+K_{j+1}y_{j+1}^{(n)},\quad n=1,\ldots,N. \\
\end{align}
\end{subequations}\\
Here we take
\begin{equation*}
\eta_{j}^{(n)} \sim \mathcal{N}(0,\Gamma), \quad \text{ i.i.d.}
\end{equation*}
The constant $s$ takes value $0$ or $1$.
When $s=1$ the $y_{j+1}^{(n)}$ are referred
to as {\em perturbed observations}.
The analysis step may be written as
\begin{equation}
v_{j+1}^{(n)}=\arg \min_v {\mathsf{J}}_{n}(v),
\end{equation}
where
\begin{equation}
\label{eq:deenz2}
{\mathsf{J}}_{n}(v) := \frac{1}{2} |y_{j+1}^{(n)} - Hv|^{2}_{\Gamma} + \frac{1}{2} |v-\widehat{v}_{j+1}^{(n)} |^{2}_{\widehat{C}_{j+1}}
\end{equation}
and the predicted mean and covariance are given by step \textcircled{p}.
Note that $\widehat{C}_{j+1}$ is typically not invertible as it is a rank
$N$ matrix and $N$ is usually less than the dimension $d$ of the space
on which $\widehat{C}_{j+1}$ acts; this is since the typical use
of ensemble methods is for high dimensional state space estimation,
with a small ensemble size.
The minimizing solution can be found
by regularizing $\widehat{C}_{j+1}$ by additions of $\epsilon I$ for
$\epsilon>0$, deriving the update equations as above for \textcircled{a},
and then letting $\epsilon \to 0.$ Alternatively, the constraint
formulation of the minimization problem, derived for the Kalman filter
in section \ref{sec:okf}, may also be used to derive the update formulae above.

\subsection{Subspace Property of EnKF}

We now give another way to think of, and exploit in algorithms, the
low rank property of $\widehat{C}_{j+1}.$
Note that ${\mathsf{J}}_{n}(v)$ is undefined unless
$$v-\widehat{v}_{j+1}^{(n)}=\widehat{C}_{j+1}a$$
for some $a \in \mathbbm{R}^\Du.$
From the structure of $\widehat{C}_{j+1}$ given in
(\ref{eq:deenz}c) it follows that
\begin{equation}
\label{eq:deenz3}
v=\widehat{v}_{j+1}^{(n)}+\frac{1}{N}\sum_{n=1}^N b_n\bigl(\widehat{v}^{(n)}_{j+1}-\widehat{m}_{j+1}\bigr)
\end{equation}
for some unknown parameters $\{b_n\}_{n=1}^N,$ to be determined.
(Note that the vector $\{b_n\}_{n=1}^N$ depends on enesemble index $n$; we have supressed this
dependence for notational convenience.)
This form for $v$ can be substituted into \eqref{eq:deenz2} to obtain
a functional ${\mathsf{I}}_{n}(b)$ to be minimized over $b \in \mathbbm{R}^N.$
We re-emphasize that $N$ will typically be much smaller than $d$, the
state-space dimension.
Once $b$ is determined it may be substituted back into
\eqref{eq:deenz3} to obtain the solution to the minimization problem.

To dig a little deeper into this calculation we define
$$e^{(n)}=\widehat{v}^{(n)}_{j+1}-\widehat{m}_{j+1}$$
and note that then
$$\widehat{C}_{j+1}=\frac{1}{N}\sum_{n=1}^N e^{(n)} \otimes e^{(n)}.$$
Since
$$\widehat{C}_{j+1} a=\frac{1}{N}\sum_{n=1}^N b_n e^{(n)}$$
we deduce that this is solved by taking
$$b_n= \langle e^{(n)}, a \rangle.$$
Now note that
$$\frac{1}{2} | v-\widehat{v}_{j+1}^{(n)} |^{2}_{\widehat{C}_{j+1}}
= \frac12 \langle a, \widehat{C}_{j+1} a \rangle=\frac{1}{2N}\sum_{n=1}^N b_n^2.$$
We define
\begin{equation}
\label{eq:z2}
{\mathsf{I}}_{n}(b) := \frac{1}{2} \Bigl|y_{j+1}^{(n)} - H\widehat{v}_{j+1}^{(n)} -\frac{1}{N}\sum_{n=1}^N b_n H(\widehat{v}_{j+1}^{(n)}-\widehat{m}_{j+1}) \Bigr|^{2}_{\Gamma}+\frac{1}{2N}\sum_{n=1}^N b_n^2.
\end{equation}
We have shown:

\begin{theorem}[Implementation of EnKF in $N$ Dimensional Subspace]
\label{eq:extra1}
Given the prediction \textcircled{p} defined by (\ref{eq:deenz}a), the Kalman
update formulae \textcircled{a} may be found
by minimizing ${\mathsf{I}}_{n}(b)$ with respect to $b$ and substituting
into \eqref{eq:deenz3}.
\end{theorem}

\subsection{Discussion and Bibliography}

The development and theory of the extended Kalman filter is
documented in the text \cite{jazwinski2007stochastic}.
A methodology for analyzing evolving probability distributions
with small variance, and establishing the validity of the
Gaussian approximation, is described in \cite{sanz2016gaussian}.
The use of the ExKF
for weather forecasting was proposed in \cite{ghilande}.
However the dimension of the state space in most geophysical
applications renders the extended Kalman filter impractical.
The ensemble Kalman filter provided an innovation with far
reaching consequences in geophysical applications, because
it allowed for the use of partial empirical correlation
information, without the computation of the full covariance.
An overview of ensemble Kalman methods may be found in the
book \cite{evensen2009data}, including a historical perspective
on the subject, originating from papers of Evensen and
Van Leeuwen in the mid
1990s \cite{evensen1994sequential,evensen1996assimilation};
a similar idea was also developed by Houtkamer
within the Canadian meteorological service, around the same time;
\cite{houtekamer1995methods,houtekamer1998data}.

The presentation of the ensemble
Kalman filter as a smart optimization tool
is also developed in \cite{law2015data}, but the derivation of
the update equations in a space whose dimension is that of
the ensemble is not described there.
The analysis of ensemble methods
is difficult and theory is only just starting to emerge.
In the linear case the method converges in the large ensemble
limit to the Kalman filter \cite{le2009large},
but in the nonlinear case the
limit does not reproduce the filtering distribution
\cite{ernst2015analysis}.
In any case the primary advantage of ensemble methods is that
they can provide good state estimation when the number of
particles is {\em not} large; this subject
is discussed in \cite{gottwald2013mechanism,kelly2014well,tong2015nonlinear,tong2016nonlinear}.

\part*{\Large{\sffamily{Introduction}}}

\noindent{\bf Overview of the Notes\\}
\vspace{0.05in}

These notes were first developed out of Caltech course ACM 159 in Fall 2017, and then substantially modified for the University of Chicago course STAT 31550 in Winter 2019. The aim of the notes is to provide a clear and concise introduction
to the subjects of Inverse Problems and Data Assimilation, and their
inter-relations, together with citations to some relevant literature
in this area

In its most basic form, inverse problem theory is the study of how to estimate model parameters from data. Often the data provides indirect information about these parameters, corrupted by noise. The theory of inverse problems,
however, is much richer than just parameter estimation. For example,
the underlying theory can be used to
determine the effects of noisy data on the accuracy of the solution; it can
be used to determine what kind of observations are needed to accurately
determine a parameter; and it can be used to study the uncertainty in a
parameter estimate and, relatedly, is useful, for example,
in the design of strategies for control or optimization under
uncertainty, and for risk analysis. The theory
thus has applications in many fields of science and engineering.\\

To apply the ideas in these notes, the starting point is a mathematical model
mapping the unknown parameters to the observations: termed the ``forward'' or
``direct'' problem, and often a subject of research in its own right. A good forward model will not only identify how the data is dependent on parameters, but
also what sources of noise or model uncertainty are present in the postulated
relationship between unknown parameters and data. For example, if  the desired
forward problem cannot be solved analytically, then the forward model may be approximated by a simulation; in this case
discretization may be considered as a source of error. Once a relationship between model parameters, sources of error, and data is clearly defined, the inverse problem of estimating parameters from data can be addressed. The theory of
inverse problems can be separated into two cases: (1) the ideal case where data is not corrupted by noise and is derived from a known perfect model; and (2) the practical case where data is incomplete and imprecise. The first case is useful for classifying inverse problems and determining if a given set of observations
can, in principle, provide exact solutions; this provides insight into
conditions needed for existence, uniqueness, and stability of a solution.
The second case is useful for the formulation of practical algorithms
to learn about parameters, and uncertainties in their estimates, and will
be the focus of these notes.\\

A model that has the properties: (a) a solution map exists, (b) is unique, and (c) its behavior changes continuously with input (stability) is termed ``well-posed''. Conversely, a model lacking any of these properties is termed ``ill-posed''. Ill-posedness is present is many inverse problems,
and mitigating it is an extensive part of the subject.
Out of the different approaches for formulating an inverse problem, the Bayesian framework naturally offers the ability to assess quality in parameter estimation,
and also leads to a form of well-posedness at the level of probability
distributions describing the solution. The goal of the Bayesian framework is to find a probability measure that assigns a probability to each possible solution for a parameter $u$, given the
data $y$. Bayes formula states that
\begin{equation*}
\operatorname{\mathbbm{P}}(u \vert y) = \frac{1}{\operatorname{\mathbbm{P}}(y)} \operatorname{\mathbbm{P}}(y \vert u)\operatorname{\mathbbm{P}}(u).
\end{equation*}
It enables calculation of the posterior probability on
$u\vert y$, $\operatorname{\mathbbm{P}}(u \vert y) $, in terms of the
product of the data likelihood $\operatorname{\mathbbm{P}}(y \vert u)$ and the
prior information on the parameter encoded in $\operatorname{\mathbbm{P}}(u)$. The likelihood
describes the probability of the observed data $y$, if the input parameter
were set to be $u$; it is determined by the forward model, and the
structure of the noise. The normalization constant $\operatorname{\mathbbm{P}}(y)$ ensures that
$\operatorname{\mathbbm{P}}(u \vert y)$  is a probability measure. There are five primary benefits to this framework: (a) it provides a clear theoretical setting in which the
forward model choice, noise model and a priori information are explicit;
(b) it provides information about the entire solution space for possible
input parameter choices; (c) it naturally leads to quantification of
uncertainty and risk in parameter estimates; (d) it is generalizable to a wide class of inverse problems, in finite and infinite dimension and comes
with a well-posedness theory useful in these contexts.

The first half of the notes is dedicated to studying the Bayesian framework for inverse problems. Techniques such as importance sampling and Markov
Chain Monte Carlo (MCMC) methods are introduced; these methods have the
desirable property that in the limit of an infinite number of samples
they reproduce the full posterior distribution.  Since it is often
computationally intensive to implement these methods, especially in
high dimensional problems, approximate techniques such as approximating
the posterior by a Dirac or a Gaussian distribution are discussed.

The second half of the notes covers data assimilation.
This refers to a particular class of inverse problems in which
the unknown parameter is the initial condition of a dynamical system,
and in the stochastic dynamics case the subsequent states of the system,
and the data comprises partial and noisy observations of that
(possibly stochastic) dynamical system. A primary use of data assimilation is in forecasting, where the purpose is to provide better future estimates than can be obtained using
either the data or the model alone.
All the methods from the first half of the course
may be applied directly, but there are other new methods which
exploit the Markovian structure to update the state of the system
sequentially, rather than to learn about the initial condition. (But
of course knowledge of the initial condition may be used to inform
the state of the system at later times.)
We will also demonstrate that methods developed in
data assimilation may be employed to study generic inverse problems,
by introducing an artificial time to generate a sequence of
probability measures interpolating from the prior to the posterior.\\

\noindent{\bf Notation}
\vspace{0.05in}

Throughout the notes we use $\mathbb{N}$ to denote the positive integers $\{1,2,3, \cdots \}$
and $\mathbb{Z}^+$ to denote the non-negative integers $\mathbb{N} \cup \{0\}=\{0,1,2,3, \cdots \}.$
The matrix $I_d$ denotes the identity on $\mathbbm{R}^\Du.$
We use $|\cdot|$ to denote the Euclidean norm corresponding to the
inner-product $\langle \cdot, \cdot \rangle.$   A square matrix $A$
is positive definite (resp. positive semi-definite) if the quadratic
form $\langle u, Au \rangle$ is positive (resp. non-negative)
for all $u \ne 0$. By
$| \cdot |_A $ we denote the weighted norm defined by $|v|_A^2  = v^T A^{-1} v $.
The corresponding weighted Euclidean inner-product is given by
$\langle \cdot,  \cdot \rangle_{A}$ and $\langle \cdot, A^{-1}\cdot \rangle.$
We use $\otimes$ to denote the outer product between
two vectors: $(a \otimes b)c=\langle b,c \rangle a.$
We let $B(u,\delta)$ denote the open ball of radius $\delta$ at $u$, in the
Euclidean norm.

Throughout we denote by $\operatorname{\mathbbm{P}} (\cdot), \operatorname{\mathbbm{P}}(\cdot | \cdot )$ the pdf of
a random variable and its conditional pdf, respectively.
We write
$$ \rho(f)  =\operatorname{\mathbb{E}}^\rho[f]=\int_{\mathbbm{R}^\Du}f(u)\rho(u)du$$
to denote expectation of $f: \mathbbm{R}^\Du \mapsto  \mathbb{R}$
with respect to probability measure with probability density function (pdf)
$\rho$ on $\mathbbm{R}^\Du.$
The distribution of the random variables in this book will often have density with respect to Lebesgue
measure, but occasional use of Dirac masses will be required; we will
use the notational convention that Dirac mass at point $v$
has ``density'' $\delta(\cdot-v)$ or $\delta_v(\cdot).$ When a random
variable $u$ has pdf $\rho$
we will write $u \sim \rho.$ We use $\Rightarrow$ to denote weak
convergence of probability measures,  that is, $\rho_n \Rightarrow \rho$ if $\rho_n(f) \to \rho(f)$ for all bounded and continuous $f: \mathbbm{R}^\Du \mapsto  \mathbbm{R}$. \\

\noindent{\bf Acknowledgements}
\vspace{0.05in}

These notes were first created out of Caltech course ACM 159 in Fall 2017 and were further developed for the University of Chicago course STAT 31550 in Winter 2019. The
notes were created in \LaTeX \, by the students in ACM 159,
based on lectures presented by the instructor Andrew Stuart, and
on input from the course TA Armeen Taeb. The individuals responsible for the notes listed in alphabetic order are: Blancquart, Paul; Cai, Karena; Chen, Jiajie; Cheng, Richard; Cheng, Rui; Feldstein, Jonathan; Huang, De; Id{\'i}ni, Benjamin; Kovachki, Nikola; Lee, Marcus; Levy, Gabriel; Li, Liuchi; Muir, Jack; Ren, Cindy; Seylabi, Elnaz, Sch{\"a}fer, Florian; Singhal, Vipul; Stephenson, Oliver; Song, Yichuan;  Su, Yu; Teke, Oguzhan; Williams, Ethan; Wray, Parker; Zhan, Eric; Zhang, Shumao; Xiao, Fangzhou. Furthermore, the following students have added content beyond the class materials: Parker Wray -- the Overview, Jiajie Chen -- alternative proof of Theorem 1.10 and proof idea for Theorem 14.3, Fangzhou Xiao -- numerical simulation of prior, likelihood $\&$ posterior, Elnaz Seylabi $\&$ Fangzhou Xiao -- catching typographical errors in a draft of these notes, Cindy Ren -- numerical simulations to enhance understanding of importance sampling in Examples 6.2 and 6.5,  Cindy Ren $\&$ De Huang -- improving the constants in Theorem 6.3 regarding the approximation error of importance sampling, Richard Cheng $\&$ Florian Sch{\"a}fer -- illustrations to enhance understanding of the coupling argument used to study convergence of MCMC algorithms by presenting the finite state-space case, and Ethan Williams $\&$ Jack Muir -- numerical simulations and illustrations of Ensemble Kalman Filter and Extended Kalman Filter.
We would also like to thank Tapio Helin (Helsinki) who used the notes in his own course and
provided very helpful feedback on an early draft.

The work of AS has been funded by the EPSRC (UK), ERC (Europe) and by AFOSR, ARL, NIH, NSF and ONR (USA). The work of DSA is funded by NSF. This funded research has helped to shape the
presentation of the material here and is gratefully acknowledged. \\[0.1in]

\vspace{0.05in}
\noindent{\bf Warning}

\vspace{0.05in}
These are rough notes, far from being perfected. They are likely
to contain mathematical errors, incomplete bibliographical information,
inconsistencies in notation and typographical errors. We hope that the notes are nonetheless useful. Please contact the authors with any feedback from typos, through mathematical errors and bibliographical omissions, to
comments on the structural organization of the material.
\section{\Large{\sffamily{The Linear-Gaussian Setting}}}
Recall the stochastic dynamics and data models introduced in the previous chapter:
\begin{align}\label{eq:models}
\begin{split}
        v_{j+1} &= \Psi(v_j) + \xi_j,   \quad \quad \,\, \, \, \;  \xi_j \sim \mathcal{N}(0, \Sigma) \text{ i.i.d.},  \\
        y_{j+1} &= h(v_{j+1}) + \eta_j,  \quad \quad \, \eta_j \sim \mathcal{N}(0, \Gamma) \text{ i.i.d.},
\end{split}
\end{align}
with \(v_0 \sim \mathcal{N}(m_0, C_0),\) $C_0, \Sigma$ and $\Gamma$ positive definite and $v_0 \perp \{\xi_j\} \perp \{\eta_j\}.$
Here we study the filtering and smoothing problems under the assumption that both the state-transition operator \(\Psi(\cdot)\) and the observation operator \(h(\cdot)\) are  linear. Throughout, we will assume the following:
\begin{assumption}
The stochastic dynamics and the data models defined by equation \eqref{eq:models} hold. Moreover,
\label{a:kf}
\begin{description}
	\item[1] Linear dynamics:  $v_{j+1} = Mv_j + \xi_j$ for some $\mtx{M}\in\mathbbm{R}^{d \times d}$;
    \item[2] Linear obervation: $y_{j+1} = H v_{j+1} + \eta_{j+1}$ for some $H \in \mathbbm{R}^{k\times d}.$
\end{description}
\end{assumption}
We will be mostly concerned with the case where $d>k.$
Under the \emph{linear-Gaussian} assumption, the filtering and smoothing distributions are  Gaussian and therefore are fully characterized by their mean and covariance.
We consider first the \emph{Kalman filter}, which gives explicit formulae for the iterative update of the mean and covariance of the filtering distribution, and then the {\emph{Kalman smoother}, which characterizes the smoothing distribution.
While the Kalman filter and the Kalman smoother only characterize the filtering and smoothing distributions in the linear-Gaussian setting, their importance extends beyond this setting, as will be demonstrated in the next two chapters.

\subsection{Kalman Filter}
The {\em filtering problem} is to estimate the state at time \(j\) given the data from the past up to the present time \(j\). That is, we want to determine the pdf \(\pi_j = \operatorname{\mathbbm{P}}(v_j|Y_j)\), where $Y_j := \{y_1,\hdots,y_j\}.$
We  define $\hat{\pi}_{j+1}=\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)$ and recall
the evolution $$\pi_{j+1} = {\mathcal{A}}_j {\mathcal{P}} \pi_j, \quad \pi_0 = \mathcal{N}(m_0, C_0),$$
which can be decomposed in terms of the prediction and analysis steps \eqref{eq:pna}.
Note that \({\mathcal{P}}\) does not depend on $j$ because the same Markov process defined by the state dynamics governs each prediction step, whereas \({\mathcal{A}}_j\) depends on $j$ because at each step $j$ the likelihood sees different data. The linear dynamics assumption implies that applying the operator ${\mathcal{P}}$ to a Gaussian distribution gives again a Gaussian, and the linear observation assumption implies that applying the operator ${\mathcal{A}}_j$ to a Gaussian gives again a Gaussian. Thefore we have the following:

\begin{theorem}(Gaussianity of Filtering Distributions)
	Under Assumption \ref{a:kf},
$\pi_0$, \(\{\pi_{j+1}\}_{j \in \mathbbm{Z}^+}\) and \(\{\hat{\pi}_{j+1}\}_{j \in \mathbbm{Z}^+}\) are all Gaussian distributions.
\end{theorem}

As a consequence, the filtering distributions can be entirely characterized by
their mean and covariance. We write
\begin{align*}
	\hat{\pi}_{j+1} &= \operatorname{\mathbbm{P}}(v_{j+1}|Y_j) = \mathcal{N}(\hat{m}_{j+1}, \hat{C}_{j+1}), && \text{(prediction)}\\
	\pi_{j+1} &= \operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1}) = \mathcal{N}(m_{j+1}, C_{j+1}), && \text{(analysis)}
\end{align*}
and aim to find update formulae for these means and covariances.
The Kalman filter algorithm achieves this.

\begin{theorem}[Kalman filter]
	\label{kalmanfilter}
Under Assumption \ref{a:kf}
for all $j \in \mathbbm{Z}^+$, $C_j$ is positive definite and
\begin{align*}
\hat{m}_{j+1} &= Mm_j, \\
\hat{C}_{j+1} &= MC_jM^T+\Sigma, \\
C_{j+1}^{-1} &= (MC_jM^T + \Sigma)^{-1} + H^T \Gamma^{-1} H,\\
C_{j+1}^{-1}m_{j+1} &= (MC_jM^T + \Sigma)^{-1} M m_j + H^T \Gamma^{-1} y_{j+1}.
\end{align*}
\end{theorem}

\begin{proof}
The proof proceeds by breaking the Kalman filter step above into the prediction and the analysis steps.

\noindent \textbf{Prediction}: The mean and variance of the prediction
step may be calculated as follows. The mean is given by:
\begin{align*}
	\hat{m}_{j+1} &= \operatorname{\mathbb{E}} \big[ v_{j+1} | Y_j \big] \\
	&= \operatorname{\mathbb{E}} \big[ M v_j + \xi_j | Y_j \big]  \\
	&= M \operatorname{\mathbb{E}} \big[ v_j | Y_j \big] + \operatorname{\mathbb{E}} \big[\xi_j | Y_j \big] \\
	&= Mm_j,
\end{align*}
where we used that \(\xi_j\) and \(Y_j\) are independent.
The covariance is given by:
\begin{align*}
	\hat{C}_{j+1} =& \operatorname{\mathbb{E}} \big[ (v_{j+1} - \hat{m}_{j+1}) \otimes (v_{j+1} - \hat{m}_{j+1}) | Y_j \big] \\
	=& \operatorname{\mathbb{E}} \big[ M(v_j - m_j) \otimes M(v_j - m_j) | Y_j \big] + \operatorname{\mathbb{E}} \big[ \xi_j \otimes\xi_j | Y_j \big] \\
	&+ \operatorname{\mathbb{E}} \big[ \xi_j \otimes M(v_j - m_j) | Y_j \big] + \operatorname{\mathbb{E}} \big[ M(v_j - m_j) \otimes\xi_j | Y_j \big] \\
	=& M \operatorname{\mathbb{E}} \big[(v_j - m_j) \otimes (v_j - m_j) | Y_j \big] M^T + \Sigma   \\
	=& M C_j M^T + \Sigma,
\end{align*}
where we used that \(\xi_j\) and \(v_j\) are independent.
Thus in the linear Gaussian setting
the prediction operator \({\mathcal{P}}\) from \(\pi_j = \mathcal{N}(m_j, C_j)\) to \(\hat{\pi}_{j+1} = \mathcal{N}(\hat{m}_{j+1}, \hat{C}_{j+1})\) is given by
\begin{align*}
	\hat{m}_{j+1} &= M m_j, \\
	\hat{C}_{j+1} &= M C_j M^T + \Sigma.
\end{align*}

\noindent \textbf{Analysis}: The analysis step may be derived as follows,
using Bayes Theorem:
\begin{align*}
	\operatorname{\mathbbm{P}}(v_{j+1} | Y_{j+1}) &= \operatorname{\mathbbm{P}}(v_{j+1} | y_{j+1},Y_{j})\\
& \propto \operatorname{\mathbbm{P}}(y_{j+1} | v_{j+1}, Y_j) \operatorname{\mathbbm{P}}(v_{j+1} | Y_j) \\
	& = \operatorname{\mathbbm{P}}(y_{j+1} | v_{j+1}) \operatorname{\mathbbm{P}}(v_{j+1} | Y_j).
\end{align*}
This gives
\begin{align}\label{eq:smoot}
\begin{split}
\operatorname{\mathbbm{P}}(v_{j+1} | Y_{j+1}) &\propto \exp\left(-\frac{1}{2} |v_{j+1} - m_{j+1}|_{C_{j+1}}^2 \right)  \\
&\propto \exp\left(-\frac{1}{2} |y_{j+1} - H v_{j+1}|_{\Gamma}^2 \right) \exp\left(-\frac{1}{2} |v_{j+1} - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2 \right) \\
	&= \exp\left(-\frac{1}{2} |y_{j+1} - H v_{j+1}|_{\Gamma}^2 -\frac{1}{2} |v_{j+1} - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2 \right).
	\end{split}
\end{align}
Taking logartihms and
matching quadratic and linear terms in $v_{j+1}$ from either side of this
identity gives the update operator \({\mathcal{A}}_j\) from \(\hat{\pi}_{j+1} = \mathcal{N}(\hat{m}_{j+1}, \hat{C}_{j+1})\) to \(\pi_{j+1} = \mathcal{N}(m_{j+1}, C_{j+1})\):
\begin{align*}
	C_{j+1}^{-1} &= \hat{C}_{j+1}^{-1} + H^T \Gamma^{-1} H, \\
	C_{j+1}^{-1} m_{j+1} &= \hat{C}_{j+1}^{-1} \hat{m}_{j+1} + H^T \Gamma^{-1} y_{j+1}.
\end{align*}
Combining the prediction operator \({\mathcal{P}}\) and update operator \({\mathcal{A}}_j\) yields the desired update formulae.

\noindent \textbf{Positive-definiteness of \(C_j\)}: It remains to show that \(C_j > 0\) for all \(j \in \mathbbm{Z}^+\). We will use induction. By assumption the
result holds true for \(j = 0\). Assume that it is true for \(C_j\). For
the prediction operator \({\mathcal{P}}\) we have, for $u \ne 0$,
\begin{align*}
	\langle u, \hat{C}_{j+1} u \rangle &= \langle u, M C_j M^T u \rangle + \langle u, \Sigma u \rangle \\
	&= \langle M^T u, C_j  M^T u \rangle + \langle u, \Sigma u \rangle \\
        & \ge \langle u, \Sigma u \rangle \\
	&> 0,
\end{align*}
where we used that $C_j>0$ and $\Sigma>0.$
Therefore \(\hat{C}_{j+1}, \hat{C}_{j+1}^{-1} > 0\). Then for the update operator \({\mathcal{A}}_j\):
\begin{align*}
	\langle u, C_{j+1}^{-1} u \rangle &= \langle u, \hat{C}_{j+1}^{-1} u \rangle + \langle u, H^T \Gamma^{-1} H u \rangle \\
	&= \langle u, \hat{C}_{j+1}^{-1} u \rangle + \langle H u, \Gamma^{-1} H u \rangle \\
        &\ge \langle u, \hat{C}_{j+1}^{-1} u \rangle \\
	&> 0,
\end{align*}
where we used that $\Gamma>0.$ Therefore, \(C_{j+1}, C_{j+1}^{-1}> 0\), which concludes the proof.
\end{proof}

\begin{remark}
The previous proof reveals two interesting facts about the structure of the Kalman filter updates. The first is that the covariance update does not involve the observed data; this can be thought of as a consequence of the fact that the posterior covariance in the linear-Gaussian setting for inverse problems does not depend on the observed data, as noted in Chapter 2.  The second is that the update formulae for the covariance are linear in the prediction step, but nonlinear in the analysis step; specifically the analysis step is linear in the precisions (inverse covariances).
\end{remark}

\subsubsection{Kalman Filter: Alternative Formulation}
We now rewrite the Kalman filter in an alternative form. This formulation
is written in terms of the covariances directly, and does not involve
the precisions. Furthermore, the formulation in Theorem \ref{kalmanfilter} involves a matrix inversion in the state space while the one that we will present here requires
only inversion the data space, namely \(S_{j+1}^{-1}\) in what follows.
In many applications the observation space dimension is much smaller
than the state space dimension (\(k \ll d\)), and then the
formulation given in this section is
much cheaper to compute than the one given in Theorem \ref{kalmanfilter}.

\begin{corollary}[Standard Form of Kalman Filter]
\label{cor:k2}
	Under Assumption \ref{kalmanfilter}, the Kalman
update formulae may be written as
	\begin{align}\label{eq:kf update}
	\begin{split}
		m_{j+1} &= \hat{m}_{j+1} + K_{j+1} d_{j+1}, \\
		C_{j+1} &= (I - K_{j+1} H) \hat{C}_{j+1},
		\end{split}
	\end{align}
where
	\begin{align*}
		d_{j+1} &= y_{j+1} - H \hat{m}_{j+1}, \\
		S_{j+1} &= H \hat{C}_{j+1} H^T + \Gamma, \\
		K_{j+1} &= \hat{C}_{j+1} H^T S_{j+1}^{-1},
	\end{align*}
and where \(\hat{m}_{j+1}, \hat{C}_{j+1}\) are defined as in the proof of Theorem \ref{kalmanfilter}.
\end{corollary}

\begin{remark} The vector \(d_{j+1}\) is known as the {\em innovation}
and the matrix \(K_{j+1}\) as the {\em Kalman gain}. Note that
\(d_{j+1}\) measures the mismatch of the predicted state from the given data.
\end{remark}

The corollary may be proved by an application of the following:

\begin{lemma}
	(Woodbury Matrix Identity) Let \(A \in \mathbbm{R}^{p \times p}\), \(U \in \mathbbm{R}^{p \times q}\), \(B \in \mathbbm{R}^{q \times q}\), \(V \in \mathbbm{R}^{q \times p}\). If \(A, B > 0\), then \(A + UBV\) is invertible and
	\[(A + UBV)^{-1} = A^{-1} - A^{-1}U (B^{-1} + VA^{-1}U)^{-1} V A^{-1}.\]
\end{lemma}

Combining the form of $d_{j+1}$ and $\hat{m}_{j+1}$ shows that the update
formula for the Kalman mean can be written as
\begin{equation}
\label{eq:kmu}
m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}, \quad \hat{m}_{j+1}=Mm_j.
\end{equation}
This update formula has the very natural interpretation that the mean update
is formed as a linear combination of the evolution of the noise-free dynamics
and of the data. Equations \eqref{eq:kf update} and \eqref{eq:kmu} show that the Kalman gain $K_{j+1}$ determines the weight given to the new observation $y_{j+1}$ in the state estimation.
The update formula \eqref{eq:kmu} may be also derived from an optimization perspective,
the topic of the next subsection.

\subsubsection{Optimization Perspective: Mean of  Kalman Filter}
\label{sec:okf}
Since \(\pi_{j+1}\) is Gaussian, its mean agrees with its mode. Thus, formulae \eqref{eq:smoot} implies that
\begin{align*}
	m_{j+1} &=  {\rm argmax}_v \pi_{j+1}(v)  \\
	& = {\rm argmin}_v {\mathsf{J}}(v),
\end{align*}
where
$${\mathsf{J}}(v):= \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2.$$
In other words, \(m_{j+1}\) is chosen to fit both the observed data
\(y_{j+1}\) and the predictions \(\hat{m}_{j+1}\) as well as possible.
The covariances \(\Gamma\) and \(\hat{C}_{j+1}\)  determine
the relative weighting between the two quadratic terms.
The solution of the minimization problem is given by \eqref{eq:kmu}, as may be verified by direct differentiation of ${\mathsf{J}}.$

An alternative derivation which is helpful in more sophisticated contexts is to cast the problem in terms of constrained minimization. Write $v'=v-\hat{m}_{j+1}$, $y'=y_{j+1} - H \hat{m}_{j+1}$ and
$C'=\hat{C}_{j+1}.$ Then
 minimization of ${\mathsf{J}}$ may be reformulated as
\begin{align*}
        m_{j+1} = \hat{m}_{j+1}+{\rm argmin}_{v'} \left( \frac{1}{2} |y' - H v'|_{\Gamma}^2 + \frac{1}{2} \langle v', b \rangle \right),
\end{align*}
where the minimization is now subject to the constraint $C'b=v'.$ Using
Lagrange multipliers we write
\begin{equation}
\label{eq:LM}
{\mathsf{I}}(v')= \frac{1}{2} |y' - H v'|_{\Gamma}^2 + \frac{1}{2} \langle v', b \rangle
+\langle \lambda, C'b-v' \rangle;
\end{equation}
computing the dervative and setting to zero gives
\begin{subequations}
\label{eq:LM_EL}
\begin{align*}
-H^T\Gamma^{-1}(y'-Hv')+\frac12 b-\lambda & =0,\\
\frac12 v'+C'\lambda &=0,\\
v'-C'b &=0.
\end{align*}
\end{subequations}
The last two equations imply that $C'(2\lambda+b)=0.$ Thus we set
$\lambda=-\frac{1}{2}b$ and drop the second equation, replacing the first by
$$-H^T\Gamma^{-1}(y'-HC'b)+b=0.$$
Solving for $b$ gives
\begin{align*}
v&=\hat{m}_{j+1}+v'\\
&=\hat{m}_{j+1}+C'b\\
&=\hat{m}_{j+1}+C'(H^T\Gamma^{-1}HC'+I)^{-1}H^T\Gamma^{-1}y'\\
&=\hat{m}_{j+1}+C'(H^T\Gamma^{-1}HC'+I)^{-1}H^T\Gamma^{-1}(y_{j+1} - H \hat{m}_{j+1})\\
&=(I-K_{j+1}H)\hat{m}_{j+1}+K_{j+1}y_{j+1},
\end{align*}
where we have defined
$$K_{j+1}=C'(H^T\Gamma^{-1}HC'+I)^{-1}H^T\Gamma^{-1}.$$
It remains to show that $K_{j+1}$ agrees with the definition given
in Corollary \ref{cor:k2}. To see this we note that if we choose $S$ to
be any matrix satisfying
$K_{j+1}=C'H^T S^{-1}$ then
$$H^T S^{-1}=(H^T\Gamma^{-1}HC'+I)^{-1}H^T\Gamma^{-1}$$
so that
$$(H^T\Gamma^{-1}HC'+I)H^T=H^T\Gamma^{-1} S.$$
Thus
$$H^T\Gamma^{-1}HC'H^T+H^T=H^T\Gamma^{-1}S$$
which may be achieved by choosing any $S$ so that
$$\Gamma^{-1}(HC'H^T+\Gamma)=\Gamma^{-1}S$$
and multiplication by $\Gamma$ gives the desired formula for $S$.

\subsubsection{Optimality of Kalman Filter}

The following theorem states that the Kalman filter gives the best
estimator of the mean in an online setting.
In the following $\operatorname{\mathbb{E}}$ denotes expectation with respect to
all randomness present in the problem statement, through the initial
condition, the noisy dynamical evolution, and the noisy data. Furthermore
$\operatorname{\mathbb{E}} [\cdot|Y_j]$ denotes conditional expectation, given
the data $Y_j$ upto time $j$.

\begin{theorem}[Optimality of Kalman Filter]
	Let \(\{m_j\}\) be the sequence computed using the Kalman filter, and \(\{z_j\}\) be any sequence in \(\mathbbm{R}^\Du\) such that $z_j$ is $Y_j$
measurable.\footnote{For practical purposes this means $z_j$ is a fixed
non-random function of given observed $Y_j.$} Then, for all $j \in \mathbbm{N},$
	\[ \operatorname{\mathbb{E}} \Big[ |v_j - m_j |^2 \mid Y_j \Big] \leq \operatorname{\mathbb{E}} \Big[ |v_j - z_j |^2 \mid Y_j \Big].\]
\end{theorem}

\begin{proof}
Note that $m_j$ and $z_j$ are fixed and non-random, given $Y_j$. Thus we have:
\begin{align*}
	\operatorname{\mathbb{E}} \Big[ |v_j - z_j |^2 \mid Y_j \Big] =& \operatorname{\mathbb{E}} \Big[ |v_j - m_j + m_j - z_j |^2 \mid Y_j \Big] \\
	=& \operatorname{\mathbb{E}} \Big[ |v_j - m_j|^2 + 2 \Big\langle v_j - m_j, m_j - z_j
\Big \rangle + |m_j - z_j|^2 \mid Y_j \Big] \\
 =& \operatorname{\mathbb{E}} \Big[ |v_j - m_j|^2 \mid Y_j \Big] + 2 \Big \langle \operatorname{\mathbb{E}} \Big[ v_j - m_j \mid Y_j \Big] , m_j - z_j \Big \rangle + |m_j - z_j|^2 \\
	=& \operatorname{\mathbb{E}} \Big[ |v_j - m_j|^2 \mid Y_j \Big] + 2 \Big \langle \operatorname{\mathbb{E}} \Big[ v_j \mid Y_j \Big]-m_j , m_j - z_j \Big \rangle + |m_j - z_j|^2 \\
	=& \operatorname{\mathbb{E}} \Big[ |v_j - m_j|^2 \mid Y_j \Big] + 0 + |m_j - z_j|^2 \\
	\geq & \operatorname{\mathbb{E}} \Big[ |v_j - m_j|^2 \mid Y_j \Big].
\end{align*}
The fifth step follows since \(m_j = \operatorname{\mathbb{E}} \big[ v_j \mid Y_j \big].\)
\end{proof}

\subsection{Kalman Smoother}
We next discuss the Kalman smoother, which refers to the smoothing problem in the linear-Gaussian setting. As with the Kalman filter, it is possible
to solve the problem explicitly because the smoothing distribution is itself a Gaussian. The explicit formulae computed help to build
intuition about the smoothing distribution more generally. We recall remark \ref{rem:marginal}, which implies that the filtering distribution at time $j=J$ determines the marginal of the Kalman smoother on its last coordinate. However, the filtering distributions do not determine the
Kalman smoother in its entirety.

\subsubsection{Defining Linear System}
Let $V= \{v_0,\ldots,v_J\}$ and $Y =\{y_1,\ldots, y_J\}$. Using Bayes theorem
and the fact that $\{\xi_j\}$, $\{\eta_j\}$ are mutually independent
i.i.d. sequences, independent of $v_0$, we have
\begin{equation*}
\operatorname{\mathbbm{P}} (V|Y)\propto \operatorname{\mathbbm{P}}(Y|V) \operatorname{\mathbbm{P}}(V)= \prod\limits_{j=1}^J\operatorname{\mathbbm{P}}(y_j|v_j)\times \prod\limits_{j=0}^{J-1} \operatorname{\mathbbm{P}} (v_{j+1}|v_j)\times \operatorname{\mathbbm{P}} (v_0).
\end{equation*}
Noting that
$$v_{j+1}|v_j \sim \mathcal{N}(Mv_j,\Sigma), \quad y_j|v_j \sim \mathcal{N}(Hv_j,\Gamma)$$
the Kalman smoothing distribution can be expressed as
\begin{equation}
\operatorname{\mathbbm{P}} (V|Y) \propto \exp\bigl(-{\mathsf{J}}(V)\bigr),
\end{equation}
where
\begin{equation}\label{eq:jformula}
{\mathsf{J}}(V):=\frac{1}{2}|v_0-m_0|^{2}_{C_0}+\frac{1}{2} \sum\limits_{j=0}^{J-1}|v_{j+1}-Mv_j|^{2}_{\Sigma}+\frac{1}{2}\sum\limits_{j=0}^{J-1}|y_{j+1}-Hv_{j+1}|^{2}_{\Gamma}.
\end{equation}

\begin{theorem}[Characterization of the Kalman Smoother]
Suppose that Assumption \ref{a:kf} holds. Then $\operatorname{\mathbbm{P}}(V | Y)$ is Gaussian with a block tridiagonal precision matrix $\Omega>0$ and  mean $m$ solving $\Omega m=r$, where
\begin{align}
\Omega  =& \begin{bmatrix}
 \Omega_{0,0} & \Omega_{0,1} &  &  &  &  \\
 \Omega _{1,0} & \Omega_{1,1} & ... &  & 0  &  \\
 0 & ... & ... &  &  &  \\
  & 0 & ... & ... & ... &  \\
  &  &  & ... & \Omega_{J-1,J-1} & \Omega_{J-1,J} \\
  &  &  &  & \Omega_{J,J-1} & \Omega_{J,J}
 \end{bmatrix}
\end{align}
with
\begin{align*}
\Omega_{0,0}&=C_0^{-1} +M^T\Sigma^{-1}M,\\
\Omega_{j,j}&=\Sigma^{-1}+M^T\Sigma^{-1}M+H^T\Gamma^{-1}H, \quad  1\leq j \leq J-1,\\
\Omega_{J,J}&=\Sigma^{-1}+H^T\Gamma^{-1}H,\\
\Omega_{j,j+1}&=-M^T\Sigma^{-1}, \quad  1\leq j\leq J-1,\\
r_0&=C_0^{-1}m_0,\\
r_j&=H^T\Gamma^{-1}y_j, \quad  1\leq j\leq J.
\end{align*}
\end{theorem}

\begin{proof}
We may write ${\mathsf{J}}(V)=\frac{1}{2}\vert \Omega^{1/2}(V-m)\vert^2 +q$ with $q$ independent of $V$, by definition. Note that  $\Omega$ is then the Hessian of ${\mathsf{J}}(V)$, and differentiating in equation \eqref{eq:jformula} we obtain that
\begin{align*}
\Omega_{0,0}&= \partial_{v_0}^2{\mathsf{J}}(V)=C_0^{-1} + M^T\Sigma^{-1}M,\\
\Omega_{j,j}&= \partial_{v_j}^2{\mathsf{J}}(V)=\Sigma^{-1} + M^T\Sigma^{-1}M+H^T\Gamma^{-1}H,\\
\Omega_{J,J}&= \partial_{v_J}^2{\mathsf{J}}(V)=\Sigma^{-1} +H^T\Gamma^{-1}H,\\
\Omega_{j-1,j}&= \Omega_{j,j-1}=\partial_{v_j,v_{j-1}}^2{\mathsf{J}}(V)=-M^T\Sigma^{-1}.
\end{align*}
Otherwise, for all other values of indices $\{k,l\}$, $\Omega_{k,l}=0.$ This proves that the matrix $\Omega$ has a block tridiagonal structure.

Now we focus on finding $m$. We have that $\nabla_V{\mathsf{J}}(V)=\Omega(V-m)$, so that $-\nabla_V{\mathsf{J}}(V)\vert_{V=0}=\Omega m$. Thus, we find $r$ as
\begin{align*}
r_0=&-\nabla_{v_0}{\mathsf{J}}(V)\vert_{V=0}=-(-C_0^{-1}m_0)=C_0^{-1}m_0,\\
r_j=&-\nabla_{v_j}{\mathsf{J}}(V)\vert_{V=0}=-(-H^T\Gamma^{-1}y_j)=H^T\Gamma^{-1}y_j.
\end{align*}
We have shown that $\Omega$ is symmetric and that  $\Omega\ge 0$; to prove that $\Omega$ is a
precision matrix, we need to show that $\Omega>0.$ Take, for the sake of argument, $Y = 0$ and $m_0=0$ in equation \eqref{eq:jformula}, so that every term in the expansion of ${\mathsf{J}}(V)$ involves $V$. It is evident that in such case ${\mathsf{J}}(V) = V^T\Omega V$. Suppose that $V^T\Omega V = 0$ for some nonzero $V$. Then by positive-definiteness of $C_0, \Sigma$, and $\Gamma$, it must be that $v_0 = 0$ and $v_{j+1} = M v_j$ for $j = 0,1,\dots,J$. Thus we must have $V = 0$. This proves that $\Omega$ is positive-definite.
\end{proof}
\begin{remark}
We note once again that since the smoothing distribution in the linear-Gaussian setting is itself Gaussian, its mean agrees with its mode. Therefore, the posterior mean found above is the unique minimizer of ${\mathsf{J}}(V),$ that is, the MAP estimator.
\end{remark}
\subsubsection{Solution of the Linear System}

We may also obtain $m$ via Gaussian elimination, using
the block tridiagonal structure of $\Omega$, as follows.
First we form the matrix sequence $\{\Omega_j\}$:
\begin{align}
\Omega_0 &=\Omega_{0,0}, \nonumber\\
\Omega_{j+1} &=\Omega_{j+1,j+1}-\Sigma^{-1}M^T\Omega_j^{-1}M^T\Sigma^{-1}, \quad j=0,\dots, J-1;
\label{eqn:Ljm}
\end{align}
and we form the vector sequence $\{z_j\}$:
\begin{align*}
z_0&= C_0^{-1}m_0, \\
z_{j+1}&= H^T\Gamma^{-1}y_{j+1}-\Sigma^{-1}M^T\Omega_j^{-1}z_j.
\end{align*}
We may then read off $m_J$ from the equation $\Omega_Jm_J=z_J$ and
finally we perform back-substitution to obtain
$$\Omega_jm_j=z_j-\Omega_{j,j+1}m_{j+1}, \quad j=J-1,\ldots,1.$$
Note that $m_J$ found this way coincides with the mean of the Kalman
filter at $j=J.$

\begin{proposition}
The matrices $\{\Omega_j\}$ in \eqref{eqn:Ljm} are positive definite.
\end{proposition}
\begin{proof}
The proof of this theorem relies on the following two lemmas:
\begin{lemma}\label{lemm1}
If $$X:=\begin{bmatrix}
X_1 &  \times& \times & \times \\
\times & X_2 & \times & \times \\
\times & \times & ... &\times  \\
\times & \times & \times & X_d
\end{bmatrix}$$ is positive-definite symmetric then $X_i$ is
positive-definite symmetric for all $i \in \{1,\dots, d\}.$
\end{lemma}
\begin{lemma} \label{lemm2}
Let $B$ be a block  lower (or upper) triangular matrix with identity on the
diagonal. Then, $B$ is an invertible matrix.
\end{lemma}
Using Lemma \ref{lemm1}, we deduce that $\Omega_0=\Omega_{0,0}$ is positive-definite symmetric. Consider the matrix $B\in\mathbbm{R}^{d(J+1)\times d(J+1)}$ defined as:
$$B= \begin{bmatrix}
 I & 0 & & 0 \\
 -\Omega_{1,0}\Omega_0^{-1} & I & ... & ... \\
 0 &  & ... & 0 \\
  &  & 0 & I
 \end{bmatrix}.$$
We compute
$$B\Omega B^T= \begin{bmatrix}
\Omega_0 & 0 & & &0 \\
0 & \Omega_1 & \Omega_{1,2} & ... & 0\\
& \Omega_{2,1} & \Omega_{2,2} & ... &\\
& & & & \\
0& & & \Omega_{J-1,J-1} &\Omega_{J-1,J} \\
0& & & \Omega_{J,J-1}& \Omega_{J,J}\\ \end{bmatrix}.$$
By Lemma \ref{lemm1}, the matrix
 $$\tilde{\Omega}=\begin{bmatrix}
\Omega_1 & \Omega_{1,2} & ... & 0\\
 \Omega_{2,1} & \Omega_{2,2} & ... &\\
 & & & \\
 & & \Omega_{J-1,J-1} &\Omega_{J-1,J} \\
 & & \Omega_{J,J-1}& \Omega_{J,J}\\ \end{bmatrix}$$ is positive definite,
and so is $\Omega_0.$

Lemma \ref{lemm1} and the positive definiteness of $\tilde{\Omega}$ imply that $\Omega_1$ is positive definite. Therefore, by Lemma \ref{lemm2} the matrix
 $$B_2=\begin{bmatrix}
I& 0 & & 0\\
-\Omega_{2,1}\Omega_1^{-1}& I & ... & ...\\
& & ... &0\\
0& & & I\\
\end{bmatrix}$$
is invertible.
Thus we have $$B_2 \tilde{\Omega}B_2^T = \begin{bmatrix}
\Omega_1 & 0 & & & 0\\
0 & \Omega_2 & \Omega_{2,3} & ... & \\
& \Omega_{3,2} & \Omega_{3,3} & & 0\\
& & & & \\
& & & \Omega_{J-1,J-1} &  \Omega_{J-1,J} \\
0& & & \Omega_{J,J-1}& \Omega_{J,J} \\
\end{bmatrix},
$$
giving the positive definiteness of $\Omega_2$. Iterating the argument shows that all the $\Omega_j$ are  positive definite.
\end{proof}

\subsection{Discussion and Bibliography}
The original paper of Kalman, which is arguably the first systematic
presentation of a  methodology to combine models with data,
is \cite{kalman1960new}. The continuous time analogue of
that work may be found in \cite{kalman1961new}.
The book \cite{harvey1990forecasting} overviews the subject in the
context of time-series analysis and economics.
The proof of Corollary \ref{cor:k2} may be found in \cite{law2015data}.
The paper \cite{sanz2015long} contains an application of the
optimality property of the Kalman filter (which applies beyond
the linear Gaussian setting to the mean of the filtering distribution
in quite general settings).

The subject of Kalman smoothing is overviewed in the text
\cite{anderson1979optimal}; see also \cite{rauch1965maximum}.
A link between the standard implementation of the smoother
and Gauss-Newton methods for MAP estimation is made in
\cite{bell1994iterated}. Fur further details on the
Kalman smoother, in both discrete and continuous time,
see \cite{law2015data} and \cite{hairer2005analysis}.

\section{\Large{\sffamily{Optimization for
Filtering and Smoothing: 3DVAR and 4DVAR}}}

In the previous chapter we showed how the mean of the Kalman
filter could be derived through an optimization principle,
once the predictive covariance is known.
In this chapter we discuss two optimization based approaches to
filtering and smoothing, namely the 3DVAR and 4DVAR methodologies.
We emphasize that the methods we present in this chapter do not provide
approximations of the filtering and smoothing probability distributions;
they simply provide estimates of the signal, given data, in the
filtering (on-line) and smoothing (off-line) data scenarios.

\subsection{The Setting}
3DVAR borrows from the Kalman filter optimization principle,
but substitutes a fixed given covariance for the predictive covariance.
Here ``VAR'' refers to variational, and encodes the concept of
optimization. The 3D and 4D, respectively, refer to three Euclidean
spatial dimensions and to three Euclidean
spatial dimensions plus a time dimension; this nomenclature reflects
the historical derivation of these problems in the geophysical sciences,
but the specific structure of fields over three dimensional Euclidean
space plays no role in the generalized form of the methods described
here. The key distinction is that 3DVAR solves a sequence of optimization
problems at each point in time (hence is an on-line filtering
method); in contrast 4DVAR solves an optimization
problem which involves data distributed over time (and is an off-line
smoothing method).

Throughout we consider the set-up in which the dynamics model
is nonlinear, but the observation operator is linear, commonly
occurring in applications. We thus have
a discrete-time dynamical system with noisy state
transitions and noisy observations given by
\begin{align*}
        \text{Dynamics Model:}  \quad v_{j+1} &= \Psi(v_j) + \xi_j, \quad j \in \mathbbm{Z}^+. \\
        \text{Data Model:}  \quad y_{j+1} &= Hv_{j+1} + \eta_{j+1}, \quad j \in \mathbbm{Z}^+, \text{ for some} \,\, H \in \mathbbm{R}^{k \times d}.  \\
        \text{Probabilistic Structure:}  \quad v_0 &\sim \mathcal{N}(m_0, C_0), \quad \xi_j \sim \mathcal{N}(0, \Sigma), \quad \eta_j \sim \mathcal{N}(0, \Gamma).\\
 \text{Probabilistic Structure:} \quad v_0 &\perp \{\xi_j\} \perp \{\eta_j\} \text{ independent.}
\end{align*}

\subsection{3DVAR}
We introduce 3DVAR by analogy with the update formula \eqref{eq:kmu}
for the Kalman filter, and its derivation through optimization from
section \ref{sec:okf}. The primary differences between 3DVAR and the Kalman
filter mean update is that $\Psi(\cdot)$ can be nonlinear for 3DVAR, and
that for 3DVAR we have no closed update formula for the covariances.
To deal with this second issue 3DVAR uses a fixed predicted covariance,
independent of time $j$, and pre-specified.
The resulting minimization problem, and
its solution, is described in Table \ref{3DVAR:t1}, making the
analogy with the Kalman filter.  Note that the minimization
itself is of a quadratic functional, and so may be solved by means of
linear algebra. The constraint formulation used for the Kalman filter,
in section \ref{sec:okf}, may also be applied and used to derive the mean
update formula.
\FloatBarrier
\begin{table}
	\begin{tabular}{ | c | c |}
		\hline
		Kalman Filter & 3DVAR  \\ \hline
		$m_{j+1} = \arg \min_v {\mathsf{J}}(v)$ & $m_{j+1} = \arg \min_v {\mathsf{J}}(v)$  \\ \hline
		${\mathsf{J}}(v) = \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2$  &  ${\mathsf{J}}(v) = \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}}^2$ \\
		\hline
		$\hat{m}_{j+1} = M m_j$ & $\hat{m}_{j+1} = \Psi(m_j)$ \\ \hline
		$m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}$ & $m_{j+1} = (I-KH) \hat{m}_{j+1} + K y_{j+1}$\\ \hline
	\end{tabular}
	\caption{Comparison of Kalman Filter and 3DVAR update formulae.}
		\label{3DVAR:t1}
\end{table}
\FloatBarrier

The Kalman gain $K$ for 3DVAR is fixed, because the predicted covariance
$\hat{C}$ is fixed. By analogy with Corollary \ref{cor:k2} we
have the following formulae for the 3DVAR gain matrix $K$, and the
update formula for the estimator $m_j$:
\begin{align*}
                S &= H \hat{C} H^T + \Gamma, \\
                K &= \hat{C} H^T S^{-1}, \\
m_{j+1} &= (I-KH) \Psi({m}_{j}) + K y_{j+1}.
        \end{align*}
The method also delivers an implied analysis covariance $C= (I - K H) \hat{C}.$
Note that the resulting algorithm which maps $m_j$ to $m_{j+1}$
may be specified directly in terms of the gain $K$, without
need to introduce $\hat{C}, C$ and $S$. In the remainder of this
section we simply view $K$ as fixed and given.
In this setting we
show that the 3DVAR algorithm produces accurate state estimation
under vanishing noise assumptions in the dynamics/data model.
The governing assumptions concerning the dynamics/data model
are encapsulated in:

\begin{assumption}
\label{a:3dvar}
Consider the dynamics/data model under the assumptions that
$\xi_j \equiv 0, \Gamma = \gamma^2 \Gamma_0, |\Gamma_0| = 1$
and assume that the data $y_{j+1}$ used in the 3DVAR
algorithm is found from observing
a true signal $v_j^{\dagger}$ given by
\begin{align*}
        \emph{Dynamics Model:}  \quad v_{j+1}^{\dagger} &= \Psi(v_j^{\dagger}), \quad j \in \mathbbm{Z}^+. \\
        \emph{Data Model:}  \quad y_{j+1} &= Hv_{j+1}^{\dagger} + \gamma  \eta^{\dagger}_{j+1,0}, \quad j \in \mathbbm{Z}^+.
\end{align*}
\end{assumption}

With this assumption of noise-free dynamics ($\xi_j \equiv 0$)
we deduce that the 3DVAR filter produces output which,
asymptotically, has an error of the same size as the observational noise
error $\gamma.$ The key additional assumption in the theorem that
allows this deduction is
a relationship between the Kalman gain $K$ and
the derivative $D\Psi(\cdot)$ of the dynamics model.
Encoded in the assumption are two ingredients: that the observation
operator $H$ is rich enough in principle to learn enough components
of the system to synchronize the whole system; and that $K$ is designed
cleverly enough to effect this synchronization. The proof of the
theorem is simply using these two ingredients and then controlling
the small stochastic perturbations, arising from Assumption \ref{a:3dvar}.

\begin{theorem}[Accuracy of 3DVAR]
Let Assumption \ref{a:3dvar} hold with $\eta^{\dagger}_{j,0} \sim \mathcal{N}(0,\Gamma_0).$ an i.i.d. sequence. Assume
that, for the gain matrix $K$ appearing in the 3DVAR method,
there exists a norm $\lVert \cdot \rVert$ on $\mathbbm{R}^\Du$ and constant $\lambda \in (0,1)$ such that, for all $v \in \mathbbm{R}^\Du,$
$$\lVert (I-KH)D \Psi (v) \rVert \leq \lambda.$$
Then there is constant $c>0$ such that the 3DVAR algorithm satisfies the following large-time asymptotic
error bound:
$$\limsup_{j\to\infty}  \operatorname{\mathbb{E}}[\lVert m_j - v^{\dagger}_j \rVert] \leqslant \frac{c\gamma}{1-\lambda},$$
where the expectation is taken with respect to the sequence $\{\eta^{\dagger}_{j,0}\}.$
\end{theorem}

\begin{proof}
We have
	\begin{align*}
		v^{\dagger}_{j+1} &= \Psi(v^{\dagger}_{j}), \\
m_{j+1} &= (I-KH)\Psi(m_j) + Ky_{j+1},
\end{align*}
and hence that
\begin{align*}
	v^{\dagger}_{j+1} &= (I-KH)\Psi(v^{\dagger}_{j}) + KH\Psi(v^{\dagger}_{j}),\\
	m_{j+1}	&= (I-KH)\Psi(m_j) + KH\Psi(v^{\dagger}_{j}) + \gamma K\eta^{\dagger}_{j+1,0}.
	\end{align*}
Define $e_j=m_j-v_j^{\dagger}.$ By subtracting the evolution
equation for $v_j^{\dagger}$ from that for $m_j$ we obtain, using
the mean value theorem,
\begin{align*} \label{3DVAR:eq3}
	e_{j+1} &= m_{j+1} - v^{\dagger}_{j+1} & \\
	&= (I-KH)(\Psi(m_j)-\Psi(v^{\dagger}_j)) + \gamma K \eta^{\dagger}_{j+1,0} & \\
	&= \left( (I-KH) \int_0^1  {D} \Psi \left(s m_j + (1-s)v_j^\dagger \right) ds \right) e_j + \gamma K \eta_{j+1,0}^\dagger. &&
\end{align*}

\noindent As a result, by the triangle inequality,
\begin{align*}
	\lVert e_{j+1} \rVert  & \leq  \left\lVert \left(  \int_0^1 (I-KH)D \Psi \left(s m_j + (1-s)v_j^\dagger \right) ds \right) e_j \right\rVert + \lVert \gamma K \eta_{j+1,0}^\dagger \rVert \\
	& \leq   \left(  \int_0^1 \left\lVert (I-KH)D \Psi \left(s m_j + (1-s)v_j^\dagger \right)  \right\rVert ds \right)  \lVert e_j \rVert  + \lVert \gamma K \eta_{j+1,0}^\dagger \rVert &  \\
	& \leq \lambda \lVert e_j \rVert + \gamma \lVert K \eta_{j+1,0}^{\dagger} \rVert. &
\end{align*}
Taking expectations on both sides, we obtain, for $c:= \operatorname{\mathbb{E}}[\lVert K \eta_{0,j+1}^{\dagger} \rVert] > 0$,
\begin{equation} \label{3DVAR:eq5}
	\begin{split}
		\operatorname{\mathbb{E}} [\lVert e_{j+1} \rVert] & \leq \lambda \operatorname{\mathbb{E}} [\lVert e_j \rVert] + \gamma \operatorname{\mathbb{E}} [\lVert K \eta_{j+1,0}^{\dagger} \rVert] \\
		& \leq \lambda \operatorname{\mathbb{E}} [\lVert e_j \rVert] + \gamma c.
	\end{split}
\end{equation}
Using the  discrete Gronwall inequality we have that:
\begin{equation} \label{3DVAR:eq6}
	\begin{split}
		\operatorname{\mathbb{E}} [\lVert e_j \rVert] & \leq \lambda^j \operatorname{\mathbb{E}} [\lVert e_0 \rVert] + \sum_{i=0}^{j-1} c \lambda^i \gamma \\
		& \leq \lambda^j \operatorname{\mathbb{E}} [\lVert e_0 \rVert] +  c \gamma \frac{1-\lambda^j}{1-\lambda},
	\end{split}
\end{equation}
where $e_0 =  m_0 - v_0$. As $\lambda<1$, the desired statement follows.
\end{proof}

\subsection{4DVAR}

Recall that 3DVAR differs from 4DVAR because, whilst also based on an
optimization principle, 4DVAR is applied in a distributed fashion over
all data in the time interval $j=1,\ldots, J$; in contrast 3DVAR is
applied sequentially from time $j-1$ to time $j,$ for $j=1,\ldots, J$.
We consider two forms of the methodology: {\em weak constraint 4DVAR (w4DVAR)}, in which the fact that the dynamics model contains randomness is allowed
for in the optimization; and {\em 4DVAR} (sometimes known as {\em strong constraint 4DVAR}) which can be derived from w4DVAR in the limit of $\Sigma \to 0$ (no randomness in the dynamics).

The objective function minimized in w4DVAR is
\begin{equation}\label{eq:objectivew4dvar}
	{\mathsf{J}}(V) = \frac{1}{2}|v_0 - m_0|^2_{C_0} + \frac{1}{2}  \sum^{J-1}_{j=0}  |v_{j+1} - \Psi(v_j)|^2_{\Sigma} +  \frac{1}{2} \sum^{J-1}_{j=0} |y_{j+1} - H{v_{j+1}}|^2_{\Gamma},
\end{equation}
where $V=\{v_{j}\}^{J}_{j=0} \in \mathbbm{R}^{d(J+1)}$,
$Y=\{y_{j}\}^{J}_{j=1} \in \mathbbm{R}^{k J}$, $v_j \in \mathbbm{R}^\Du$, $y_j \in \mathbbm{R}^\Dy$, $H$
is the observation operator, $\Sigma$ is the random dynamical system covariance, $\Gamma$ is the data noise covariance and $m_0$ and $C_0$ are the mean and
covariance of the initial state.
The three terms in the objective function enforce, in turn, the initial
condition $v_0$, the dynamics model and the data model. Note that, because
$\Psi$ is nonlinear, the objective is not quadratic and cannot be optimized
in closed form. In contrast, each step of 3DVAR required solution of a
quadratic optimization, tractable in closed form.

\begin{theorem}[Minimizer Exists for w4DVAR]
Assume that $\Psi$ is bounded and continuous. Then ${\mathsf{J}}$ has a minimizer, which is a MAP estimator for the smoothing problem.
\end{theorem}

\begin{proof}
Recall Theorem \ref{thm:achievable-obj}, which shows that the MAP estimator based on the smoothing distribution $\operatorname{\mathbbm{P}}(V|Y) \propto \exp\bigl( -{\mathsf{J}}(V) \bigr)$ is attained provided that ${\mathsf{J}}$ is guaranteed to be non-negative, continuous and satisfy ${\mathsf{J}}(V) \to \infty$ as $|V| \to \infty.$ Now, the objective ${\mathsf{J}}$ defined by equation \eqref{eq:objectivew4dvar} is clearly non-negative, and it is continuous since $\Psi$ is assumed to be continuous. It remains to show that ${\mathsf{J}}(V) \to \infty$ as $|V| \to \infty.$
Let $R$ be a bound for $\Psi,$ so that $|\Psi(v_j)|_\Sigma \le R$ for all $v_j \in \mathbbm{R}^\Du.$
Then, since
$${\mathsf{J}}(V) \ge \frac{1}{2}|v_0|_{C_0}^2 - 2|v_0|_{C_0} |m_0|_{C_0}  + \frac{1}{2} \sum_{j=0}^{J-1} \Bigl( |v_{j+1} |^2_\Sigma
- 2R|v_{j+1}|_{\Sigma}  \Bigr)$$
it follows that ${\mathsf{J}}(V) \to \infty$ as $|V| \to \infty$ and the proof is complete.
\end{proof}

We now consider the vanishing dynamical noise limit of w4DVAR.
This is to minimize
\begin{equation*}
	{\mathsf{J}}_0(V) = \frac{1}{2}|v_0 - m_0|^2_{C_0} +  \frac{1}{2}  \sum^{J-1}_{j=0} |y_{j+1} - H{v_{j+1}}|^2_{\Gamma}
\end{equation*}
subject to the hard constraint that
$$v_{j+1}=\Psi(v_j), \quad j=0,\ldots, J-1.$$
This is 4DVAR.
Note that by using the constraint 4DVAR can be written as a minimization over
$v_0$, rather than over the entire sequence $\{v_j\}_{j=0}^{J}$ as is required
in w4DVAR.

We let ${\mathsf{J}}_{\sigma}$ denote the objective function ${\mathsf{J}}$ from w4DVAR
in the case where $\Sigma \to \sigma^2\Sigma_0$.
Roughly speaking, the following result shows that minimizers of ${\mathsf{J}}_{\sigma}$
converge as $\sigma \to 0$ to points in ${\mathbbm{R}}^{k(J+1)}$ which
satisfy the hard constraint associated with 4DVAR.

\begin{theorem}[Small Signal Noise Limit of w4DVAR]
Suppose that $\Psi$ is bounded and continuous and let $V^{\sigma}$ be a minimizer of ${\mathsf{J}}_{\sigma}.$
Then as $\sigma \to 0$ there is a convergent
subsequence of $V^{\sigma}$ with
limit $V^*$ satisfying $v^*_{j+1} = \Psi(v^*_j)$.
\end{theorem}

\begin{proof}
Throughout this proof $c$ is a constant which may change from
instance to instance, but is independent of $\sigma.$
Consider $V \in \mathbbm{R}^{d(J+1)}$ defined by  $v_0=m_0$
and  $v_{j+1}=\Psi(v_{j})$. Then $V$ is bounded, as $\Psi(\cdot)$ is bounded,
and the bound is independent of $\sigma$. Furthermore
\begin{equation*}
	{\mathsf{J}}_{\sigma}(V) =  \frac{1}{2} \sum^{J-1}_{j=0} |y_{j+1}-Hv_{j+1}|^2_\Gamma \leq c,
\end{equation*}
where $c$ is independent of $\sigma$.
It follows that
\begin{equation*}
	{\mathsf{J}}_{\sigma}(V^{\sigma}) \leq {\mathsf{J}}_{\sigma}(V) \leq c.
\end{equation*}
Thus
\begin{align*}
	\frac{1}{2}|v^{\sigma}_{j+1} - \Psi(v^{\sigma}_{j})|^2_{\Sigma_0} &= \frac{\sigma^2}{2}|v^{\sigma}_{j+1} - \Psi(v^{\sigma}_{j})|^2_{\Sigma} = \sigma^2 {\mathsf{J}}_\sigma(V^\sigma) \leq \sigma^2 c, \\
	 \frac{1}{2}|v^{\sigma}_{0} - m_0|^2_{C_0} &\leq  {\mathsf{J}}_\sigma(V^\sigma) \le  c .
\end{align*}
Since $\Psi$ is bounded these bounds imply that $|V^{\sigma}| \leq c$.
Therefore
there is a limit $V^{*} : V^{\sigma} \to V^*$ along
a subsequence. By continuity
\begin{equation*}
	0 \leq \frac{1}{2}|v^{*}_{j+1} - \Psi(v^{*}_{j})|^2_{\Sigma_0} \leftarrow \frac{1}{2}|v^{\sigma}_{j+1} - \Psi(v^{\sigma}_{j})|^2_{\Sigma_0} \leq \sigma^{2}c. \\
\end{equation*}
Letting $\sigma \to 0$ we obtain that $v^*_{j+1} = \Psi(v^*_j).$
\end{proof}

\subsection{Discussion and Bibliography}

The 3DVAR and 4DVAR methodologies, in the context of weather forecasting,
are discussed in \cite{lorenc1986analysis} and \cite{fisher2009data}
respectively.
The accuracy analysis presented here is similar to
that which first appeared in the papers
\cite{brett2013accuracy}, \cite{moodey2013nonlinear}
and was developed further in \cite{law2012analysis,law2016filter}.
It arises from considering stochastic perturbations of the
seminal work of Titi and co-workers, exemplified by the paper
\cite{hayden2011discrete}. For an overview of variational methods,
and their links to problems in physics and mechanics, see the book
\cite{abarbanel2013predicting}, and the references therein; see
also the paper \cite{broecker2017existence}.

\section{\Large{\sffamily{The Gaussian Approximation}}}

Recall the inverse problem of finding $u$ from $y$ given by \eqref{eq:jc0},
and the Bayesian formulation which follows from Assumption \ref{a:jc1}.
In the previous chapter we explored the idea of obtaining a point estimator using an optimization perspective arising
from maximizing the posterior pdf. We related this idea to finding the center of a ball of radius $\delta$ with maximal probability in the limit $\delta \to 0.$
Whilst the idea is intuitively appealing, and reduces the complexity of Bayesian inference from determination of an entire distribution to
determination of a single point, the approach has a number of limitations,
in particular for noisy, multi-peaked or high dimensional
posterior distributions; the examples at the end of the previous chapter
illustrated this.

In this chapter we again adopt an optimization approach to the problem of Bayesian inference, but instead seek a Gaussian distribution $p = \mathcal{N}(\vct{\mu}, \vct{\Sigma})$ that minimizes the Kullback-Leibler divergence from  the posterior $\pi^y(\vct{u})$; since the Kullback-Leibler divergence is not symmetric this leads to two distinct problems, both of which we will study.

\subsection{The Kullback-Leibler Divergence}
\begin{definition}
Let $\pi,  \pi' >0$ be two probability distributions on $\mathbbm{R}^\Du$. The
{\em Kullback-Leibler (K-L) divergence}, or {\em relative entropy},
of $\pi$ with respect to $\pi'$ is defined by
\begin{equation*}
\begin{split}
d_{\mbox {\tiny{\rm KL}}}(\pi \| \pi') &:=  \int_{\mathbbm{R}^\Du} \log{\left(\frac{\pi(u)}{\pi'(u)}\right) \pi(u)du  }\\
 &= \operatorname{\mathbb{E}}{^\pi\left[ \log{\left(\frac{\pi}{\pi'}\right) }\right]} \\
 &= \operatorname{\mathbb{E}}{^{\pi'}\left[ \log{\left(\frac{\pi}{\pi'}\right)\frac{\pi}{\pi'} }\right]}.
\end{split}
\end{equation*}
\end{definition}

Kullback-Leibler is a divergence in that $d_{\mbox {\tiny{\rm KL}}}(\pi, \pi') \ge 0,$ with equality if and only if $\pi = \pi'.$ However, unlike Hellinger and total variation it is not a distance. In particular,
the K-L divergence is not symmetric: in general
$$ d_{\mbox {\tiny{\rm KL}}}(\pi \| \pi') \neq d_{\mbox {\tiny{\rm KL}}}(\pi' \| \pi),$$
a fact that will be important in this chapter.
Nevertheless, it is useful for at least four reasons: (1) it provides an upper bound for many distances; (2) its logarithmic structure allows explicit computations
that are difficult using actual distances; (3) it satisfies many convenient analytical properties such as being convex in both arguments and lower-semicontinuous in the topology of weak convergence (4) it has an information theoretic and physical interpretation.

\begin{lemma} \label{l:kllemma}
The K-L divergence provides
the following upper bounds for Hellinger and total variation distance:
$$ d_{\mbox {\tiny{\rm H}}}(\pi, \pi')^2 \leq \frac{1}{2}d_{\mbox {\tiny{\rm KL}}}(\pi \| \pi'),
\quad d_{\mbox {\tiny{\rm TV}}}(\pi, \pi')^2 \leq d_{\mbox {\tiny{\rm KL}}}(\pi \| \pi').$$
\end{lemma}

\begin{proof}

The second inequality follows from the first one by Lemma \ref{l:dh2}; thus
we prove only the first inequality.
Consider the function $\phi:\mathbbm{R}^+ \mapsto \mathbbm{R}$
defined by
$$\phi(x) = x - 1 - \log{x}.$$
Note that
\begin{equation*}
\begin{split}
\phi'(x) &= 1 - \frac{1}{x}, \\
\phi''(x) &= \frac{1}{x^2}, \\
\phi(\infty) & = \phi(0) =\infty.
\end{split}
\end{equation*}
Thus the function is convex on its domain.
As the minimum of $\phi$ is attained at $x = 1$, and as $\phi(1) = 0$,
we deduce that $\phi(x) \geq 0$  for all $x \in (0, \infty).$
Hence,
\begin{equation*}
\begin{split}
x-1 &\geq \log{x} \hspace{1cm} \text{ for all } x \ge 0, \\
\sqrt{x} - 1&\geq \frac{1}{2}\log{x} \hspace{1cm} \text{for all } x  \ge 0.
\end{split}
\end{equation*}
We can use this last inequality to bound the Hellinger distance:
\begin{equation*}
\begin{split}
d_{\mbox {\tiny{\rm H}}}(\pi,\pi')^2 &= \frac{1}{2}\int \left( 1 - \sqrt{\frac{\pi'}{\pi}} \right)^2 \pi du \\
&=  \frac{1}{2}\int \left( 1 + \frac{\pi'}{\pi} -  2\sqrt{\frac{\pi'}{\pi}} \right) \pi du \\
&= \int \left( 1 - \sqrt{\frac{\pi'}{\pi}} \right) \pi du \leq -\frac{1}{2}\int \log\Bigl({\frac{\pi'}{\pi}}\Bigr)\pi du = \frac{1}{2}d_{\mbox {\tiny{\rm KL}}}(\pi \| \pi').
\end{split}
\end{equation*}
\end{proof}

\subsection{Best Gaussian Fit By Minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$}
\label{sec:existence}

In this section we prove the existence of an approximating Gaussian.
To streamline the exposition we work under the following assumption, which can be relaxed:

\begin{assumption}\label{asp:1}
The posterior distribution is constructed under the following assumptions:
\begin{itemize}
\item The loss function $\mathsf{L}(u):= -\log\nu\bigl(y-G(u)\bigr)$ is
non-negative and bounded above.
\item The prior is a centered isotropic Gaussian: $\rho \sim \mathcal{N}(0, \lambda^{-1}\Id).$
\end{itemize}
\end{assumption}
Let $\mathcal{A}$ be the set of Gaussian distributions on $\mathbbm{R}^\Du$ with positive definite covariance,
$$ \mathcal{A} = \{\mathcal{N}(\vct{\mu},\mtx{\Sigma}): \vct{\mu} \in \mathbbm{R}^\Du, \mtx{\Sigma} \in \mathbbm{R}^{d\times d} \text{ positive-definite symmetric}\}.$$

We have the following theorem:
\begin{theorem}[Best Gaussian Approximation]
\label{t:bi1}
Under Assumption \ref{asp:1}, there exists at least one probability distribution $p \in \mathcal{A}$ at which the infimum
$$\inf_{p \in \mathcal{A}} d_{\mbox {\tiny{\rm KL}}} (p \| \pi) $$
is attained.
\end{theorem}

\begin{proof}
The K-L divergence can be computed explicitly as
\begin{equation*}
\begin{split}
d_{\mbox {\tiny{\rm KL}}} (p \| \pi) = &\operatorname{\mathbb{E}}^p{\log{p}} - \operatorname{\mathbb{E}}^p \log \pi \\
 = &\operatorname{\mathbb{E}}^p
\left(
-\frac{1}{2}\vert \vct{u} - \vct{\mu} \vert^2_{\vct{\Sigma}}
- \frac{1}{2}\log \left( (2\pi)^d \text{det} \vct{\Sigma} \right) \right. + \left. \mathsf{L} (\vct{u}) + \frac{\lambda}{2} \vert \vct{u} \vert^2 +\log Z
\right).
\end{split}
\end{equation*}
Note that $Z$ is the normalization constant for $\pi$ and is independent
of $p$ and hence of $\vct{\mu}$ and $\vct{\Sigma}.$
We can represent a given random variable $\vct{u} \sim p$ by writing
 $\vct{u} = \vct{\mu} + \vct{\Sigma}^{1/2}\xi$, where $\xi \sim \mathcal{N}(\vct{0}, \vct{\Id})$,
and hence
$$ \vert \vct{u} \vert^2 = \vert \vct{\mu} \vert^2 + \vert \Sigma^{1/2}\xi|^2 + 2\langle \vct{\mu}, \vct{\Sigma}^{1/2}\xi \rangle$$
to obtain
\begin{equation*}
\begin{split}
d_{\mbox {\tiny{\rm KL}}} (p \| \pi) &=  - \frac{d}{2} - \frac{d}{2}\log(2\pi) - \frac{1}{2}\log \text{det}\vct{\Sigma}   +\operatorname{\mathbb{E}}^p \mathsf{L} (\vct{u}) +  \frac{\lambda}{2} \vert \vct{\mu} \vert^2+ \frac{\lambda}{2} \text{tr}(\vct{\Sigma})   + \log Z.\\
\end{split}
\end{equation*}
 Define ${\mathsf{I}}(\mu,\Sigma) = \operatorname{\mathbb{E}}^p \mathsf{L} (\vct{u}) + \frac{\lambda}{2} \vert \vct{\mu} \vert^2 + \frac{\lambda}{2} \text{tr}(\vct{\Sigma}) - \frac{1}{2}\log \text{det}\vct{\Sigma}.$
Clearly, there is a correspondence between minimizing $d_{\mbox {\tiny{\rm KL}}}(p \| \pi)$ over $p \in {\cal A}$ and minimizing ${\mathsf{I}}(\mu,\Sigma)$ over $\mu \in \mathbbm{R}^d$ and positive definite $\Sigma.$
Note that:
\begin{itemize}
\item ${\mathsf{I}}(0,\Id) <\infty.$
\item For any $\Sigma,$ ${\mathsf{I}}(\mu,\Sigma) \to \infty$ as $|\mu|\to \infty.$
\item For any $\mu,$ ${\mathsf{I}}(\mu,\Sigma) \to \infty$ as $\text{tr}(\Sigma)\to 0$ or $\text{tr}(\Sigma) \to \infty.$
\end{itemize}
Therefore, there are $M,r,R>0$ such that the infimum of ${\mathsf{I}}(\mu,\Sigma)$ over $\mu\in \mathbbm{R}^d$ and positive definite $\Sigma$  is equal to the infimum of ${\mathsf{I}}(\mu,\Sigma)$ over
$$\tilde{\mathcal{A}} := \{(\vct{\mu},\mtx{\Sigma}): \vct{\mu} \in \mathbbm{R}^\Du, \mtx{\Sigma} \in \mathbbm{R}^{d\times d} \text{ positive-definite symmetric}, |\mu|\le M, r\le \text{tr}(\Sigma) \le R  \}.$$
Since ${\mathsf{I}}$ is continuous in $\tilde{\mathcal{A}}$ it achieves its infimum and the proof is complete.
\end{proof}

We remark that the theorem establishes the existence of a  best Gaussian approximation. However, minimizers need not be unique.

\subsection{Best Gaussian Fit By Minimizing $d_{\mbox {\tiny{\rm KL}}}(\pi\|p)$}
In this section we show that the best Gaussian approximation in Kullback-Leibler with respect to its second argument is unique and given by moment matching.
\label{sec:existence}
\begin{theorem}[Best Gaussian by Moment Matching]
Assume that $\bar{\mu} := \operatorname{\mathbb{E}}^{\pi}[u]$ is finite and that
$\bar{\Sigma} := \operatorname{\mathbb{E}}^{\pi}[(u-\bar{\mu})\otimes(u-\bar{\mu})]$ is
positive-definite. Then the
infimum $$\inf_{p \in \mathcal{A}} d_{\mbox {\tiny{\rm KL}}} (\pi\|p)$$
is attained at the element in $\mathcal{A}$ with mean $\bar{\mu}$ and
covariance $\bar{\Sigma}.$
\end{theorem}

\begin{proof}
By definition
\begin{align}
d_{\mbox {\tiny{\rm KL}}}(\pi \|p) = -\operatorname{\mathbb{E}}^{\pi}{[\log{p}]} + \operatorname{\mathbb{E}}^{\pi}{[\log{\pi}]}.
\label{Dkl}
\end{align}
Since the second term does not involve $p$, we study minimization of
\begin{align*}
-\operatorname{\mathbb{E}}^{\pi}{[\log{p}]} &= -\operatorname{\mathbb{E}}^{\pi}{\left[ \log\left(\frac{1}{\sqrt{(2\pi)^d\text{det}\Sigma}}\text{exp}\left(-\frac{1}{2}\big|u-\mu\big|_{\Sigma}^2\right)\right)\right]}\\
 &= \frac{1}{2}\operatorname{\mathbb{E}}^{\pi}{\left[\big|u-\mu\big|_{\Sigma}^2\right]} +\frac{1}{2}\log{\text{det}\Sigma}+\frac{d}{2}\log{2\pi}.
\end{align*}
Let $\Omega = \Sigma^{-1}$. Then our task is equivalent to minimizing the
following function of $\mu$ and $\Omega$:
\begin{align*}
{\mathsf{I}}(\mu,\Omega) = \frac{1}{2}\operatorname{\mathbb{E}}^{\pi}{\left[ \langle u-\mu, \Omega(u-\mu)\rangle\right]-\frac{1}{2}\log{\text{det}\Omega}}.
\end{align*}
First we find the critical points of ${\mathsf{I}}$ by taking its first order partial derivative with respect to $\mu$ and $\Omega$ and setting both to zero:
\begin{align*}
\partial_\mu{\mathsf{I}} &= -\operatorname{\mathbb{E}}^{\pi}\left[\Omega(u-\mu)\right] = 0;\\
\partial _\Omega{\mathsf{I}} &= \frac{1}{2}\partial_\Omega \left( \operatorname{\mathbb{E}}^{\pi}\left[(u-\mu)\otimes(u-\mu):\Omega\right]\right)-\frac{1}{2\text{det}\Omega}\partial_\Omega \text{det}\Omega\\
& = \frac{1}{2}\operatorname{\mathbb{E}}^{\pi}\left[(u-\mu)\otimes(u-\mu)\right] - \frac{1}{2}\Omega^{-1} = 0; \quad\quad
\end{align*}
where we have used the relation $\partial_\Omega\text{det}\Omega = \text{det}{\Omega}\cdot \Omega^{-1}.$ Solving the above two equations gives us the critical point, expressed in terms
of mean and covariance,
$$(\bar{\mu},\bar{\Sigma}) = (\operatorname{\mathbb{E}}^{\pi}[u], \operatorname{\mathbb{E}}^{\pi}[(u-\bar{\mu})\otimes(u-\bar{\mu})]).$$

Next we will show $(\bar{\mu},\bar{\Sigma}^{-1})$ is a minimizer of ${\mathsf{I}}$ or,
equivalently, that $(\bar{\mu},\bar{\Sigma})$ is a minimizer of the expression
given in \eqref{Dkl}. To this end, if we deal with a vector $\theta$ parametrizing the distribution $p$, it will be sufficient to show that the Hessian of
equation~\eqref{Dkl} with respect to $\theta$ is positive definite. Recalling
the label $d$ of the dimension of the parameter $u$ of unknowns we then have:
\begin{align*}
p_{\theta}(u) &= \sqrt{\frac{\text{det}\Omega}{(2\pi)^d}}\text{exp}\left(-\frac{1}{2}(u-\mu)^{T}\Omega(u-\mu)\right)\\
&=\sqrt{\frac{\text{det}\Omega}{(2\pi)^d}}\text{exp}\left[-\left(\frac{1}{2}u^{T}\Omega u-\mu^{T}\Omega u+\frac{1}{2}\mu^{T}\Omega \mu\right)\right]\\
&= \sqrt{\frac{\text{det}\Omega }{(2\pi)^d}}\text{exp}\left(-\frac{1}{2}\mu^{T}\Omega \mu\right)\text{exp}\left(-\frac{1}{2}u^{T}\Omega u+\mu^{T}\Omega u\right).
\end{align*}
Utilizing vectorization of matrices we may write:
\begin{align*}
u^{T}\Omega u &= \Omega :uu^{T} = [\text{vec}(\Omega )]^{T}\text{vec}(uu^{T})\\
\mu^{T}\Omega u &= (\Omega \mu)^{T}u\\
\Rightarrow -\frac{1}{2}u^{T}\Omega u+\mu^{T}\Omega u & = \begin{bmatrix} \Omega \mu \\ -\frac{1}{2}\text{vec}(\Omega ) \end{bmatrix}^{T} \begin{bmatrix} u\\ \text{vec}(uu^{T}) \end{bmatrix}.
\end{align*}
Then we let:
\begin{align*}
\theta =  \begin{bmatrix} \Omega \mu \\ -\frac{1}{2}\text{vec}(\Omega ) \end{bmatrix},\quad\quad T(u) = \begin{bmatrix} u\\ \text{vec}(uu^{T}) \end{bmatrix}.
\end{align*}
The pdf $p_{\theta}(u)$ can then be written in the following form:
\begin{align}
\label{expo}
p_{\theta}(u) &= \frac{1}{Z(\theta)}\text{exp}\left(\theta^{T}T(u)\right)\\
\text{with}\quad Z(\theta)  &= \int_{\mathbbm{R}^\Du}\text{exp}\left(\theta^{T}T(u)\right)du.
\end{align}
Using equation~\eqref{expo} we can rewrite equation \eqref{Dkl} as:
\begin{align}
H(\theta) = d_{\mbox {\tiny{\rm KL}}}(\pi\|p_{\theta}) = -\theta^{T}\operatorname{\mathbb{E}}^{\pi}\left[T(u)\right]+\log\left(Z(\theta)\right)+\operatorname{\mathbb{E}}^{\pi}\left[\pi(u)\right].
\end{align}
Notice that
\begin{align*}
\nabla_\theta \log\left(Z(\theta)\right) &= \frac{1}{Z(\theta)}\int_{\mathbbm{R}^\Du}\nabla_\theta \left[\text{exp}\left(\theta^{T}T(u)\right)\right]du\\
&= \frac{1}{Z(\theta)}\int_{\mathbbm{R}^\Du}T(u)\text{exp}\left(\theta^{T}T(u)\right)du\\
&= \operatorname{\mathbb{E}}^{p_\theta}\left[T(u)\right].
\end{align*}
Therefore we can calculate the gradient and Hessian of $H(\theta)$ as follows:
\begin{align*}
\nabla_\theta H(\theta) &= -\operatorname{\mathbb{E}}^{\pi}\left[T(u)\right]+\operatorname{\mathbb{E}}^{p_\theta}\left[T(u)\right],\\
\left[\nabla^{2}_\theta H(\theta) \right]_{ij}&= \frac{\partial \log\left(Z(\theta)\right)}{\partial \theta_i \theta_j}= \frac{\partial}{\partial\theta_j}\left( \frac{1}{Z(\theta)}\int_{\mathbbm{R}^\Du}T_i(u) e^{\theta^{T}T(u)}du\right)\\
&= \frac{1}{Z(\theta)}\frac{\partial}{\partial \theta_j}\int_{\mathbbm{R}^\Du} T_i(u)e^{\theta^{T}T(u)}dx +\int_{\mathbbm{R}^\Du} T_i(u)e^{\theta^{T}T(u)}du \cdot \frac{\partial}{\partial \theta_j}Z(\theta)^{-1}\\
&= \frac{1}{Z(\theta)}\int_{\mathbbm{R}^\Du}T_i(u)T_j(u)e^{\theta^{T}T(u)}du\\
&\quad\quad\quad -\frac{1}{Z(\theta)^2}\int_{\mathbbm{R}^\Du}T_i(u)e^{\theta^{T}T(u)}du \cdot \int_{\mathbbm{R}^\Du}T_j(u)e^{\theta^{T}T(u)}du\\
&= \operatorname{\mathbb{E}}^{p_\theta}\left[T_iT_j\right] - \operatorname{\mathbb{E}}^{p_\theta}\left[T_i\right]\operatorname{\mathbb{E}}^{p_\theta}\left[T_j\right]\\
&= \left[\text{Cov}^{p_\theta}(T)\right]_{ij}.
\end{align*}
Therefore the Hessian of the objective function is
the covariance matrix of $T(u)$ under the multivariate normal distribution $p_\theta$ and by construction, is positive semi-definite. Therefore $d_{\mbox {\tiny{\rm KL}}}(\pi\|p_\theta)$ is convex in $\theta$ and the critical point $(\bar{\mu},\bar{\Sigma})$ is a corresponding minimizer.
\end{proof}

\begin{remark}
Notice that the preceding proof of convexity
holds for any distribution $p$ that can alternatively be parametrized by a vector $\theta$ with the general form of equation \eqref{expo}. In particular
for such problems the convexity of $d_{\mbox {\tiny{\rm KL}}}(\pi\|p_\theta)$ follows,
ensuring the existence of a minimizer. In fact, equation \eqref{expo}
is a reduced form of the following more general expression:
\begin{align}
p_\theta(u) &= h(u)\text{exp}\left(\theta^{T}T(u)-A(\theta)\right)
\label{general_expo}\\
\text{with}\quad\quad A(\theta) &= \log\left[\int_{\mathbbm{R}^\Du}h(u)\text{exp}\left(\theta^{T}T(u)\right)du\right].
\end{align}
Since $h(u)$ is independent of $\theta$, the conclusion of the
previous theorem carries over to distributions with the form of
\eqref{general_expo}. Such distributions are said to termed
the {\em exponential family} in the statistics literature, and
$\theta$ is called the natural parameter, $T(u)$ the  sufficient statistic, $h(u)$ the base measure, and $A(\theta)$ the log-partition. The Gaussian
distribution is a special case in which $h(u)$ is constant with respect to $u$.
In what follows we show some examples of other exponential family members.
\end{remark}

\begin{example}\textbf{The Bernoulli distribution}\\[0.05in]
Let $u$ be a random variable following the Bernoulli distribution with $\operatorname{\mathbbm{P}}(u=1) = q$ and $\operatorname{\mathbbm{P}}(u=0) = 1-q$. Then the point mass function describing
the distribution of $u$ is:
\begin{align*}
p(u;q) &= q^u(1-q)^{1-u}\\
&= \text{exp}\left[u\log\left(\frac{q}{1-q}\right)+\log(1-q)\right]\\
&= (1-q)\text{exp}\left[u\log\left(\frac{q}{1-q}\right)\right].
\end{align*}
Therefore $\theta = \log\left(\frac{q}{1-q}\right)$, $T(u) = u$, $A(\theta) = \log(1+e^{\theta})$, and $h(u) = 1$.
\end{example}

\begin{example}\textbf{The Poisson distribution}\\[0.05in]
The point mass function describing
the distribution of $u$ is, in this case,
\begin{align*}
p(u;\lambda) &= \frac{\lambda^{u}e^{-\lambda}}{u!}\\
& = \frac{1}{u!}e^{u\log\lambda-\lambda}.
\end{align*}
Therefore $\theta = \log{\lambda}$, $T(u) = u$, $A(\theta) = e^{\theta}$ and $h(u) = \frac{1}{u!}$.\\
\end{example}

\subsection{Comparison between $d_{\mbox {\tiny{\rm KL}}}(\pi \|p)$ and $d_{\mbox {\tiny{\rm KL}}}(p \|\pi)$}
It is instructive to compare the two different minimization
problems, both leading to a ``best Gaussian'', that we described
in the preceding two sections.
We write the two relevant divergences as follows and then explain the
nomenclature:
\begin{align*}
&d_{\mbox {\tiny{\rm KL}}}(p\|\pi) = \operatorname{\mathbb{E}}^{p}{\left[\log\left(\frac{p}{\pi}\right)\right]}= \operatorname{\mathbb{E}}^{p}{[\log{p}]} - \operatorname{\mathbb{E}}^{p}{[\log{{\pi}}]} \quad \text{"Mode-seeking"}\\
&d_{\mbox {\tiny{\rm KL}}}(\pi\|p) = \operatorname{\mathbb{E}}^{\pi}{\left[\log\left(\frac{\pi}{p}\right)\right]}= \operatorname{\mathbb{E}}^{\pi}{[\log{\pi}]} - \operatorname{\mathbb{E}}^{\pi}{[\log{{p}}]} \quad \text{"Mean-seeking"}
\end{align*}

\begin{figure}[!tbp]
\hspace*{\fill}%
  \centering
  \subfloat[Minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$]{\includegraphics[width=0.3\textwidth]{figures/lec5_mode.png}}
  \hfill
  \subfloat[Minimizing $d_{\mbox {\tiny{\rm KL}}}(\pi\|p)$]{\includegraphics[width=0.3\textwidth]{figures/lec5_mean.png}}
  \hspace*{\fill}
  \caption{(a) Minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ can lead to serious information loss while (b) minimizing $d_{\mbox {\tiny{\rm KL}}}(\pi\|p)$ ensures a comprehensive consideration of all components of $\pi$.}
\label{f:mvm}
\end{figure}

Note that when minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ we want $\log{\frac{p}{\pi}}$ to be small, which can happen when $p \simeq \pi$ or $p \ll \pi$. This illustrates
the fact that minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ may miss out components of $\pi$.
For example, in Figure \ref{f:mvm}(a)  $\pi$ is a bi-modal like distribution
but minimizing $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ over Gaussians $p$ can only give a single mode
approximation which is achieved by matching one of the modes;
we may think of this as  ``mode-seeking''. In contrast, when minimizing
$d_{\mbox {\tiny{\rm KL}}}(\pi\|p)$ over Gaussians $p$ we want $\log{\frac{\pi}{p}}$ to be small where $p$ appears as the denominator. This implies that wherever $\pi$ has some mass we must let $p$ also have some mass there in order to keep $\frac{\pi}{p}$ as close as possible to one. Therefore the minimization is carried out by allocating the mass of $p$ in a way such that on average the divergence between $p$ and $\pi$ attains its minimum as shown in Figure ~\ref{f:mvm}(b); hence
the label ``mean-seeking.''  Different applications will favor different
choices between the mean and mode seeking approaches to Gaussian approximation.
\label{sec:difference}

\subsection{Variational Formulation of Bayes Theorem}
\label{sec:posterior}
This chapter has been concerned with finding the best Gaussian approximation
to a measure with respect to KL  divergences. Bayes Theorem \ref{t:bayes}
itself can be formulated through a closely related minimization
principle.  Consider a posterior $\pi^y(u)$ in the following form:
\begin{align*}
\pi^y(u) = \frac{1}{Z}\text{exp}\bigl(-\mathsf{L}(u)\bigr)\rho(u),
\end{align*}
where $\rho(u)$ is the prior, $\mathsf{L}(u)$ is the negative log-likelihood,
and $Z$ the normalization constant. We assume here for exposition that all densities are positive. Then we can express $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ in terms of the prior as follows:
\begin{align*}
d_{\mbox {\tiny{\rm KL}}}(p\|\pi) &= \int_{\mathbbm{R}^\Du} \text{log}\left( \frac{p}{\pi}\right)p du\\
&= \int_{\mathbbm{R}^\Du} \text{log}\left( \frac{p}{\rho}\frac{\rho}{\pi}\right)p du\\
&=  \int_{\mathbbm{R}^\Du} \text{log}\left( \frac{p}{\rho}\text{exp}\bigl(\mathsf{L}(u)\bigr)Z\right)p du\\
&= d_{\mbox {\tiny{\rm KL}}}(p\|\rho) + \operatorname{\mathbb{E}}^{p}[\mathsf{L}(u)] +\log{Z}.
\end{align*}
If we define
$${\mathcal J}(p) =  d_{\mbox {\tiny{\rm KL}}}(p\|\rho) + \operatorname{\mathbb{E}}^{p}[\mathsf{L}(u)]$$
then we have the following:
\begin{theorem}[Bayes Theorem as an Optimization Principle]
The posterior distribution $\pi$ is given by the following minimization principle:
$$\pi = \text{\rm argmin}_{p\in {\mathscr{P}}} {\mathcal J}(p),$$
where ${\mathscr{P}}$ contains all probability densities on $\mathbbm{R}^\Du$.
\end{theorem}
\begin{proof}
Since $Z$ is the normalization constant for $\pi$ and is independent of $p$,
the minimizer of $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ will also be the minimizer of ${\mathcal J}(p).$ Since the global minimizer of $d_{\mbox {\tiny{\rm KL}}}(p\|\pi)$ is
attained at $p=\pi$ the result follows.
\end{proof}

Why is it useful to view the posterior as the minimizer of an energy? There are at least three advantages of this viewpoint. First, the variational formulation provides a natural way to approximate the posterior by restricting the minimization problem to distributions satisfying some computationally desirable property. For instance, variational Bayes methods often restrict the minimization to densities with product structure and in this chapter we have studied restriction to the class of Gaussian distributions. Second, variational formulations allow to show convergence of posterior distributions indexed by some parameter of interest by studying the Gamma-convergence of the associated objective functionals. Third, variational formulations provide natural paths, defined by a gradient flow, towards the posterior. Understanding these flows and their rates of convergence is helpful in the choice of sampling algorithms.

\subsection{Discussion and Bibliography}
The definition of the K-L divergence, and upper-bounds
in terms of probability metrics, can be found in \cite{gibbs2002choosing}.
For a basic introduction to variational methods,
and the moment-matching version of Gaussian
approximation, see \cite{bishop}.
The problem of finding a Gaussian approximation of a general
finite dimensional probability distribution is studied in \cite{lu2016gaussian}, and
infinite dimensional formulations are considered in \cite{pinski2015kullback} and the
companion paper \cite{pinski2015algorithms}. The approximation in Theorem \ref{t:bi1}
consists of a single Gaussian distribution.
If the posterior has
more than one mode, a single Gaussian may not be appropriate.
For an approximation composed of Gaussian mixtures,
the reader is referred to \cite{lu2016gaussian}.
The paper \cite{trillos2019variational} highlights how minimization of K-L divergence arises naturally in the optimization of local entropy and heat regularized costs in deep learning.
The formulation of Bayes Theorem as an optimization principle is
well-known and widely used in the machine learning community
where it goes under the name ``variational Bayes''; see
the book \cite{mackay2003information} and the
paper \cite{bissiri2016general} for clear expositions of this subject. The variational formulation has been used to establish convergence of Bayesian procedures in \cite{trillos2017consistency} and \cite{garcia2018continuum} and to define gradient flows that converge to the posterior as a way to develop and analyze Markov chain Monte Carlo proposals in \cite{trillos2018bayesian}.
 For more information about the properties and examples of exponential family distributions, we refer to \cite{nielsen2009statistical}. and for background on matrix calculations that were used in this chapter we refer to  \cite{petersen2008matrix}.

\section{\Large{\sffamily{Filtering Approach to the Inverse Problem}}}

In this final chapter we demonstrate how the two separate themes that
underpin this course, inverse problems and data assimilation, may be
linked. This opens up the possibility of transferring ideas from filtering
into the setting of quite general inverse problems.
In the first section we describe the general, abstract, connection
and introduce the revolutionary idea of sequential Monte Carlo
methods (SMC).
In the second section we analyze the concrete case of applying the EnKF
to solve an inverse problem, leading to ensemble Kalman inversion (EKI).
In the final section we link the EKI methodology to SMC.

\subsection{General Formulation}

Recall the inverse problem of finding $u \in \mathbbm{R}^\Du$ from $y \in \mathbbm{R}^\Dy$ where
\begin{align}
\label{eq:need99}
y=G(u)+\eta,\; \eta\sim \mathcal{N}(0,\Gamma_0)
\end{align}
and the related loss function
\begin{equation*}
\phi_0(u)=\frac{1}{2}|y-G(u)|^2_{\Gamma_0}.
\end{equation*}
The reason for writing $\Gamma_0$, rather than $\Gamma$, will become apparent
below and will also be exploited in the third section of this chapter where
we study EKI.

If we put a prior $\rho$ on the unknown $u$ then the posterior takes
the form
$$\pi(u)=\frac{1}{Z}\exp(-\phi_0(u))\rho(u).$$
Let $J\in\mathbbm{N}$ and choose $h$ so that $Jh=1$. Then
define the family of pdfs $\{\pi_j\}_{j=0}^J$ by
$$\pi_j(u)=\frac{1}{Z_j}\exp(-jh\phi_0(u))\rho(u).$$
It follows that $\rho_0 = \rho$ and $\rho_J=\pi$ and, furthermore, we may update the
sequence $\{\pi_j\}_{j=0}^J$ sequentially using the formula
$$\pi_{j+1}(u)=\frac{Z_j}{Z_{j+1}}\exp\bigl(-h\phi_0(u)\bigr)\pi_j(u).$$
This simply corresponds to application of Bayes Theorem to the
inverse problem
\begin{align}
\label{eq:need00}
y=G(u)+\eta,\; \eta\sim \mathcal{N}(0,\Gamma),
\end{align}
with $\Gamma=\frac{1}{h}\Gamma_0$
and with prior $\pi_j.$ With this choice of $\Gamma$ the identity \eqref{eq:need00}
gives the likelihood proportional to $\exp(-h\phi_0(u))$. The update
may be written
$$\pi_{j+1}(u)={\mathcal{A}}\pi_j,$$
noting that the analysis operator ${\mathcal{A}}$ is nonlinear, corresponding
to multiplication by $\exp(-h\phi_0(u))$ and then normalization to
a pdf.

If we let ${\mathcal{P}}_j$ denote any Markov kernel for which $\pi_j$ is
invariant then we obtain
$$\pi_{j+1}={\mathcal{A}}{\mathcal{P}}_j \pi_j.$$
This update formula should be compared with the filtering update formula
\begin{align*}
\pi_{j+1} = {\mathcal{A}}_j {\mathcal{P}}\pi_j
\end{align*}
introduced and used in earlier chapters.

Whilst the $j-$dependence in Bayes rule and the Markov kernel
is interchanged between the inverse problem and the filtering problem,
this makes little material difference to implementation.
Firstly, once a Markov kernel $Pred$ is identified under which $\pi=\pi_J$
is invariant, it can be easily adapted to find a family of kernels
${\mathcal{P}}_j$ under which $\pi_j$ is invariant, simply by rescaling the
observation covariance. Secondly the fact that the data $y$ is fixed,
rather than changing at each step, makes little difference to
the implementation.
This perspective opens up an entire field, known as {\em sequential
Monte Carlo}, in which ideas from filtering
may be transferred to other quite different problems, including Bayesian
inversion, as explained here.

Note however (see the discussion at the end of section \ref{sec:overview})
that of the filtering methods we have introduced so far only the bootstrap filter
applies directly to the case of {\em nonlinear} observation operators; since
$G$ is in general nonlinear this means that extra ideas are required to
implement the optimal particle filter, 3DVAR, ExKF and EnKF. One approach is
to replace the sequential optimization principles by the non-quadratic
optimization problem required for nonlinear $G$; we do not discuss this idea
in any detail but it is a viable option. Another approach is to use the linearization
technique which we now outline in the context of EKI.

\subsection{Ensemble Kalman Inversion}

We now make a detour into the subject of how to use
filters to estimate parameters $u$ from data $y$ satisfying
\eqref{eq:need00}. The approach we study,
and which we will relate to sequential Monte Carlo in
the next section, is to introduce an artificial
time dynamic. This can be done quite simply as follows: we write
\begin{align*}
u_{j+1} = &  u_j,\\
y_{j+1} = &  G(u_{j+1}) + \eta_{j+1},
\end{align*}
and we can think of finding the filtering distribution on $u_j|Y_j.$
We discuss how to relate the $y_j$ to the one data point $y$ below.
For now we take $\eta_j \sim \mathcal{N}(0,\Gamma)$ but we will revisit this
choice in the next section, linking the problem to solution of \eqref{eq:need99}.
Because the observation operator $G$ is, in general, nonlinear, this
does not render our system in a form where we can readily apply the
EnKF. To this end we introduce a new variable $w_{j}$ and rewrite the
filter as:
\begin{align*}
u_{j+1} = &  u_j,\\
w_{j+1} = &  G(u_{j}),\\
y_{j+1} = &  w_{j+1} + \eta_{j+1}.
\end{align*}

We introduce the new variables $v=(u,w)^T$, nonlinear map $\Psi(v)=(u, G(u))^T$
and linear operators $H =  [0, I], H^{\perp} = [I,0].$  Then if we
write $v_j=(u_j,w_j)^T$ we may write the dynamical system in the form
\begin{subequations}
\label{eq:new}
\begin{align}
v_{j+1} = & {} \Psi(v_j)\\
y_{j+1} = & {} Hv_{j+1} + \eta_{j+1}.
\end{align}
\end{subequations}
We note that
$$Hv= w, \quad H^{\perp}v=u.$$

\begin{remark}
Typically either of the following  are used to construct artificial
data $\{y_j\}$ for the filtering algorithm, given a single instance of data $y$:
 $$y_{j+1}=
  \begin{cases}
  & y \text{ (unperturbed obervations)}\\
  & y+\bar{\eta}_{j+1}, \bar{\eta}_{j+1}\sim \mathcal{N}(0,\Gamma)\text{ (perturbed observations)}
  \end{cases}$$
The first choice is natural if viewing the algorithm as a sequential
optimizer; the latter, in the linear case, is natural when seeking to draw
samples from the posterior.\qed
\end{remark}

We now apply the EnKF to the dynamics/data model \eqref{eq:new}.
We obtain, for $n = 1,\dots,N$,
\begin{align}
\label{eq:eq1}
\hat{v}_{j+1}^{(n)}&=\Psi(v_j^{(n)}),\\
\bar{v}_{j+1}&= \frac{1}{N}\sum_{n=1}^N\hat{v}_{j+1}^{(n)},\\
\widehat{C}_{j+1}&=\frac{1}{N}\sum_{n=1}^N(\hat{v}_{j+1}^{(n)}-\bar{v}_{j+1})\otimes(\hat{v}_{j+1}^{(n)}-\bar{v}_{j+1}),\\
v_{j+1}^{(n)}&=
(I-K_{j+1}H)\hat{v}_{j+1}^{(n)}+ K_{j+1}y_{j+1}^{(n)}
\end{align}
with the Kalman gain
\begin{align*}
K_{j+1}& =\widehat{C}_{j+1}H^T S_{j+1},\\
S_{j+1}& =(HC_{j+1}H^T+\Gamma)^{-1}.
\end{align*}
Now we may simplify these expressions by using the specific $\Psi$, $v$, $H$ arising in the inverse problem:
\begin{align*}
\widehat{C}_{j+1}=
\begin{bmatrix}
C_{j+1}^{uu} & C_{j+1}^{uw}\\
(C_{j+1}^{uw})^T & C_{j+1}^{ww}
\end{bmatrix}
\;\;
\bar{v}_{j+1}=
\begin{pmatrix}
\bar{u}_{j+1}\\
\bar{w}_{j+1}
\end{pmatrix}
\end{align*}
Here $$\bar{u}_{j+1} = \frac{1}{N}\sum_{n=1}^N u_{j}^{(n)},\;\bar{w}_{j+1} = \frac{1}{N}\sum_{n=1}^N G(u_{j}^{(n)}):=\bar{G}_j$$
and $$C_{j+1}^{uw} = \frac{1}{N}\sum_{n=1}^N(u_{j}^{(n)}-\bar{u}_{j+1})\otimes(G(u_{j}^{(n)})-\bar{G}_{j}), \; C_{j+1}^{ww} = \frac{1}{N}\sum_{n=1}^N(G(u_{j}^{(n)})-\bar{G}_{j})\otimes(G(u_{j}^{(n)})-\bar{G}_{j}).$$
There is a similar expression for $C_{j+1}^{uu}$, but as we will show in what
follows it is not needed for the unknown parameter $u$ update formula.
Noting that, because of the structure of $H$, $S_{j+1}=(C_{j+1}^{ww}+\Gamma)^{-1}$
we obtain
\begin{align}
K_{j+1}=
\begin{pmatrix}\label{eq:2}
C_{j+1}^{uw}(C_{j+1}^{ww}+\Gamma)^{-1}\\
(C_{j+1}^{ww})(C_{j+1}^{ww}+\Gamma)^{-1}
\end{pmatrix}.
\end{align}
Combining equation \eqref{eq:2} with the update equation within \eqref{eq:eq1}
it follows that
\begin{align*}
\{v_j^{(n)}\}_{n\in \{1,N\}} \rightarrow \{v_{j+1}^{(n)}\}_{n\in \{1,N\}}
\end{align*}
and
\begin{align*}
\{H^{\perp}v_j^{(n)}\}_{n\in \{1,N\}} \rightarrow \{H^{\perp}v_{j+1}^{(n)}\}_{n\in
\{1,N\}}
\end{align*}
and hence that
\begin{align*}
u_{j+1}^{(n)} = H^{\perp}v_{j+1}^{(n)} = u_j^{(n)}+ C_{j+1}^{uw}(C_{j+1}^{ww}+\Gamma)^{-1}(y_{j+1}^{(n)}-G(u_j^{(n)})).
\end{align*}
Thus we have derived the EKI step \begin{equation}
\label{eq:EnKF}
u_{j+1}^{(n)}  = u_j^{(n)}+ C_{j+1}^{uw}\bigl(C_{j+1}^{ww}+\Gamma)^{-1}(y_{j+1}^{(n)}-G(u_j^{(n)})\bigr).
\end{equation}
The full algorithm is described below
\FloatBarrier
\begin{algorithm}
\caption{Algorithm for Ensemble Kalman Inversion}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Initial distribution $\rho_0 = \rho$, observations $Y_{J}$, number of particles $N$
\STATE {\bf Initial Sampling}: Draw $N$ particles $u_0^{(n)} \sim \rho$ so that $\rho^N=S^N\rho$
\vspace{0.04in}
\STATE{\bf Subsequent Sampling} For $j = 0,1,\dots,J-1$, perform
\begin{enumerate}
\item Set $\bar{u}_{j+1} = \frac{1}{N}\sum_{n=1}^N u_{j}^{(n)}.$
\item Set $\bar{G}_j= \frac{1}{N}\sum_{n=1}^N G(u_{j}^{(n)}).$
\item Set $C_{j+1}^{uw} = \frac{1}{N}\sum_{n=1}^N(u_{j}^{(n)}-\bar{u}_{j+1})\otimes(G(u_{j}^{(n)})-\bar{G}_{j}).$
\item Set $C_{j+1}^{uw} = \frac{1}{N}\sum_{n=1}^N(u_{j}^{(n)}-\bar{u}_{j+1})\otimes(G(u_{j}^{(n)})-\bar{G}_{j}), \; C_{j+1}^{ww} = \frac{1}{N}\sum_{n=1}^N(G(u_{j}^{(n)})-\bar{G}_{j})\otimes(G(u_{j}^{(n)})-\bar{G}_{j}).$
\item $u_{j+1}^{(n)}  = u_j^{(n)}+ C_{j+1}^{uw}\bigl(C_{j+1}^{ww}+\Gamma)^{-1}(y_{j+1}^{(n)}-G(u_j^{(n)})\bigr).$
\end{enumerate}
\vspace{0.04in}
\STATE {\bf Output}: $N$ particles $u_{J}^{(1)},u_{J}^{(2)}, \dots, u_{J}^{(N)}$
\end{algorithmic}
\label{alg:EKI}
\end{algorithm}
\FloatBarrier

\begin{remark}
This algorithm may be viewed as a derivative-free optimization method,
within the broad class that contains genetic, swarm or ant colony
optimization. \qed
\end{remark}

The algorithm has the following invariant subspace property.

  \begin{theorem}(Space of Ensemble)
\label{t:cite}
  Define $\mathcal{A} = span(u^{(n)}_{0})_{n \in \{ 1,\cdots, N \}}$, then
for all $j$ in $\{0,\dots,J\}$ and for all $n$ in $\{1,\cdots,N\}$,
$u^{(n)}_{j}$ defined by the iteration \eqref{eq:EnKF} lie
in $\mathcal{A}$.
  \end{theorem}
  \begin{proof}
  The proof proceeds by induction. We let $\mathfrak{u}_j \in {\mathbbm{R}^\Du}^N$
denote the collection of $\{u^{(n)}_{j}\}_{n \in \{1,\dots,N\}}.$
  At first, let $j$ be in $\{0,\dots,J\}$, $n$ and $m$ in $\{1,\dots,N\}$
and define the following two quantities:
  \begin{align*}
  d^{(n)}_{j}&:=(C^{ww}_{j+1}+\Gamma)^{-1}(y^{(n)}_{j+1}-G(u^{(n)}_{j})) \\
  D_{mn}(\mathfrak{u}_j)&:=-\langle d^{(n)}_{j},G(u^{(m)}_{j})-\bar{G_j} \rangle
  \end{align*}
  Notice that the $d^{(n)}_{j}$ are elements of the data space ${\mathbb R}^J$,
whereas the $D_{mn}(\mathfrak{u}_j)$ are scalar quantities. Furthermore notice that
$$u^{(n)}_{j+1}=u^{(n)}_{j}-\frac{1}{N} \sum\limits_{n=1}^N D_{mn}(\mathfrak{u}_j)(u^{(m)}_{j}-\bar{u_j}).$$
From the definition of $\bar{G_j}$ and bilinearity of the inner product
it follows that $\sum\limits_{n=1}^N D_{mn}(\mathfrak{u}_j) = 0$. Therefore the
update expression may be rewritten as,
for all $j \in \{0,\dots,J\}$ and $n \in \{1,\dots,N\}$,
$$u^{(n)}_{j+1}=u^{(n)}_{j}-\frac{1}{N} \sum\limits_{n=1}^N D_{mn}(\mathfrak{u}_j)u^{(m)}_{j}.$$
Hence if the property holds for all the particles of the time step $j$, it will clearly be the case for all the particles at time step $j+1$.
  \end{proof}

\begin{remark}
The choice of the initial ensemble of particles
$\{u^{(n)}_{0}\}_{n \in \{1,\dots,N\}}$ is thus crucial to the
performance of the EnKF,
since the algorithm remains in the initial
space of the initial ensemble.
In the setting of Bayesian inverse problems the initial ensemble
is frequently created by drawing from the prior.
Alternatively, if the prior is Gaussian, the first $N$ Karhunen-Loeve
eigenfunctions may be used, ordered by decreasing variance contribution.
More generally any truncated basis for the space $\mathbbm{R}^\Du$ is
a natural initial ensemble. However the question of how to adaptively
learn a good choice of ensemble subspace, in
response to observed data, is unexplored and potentially fruitful. \qed
\end{remark}

\begin{remark}
We describe an alternative way to approach the derivation of the EKI update
formulae. We apply Theorem \ref{eq:extra1} with the specific structure
arising from the dynamical system used in EKI. To this end we define
\begin{equation}
\label{eq:z2}
{\mathsf{I}}_{n}(b) := \frac{1}{2} \mid y_{j+1}^{(n)} - G({u}_{j}^{(n)})
-\frac{1}{N}\sum_{n=1}^N b_n \bigl(G({u}_{j}^{(n)})-\bar{G}_j\bigr) \mid ^{2}_{\Gamma}+\frac{1}{2N}\sum_{n=1}^N b_n^2.
\end{equation}
Once this quadratic form has been minimized with respect to $b$ then the
upate formula  \eqref{eq:deenz3} gives
\begin{subequations}
\label{eq:deenz33}
\begin{align*}
u_{j+1}^{(n)}&={u}_{j}^{(n)}+\frac{1}{N}\sum_{n=1}^N b_n\bigl({u}^{(n)}_{j}-\bar{u}_{j}\bigr),\\
w_{j+1}^{(n)}&=G({u}_{j}^{(n)})+\frac{1}{N}\sum_{n=1}^N b_n\bigl(G({u}^{(n)}_{j})-\bar{G}_{j}\bigr),\\
\end{align*}
\end{subequations}
(Note that the vector $b$ depends on $n$; we have supressed this dependence
for notational convenience). \color{red} Revise \normalcolor
Theorem \ref{t:cite} is an immediate consequence of this structure.
\qed
\end{remark}

\subsection{Linking Ensemble Kalman Inversion and SMC}

He we link the two preceding sections.
In the first subsection
we describe how ensemble Kalman inversion may be linked with
SMC through a particular scaling of $\Gamma$ with $h$. We then
take the limit $h \to 0$ and obtain a system or ordinary
differential equations.
In the second subsection we study the resulting algorithm in
the case in which $G$ is linear, leading to insight into the
EKI algorithm in the iterated context.

\subsubsection{Continuous Time Limit}

Although we have emphasized the optimization perspective on ensemble methods,
we may think of one-step of ensemble Kalman inversion as approximating
the filtering mapping $\pi_j=\operatorname{\mathbbm{P}}(u_j|Y_j) \mapsto  \pi_{j+1}=
\operatorname{\mathbbm{P}}(u_{j+1}|Y_{j+1});$
this will be a good approximation with measures are close to Gaussian and
when the number of ensemble members $N$ is large.
In order to link this to the mapping $\pi_{j+1}={\mathcal{A}}\pi_j$ in the
presentation of SMC in the first section of this chapter,
we set $\Gamma=\frac{1}{h}\Gamma_0.$
Let $\mathfrak{u} \in {\mathcal {X}}^N$ denote the
collection of $\{u^{(n)}\}_{n \in \{1,\dots,N\}}.$
Now define
$$D^0_{mn}(\mathfrak{u}):=\langle G(u^{(n)})-y,G(u^{(m)})-\bar{G} \rangle_{\Gamma_0},$$
where
$$\bar{G}:=\frac{1}{N}\sum_{m=1}^N G(u^{(m)}).$$
Note that then, to leading order in $h \ll 1$,
$$D_{mn}(\mathfrak{u})\approx h\langle G(u^{(n)})-y,G(u^{(m)})-\bar{G} \rangle_{\Gamma_0} = hD^0_{mn}(u).$$ It follows that, also to leading order in $h$,
\begin{equation*}
u^{(n)}_{j+1}\approx u^{(n)}_{j}-\frac{h}{N} \sum\limits_{m=1}^N D^0_{mn}(\mathfrak{u}_j)u^{(m)}_{j}
\end{equation*}
where $\mathfrak{u}_j$ is as defined analogously to the previous section
to denote the collection of particles at discrete time $j$.
Then letting $h\longrightarrow 0$ yields:
\begin{equation}
\label{eq:has}
\frac{du^{(n)}}{dt}=-\frac{1}{N}\sum\limits_{m=1}^N D^0_{mn}(\mathfrak{u})u^{(m)}
\end{equation}

\begin{remark}
Notice that equation \eqref{eq:has}
has families of fixed points where: (i) either the particles fit
the data exactly (which corresponds to the left hand side in the inner product
defining $D^0_{mn}(u)$ being zero);  or (ii) all particles collapse on their
mean value (which corresponds to the right hand side of the same inner product).
This suggests that the system of ordinary differential equations which
describes the behaviour of ensemble Kalman inversion is driven by
two desirable attributes: matching the data and achieving consensus. \qed
\end{remark}

\subsubsection{Linear Setting}
Now suppose $G(\cdot)$ is a linear map denoted $A\cdot$,
and define
$$\bar{u}=\frac{1}{N}\sum\limits_{m=1}^N u^{(m)}.$$
In this setting we have
\begin{align*}
\frac{du^{(n)}}{dt}&=-\frac{1}{N}\sum\limits_{m=1}^N\langle Au^{(n)}-y,A(u^{(m)}-\bar{u}) \rangle_{\Gamma_0}u^{(m)} \\
&=-\frac{1}{N}\sum\limits_{m=1}^N\langle Au^{(n)}-y,A(u^{(m)}-\bar{u}) \rangle_{\Gamma_0}(u^{(m)}-\bar{u}) \\
&= -\frac{1}{N}\sum\limits_{m=1}^N\langle A^T\Gamma_0^{-1}(Au^{(n)}-y),u^{(m)}-\bar{u} \rangle (u^{(m)}-\bar{u}) \\
&= -\frac{1}{N}\sum\limits_{m=1}^N (u^{(m)}-\bar{u})\otimes (u^{(m)}-\bar{u})(A^T\Gamma_0^{-1}(Au^{(n)}-y)).
\end{align*}
If we define
\begin{equation}
C(\mathfrak{u})=\frac{1}{N}\sum\limits_{m=1}^N (u^{(m)}-\bar{u})\otimes (u^{(m)}-\bar{u})
\end{equation}
then we find that, for $n=1, \dots, N$,
\begin{align*}
\frac{du^{(n)}}{dt}&=-C(\mathfrak{u}) (A^T\Gamma_0^{-1}(Au^{(n)}-y)\\
&=-C(\mathfrak{u})\nabla \Phi (u^{(n)}),
\end{align*}
where
$$\Phi(u)=\frac12|y-Au|_{\Gamma_{0}}^2.$$
This corresponds to a gradient descent in the subspace defined by the
initial ensemble. In particular
\begin{align*}
\frac{d}{dt}\Phi (u^{(n)}(t))&=\langle \nabla \Phi (u^{(n)}(t)),\frac{du^{(n)}}{dt}(t) \rangle \\
&= -|C(u)^{1/2}\nabla \Phi (u^{(n)}(t))|^{2} \leq 0
\end{align*}
demonstrating that the loss function $\Phi$ is decreasing along each trajectory
associated to each ensemble member.

We note also that, if we define $e^{(n)}=u^{(n)}-\bar{u}$, then
$$\frac{de^{(n)}}{dt}=-C(\mathfrak{u}) A^T\Gamma_0^{-1}Ae^{(n)}.$$
Because
$$C:=C(\mathfrak{u})=\frac{1}{N}\sum_{n=1}^N e^{(n)} \otimes e^{(n)}$$
it follows that
$$\frac{dC}{dt}=-2CA^T\Gamma_0^{-1}AC.$$
Note further that if $C$ is invertible (requires $N$ at least as big as the dimension
$d$) then the inverse $P$ satisfies
$$\frac{dP}{dt}=2A^T\Gamma_0^{-1}A$$
so that $\|P\| \to 0$ as $t \to \infty.$ This fact, suitably interpreted,
even in the case where $C$ is only invertible on a subspace,
drives the covariance to zero and causes ensemble collapse -- consensus -- and
at the same time $\Phi$ is minimized over an appropriate subspace.

\subsection{Discussion and Bibliography}

The idea of using particle filters to sample general distributions,
including those arising in Bayesian inversion, maybe be found in
\cite{del2006sequential}; a recent application to a Bayesian
inverse problem, which demonstrated the potential of the
methodology in that context, is \cite{kantas2014sequential}.
A simple proof of convergence of the method may be found in
\cite{beskos2015sequential}; it is based on the proof described in
\cite{rebeschini2015can} for the standard bootstrap particle filter.

The use of the ensemble Kalman filter for parameter
estimation was introduced in the papers \cite{lorentzen2001underbalanced,anderson2014kalman} in which a physical dynamical model was
appended with trivial dynamics for the parameters in order to estimate them;
the idea was extended to learn an entire field of parameter values
in \cite{naevdal2002near}.
The paper \cite{skjervheim2011ensemble} was the first to do what we do
here, namely to consider all the data at once and a single mapping
of unknowns parameters to data. In that paper only one iteration is
used; the papers \cite{chen2012ensemble,emerick2013ensemble}
demonstrated how iteration could be useful.
See also the book \cite{oliver2008inverse} for the use of ensemble
Kalman methods in oil reservoir simulation.

Development of the method
for general inverse problems was undertaken in \cite{iglesias2013ensemble},
and further development of iterative methods is described in
\cite{iglesias2015iterative,iglesias2016regularizing}.
Analysis of ensemble inversion was undertaken in
\cite{schillings2017analysis}, including the continuous-time limit
and gradient flow structure described here. The potential for
ensemble inversion in the context of hierarchical Bayesian
methods is demonstrated in \cite{chada2017parameterizations}.
The idea that we explain here, of obtaining an evolution equation for
the covariance which is satisfied exactly by ensemble methods, appears in
the remarkable papers
\cite{reich2011dynamical, bergemann2012ensemble}, and in the other
papers of Bergemann and Reich referenced therein. Their work is at the heart of
the analysis in \cite{schillings2017analysis} which demonstrates
the ensemble collapse and approximation properties of the ensemble
method when applied to linear inverse problems.

\section{\Large{\sffamily{Monte Carlo and Importance Sampling }}}
In this chapter we introduce Monte Carlo sampling and importance sampling.
They are two general techniques for estimating expectations computed
with respect to a particular distribution by generating samples from
(a possibly different) distribution; they may also be viewed as
approximations of a given measure via sums of Dirac measures.

Throughout this chapter we focus interest on
computing expectations with respect to
a probability distribution with pdf $\pi$ given in the form
\begin{equation}
\label{eq:MC1}
\pi(u) = \frac{1}{Z}g(u)\rho(u),
\end{equation}
where $Z$ is a normalizing constant.
Monte Carlo sampling will use samples from $\pi$ itself; importance
sampling will use samples from $\rho$. A particular application of
this setting is where $\rho$ is the prior, $\pi$ the posterior
and $g$ the likelihood.
Let $f:\mathbbm{R}^\Du\longrightarrow\mathbbm{R}$ denote the
function whose expectation we are interested in computing.
For any pdf $p$ on $\mathbbm{R}^\Du$
we write
\begin{equation}
\label{eq:meas2}
p(f)=\operatorname{\mathbb{E}}^{p}\left[f(u)\right]=\int_{\mathbbm{R}^\Du} f(u)p(u)du.
\end{equation}
In this chapter we will generalize the concept of pdf to include
Dirac mass distributions. A Dirac mass at $v$ will be viewed as
having pdf $\delta(\cdot-v)$ where $\delta(\cdot)$ integrates to one
and takes the value zero everywhere except at the origin.

\subsection{Monte Carlo Sampling}
\label{sec:MCdefn&thm}
If we have $N$ random samples $u^{(1)},\ldots,u^{(N)}$, generated i.i.d. according to $\pi$, then we can estimate $\pi$ by the Monte Carlo estimator $\pi_{MC}^N$, which is defined as
\begin{equation}
\label{eq:indeed}
\pi^{N}_{MC} := \frac{1}{N} \sum_{n=1}^{N}\delta(u-u^{(n)}).
\end{equation}
This gives rise to the following estimator of $\pi(f):$
\begin{equation*}
\pi^{N}_{MC}(f) = \frac{1}{N}\sum_{n=1}^{N}f(u^{(n)}),\,\, u^{(n)} \sim \pi \quad {\rm i.i.d.}
\end{equation*}

\newcommand*\diff{\mathop{}\!\mathrm{d}}
      \begin{theorem}[Monte Carlo Error]
\label{t:MC}
       For $f:\mathbbm{R}^\Du\longrightarrow\mathbb{R}$ denote $|f|_\infty := \sup_{u\in \mathbbm{R}^\Du} |f(u)|.$ We have
       \begin{equation*}
        \begin{split}
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\pi^{N}_{MC}(f)-\pi(f)\right]}\right|  = 0,\\
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{MC}(f)-\pi(f)\right)^2\right]}\right|  \leq \frac{1}{N}.
        \end{split}
        \end{equation*}
      \end{theorem}

      \begin{proof}
Define
$$\bar{f}(u) = f(u) - \pi(f).$$
To prove the first result, namely that the estimator is unbiased, note that
        \begin{equation*}
        \begin{split}
        &\operatorname{\mathbb{E}}{\left[\pi^{N}_{MC}(f)-\pi(f)\right]}\\
        & = \frac{1}{N} \sum_{n=1}^{N} \operatorname{\mathbb{E}}{\left[f\left(u^{(n)}\right) - \pi(f)\right]}\\
         &=\frac{1}{N} \sum_{n=1}^{N} \bigl(\pi\left(f\right)-\pi\left(f\right)\bigr)\\
         &= \frac{1}{N}\cdot 0 = 0.
        \end{split}
        \end{equation*}
Therefore the supremum of its absolute value is also zero.
For the second result, which bounds the variance of the estimator, we observe that $\operatorname{\mathbb{E}}{[\bar{f}]} = 0$ and, then,
        \begin{equation*}
        \begin{split}
        \operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{MC}(f)-\pi(f)\right)^2\right]} &= \frac{1}{N^2}\sum_{n=1}^{N}\sum_{n=1}^{N}\operatorname{\mathbb{E}}{\left[\bar{f}\left(u^{(n)}\right)\bar{f}\left(u^{(n)}\right)\right]}\\
        &= \frac{1}{N^2}\sum_{n=1}^{N} \operatorname{\mathbb{E}}{\left[\bar{f}\left(u^{(n)}\right)^2\right]} \\
        &= \frac{1}{N}\operatorname{\mathbb{E}}{\left[\bar{f}\left(u^{(1)}\right)^2\right]} = \frac{1}{N}\text{Var}_{\pi}[f]
        \end{split}
        \end{equation*}
        since $u^{(n)}$ are i.i.d.
In particular we have
\begin{equation}
\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{MC}(f)-\pi(f)\right)^2\right]}
= \frac{1}{N}  \text{Var}_{\pi}[f] \le \frac{1}{N}\pi(f^2)
\label{eq:refer}
\end{equation}
since
        \begin{equation*}
        \text{Var}_{\pi}[f] = \pi(f^2) -\pi(f)^2 \leq \pi(f^2).
        \end{equation*}
        Therefore
        \begin{equation*}
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{MC}(f)-\pi(f)\right)^2\right]}\right| = \sup_{|f|_\infty\leq1} \left| \frac{1}{N}\text{Var}_{\pi}[f]\right| \leq \frac{1}{N}.
        \end{equation*}
      \end{proof}

The theorem shows that the Monte Carlo estimator  $\pi^{N}_{MC}$ is
an unbiased approximation for the posterior $\pi$ and that, by choosing
$N$ large enough,
expectation of any bounded function $f$
can in principle be approximated by Monte Carlo
sampling to arbitrary accuracy. Furthermore, although the
convergence is slow with respect to $N$ there is no dependence on the
dimension of the problem or on the properties of $f$, other than its
supremum.
~

      \begin{example}[Approximation of an Integral]
\label{ex:q}
       Let $f:\mathbb{R}\longrightarrow\mathbb{R}$ be a sigmoid function defined on $\mathbb{R}$ and shown in Figure \ref{F:MC1}(a) below as the blue solid curve.
Let $u\sim \pi=0.1~\mathcal{N}(-5,1) +0.9~\mathcal{N}(5,1)$ be a Gaussian mixture consisting of two Gaussian distributions. We wish to approximate the expected value of $f(u) \times \mathbb{I}_{[a,b]}(u)$ and $\mathbb{I}_{[a,b]}(u) = \begin{cases} 1 & \text{ if } u \in [a,b] \\ 0 & \text{ otherwise} \end{cases}$. We use Monte Carlo sampling to generate $N$ random samples $u^{(1)},\ldots,u^{(N)}$ and compute the error between the actual integral and Monte Carlo estimator. The integral and estimator are in the form:
        \begin{equation*}
        \begin{split}
        &\pi(f) = \int_{a}^{b} f(u)\pi(u) \diff u,\\
        &\pi^{N}_{MC}(f) = \frac{1}{N} \sum_{n=1}^{N}f(u^{(n)})\mathbb{I}_{[a,b]}(u^{(n)}).
        \end{split}
        \end{equation*}
The results of a set of numerical experiments with $a=-5, b=5$ and varying $N$ are shown
in Figure \ref{F:MC1}(b).
A randomly chosen subset of the samples used when $N=100$ is displayed in Figure \ref{F:MC1}(a).

       \begin{figure}[thbp]
        \centering
        \begin{tabular}{@{}c@{}}
            \includegraphics[width=0.5\textwidth]{figures/lec6_pic2.png}
        \end{tabular}
        \hspace{0.0in}
        \begin{tabular}{@{}c@{}}
            \includegraphics[width=0.55\textwidth]{figures/lec6_pic3.png}
        \end{tabular}
        \caption{Large sampling number reduces the estimation error by Monte Carlo method.}
        \label{F:MC1}
       \end{figure}

      \end{example}

\subsection{Importance Sampling}
\label{sec:ISdefn&thm}
Standard Monte Carlo sampling can only be used when it is possible to sample from the desired
{\em target} distribution $\pi$. When it is not possible to sample from $\pi$,
we can draw samples from another {\em proposal} distribution $\rho$ instead.
We then need to evaluate $g$ in \eqref{eq:MC1} at each sample and use
it as an importance weight in the approximation of the desired distribution.
This is the idea of importance sampling.

Consider $\pi$ as in equation \eqref{eq:MC1}. Given a test function $f: \mathbbm{R}^d \to \mathbbm{R}$ we can rewrite its expectation with respect to $\pi$ in terms of expected values with respect to $\rho$ as follows:
\begin{equation*}
\pi(f) = \frac{\rho(fg)}{\rho(g)},
\end{equation*}
where we used that $Z = \rho(g).$
Approximating with standard Monte Carlo the numerator and the denominator gives
\begin{align*}
\pi(f) & \approx  \sum_{n=1}^{N }w^{(n)} f\left(u^{(n)}\right),\,\, u^{(n)} \sim \rho \,\, {\rm i.i.d.}  \\
&= \pi^{N}_{IS}(f),
\end{align*}
where
\begin{equation*}
w^{(n)} := \frac{g\left(u^{(n)}\right)}{\sum_{m=1}^{N}g\left(u^{(m)}\right)}, \quad \quad \pi^{N}_{IS} := \sum_{n=1}^{N }w^{(n)}\delta(u-u^{(n)}).
\end{equation*}
Thus, given $N$ samples $u^{(1)}, \ldots, u^{(N)}$ generated i.i.d. according to $\rho$ we can estimate $\pi$ with the \emph{particle approximation measure} $\pi^{N}_{IS}.$
The quality of this estimator may be assessed by considering the worst-case bias and mean squared error when using the particle approximation measure to estimate expectations over the class of bounded test functions $\{f: \mathbbm{R}^\Du \to \mathbbm{R}: |f|_\infty =1\}.$  We will show that worst-case error upper-bounds can be obtained in terms of the $\chi^2$ divergence between the target and the proposal, suggesting that the performance of importance sampling depends on the closeness between target and proposal.

\begin{definition}
Let $\pi>0$ and $\pi'>0$ be two probability distributions on $\mathbbm{R}^\Du.$ The \emph{$\chi^2$ divergence} of $\rho$ with respect to $\rho'$ is
\begin{equation}\label{eq:chi2}
d_{\mbox {\tiny{$ \chi^2$}}}(\pi \| \pi') := \int_{\mathbbm{R}^\Du} \Bigl( \frac{\pi(u)}{\pi'(u)} -1  \Bigr)^2 \, \pi'(u) du.
\end{equation}
\end{definition}
The $\chi^2$  is a divergence but not a distance as it is, in general, not symmetric. The next result shows that,  similarly as for standard Monte Carlo, the mean squared error of $\pi^{N}_{IS}(f)$ as an estimator of $\pi(f)$ is order $N^{-1}.$ However, there are two main differences: the estimator is now biased, and the constant in the mean squared error depends on the $\chi^2$ divergence between the target and the proposal.

    \begin{theorem}[Importance Sampling Error]
      \label{IS}
       We have
       \begin{equation*}
        \begin{split}
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\pi^{N}_{IS}(f)-\pi(f)\right]}\right|  \leq 2 \frac{1 + d_{\mbox {\tiny{$ \chi^2$}}}(\pi\|\rho)}{N},\\
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f)-\pi(f)\right)^2\right]}\right|  \leq 4 \frac{1 + d_{\mbox {\tiny{$ \chi^2$}}}(\pi\|\rho)}{N}.
        \end{split}
       \end{equation*}
      \end{theorem}

      \begin{proof}
        Given
        \begin{equation*}
        \pi(u) = \frac{1}{Z}g(u)\rho(u) = \frac{1}{\rho(g)}g(u)\rho(u),
        \end{equation*}
        we have that $d_{\mbox {\tiny{$ \chi^2$}}}(\pi\|\rho) = \frac{\rho(g^2)}{\rho(g)^2} -1.$ To ease the notation we denote $\zeta: = \frac{\rho(g^2)}{\rho(g)^2}.$
        We rewrite
        \begin{equation*}
        \begin{split}
        \pi(f) &= \frac{\rho(gf)}{\rho(g)} \simeq \frac{\rho^{N}_{MC}(gf)}{\rho^{N}_{MC}(g)} =\pi^{N}_{IS}(f).\\
        \end{split}
        \end{equation*}
       Then we have
       \begin{equation}\label{eq:splitting}
        \begin{split}
        \pi^{N}_{IS}(f) -\pi(f) &= \pi^{N}_{IS}(f) -  \frac{\rho(gf)}{\rho(g)}\\
        &=  \frac{\pi^{N}_{IS}(f)\left(\rho(g)-\rho^{N}_{MC}(g)\right)}{\rho(g)} - \frac{\left(\rho(gf)-\rho^{N}_{MC}(gf)\right)}{\rho(g)}.
        \end{split}
        \end{equation}
The expectation of the second term is zero and hence
       \begin{equation*}
       \begin{split}
\left| \operatorname{\mathbb{E}}{\left[\pi^{N}_{IS}(f) -\pi(f)\right]}\right|
&= \frac{1}{\rho(g)}\left|\operatorname{\mathbb{E}}{\left[\pi^{N}_{IS}(f)\left(\rho(g)-\rho^{N}_{MC}(g)\right)\right]}\right|\\
        &\leq \frac{1}{\rho(g)}\left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f)-\pi(f)\right)\left(\rho(g)-\rho^{N}_{MC}(g)\right)\right]}\right|,\\
        \end{split}
        \end{equation*}
       since $\operatorname{\mathbb{E}}{\left[\rho(g)-\rho^{N}_{MC}(g)\right]}=0$.
       Using the Cauchy-Schwarz inequality, the second result
from this theorem (whose proof follows) and \eqref{eq:refer} from the proof of
Theorem \ref{t:MC} we have, for all $|f|_\infty \leq 1$,

       \begin{equation*}
       \begin{split}
\left| \operatorname{\mathbb{E}}{\left[\pi^{N}_{IS}(f) -\pi(f)\right]}\right|
&\leq \frac{1}{\rho(g)} \left(\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f)-\pi(f)\right)^2\right]}\right)^{1/2}\left(\operatorname{\mathbb{E}}{\left[\left(\rho(g)-\rho^{N}_{MC}(g)\right)^2\right]}\right)^{1/2}\\
        &\leq \frac{1}{\rho(g)}\left(\frac{4\zeta}{N}\right)^{1/2}\left(\frac{\rho(g^2)}{N}\right)^{1/2} = \frac{2\zeta}{N}.
       \end{split}
       \end{equation*}

\begin{comment}
       The expectation of the second term is derived as
       \begin{equation*}
       \begin{split}
        \left| \operatorname{\mathbb{E}}{\left[\frac{\rho(g)-\rho^{N}_{MC}(g)}{\rho(g)}\right]}\right| &= \frac{1}{\rho(g)}\left|\operatorname{\mathbb{E}}{\left[\rho(g)-\rho^{N}_{MC}(g)\right]}\right| =0.
       \end{split}
       \end{equation*}

       Therefore,
       \begin{equation*}
       \begin{split}
        \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\pi^{N}_{IS}(f)-\pi(f)\right]}\right| &=  \sup_{|f|_\infty\leq1}\left| \operatorname{\mathbb{E}}{\left[\frac{\pi^{N}_{IS}(f)\left(\rho(g)-\rho^{N}_{MC}(g)\right)}{\rho(g)}\right]}-0\right|
        \leq \frac{2r}{N}.
       \end{split}
       \end{equation*}
\end{comment}

We now prove the second result. We use the splitting of
$\pi^{N}_{IS}(f) -\pi(f)$ into the sum of two terms
as derived in equation \ref{eq:splitting}. Using \eqref{eq:refer}, the basic inequality $(a-b)^2 \le 2(a^2 + b^2)$ and that for all $|f|_\infty \leq 1, |\pi^{N}_{IS}(f)| \leq 1$, we have, for all $|f|_\infty \leq 1$,
        \begin{equation*}
        \begin{split}
        &\left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f) -\pi(f)\right)^2\right]}\right|\\
        &\leq \frac{2}{\rho(g)^2} \left( \operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f)\right)^2\left(\rho(g)-\rho^{N}_{MC}(g)\right)^2\right]}+ \operatorname{\mathbb{E}}{\left[\left(\rho(gf)-\rho^{N}_{MC}(gf)\right)^2\right]} \right)\\
        &\leq \frac{2}{\rho(g)^2} \left( \operatorname{\mathbb{E}}{\left[\left(\rho(g)-\rho^{N}_{MC}(g)\right)^2\right]}+ \operatorname{\mathbb{E}}{\left[\left(\rho(gf)-\rho^{N}_{MC}(gf)\right)^2\right]} \right)\\
        &= \frac{2}{\rho(g)^2N}\bigl(\text{Var}_{\rho}\left[g\right] +\text{Var}_{\rho}\left[gf\right] \bigr)\\
        &\leq \frac{2}{\rho(g)^2N} \left( \rho(g^2)+\rho(g^2f^2)\right)\\
        &\leq  \frac{4\rho(g^2)}{\rho(g)^2N} = \frac{4\zeta}{N}.
        \end{split}
        \end{equation*}
        Therefore,
        \begin{equation*}
        \begin{split}
         \sup_{|f|_\infty\leq1} \left|\operatorname{\mathbb{E}}{\left[\left(\pi^{N}_{IS}(f)-\pi(f)\right)^2\right]}\right|  \leq \frac{4\zeta}{N}.
        \end{split}
        \end{equation*}

      \end{proof}

This theorem shows that, unlike Monte Carlo, the Importance Sampling estimator $\pi^{N}_{IS}$ is biased for $\pi$.
The rate of convergence of the bias is twice that of the standard deviation,
however.
As for Monte Carlo the rate of convergence of the variance of the error
is governed by the inverse of $N$, but is independent of the dimension $d$
of the state space. However
for Importance Sampling to be accurate (with a limited number of samples $N$)
it is important that the $\chi^2$ divergence between the target and the proposal is not too large.

      \begin{example}[Change of Measurement]
\label{ex:q2}
We consider a similar set-up as in Example \ref{ex:q}, integrating
a sigmoid function, shown in blue in Figure \ref{Fig. 3}, with respect to
a probability measure $\pi$ which is bimodal, shown in red in
Figure \ref{Fig. 3}; we again restrict the support of the desired integral.
We estimate the integral using Importance Sampling based on $N$ random
samples $u^{(1)},\ldots,u^{(N)}$ from the measure
$\rho=\mathcal{N}(\mu,\,\sigma^{2})$, shown in green in Figure \ref{Fig. 3}.
The estimator of the integral is given by
        \begin{align*}
        \begin{split}
        \pi^{N}_{IS}(f) &= \sum_{n=1}^{N}w^{(n)} f(u^{(n)}){\mathbbm{1}}_{[a,b]}(u^{(n)}),\\
        w^{(n)}&=\frac{g(u^{(n)})}{\sum_{m=1}^{N}g(u^{(m)})}.
        \end{split}
        \end{align*}
Here $g$ is a function proportional to the ratio of the densities of
$\pi$ and $\rho.$
If $\pi(u^{(n)})>\rho(u^{(n)})$, the samples should have been denser,
so we raise the weight on $f(u^{(n)})$ in proportion to
$\frac{\pi(u^{(n)})}{\rho(u^{(n)})}> 1$. If $\pi(u^{(n)})<\rho(u^{(n)})$, the samples should have been less dense, so we lower the weight on $f(u^{(n)})$ in proportion to $\frac{\pi(u^{(n)})}{\rho(u^{(n)})}< 1$.

        \FloatBarrier
        \begin{figure}[thbp]
        \centering
        \begin{tabular}{@{}c@{}}
            \includegraphics[width=0.5\textwidth]{figures/lec6_pic4a.png}
        \end{tabular}
        \hspace{0.0in}
        \begin{tabular}{@{}c@{}}
            \includegraphics[width=0.5\textwidth]{figures/lec6_pic4b.png}
        \end{tabular}
        \caption{Importance Sampling is a change of measurement via the importance weights. The red curve shows a bimodal distribution $\pi$ and the green curve shows a Gaussian distribution $\rho$. The blue curve is the function to be integrated, on its support $[-5,5].$ The upper figure shows samples
from the posterior $\pi$ itself; these would be used for Monte Carlo
sampling; the lower curve shows samples from the prior $\rho$, as used
for Importance Sampling. The importance weights capture and compensate for the difference of sampling from these two distributions.}
          \label{Fig. 3}
		\end{figure}
		\FloatBarrier
      \end{example}

\subsection{Discussion and Bibliography}
A classic reference on the Monte Carlo method
which includes discussion of several variance-reduction techniques is \cite{hammersley1964percolation}. The
chapter notes \cite{anderson2014monte} give a comparison of Monte Carlo and Importance Sampling with examples. The paper \cite{kawai2017adaptive} further explores advanced Importance Sampling via adaptive algorithms.
When $N$ is large enough, $\pi^{N}_{MC}$ arising from a Monte Carlo simulation should be close to $\pi$ \cite{evans1995methods}.
In practice, all probabilities, integrals and summations can be approximated by the Monte Carlo method \cite{tokdar2010importance}.
A review of importance sampling, from the perspective of filtering
and sequential importance resampling, may be found in
\cite{agapiou2017importance}; the proofs in this chapter closely follow the presentation in that paper. Necessary sample size results for importance sampling in terms of several divergences between target and proposal were established in \cite{sanz2018importance}.
The subject of multi level Monte Carlo (MLMC) has made the use of
Monte Carlo methods practical in new areas of application;
see \cite{giles2015multilevel} for an overview. The methodology applies
when approximating expectations over infinite dimensional spaces,
and distributes the computational budget over different levels of
approximation, with the goal of optimizing the cost per unit
error, noting that the latter balances sampling and approximation based
sources.

\section{\Large{\sffamily{Filtering and Smoothing Problems and Well-Posedness}}}

In this chapter we introduce data assimilation problems in which
the model of interest, and the data associated with it, have
a time-ordered nature. We distinguish between the filtering problem
(on-line) in which the data is incorporated sequentially as it
comes in, and the smoothing problem (off-line) which is a specific instance
of the inverse problems that have been the subject of the preceding
chapters.

\subsection{Formulation of Filtering and Smoothing Problems}
\label{sec:filtering}
Consider the {\em stochastic dynamics model} given by
\begin{align*}
v_{j+1} &= \Psi(v_j) + \xi_j, \: j \in \mathbbm{Z}^+, \notag \\
v_0 &\sim \mathcal{N}(m_0, C_0), \: \xi_j \sim \mathcal{N}(0, \Sigma) \: \text{i.i.d.} \notag,
\end{align*}
where we assume that $v_0$ is independent of the sequence $\{ \xi_j \}$;
this is often written as $v_0 \perp \{ \xi_j \}.$
Now we add the {\em data model} given by
\begin{align*}
y_{j+1} &= h(v_{j+1}) + \eta_{j+1} , \: j \in \mathbbm{Z}^+, \notag \\
\eta_j &\sim \mathcal{N}(0, \Gamma) \: \text{i.i.d.} \notag,
\end{align*}
where we assume that $\{\eta_j\} \perp v_0$ for all $j$ and
$\eta_k \perp \xi_j $ for all $j,k$.
The following will be assumed in the remainder of this book.
\begin{assumption}
\label{a:fas}
The matrices $C_0,$ $\Sigma$ and $\Gamma$ are positive-definite.
Further, we have $\Psi \in C(\mathbbm{R}^\Du, \mathbbm{R}^\Du)$ and $h \in C(\mathbbm{R}^\Du, \mathbbm{R}^\Dy)$.
\end{assumption}

\noindent
We define
\begin{equation}
V := \{v_0, \ldots, v_J\}, \: Y := \{y_1, \ldots, y_J\},
\: \text{and} \: Y_j := \{y_1, \ldots, y_j\}.  \notag
\end{equation}
The sequence $V$ is often termed the {\em signal} and the sequence
$Y$ the {\em data}.

\begin{definition}[The Smoothing Problem]
The {\em smoothing problem} is to find the probability density
$\Pi(V):=\operatorname{\mathbbm{P}}(V|Y) = \operatorname{\mathbbm{P}}(\{v_0, \ldots, v_J\}|\{y_1, \ldots, y_J\})$ on $\mathbbm{R}^{d(J+1)}$
for some fixed integer $J.$ We refer to $\Pi$ as the {\em  smoothing distribution}.
\end{definition}
\begin{definition}[The Filtering Problem]
The {\em filtering problem} is to find the probability densities
$\pi_j(v_j):=\operatorname{\mathbbm{P}}(v_j|Y_j)$ on $\mathbbm{R}^\Du$ for $j=1, \dots, J.$
We refer to $\pi_j$ as the {\em filtering distribution at time $j.$}
\end{definition}

The key conceptual issue to appreciate concerning the filtering problem, in
comparison with the smoothing problem, is that interest is focused
on characterizing, or approximating, a sequence of probably distributions,
defined  in an iterative fashion as the data is acquired sequentially.

\begin{remark}\label{rem:marginal}
We note the following identity:
\begin{align*}
\int \Pi(v_0, \ldots, v_J)dv_0dv_1 \ldots dv_{J-1} = \pi_J(v_J).
\end{align*}
This expresses the fact that the marginal of the smoothing distribution at time $J$ corresponds to the
filtering distribution at time $J$. Note also that, in general, for $j<J$
\begin{equation}
\int \Pi(v_0, \ldots , v_J) dv_0\ldots dv_{j-1}\hat{dv_{j}}dv_{j+1} \ldots dv_{J} \neq \pi_j(v_j), \notag
\end{equation}
since the expression on the left-hand side of the equation depends on data
$Y_J$, whereas that on the right-hand side depends only on $Y_j$, and $j < J$.
\end{remark}

\subsection{The Smoothing Problem}
\label{sec:smooth}

\subsubsection{Formula for pdf of the Smoothing Problem}
The smoothing distribution can be found by combining a prior on $v$ and a likelihood function using Bayes formula.
The prior is the probability distribution on $v$ implied by the distribution of $v_0$ and the stochastics dynamics model; the likelihood function is defined by the data model. We now derive the prior and the likelihood separately.

The prior distribution can be derived as follows:
\begin{subequations}
\begin{align}
\operatorname{\mathbbm{P}}(V) &= \operatorname{\mathbbm{P}}(v_J, v_{J-1}, \ldots , v_0) \notag \\
&= \operatorname{\mathbbm{P}}(v_J|v_{J-1}, \ldots , v_0)\operatorname{\mathbbm{P}}(v_{J-1}, \ldots , v_0) \notag \\
&= \operatorname{\mathbbm{P}}(v_J|v_{J-1})\operatorname{\mathbbm{P}}(v_{J-1}, \ldots , v_0). \notag
\end{align}
\end{subequations}
The third equality comes from the Markov, or memoryless, property which
follows from the independence of the elements of the sequence $\{\xi_j\}.$
By induction, we have:
\begin{subequations}
\begin{align}
\operatorname{\mathbbm{P}}(V) &= \prod_{j=0}^{J-1}\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\operatorname{\mathbbm{P}}(v_0) \notag \\
&= \frac{1}{Z_\rho}\exp\bigl(-\mathsf{R}(V)\bigr) \notag \\
&=: \rho(V), \notag
\end{align}
\end{subequations}
where $Z_\rho>0$ is a normalizing constant and
\begin{equation}
\mathsf{R}(V) := \frac{1}{2}|v_0-m_0|^2_{C_0} + \frac{1}{2}  \sum_{j=0}^{J-1} |v_{j+1} - \Psi(v_j)|^2_{\Sigma}. \notag
\end{equation}

The likelihood function, which incorporates the measurements gathered from observing the system, depends only on the measurement model and may be derived as follows:
\begin{align}
\operatorname{\mathbbm{P}}(Y|V) &= \prod_{j=0}^{J-1} \operatorname{\mathbbm{P}}(y_{j+1}|v_0,\ldots , v_{J}) \notag \\
&= \prod_{j=0}^{J-1}\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1}) \notag \\
& \propto \exp\bigl(-\mathsf{L}(V;Y)\bigr), \notag
\end{align}
where
\begin{equation}
\mathsf{L}(V;Y) :=  \frac{1}{2} \displaystyle{\sum_{j=0}^{J-1}}|y_{j+1} - h(v_{j+1})|_{\Gamma}^2. \notag
\end{equation}
The factorization of $\operatorname{\mathbbm{P}}(Y|V)$ in terms of the product of the
$\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})$ follows from the independence of the
elements of $\{\eta_j\}$ and the fact that the observation at time $j+1$
depends only on the state at time $j+1.$

Using Bayes Theorem \ref{t:bayes} we find the smoothing distribution by combining the likelihood and the prior
\begin{align*}
\Pi(V) & \propto \operatorname{\mathbbm{P}}(Y|V)\operatorname{\mathbbm{P}}(V)\\
&  =\frac{1}{Z}\exp\bigl(-\mathsf{R}(V)-\mathsf{L}(V;Y)\bigr).  \notag
\end{align*}
Note that $V \in \mathbbm{R}^{d(J+1)}$, and $Y \in \mathbbm{R}^{k J}$.

\subsubsection{Well-Posedness of the Smoothing Problem}
Now we study the well-posedness of the smoothing problem with respect
to perturbations in the data. To this end we consider two smoothing
distributions corresponding to different observed data sequences $Y,Y'$:
\begin{align*}
\Pi(V)&:=\operatorname{\mathbbm{P}}(V|Y) = \frac{1}{Z}\exp\bigl(-\mathsf{R}(V)-\mathsf{L}(V;Y)\bigr), \notag \\
\Pi'(V)&:=\operatorname{\mathbbm{P}}(V|Y')= \frac{1}{Z'}\exp\bigl(-\mathsf{R}(V)-\mathsf{L}(V;Y')\bigr). \notag
\end{align*}
We make the following assumptions:
\begin{assumption}
There is finite non-negative constant $R$ such that the
data $Y, Y'$ and the observation function $h$ satisfy:
\label{a:swp}
\begin{itemize}
\item $|Y|, |Y'| \le R;$
\item Letting $\phi(V) :=\Bigl( \sum_{j=1}^J (|h(v_j)|^2) \Bigr)^{1/2}$, it holds that $\operatorname{\mathbb{E}}^{\rho}[\phi(V)]<\infty$.
\end{itemize}
\end{assumption}

The following theorem shows well-posedness of the smoothing problem.

\begin{theorem}[Well-posedness of Smoothing]
\label{t:wps}
Under Assumption \ref{a:swp}, there is $\kappa \in [0, \infty),$ independent of $Y$ and $Y'$
such that $$d_{\mbox {\tiny{\rm H}}}(\Pi, \Pi') \leq \kappa|Y-Y'|.$$
\end{theorem}

\begin{proof}
We show that the proof of Theorem \ref{thm1}, which established well-posedness for Bayesian inverse problems under Assumption \ref{a:dh1}, applies in the smoothing context as well. To do so we rewrite the problem in the same notation used in Chapter 1, and show that Assumption \ref{a:swp} above implies Assumption \ref{a:dh1}.
Write
\begin{align*}
\Pi(V)&= \frac{1}{Z}\exp\bigl(-\mathsf{L}(V;Y)\bigr) \rho(V)= \frac{1}{Z} g(V;Y) \rho(V),\\
\Pi'(V)& =  \frac{1}{Z'}\exp\bigl(-\mathsf{L}(V;Y')\bigr)  \rho(V) = \frac{1}{Z'} g(V;Y') \rho(V), \notag
\end{align*}
where $Z,Z'>0$ are normalization constants.
Here $ |Y-Y'|$ plays the role of $\delta$ in Theorem \ref{thm1}.
Since $g(V;Y) = \exp\bigl(-\mathsf{L}(V;Y)\bigr)$ and $\mathsf{L}(V;Y)$ is positive we have that
$$\sup_{v} \Bigl|\sqrt{g(V;Y)}\Bigr| + \Bigl|\sqrt{g(V;Y')}\Bigr|  \le 2,  $$
and so Assumption \ref{a:dh1} (ii) is satisfied. To see that Assumption \ref{a:dh1} (i) is also satisfied, note that $e^{-x}$ is Lipschitz-1. Therefore, using the Cauchy-Schwarz inequality and some algebraic manipulations, there is  $\kappa$ independent of $Y$ and $Y'$ such that
\begin{align*}
\Bigl|\sqrt{g(V,Y)} - \sqrt{g(V;Y')} \Bigr|&\le  \frac{1}{2} \Bigl| \mathsf{L}(V;Y) - \mathsf{L}(V;Y')  \Bigr|  \\
&=\frac{1}{2} \biggl| \sum_{j=0}^{J-1}\frac12 \Bigl( |y_{j+1} - h(v_{j+1})|_\Gamma^2 - |y'_{j+1} - h(v_{j+1})|_\Gamma^2   \Bigr)  \biggr|  \\
&\le \kappa \sum_{j=0}^{J-1} |y_{j+1} - y'_{j+1}|_\Gamma  |y_{j+1} + y'_{j+1} -2 h(v_{j+1})|_\Gamma
   \\
   &\le \kappa |Y-Y'| \phi(V),
\end{align*}
where $\phi$ is defined in Assumption \ref{a:swp}.
This shows that under Assumption \ref{a:swp} the likelihood function of the smoothing problem satisfies Assumption \ref{a:dh1} with $\delta =|Y-Y'|$; Theorem \ref{t:wps} follows from Theorem \ref{thm1}.
\end{proof}

\subsection{The Filtering Problem}
\subsubsection{Formula for pdf of the Filtering Problem}
Filtering consists on the iterative updating of a sequence of probability distributions, as new data arrives.
This rests on the formula
\begin{equation}
\pi_{j+1} = {\mathcal{A}}_j {\mathcal{P}}\pi_j, \notag
\end{equation}
where ${\mathcal{P}}$ is a {\em linear} Markov map and ${\mathcal{A}}_j$ is a {\em nonlinear}
likelihood map (Bayes theorem).
We introduce $\hat{\pi}_{j+1}=\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)$ and recall
that ${\pi}_{j}=\operatorname{\mathbbm{P}}(v_{j}|Y_j).$
Then the maps ${\mathcal{P}}$ and ${\mathcal{A}}_j$ are defined by
\begin{eqnarray}
\begin{aligned}
{ \text{\bf Prediction Step:}} ~~~~~  \hat{\pi}_{j+1} = {\mathcal{P}} \pi_j. \\
{\text{ \bf Analysis Step:}} ~~~~~  \pi_{j+1} = {\mathcal{A}}_j \ \hat{\pi}_{j+1}.
\end{aligned}
\label{eq:pna}
\end{eqnarray}

The map ${\mathcal{P}}$ is sometimes termed {\em prediction}; the map ${\mathcal{A}}_j$ is
termed {\em analysis}. Throughout this section, all integrals are over $\mathbbm{R}^\Du.$
First we derive the linear Markov map ${\mathcal{P}}$. By the Markov property of the
stochastic dynamics model we have
\begin{align}
\hat{\pi}_{j+1}(v_{j+1}) &= \operatorname{\mathbbm{P}}(v_{j+1}|Y_j)\\
&= \int \operatorname{\mathbbm{P}}(v_{j+1}|Y_j, v_j)\operatorname{\mathbbm{P}}(v_j|Y_j)dv_j \notag \\
&= \int\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\operatorname{\mathbbm{P}}(v_j|Y_j)dv_j \notag \\
&= \int\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\pi_j(v_j)dv_j \notag \\
&= \frac{1}{(2\pi)^{d/2} ({\rm det}\Sigma)^{1/2}}\int\exp\Bigl(-\frac{1}{2}|v_{j+1}-\Psi(v_j)|^2_\Sigma \Bigr)\pi_j(v_j)dv_j. \notag
\end{align}
This defines the linear integral operator ${\mathcal{P}}$.

Now we derive the nonlinear likelihood map ${\mathcal{A}}_j$ by using Bayes theorem:
\begin{equation}
 \pi_{j+1} = {\mathcal{A}}_j\hat{\pi}_{j+1}. \notag
 \end{equation}
We note that
\begin{align}
\pi_{j+1}(v_{j+1}) &= \operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1})\\
&= \operatorname{\mathbbm{P}}(v_{j+1}|Y_j, y_{j+1}) \notag \\
&= \frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1}, Y_j)\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)}{\operatorname{\mathbbm{P}}(y_{j+1}|Y_j)} \notag \\
&= \frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)}{\operatorname{\mathbbm{P}}(y_{j+1}|Y_j)} \notag \\
&= \frac{\exp(-\frac{1}{2}|y_{j+1}-h(v_{j+1})|^2_{\Gamma})\hat{\pi}_{j+1}(v_{j+1})}{\int \exp(-\frac{1}{2}|y_{j+1}-h(v_{j+1})|^2_{\Gamma})\hat{\pi}_{j+1}(v_{j+1})dv_{j+1}}.\notag
\end{align}
This defines the nonlinear map ${\mathcal{A}}_{j+1}$ through multiplication
by the likelihood, and then normalization to a probability measure.

\subsubsection{Well-Posedness of the Filtering Problem}
Now we establish the well-posedness of the filtering problem. We let
\begin{equation*}
\pi_J = \operatorname{\mathbbm{P}}(v_J|Y), \quad \quad \pi_J' = \operatorname{\mathbbm{P}}(v_J|Y')
\end{equation*}
be two filtering distributions arising from observed data $Y = Y_J$ and $Y' = Y_J'.$ As noted in Remark \ref{rem:marginal}, the filtering distribution at time $J$ is the $J$-th marginal of the smoothing distribution; using this observation, the well-posedness of the filtering problem is a direct consequence of the well-posedness of the smoothing problem.
\begin{corollary}[Well-posedness of Filtering]
Under Assumption \ref{a:swp},
there exists $\kappa = \kappa(R)$, such that $d_{\mbox {\tiny{\rm TV}}}(\pi_J, \pi_J') \leq \kappa |Y-Y'|$.
\end{corollary}

\begin{proof}
Let $\Pi, \Pi'$ be the posterior distributions from the smoothing problems $\Pi = \operatorname{\mathbbm{P}}(V|Y)$ and $\Pi' = \operatorname{\mathbbm{P}}(V|Y')$. We
note that there exists $\kappa$ such that $d_{\mbox {\tiny{\rm TV}}}(\Pi,\Pi') \leq \kappa|Y-Y'|$
by Theorem \ref{t:wps} and by the fact that the Hellinger metric bounds
the total variation metric (Lemma \ref{l:dh2}).
Let $f: \mathbbm{R}^\Du \to \mathbbm{R}$ and
$F: \mathbbm{R}^{d(J+1)} \mapsto \mathbbm{R}.$ Then
\begin{align}
d_{\mbox {\tiny{\rm TV}}} (\pi_J, \pi_{J}') &= \frac{1}{2} \sup_{|f|_\infty \leq 1 }\Bigl|\operatorname{\mathbb{E}}^{\pi_J}[f(v_J)] - \operatorname{\mathbb{E}}^{\pi_{J'}}[f(v_J)] \Bigr| \notag \\
&= \frac{1}{2} \sup_{|f|_{\infty} \leq 1} \Bigl|\operatorname{\mathbb{E}}^\Pi[f(v_J)] - \operatorname{\mathbb{E}}^{\Pi'}[f(v_J)] \Bigr| \notag \\
&\leq \frac{1}{2} \sup_{|F|_{\infty} \leq 1} \Bigl| \operatorname{\mathbb{E}}^\Pi[F(V)] - \operatorname{\mathbb{E}}^{\Pi'}[F(V)]  \Bigr|\notag \\
&= d_{\mbox {\tiny{\rm TV}}}(\Pi, \Pi') \notag \\
&\leq \kappa|Y-Y'|. \notag
\end{align}
Here the first inequality follows from the fact that $\{|f|\leq 1\}$ can be viewed as a subset of $\{|F| \leq 1\}$.
\end{proof}

\subsection{Discussion and Bibliography}

The book \cite{law2015data} gives a mathematical introduction to
data assimilation;
for further information on the smoothing problem as presented here,
see section 2.3 in that book;
for further information on the filtering problem as presented here,
see section 2.4.
The books \cite{abarbanel2013predicting,reich2015probabilistic} give
alternative foundational presentations of the subject.
The books \cite{kalnay2003atmospheric,oliver2008inverse,majda2012filtering,carrassi2018data}
study data assimilation in the context of weather forecasting, oil reservoir
simulation, turbulence modeling and geophysical sciences, respectively.

\section{\Large{\sffamily{Markov Chain Monte Carlo}}}
In this chapter we study Markov chain Monte Carlo (MCMC),
a methodology that allows to obtain approximate samples from a given {\em target} distribution $\pi.$ As with Monte Carlo and importance sampling, MCMC may be viewed as approximating the target distribution by a sum of Dirac measures, thus allowing to approximate expectations with respect to the target. Implementation of Monte Carlo pressuposes that independent samples from the target can be obtained. Importance sampling and MCMC bypass this restrictive assumption: importance sampling by appropriately weighting independent samples from a proposal distribution, and MCMC by drawing correlated samples from a Markov kernel that has the target as invariant distribution. After some discussion of the general MCMC methodology, we will specify to the case where $\pi$ is a posterior distribution given via Bayes theorem from the product of the likelihood function and the prior distribution. In this context, we will analyze the convergence of the pCN algorithm, a popular MCMC method for computing posterior expectations in high dimensional inverse problems which uses these two ingredients, the prior and the likelihood, as part of its design.

\subsection{The Idea Behind MCMC}

The idea of MCMC is that, given a target distribution $\pi,$ it is possible to construct a Markov kernel that can be sampled from and has $\pi$ as its invariant distribution (a formal definition of the invariance property will be given below). Samples  $\{ u^{(n)} \}_{n=1}^{N}$  drawn iteratively from the kernel may be used to approximate posterior expectations
$$\pi(f) \approx \frac{1}{N} \sum_{n=1}^N f(u^{(n)}).$$
The samples are given uniform weights $1/N$ but, in contrast to standard Monte Carlo, they are not independent and they are not drawn from the target $\pi.$ However,  if the chain is guaranteed to satisfy certain {\em sample path ergodicity} then the estimator in the right hand-side is asymptotically unbiased for $\pi(f)$ and satisfies a central limit theorem for suitable test functions $f.$

Addressing the design and analysis of MCMC methods in generality and depth is beyond the scope of a single chapter;  entire books are devoted to this subject. We will restrict our discussion to a particular class of MCMC methods, known as Metropolis-Hastings algorithms. We will prove that the Metropolis-Hastings kernel is invariant with respect to the desired target distribution, and we will show \emph{geometric ergodicity} of the pCN Metropolis-Hastings algorithm, meaning that the distribution $\pi_n$ of the $n$-th sample approaches the invariant distribution exponentially fast in total variation distance. The idea is illustrated in Figure \ref{fig:MCMC_chain}: after
an initial number of {\em burn-in} steps the samples from
the chain start to concentrate in regions where the target distribution has greatest mass. We will not discuss sample path ergodicity, noting simply that a general abstract theory exists to deduce it from geometric ergodicity.

  \FloatBarrier
    \begin{figure}[htp]
      \centering
      \includegraphics[width=0.75\columnwidth]{figures/lec7_MCMC_chain1.png}
      \caption{The Markov chain samples points from distribution $\pi_n$ at step $n$, and the sampling distribution converges towards the target distribution
$\pi$ whose high density regions are represented by the dashed circles.}
      \label{fig:MCMC_chain}
    \end{figure}
    \FloatBarrier

\subsection{The Metropolis-Hastings Algorithm}

Here we outline the Metropolis-Hastings algorithm. The
algorithm has two ingredients: a {\em proposal distribution} $q(u,v)$,
which is a Markov transition kernel; and an acceptance probability
$a(u,v)$ that will be used to convert the proposal kernel into a kernel $p_{\mbox {\tiny{\rm MH}}}(u,v)$ that is invariant with respect to the given target $\pi.$ Given the $n^{\rm th}$ sample $u^{(n)},$ we generate
$u^{(n+1)}$ by drawing $v^\star$ from the distribution $q(u^{(n)}, \cdot)$ and
accepting the result, which means setting $u^{(n+1)}=v^\star$,
with probability $a(u^{(n)}, v^{\star})$, or instead setting $u^{(n+1)} = u^{(n)}$ with
the remaining probability $1 - a(u^{(n)}, v^{\star})$.
The acceptance probability is given by
\begin{eqnarray}
\label{eq:MHA}
a(u,v) = \text{min} \Bigg( \frac{\pi(v) q(v, u)}{\pi(u) q(u, v)},\, 1\Bigg).\end{eqnarray}
Algorithm \ref{algMH} makes precise the steps just described:

\FloatBarrier
\begin{algorithm}
\caption{\label{algMH} Metropolis-Hastings Algorithm}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Target distribution $\pi$, initial distribution $\pi_0,$ Markov kernel $q(u,v)$, number of samples $N.$  \\
\vspace{.04in}
\STATE {\bf Initial Draw}: Draw initial sample $u^{(0)} \sim \pi_0.$
 \\
\vspace{.04in}
\STATE{{{\bf Subsequent Samples}}}: For $n = 0,1,\dots,N -1$, perform the following steps:
\begin{enumerate}
\item Sample $v^{\star} \sim q(u^{(n)},\cdot).$
\item Calculate the acceptance probability $a_n:=a(u^{(n)},v^{\star}).$
\item Update
\begin{align*}
u^{(n+1)} =
\begin{cases}
 v^{\star}, & \text{w.p.} ~~~ a_n, \\ u^{(n)}, & \text{otherwise}.
\end{cases}
 \end{align*}
\end{enumerate}
\STATE{\bf Output} Samples $u^{(1)},u^{(2)},\dots,u^{(N)}.$
\label{alg_1}
\end{algorithmic}
\end{algorithm}
\FloatBarrier

The Metropolis-Hastings algorithm implicitly defines a Markov kernel $p_{\mbox {\tiny{\rm MH}}}(u,\cdot)$ which specifies the density of the $(n+1)^{\rm th}$ sample given that the $n^{\rm th}$ sample is located at $u.$ For $u\neq v,$  the Metropolis-Hastings kernel has the following simple expression in terms of the proposal kernel and the acceptance probability
\begin{equation}\label{eq:pmh}
p_{\mbox {\tiny{\rm MH}}}(u,v) =  a(u,v) q(u,v);
\end{equation}
this expression may be deduced noting that in order to move to a new location, the proposal needs to be accepted.

\begin{remark}
\begin{itemize}
\item In order to implement the Metropolis-Hastings algorithm one needs to be able to sample from the proposal kernel $q(u,\cdot)$ and evaluate the acceptance probability $a(u,v).$ Importantly, the target distribution only appears in the acceptance probability $a(u,v),$ and only the ratio $\pi(v)/\pi(u)$ is involved. Therefore the Metropolis-Hastings algorithm may be implemented for target distributions that are only specified up to an unknown normalizing constant.
\item If $q(u,v) = q(v,u)$ the acceptance probability simplifies to $\pi(v)/\pi(u).$ In such a case, moves to regions of higher target density are always accepted, while moves to regions of smaller but non-zero target density are accepted with positive probability in order to ensure exploration of the target space. If $q$ is not symmetric, the method favors moves that are easier to be reversed, namely moves for which  $q(v,u) >q(u,v).$
\item The Metropolis-Hastings algorithm is extremely flexible due to the freedom in the choice of proposal kernel $q(u,v)$. The ergodic behavior of the algorithm is heavily dependent on the choice of proposal.
\item The accept-reject step may be implemented by drawing, independently
from the proposal, a uniformly distributed random variable $\theta_n$ in the interval $[0,1].$ If $\theta_n \in [0,a_n)$ then the proposal is accepted ($u^{(n+1)}=v^\star$); it is rejected ($u^{(n+1)}=u^{(n)})$ otherwise.
\end{itemize}
\end{remark}

\subsection{Invariance of the Target Distribution \texorpdfstring{$\pi$}{}}
In this section we show that the Metropolis-Hastings kernel is invariant with respect to the target $\pi.$ We start by introducing formal definitions of detailed balance and invariance of a Markov kernel, and showing that the former implies the latter. We then prove that the Metropolis-Hastings kernel satisfies detailed balance with respect to $\pi$, and hence it is invariant.

\subsubsection{Detailed Balance and its Implication}
The following definition makes precise the notion of detailed balance and invariant distribution of a Markov kernel.

\begin{definition}
A Markov kernel $p(u,v)$ satisfies detailed balance with respect to $\pi$ if, for any $u, v \in \mathbbm{R}^\Du,$
\begin{equation*}
\pi(u) p(u,v) = \pi(v) p(v,u).
\end{equation*}
We say that $\pi$ is an invariant distribution of the Markov kernel $p(u,v)$ if, for any $v\in \mathbbm{R}^\Du,$
\begin{equation}
\int_{\mathbbm{R}^\Du} \pi(u) p(u,v)\, du = \pi(v).
\end{equation}
\end{definition}

Detailed balance of $p(u,v)$ with respect to $\pi$ implies that $\pi$ is invariant for $p(u,v)$. To see this, note that if $p(u,v)$ satisfies detailed balance with respect to $\pi$ then
$$\int_{\mathbbm{R}^\Du} \pi(u) p(u,v)\, du = \pi(v) \int_{\mathbbm{R}^\Du} p(v,u)\, du = \pi(v).$$
Invariance guarantees that, if the chain is distributed according to $\pi$ at a given  step, then it will also be distributed according to $\pi$ in the following step. Detailed balance guarantees that the in/out flow between any two states is preserved, which is a stronger condition. The concept of detailed balance is illustrated through the flow diagram shown in Figure \ref{fig:DB4}. The flow to/from any node must be the same under detailed balance, so the density at each node must remain constant at each step, as shown in Figure \ref{fig:DB4}(b); Figure \ref{fig:DB4}(a) fails
to satisfy this.

\subsubsection{Detailed Balance and the Metropolis-Hastings Algorithm}

Consider a generic Markov kernel $q(u,v)$ as shown in Figure \ref{fig:DB4}(a),
which does not satisfy detailed balance, which we accept-reject
according to the Metropolis algorithm.
Intuitively, this Metropolis-Hastings kernel, limits the ``flow'' between nodes such that detailed balance is satisfied with respect to the distribution $\pi$, as shown in Figure \ref{fig:DB4}(b). There the red-highlighted numbers represent where the transition probabilities of the kernel have been reduced. The following theorem establishes the detailed balance of the Metropolis-Hastings kernel with respect to the target $\pi$; it implies, as a consequence, that the Metropolis-Hastings kernel is invariant with respect to the target.

      \begin{theorem}[Metropolis-Hastings and Detailed Balance]
       The Metropolis-Hastings kernel satisfies detailed balance with
respect to the distribution $\pi$.
      \end{theorem}

      \begin{proof}
      We need to show that, for any $u,v \in \mathbbm{R}^d,$
      $$\pi(u) p_{\mbox {\tiny{\rm MH}}}(u,v) = \pi(v) p_{\mbox {\tiny{\rm MH}}}(u,v).$$
If $u=v$ the equality is trivial. Let us assume henceforth that  $u\neq v,$ so that by equation \eqref{eq:pmh} we have
$$p_{\mbox {\tiny{\rm MH}}} (u,v) = a(u,v) q(u,v).$$
We rewrite
\begin{align*}
	p_{\mbox {\tiny{\rm MH}}}(u, v) &= \text{min }\biggl( \frac{\pi(v) q(v,u)}{\pi(u) q(u,v)}  ,1   \biggr) ~ q(u, v)\\
	&= \frac{1}{\pi(u)} \times \text{min }\Bigl( \pi(u) q(u,v), \pi(v) q(v,u)\Bigr).
\end{align*}
Thus, invoking symmetry,
	$$ \pi(u) p_{\mbox {\tiny{\rm MH}}}(u, v) = \text{min }\bigl( \pi(u) q(u,v), \pi(v) q(v,u)\bigr) = \pi(v) p_{\mbox {\tiny{\rm MH}}}(v, u).$$
	 \end{proof}

The invariance of the Metropolis-Hastings $p_{\mbox {\tiny{\rm MH}}}$ with respect to $\pi$ implies that if the initial sample is drawn from the target ($\pi_0 = \pi)$, then all subsequent samples are also distributed according to the target $(\pi_n = \pi).$

    \begin{figure}
      \centering
      \includegraphics[width=0.85\columnwidth]{figures/lec7_DB4.png}
      \caption{Toy representation of MCMC with 3 states ($u, v, w$). The numbers associated with each term represent an example case. (a) The top chain represents a Markov chain with Markov kernel $q$ which does not satisfy detailed balance with respect to $\pi$.  (b) The bottom chain utilizes Markov kernel $p$, which satisfies detailed balance by modifying the transition probabilities of $q$.}
      \label{fig:DB4}
    \end{figure}

\subsection{Convergence to the Target Distribution}
In the previous section we showed that if we initialize the Metropolis-Hastings algorithm with distribution $\pi$, all samples produced by the algorithm will be distributed according to $\pi.$
But our original problem was exactly that we were not able to sample from $\pi$.
Our aim in this section is to show that, for certain Metropolis-Hastings methods, the law $\pi_n$ of the $n$-th sample converges to $\pi$ regardless of the initial distribution $\pi_0.$
This is a strong form of  {\em ergodic} behavior which does not hold in general, as illustrated by the chain depicted in Figure~\ref{fig-convergence}.

\begin{figure}
  \centering
  \includegraphics[scale=0.70]{figures/lec7_period.jpg}
  \caption{The arrows represent transitions with probability one in a four
state Markov chain. The invariant distribution is the uniform distribution but for $\pi_{0} = \delta_{A}$, we do not have ergodicity.}\label{fig-convergence}
\end{figure}

In order to understand the mechanisms behid ergodicity we will
first consider Markov chains with finite state space. We then study a specific Metropolis-Hastings
algorithm, known as the {\em preconditioned Crank-Nicolson  (pCN)} method,
which applies to targets $\pi$ defined by their density with respect to
a Gaussian distribution.

\subsubsection{Finite State Space}

We consider a Markov chain on the finite state space
$S = \left\{ 1 , \cdots , d \right\}.$
We illustrate a {\em coupling} approach to proving ergodicity and then,
in the next subsection, use it to study the pCN method on a continuous
state space.

\begin{theorem}[Ergodicity in Finite State Spaces]
Let $u_{n}$ be a Markov chain with state space $S$, transition kernel $p$,
and initial distribution $\pi_{0}$. Assume that
  \begin{equation}
    \label{eqn-unif}
    \varepsilon \coloneqq \min_{i,j \in S} p\left( i,j \right) > 0.
  \end{equation}
  Then, $p$ has a unique invariant distribution $\pi$, and the following convergence result holds:
  \begin{equation}\label{eq:tvbound}
    d_{\mbox {\tiny{\rm TV}}}( \pi_{n}, \pi) \leq \left( 1 - \varepsilon \right)^{n},
  \end{equation}
  where $\pi_n$ is the law of $u_n.$
\end{theorem}
\begin{proof}
  First note that the Markov kernel $p$ maps a probability distribution on $S$
into another probability distribution on $S$; it thus maps a compact, convex set into
itself. By Brouwer's fixed point theorem it follows that $p$ has a fixed point
in this space, ensuring that an invariant distribution exists. We will now show that for any invariant distribution $\pi$ equation \eqref{eq:tvbound} holds, which then implies the uniqueness of the invariant distribution.

Let $\pi$ be an invariant distribution, a probability vector on $S$. Proving
convergence to equilibrium amounts to ``forgetting the past'', to show that the long time behavior of the Markov chain does not depend on the initial distribution $\pi_0$ and in fact converges to $\pi$. In general, $u_{n+1}$ will be strongly dependent on $u_{n}$, but
the condition given in \eqref{eqn-unif} implies that there is always some residual chance that the chain jumps to any new state, at each step, independently
of where it is currently located, $u_{n}$. We will show that this residual
probability of the chain to make a ``totally random'' move will diminish the
stochastic dependence on $u_{0}$ as $n$ increases.

To formalize this idea, let $b_n$ be i.i.d. Bernoulli random variables with $\operatorname{\mathbbm{P}}\left( b_n = 1 \right) = \varepsilon$ and $\operatorname{\mathbbm{P}}\left( b_n = 0 \right) = 1-\varepsilon;$ furthermore assume that the sequence $\{b_n\}$ is independent of the
randomness defining draws from $\{p(u_n,\cdot)\}$.
Because of the lower bound on $p$ we may define a new Markov chain as follows:
  \begin{equation} \label{eq:kernel}
    w_{n + 1} \sim
    \begin{cases}
      &s\left(w_{n}, \cdot \right), \hfill \text{ for } b_n = 0,\\
      &r\left(w_{n}, \cdot \right), \hfill \text{ for } b_n = 1,
    \end{cases}
  \end{equation}
  where
  \begin{equation*}
    s\left( i, j \right) := \frac{p\left(i, j \right) - \varepsilon r\left(i, j \right)}{1- \varepsilon}
  \end{equation*}
and  $r$ is the uniform transition kernel with equal probability of
transitioning to each state in $S: r(i,j)=d^{-1}$ for all $(i,j) \in S \times S.$
We may now compute
  \begin{align*}
    \operatorname{\mathbbm{P}}\left( w_{n + 1} = j | w_{n} = i \right) &= \varepsilon \operatorname{\mathbbm{P}}\left( w_{n + 1} = j | w_{n} = i, b_n = 1 \right) + \left( 1 - \varepsilon \right) \operatorname{\mathbbm{P}}\left( w_{n + 1} = j | w_{n} = i, b_n = 0 \right)\\
    &= \varepsilon r\left( i,j \right) + p\left( i,j \right) - \varepsilon r\left( i,j \right) \\
    &= p\left( i,j \right).
  \end{align*}
  Thus the kernel defined by \eqref{eq:kernel} is equivalent in law to $p$.
  However, by introducing the ancillary random variables $b_n$, we have
made explicit the concept of ``forgetting the past entirely, with a small probability'' at every step. We may now use this to complete the proof.
Let $f:S \mapsto \mathbbm{R}$ be an arbitrary test function with $|f|_\infty \leq 1$ and $\tau \coloneqq \min\left( n \in \mathbbm{N}: b_n = 1 \right).$
Then, regardless of how $w_n$ is initialized,
  \begin{align*}
    \operatorname{\mathbb{E}}\left[ f\left( w_{n} \right) \right]
    &= \operatorname{\mathbb{E}}\left[ f\left( w_{n} \right) | \tau \geq n \right]\operatorname{\mathbbm{P}}\left( \tau \geq n \right)  + \sum \limits_{l = 0 }^{n-1} \operatorname{\mathbb{E}}\left[f \left(w_{n}\right) | \tau = l  \right] \operatorname{\mathbbm{P}}\left( \tau =l \right)  \\
    &=\underbrace{\operatorname{\mathbb{E}}\left[ f\left( w_{n} \right) | \tau \geq n \right]\operatorname{\mathbbm{P}}\left( \tau \geq n \right)}_{|\cdot| \leq \left( 1-\varepsilon \right)^{n}}  + \underbrace{\sum \limits_{l = 0 }^{n-1} \operatorname{\mathbb{E}}_{u_{0} \sim \mathfrak{u}\left( \cdot \right)}\left[f\left(w_{n-l}\right) \right] \operatorname{\mathbbm{P}}\left( \tau =l \right)}_{\text{independent of original initial distribution}},
  \end{align*}
where $\mathfrak{u}$ denotes the uniform distribution on $S.$

Now consider two Markov chains $w_n$ and $w_n'$ with kernel \eqref{eq:kernel}, the first initialized from $\pi_0$ and the second from an invariant distribution $\pi.$ The law of $w_n$ agrees with the law $\pi_n$ of the original chain $u_n,$ while for the second chain it follows from invariance that $\pi_n' = \pi.$ We will use the variational characterization of the total variation distance established in Lemma \ref{lemmatv}. Employing the preceding identity and noting that the contribution which is independent of the initial distribution will cancel in the two different Markov chains, we obtain
  \begin{align*}
    d_{\mbox {\tiny{\rm TV}}}( \pi_{n}, \pi_n') = \frac{1}{2} \sup \limits_{|f|_\infty \leq 1} \left| \operatorname{\mathbb{E}}^{\pi_{n}}\left[ f\left( u \right) \right] - \operatorname{\mathbb{E}}^{\pi_n'}\left[ f\left( u \right) \right]\right| \leq \left( 1 - \varepsilon \right)^{n}.
  \end{align*}
Since  $\pi_n' = \pi$ the desired result follows.
\end{proof}

Before extending the above argument to a setting with continuous state
space, we make two remarks:
\begin{remark}
The coupling proof we have just exhibited may be generalized in a number of
ways; in particular:

  \begin{itemize}
    \item The distribution $r$ need not be uniform; it  was only chosen so
for convenience. What is important is that $r_{i,j}$ is lower bounded,
independently of $i$, for all $j$. Adapting $r$ to the $p$ at hand, might
in some cases greatly improve the above bound -- a larger $\epsilon$ might
be identified.
    \item Convergence to equilibrium can also be shown if condition~\eqref{eqn-unif} holds with $p$ replaced by the $n$-step transition kernel $p^{n}$.
Again, for some chains this may yield faster bounds on the convergence
to equilibrium.
  \end{itemize}
  \end{remark}

\subsubsection{The pCN Method}

The coupling argument used in the previous subsection for Markov chains with finite state space may also be employed to study ergodicity of Markov chains on a continuous state space.
To illustrate this we consider a particular Metropolis-Hastings algorithm,
the pCN method, applied to a specific Bayesian inverse problem.
Before we get into the details of the inverse problem we describe the
idea behind the pCN method at a high level.
The idea is this. If the desired target distribution has the form
\begin{equation}\label{eq:formoftarget}
\pi(u)= \frac{1}{Z}  \tilde{g}(u)  \tilde{\rho}(u)
\end{equation}
and if the Metropolis-Hastings proposal kernel $q$ satisfies detailed balance with respect to $\tilde{\rho}$, then \eqref{eq:MHA} simplifies to give
\begin{eqnarray}
\label{eq:MHA2}
a(u,v) = \text{min} \Bigg( \frac{\tilde{\rho}(v)\tilde{g}(v) q(v, u)}{\tilde{\rho}(u)\ \tilde{g}(u) q(u, v)}, 1\Bigg)= \text{min} \Bigg( \frac{  \tilde{g}(v)}{\tilde{g}(u)}, 1\Bigg).\end{eqnarray}
We will apply and study this idea in the case where $\tilde{\rho}$ is a Gaussian distribution, in which case it is straightforward to construct a proposal kernel that satisfies detailed balance with respect to $\tilde{\rho}$. This scenario arises naturally in Bayesian inverse problems where the the prior is defined in terms of a Gaussian. We now formalize the inverse problem setting that we consider by imposing certain assumptions on the likelihood and the prior, and then relate both to the functions $\tilde{g}$ and $\tilde{\rho}$ in Equation \ref{eq:formoftarget}.

\begin{assumption}
\label{a:erg}
We make the following assumptions on the Bayesian inverse problem:
\begin{itemize}
    \item Bounded likelihood: there are $g^-, g^+ >0$ such that, for all $u \in \mathbbm{R}^\Du,$
    $$0 <g^- <g(u) < g^+.$$
    \item Truncated Gaussian prior: there is a compact set $B \subset \mathbbm{R}^\Du$ with positive Lebesgue measure such that   $\rho(u) =\mathbb{1}_{B}(u)   z(u),$ where $z=\mathcal{N}\left( 0,\hat{C} \right)$.
\end{itemize}
\end{assumption}

Under Assumption \ref{a:erg} we obtain for the posterior density
\begin{equation*}
 \pi\left( u \right)  \propto g(u) \mathbb{1}_{B}\left( u \right) z(u),
\end{equation*}
which is of the form in Equation \eqref{eq:formoftarget} with $\tilde{g}(u) = g(u) \mathbb{1}_B(u)$ and $\tilde{\rho}(u) = z(u).$

The pCN method has proposal kernel which is invariant with respect to
the Gaussian $z =\mathcal{N}\left( 0,\hat{C} \right)$ defined by
\begin{equation}\label{eq:pcnp}
q(u,\cdot) \sim \mathcal{N}\Bigl( (1-\beta^2)^{1/2} u, \beta^2 \hat{C}  \Bigr).
\end{equation}
Thus, given the sample $u^{(n)},$ the pCN proposes a new sample
\begin{equation*}
  v^\star \sim \left( 1 - \beta^{2} \right)^{1/2}u^{(n)} + \beta \xi^{(n)},
\quad \xi^{(n)}\sim \mathcal{N}\left( 0, \hat{C} \right),
\end{equation*}
which only requires to sample a Gaussian.

\begin{theorem}[Ergodicity for pCN Method]
\label{t:pCN}
Assume that we apply the pCN method to sample from a
posterior density $\pi$ arising from Assumptions \ref{a:erg} with initial
condition drawn from any density supported on $B$. Then
there exists a constant $\varepsilon \in \left( 0,1 \right)$ such that
  \begin{equation*}
    d_{\mbox {\tiny{\rm TV}}} ( \pi_{n} , \pi) \leq \left( 1 - \varepsilon \right)^{n},
  \end{equation*}
where $\pi_{n}$ is the law of the $n$-th sample from the pCN Metropolis-Hastings algorithm.
\end{theorem}
Recall the notation for the covariance weighted inner-product and
resulting norm described in the introduction to these notes.
Before proving the above theorem, we will prove an auxiliary lemma.

\begin{lemma}
Let the assumptions of Theorem \ref{t:pCN} hold.  The proposal kernel \eqref{eq:pcnp} satisfies detailed balance with respect to the Gaussian $z =\mathcal{N}\left( 0,\hat{C} \right)$  and we have the following expression for the pCN accept-reject probability:
  \begin{align*}
    a\left( u,v \right)
         =\min\left( \frac{g(v)}{g(u)}  \mathbb{1}_{B} \left( v \right) ,1  \right).
  \end{align*}
\end{lemma}
\begin{proof}
We need to show that $z(v) q(v,u)$ is symmetric in $u$ and $v.$ By direct calculation,
\begin{align*}
-\log\bigl( z(v) q(v,u) \bigr)  &=\frac{1}{2}|v|^{2}_{\hat{C}} + \frac{1}{2 \beta^{2}} \left| u - \left( 1 - \beta^{2} \right)^{1/2} v\right|_{\hat{C}}^{2}\\
  &=\left( \frac{1}{2} + \frac{\left( 1 - \beta^{2} \right)}{2 \beta^{2}}  \right) |v|^{2}_{\hat{C}} + \frac{1}{2\beta^{2}} |u|^{2}_{\hat{C}} - \frac{\left( 1 - \beta^{2} \right)^{1/2}}{\beta^{2}} \langle u , v \rangle_{\hat{C}}\\
  &= \frac{1}{2 \beta^{2}}\left( |v|^{2}_{\hat{C}} + |u|^{2}_{\hat{C}} \right) - \frac{\left( 1 - \beta^{2} \right)^{1/2}}{\beta^{2}} \langle u , v \rangle_{\hat{C}}.
\end{align*}
For the expression for the acceptance probability note that since $u_{0} \in B$, we have $u_{n} \in B$ for all $n$ as any proposed move out of $B$ will be rejected. Thus $\mathbb{1}_{B}\left( u \right)$ may be dropped from the formula for the acceptance probability.
\end{proof}

Using this lemma, we can prove ergodicity much as we did in the previous
subsection in the finite state space setting.
The main idea is that, restricted to the bounded set $B$, the probability density of the transition kernel will be bounded away from zero by some $\varepsilon$.
Splitting off a ``forgetful part'' that is triggered with probability $\varepsilon$ will then yield the result.

\begin{proof}[Proof of Theorem \ref{t:pCN}]
Note again that since $u_0 \in B$ we have $u_n \in B$ for all $n \ge 1.$
Note further that since $B$ is compact and $q$ is continuous in both of its arguments, there is $q^- >0$ such that, for any $u, v \in B,$
$$q(u,v) \ge q^-.$$
Let $p$ be the Markov kernel defined by the pCN Metropolis-Hastings algorithm. It follows that, for $u, v \in B,$
\begin{align*}
p(u,v) & \ge q(u,v) a(u,v) \\
& \ge q^- \frac{g^-}{g^+} =: \varepsilon \text{Leb} (B),
\end{align*}
where the last equation defines $\varepsilon$ and $\text{Leb}(B)$ denotes the Lebesgue measure of $B$ (which is assumed to be positive).
Analogously to the discrete proof, we now define $b_n$ to be i.i.d.
Bernoulli random variables with $\operatorname{\mathbbm{P}}\left( b_n = 1 \right) = \varepsilon$, independently of all other randomness, and consider the transition rule
  \begin{equation*}
    u_{n +1} \sim
    \begin{cases}
      &s\left( u_{n} , \cdot \right), \hfill \text{ for } b_n = 0,\\
      &r\left( \cdot \right), \hfill \text{ for } b_n = 1,
    \end{cases}
  \end{equation*}
  where $r$ denotes the uniform distribution on $B$ and, for $A\subset B$ and $u \in B,$
  \begin{equation*}
    s\left( u, A \right) \coloneqq \frac{p\left(u, A \right) - \varepsilon r\left( A \right)}{1- \varepsilon}.
  \end{equation*}
Just as in the discrete case, one can check that the resulting Markov kernel is equal to the pCN Metropolis-Hastings kernel $p(\cdot,\cdot)$. Exponential convergence is then concluded in exactly the
same way as in the discrete case.
\end{proof}

\subsection{Discussion and Bibliography}

The book \cite{gamerman2006markov} is a useful
basic introduction to  MCMC and the book
\cite{brooks2011handbook} presents state of the art as of 2010.
The paper \cite{cotter2013} overviews the pCN method
and related MCMC algorithms specifically designed for inverse
problems and other sampling problems in high dimensional
state spaces.
The book \cite{lindvall2002chapters} describes the coupling method
in a general setting.
The book \cite{meyn2012markov} contains a wide-ranging presentation
of Markov chains, and their long-time behavior, including ergodicity,
and coupling. Furthermore the book describes the general methodology
for going from convergence of expectations in (possibly weighted)
total variation metrics to sample path ergodicity and almost sure convergence
of time averages, a topic we did not cover in this chapter.
The paper \cite{mattingly2002ergodicity} describes the
coupling methodology in the context of stochastic differential
equations and their approximations.

\section{\Large{\sffamily{The Linear-Gaussian Setting }}}
\label{sec: posteriordistribution}

Recall the inverse problem of estimating an unknown parameter $\vct{u} \in \mathbbm{R}^\Du$ from data $y\in \mathbbm{R}^\Dy$ under the model assumption
\begin{equation}\label{eq:yurelation}
y = G(u) + \eta.
\end{equation}
In this chapter we study the linear-Gaussian setting, where the forward map $G(\cdot)$ is linear and both the prior on $u$ and the distribution of the observation noise $\eta$ are Gaussian. This setting is highly amenable to analysis and arises frequently in applications. Moreover, as we will see throughout this book, many methods employed in nonlinear or non-Gaussian settings build on ideas from the linear-Gaussian case by performing linearization or invoking Gaussian approximations.
Having established a formula for the posterior, we investigate the effect that the choice of prior has on our solution. We do this by quantifying the spread of the posterior distribution in the small noise (approaching zero) limit. This provides intuitive understanding concerning the impact of the prior for overdetermined, determined, and underdetermined regimes, corresponding to $d <k,d =k,$ and $d>k,$ respectively.

The following will be assumed throughout this chapter.
\begin{assumption}
The relationship between unknown $u\in \mathbbm{R}^\Du$ and data $y\in \mathbbm{R}^\Dy$ defined by equation \eqref{eq:yurelation} holds. Moreover,
\label{a:sz1}
\begin{description}
	\item[1] Linearity of the forward model: $G(\vct{u})=\mtx{A}\vct{u}$, for some $\mtx{A}\in\mathbbm{R}^{k \times d}$;
    \item[2] Gaussian prior: $\vct{u}\sim \rho(\vct{u})=\mathcal{N}(\hat{m},\hat{C})$, where $\hat{C}$ is positive definite;
    \item[3] Gaussian noise: $\vct{\eta}\sim \nu(\vct{\eta})=\mathcal{N}(\zerovct,\mtx{\Gamma})$, where $\mtx{\Gamma}$ is positive definite;
    \item[4] $u$ and $\eta$ are independent.
\end{description}
\end{assumption}

\subsection{Derivation of the Posterior Distribution}

Under Assumption  \ref{a:sz1} the likelihood on $\vct{y}$ given $\vct{u}$ is a Gaussian,
\begin{align}
	\vct{y}|\vct{u}\sim \mathcal{N}(\mtx{A}\vct{u},\mtx{\Gamma}).
\end{align}
Therefore, using Bayes formula \eqref{eq:bayesformula} we see that the posterior $\pi^y(u)$ is given by
\begin{align*}
	\pi^{\vct{y}}(\vct{u})&=\frac{1}{Z}\nu(\vct{y}-\mtx{A}\vct{u})\rho(\vct{u})\\
	&=\frac{1}{Z}\exp\left(-\frac{1}{2}|\vct{y}-\mtx{A}\vct{u}|_{\mtx{\Gamma}}^2\right)\exp\left(-\frac{1}{2}|\vct{u}-\hat{m} |_{\hat{C}}^2\right)\\
	&=\frac{1}{Z}\exp\left(-\frac{1}{2}|\vct{y}-\mtx{A}\vct{u}|_{\mtx{\Gamma}}^2-\frac{1}{2}|\vct{u}-\hat{m}|_{\hat{C}}^2\right)\\
	&=\frac{1}{Z}\exp\bigl(-{\mathsf{J}}(\vct{u})\bigr),
\end{align*}
with
\begin{equation}
\label{eq:ju}
{\mathsf{J}}(\vct{u})=\frac{1}{2}|\vct{y}-\mtx{A}\vct{u}|_{\mtx{\Gamma}}^2+\frac{1}{2}|\vct{u} - \hat{m}|_{\hat{C}}^2.
\end{equation}
Since the posterior distribution can be written as the exponential of a quadratic in $u$ it follows that the posterior is Gaussian. Its mean and covariance are given in the following result.

\begin{theorem}[Posterior is Gaussian]
\label{t:sz1}
   	Under Assumption \ref{a:sz1}
the posterior distribution is  Gaussian,
   		\begin{align}
			\vct{u}|\vct{y} \sim \pi^{\vct{y}}(\vct{u}) =\mathcal{N}(m,C).\label{eq: posterior}
		\end{align}
The posterior mean $m$ and covariance $C$ are given by the
following formulae:
		\begin{align}
			m&=(\mtx{A}^{T}\mtx{\Gamma}^{-1}\mtx{A}+\hat{C}^{-1})^{-1}(\mtx{A}^{T}\mtx{\Gamma}^{-1}\vct{y} + \hat{C}^{-1}\hat{m}),\label{eq: postmean} \\
			C&=(\mtx{A}^{T}\mtx{\Gamma}^{-1}\mtx{A}+\hat{C}^{-1})^{-1}.
			\label{eq: postvar}
		\end{align}

	\label{thm: posteriorgaussian}
\end{theorem}
\begin{proof}
	Since $\pi^{\vct{y}}(\vct{u})=\frac{1}{Z}\exp\bigl(-{\mathsf{J}}(\vct{u})\bigr)$
with ${\mathsf{J}}(u)$ given by \eqref{eq:ju},
a quadratic function of $\vct{u}$, it follows that
the posterior distribution $\pi^{\vct{y}}(\vct{u})$ is Gaussian.
Denoting the mean and variance of $\pi^{\vct{y}}(\vct{u})$ by
$m$ and $C$, we can write ${\mathsf{J}}(\vct{u})$ in the following form
	\begin{align}
		{\mathsf{J}}(\vct{u})&=\frac{1}{2}|\vct{u}-m|_{C}^2+ q ,
		\label{eq: ju2}
	\end{align}
where the term $q$ does not depend on $\vct{u}$.
Now matching the coefficients of the quadratic and linear terms
in equations \eqref{eq:ju} and \eqref{eq: ju2}, we get
	\begin{align*}
		C^{-1}&=\mtx{A}^{T}\mtx{\Gamma}^{-1}\mtx{A}+\hat{C}^{-1}, \\
		C^{-1}m&=\mtx{A}^{T}\mtx{\Gamma}^{-1}\vct{y} + \hat{C}^{-1}\hat{m}.
	\end{align*}
Therefore equations \eqref{eq: postmean} and \eqref{eq: postvar} follow.
\end{proof}

Equation \eqref{eq: ju2} shows that the posterior mean $m$ minimizes ${\mathsf{J}}(u)$ given by \eqref{eq:ju}. This demonstrates that the posterior mean is found by compromising between maximizing the likelihood (by making small the \emph{loss} term $\frac{1}{2}|\vct{y}-\mtx{A}\vct{u}|_{\mtx{\Gamma}}^2$) and minimizing deviations from the prior mean (by making small the \emph{regularization} term $\frac{1}{2}|\vct{u} -\hat{m}|_{\hat{C}}^2$). The relative importance given to both objectives is determined by the relative size of the prior covariance $\hat{C}$ and the noise covariance $\Gamma.$
An important feature of the linear-Gaussian setting is that the posterior covariance $C$ does not depend on the data $y;$ this is not true in general.

As we saw in the previous chapter, the posterior mean estimator and the MAP estimator are typically different. However, in the linear-Gaussian setting the posterior is Gaussian, and therefore both estimators agree.

\begin{corollary}[Characterization of  Bayes Estimators]
The posterior mean and MAP estimators under Assumptions \ref{a:sz1} agree, and are given by $u_{\mbox {\tiny{\rm MAP}}} = u_{\mbox {\tiny{\rm PM}}} = m$ defined in equation \eqref{eq: postmean}.
\end{corollary}

\begin{example}	\label{exp: map}
	Let $\mtx{\Gamma}=\gamma^2\Id$, $\hat{C}=\sigma^2\Id,$ $\hat{m} =0,$ and
set $\lambda=\frac{\sigma^2}{\gamma^2}$. Then
	$${\mathsf{J}}_\lambda(\vct{u})\triangleq\gamma^2{\mathsf{J}}(\vct{u})=\frac{1}{2}|\vct{y}-\mtx{A}\vct{u}|^2+\frac{\lambda}{2}|\vct{u}|^2.$$
Since $m$ minimizes ${\mathsf{J}}_\lambda(\cdot)$ it follows that
	\begin{align}
		(\mtx{A}^{T}\mtx{A}+\lambda\Id)m=\mtx{A}^{T}\vct{y}.
		\label{eq: expmean}
	\end{align}
\end{example}

Example \ref{exp: map} provides a link between Bayesian inversion and
optimization approaches to inversion: ${\mathsf{J}}_\lambda(\vct{u})$ can be seen as
the objective functional in a linear regression model with a regularizer
$\frac{\lambda}{2}|\vct{u}|^2$, as used in ridge regression. Equation \eqref{eq: expmean} for $m$ is exactly the normal equation
with regularizer in the least square problem.
In fact, in the general case, equation \eqref{eq: postmean} can be also viewed as a generalized normal equation. This point of view helps us
understand the structure of Bayesian regularization by linking it to the deep understanding of optimization approaches to inverse problems. A more extensive account of the optimization perspective and its interplay with Bayesian formulations will be given in the following chapter.

 \subsection{Small Noise Limit of the Posterior Distribution}
 In this section we study the small observational noise limit of the posterior distribution in the linear-Gaussian setting. While most of the ideas can be extended beyond this setting, the explicit calculations that the linear-Gaussian setting allows for provide helpful intuition. Throughout this section we assume the following.

 \begin{assumption}
\label{a:sz2}
In addition to the linear-Gaussian setting described in Assumption \ref{a:sz1},
$\vct{\eta}:=\gamma \vct{\eta}_0$ where $\vct{\eta}_0 \sim \mathcal{N}(\zerovct,\mtx{\Gamma}_0)$ so that $\Gamma=\gamma^2\Gamma_0.$
\end{assumption}

Note that substituting $\Gamma = \gamma^2 \Gamma_0$  into \eqref{eq: postmean} and \eqref{eq: postvar} we obtain that
 \begin{align}
m =& (A^{T} \Gamma_0^{-1} A + \gamma^2 \hat{C}^{-1})^{-1} (A^{T}\Gamma_0^{-1}y +\gamma^2 \hat{C}^{-1} \hat{m}), \label{eq:meansmallnoise}\\
C =&  \gamma^2 (A^{T} \Gamma_0^{-1} A + \gamma^2 \hat{C}^{-1})^{-1}. \label{eq:covsmallnoise}
 \end{align}

In the next three subsections we study the small noise limiting behavior as $\gamma \to 0$ of the posterior mean and covariance in the overdetermined, determined, and underdetermined regimes.  We recall that $\Rightarrow$ denotes weak convergence of probability measures. We will use repeatedly that weak convergence of Gaussian distributions is equivalent to the convergence of their means and covariances. In particular, the weak limit of a sequence of Gaussians with means converging to $m^+$ and covariance matrices converging to zero is a Dirac mass $\delta_{m^+}.$

 \subsubsection{Overdetermined Case}
 We start with the overdetermined case $d<k$.
\begin{theorem}[Small Noise Limit of Posterior Distribution - Overdetermined] \label{lec2:thm1}
Suppose that Assumption \ref{a:sz2} holds, that ${\rm Null}(A)= {0}$ and that $d<k.$
Then in the limit $\gamma^2 \to 0,$
$$\pi^y \Rightarrow \delta_{m^+},$$
where $m^+$ is the solution of
the least-squares problem
\begin{equation}\label{lec1:thm1_1}
m^+  =\arg\min_{u \in \mathbbm{R}^\Du} |\Gamma_0^{-1/2} (y- Au)   |^2.
\end{equation}
\end{theorem}

\begin{proof}

Since Null$(A)= {0}$ and $\Gamma_0$ is invertible
we deduce that there is $\alpha>0$ such that, for all $u \in \mathbbm{R}^\Du,$
 \[
  \langle u, A^{T} \Gamma^{-1}_0 Au \rangle = | \Gamma_0^{-1/2} Au |^2 \geq \alpha |u|^2.
 \]
 Thus $A^{T} \Gamma_0^{-1} A$ is positive definite and invertible. It follows that as $\gamma \to 0$, the posterior covariance converges to the zero matrix, $C \to 0,$ and the posterior mean satisfies the limit
 \[
 m \to  m^{*} =   (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1} y.
 \]
This proves the weak convergence of $\pi^y$ to $\delta_{m^{*}}$. It remains to characterize $m^{*}$.  Since Null$(A)= {0}$, the minimizers of
 \[
 \mathsf{L}(u) :=\frac{1}{2} | \Gamma_0^{-1/2}(y - Au)|^2
  \]

  are unique and satisfy the normal equations $A^{T}\Gamma_0^{-1}Au = A^{T} \Gamma_0^{-1}y$. Hence $m^{*}$ solves the desired least-squares problem and coincides with $m^+$ given in \eqref{lec1:thm1_1}.
 \end{proof}

\begin{remark}
In the overdetermined case where $A^{T}\Gamma_0 A$ is invertible, the small observational noise limit leads to a posterior which is a Dirac, centered at the solution of the least-square problem \eqref{lec1:thm1_1}.   Therefore, in this limit the prior plays no role in the Bayesian inference.
\end{remark}

\begin{theorem}[Posterior Consistency]
Suppose that the assumptions of Theorem \ref{lec1:thm2_1} hold and that the data satisfies
\begin{equation} \label{eq:datapostcons}
y=Au^\dagger+\gamma \eta_0^\dagger, \quad\quad {\rm for \,\, fixed} \quad u^\dagger \in \mathbbm{R}^\Du, \eta_0^\dagger \in \mathbbm{R}^\Dy.
\end{equation}
Then, for any sequence $M(\gamma)\rightarrow\infty$ as $\gamma\rightarrow0$,
	\begin{align}
		\operatorname{\mathbbm{P}} ^{\pi^y} \bigl\{|\vct{u}-\vct{u}^\dagger|^2>M(\gamma)\gamma^2\bigr\}\rightarrow0,
		\label{eq: maincovergence}
	\end{align}
	where $\operatorname{\mathbbm{P}}^{\pi^y}$ denotes probability under the posterior distribution.
	\end{theorem}

\begin{remark}
	For any $\varepsilon>0$, set $M(\gamma)=\frac{\varepsilon^2}{\gamma^2}$ in Theorem \ref{thm: maintheorem} to obtain
	$$\operatorname{\mathbbm{P}}^{\pi_y} \bigl\{|\vct{u}-\vct{u}^\dagger|>\varepsilon \bigr\}\rightarrow0.$$
\end{remark}

\begin{proof}(Theorem \ref{lec2:thm1})
Throughout this proof we let $c$ be a constant independent of $\gamma$ that may change from line to line, and we denote by $\operatorname{\mathbb{E}}$ expectation with respect to the posterior distribution, which is Gaussian with mean $m$ and covariance $C$ given by equations \eqref{eq:meansmallnoise} and \eqref{eq:covsmallnoise}. Denote
$$m^{*} =   (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1} y$$
as in the proof of the previous theorem.
We have that
\begin{equation}
\operatorname{\mathbb{E}}|u-u^\dagger|^2 \le c\Bigl( \operatorname{\mathbb{E}}|u-m|^2 + |m - m^{*}|^2 + |m^{*}-u^\dagger|^2  \Bigr).
\end{equation}
We now bound each of the three terms in the right-hand side.

For the first one,
\begin{align*}
\operatorname{\mathbb{E}}|u-m|^2 &= \operatorname{\mathbb{E}}\bigl[(u-m)^{T} (u-m)  \bigr]  = \operatorname{\mathbb{E}}\bigl[ {\rm Tr}[(u-m)\otimes(u-m)]   \bigr] \\
 &={\rm Tr} \operatorname{\mathbb{E}}\bigl[ (u-m)\otimes(u-m)   \bigr]  \\
&=  {\rm Tr}(C) \le \gamma^2 {\rm Tr}(A^{T} \Gamma_0^{-1} A).
\end{align*}
For the second term, note that
\begin{align*}
(A^{T} \Gamma_0^{-1} A) m^{*} &=  A^{T}\Gamma_0^{-1}y, \\
(A^{T} \Gamma_0^{-1} A + \gamma^2 \hat{C}^{-1}) m &= A^{T}\Gamma_0^{-1}y +\gamma^2 \hat{C}^{-1} \hat{m}.
\end{align*}
Therefore
\[ m-m^{*} = \gamma^2 (A^{T}\Gamma_0 A)^{-1} (\hat{C}^{-1} \hat{m}   -\hat{C}^{-1}m).\]
Since $m$ converges it is bounded, and so there is $c>0$ such that
\[ |m-m^{*}|^2 \le c \gamma^4. \]
Finally, for the third term we write
\begin{align*}
m^{*} &=   (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1} Au^\dagger  + \gamma (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1}  \eta_0^{\dagger} \\
 &= u^\dagger  + \gamma (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1}  \eta_0^{\dagger},
\end{align*}
which gives
\[ |m^{*} - u^\dagger|^2 \le c \gamma^2. \]
Using Markov's inequality and the three bounds above,
 $$\operatorname{\mathbbm{P}}^{\pi^y} \bigl\{|\vct{u}-\vct{u}^\dagger|^2>M(\gamma)\gamma^2 \bigr\}\leq\frac{\operatorname{\mathbb{E}}(|\vct{u}-\vct{u}^\dagger|^2)}{M(\gamma)\gamma^2}\leq\frac{c}{M(\gamma)}\rightarrow0,~{\rm as}~\gamma\rightarrow0.$$
\end{proof}

\subsubsection{Determined Case}
As a byproduct of the proof of Theorem \ref{lec2:thm1}, we can determine the limiting behavior of $\pi^y$ in the boundary case $d = k$.
\begin{theorem}[Small Noise Limit of Posterior Distribution - Determined]\label{lec1:thm1.5}
Suppose that Assumption \ref{a:sz2} holds, ${\rm Null}(A)= 0,$ and $d=k$. Then in the small noise limit $\gamma^2 \to 0$, $$\pi^y \Rightarrow \delta_{A^{-1 } y}.$$
\end{theorem}
\begin{proof}
In the proof of Theorem \ref{lec2:thm1}, the assumption $d < k$ is used only in that $A$ is not a square matrix and thus $A, A^{T}$ are not invertible. Denote by $(m, C)$ the mean and variance of the posterior $u|y$. Using the same argument, we have $C \to 0$ and
\[
m \to m^{*} =  (A^{T}\Gamma_0^{-1}A)^{-1} A^{T}\Gamma_0^{-1} y.
\]
Using that $A, A^{T}$ are square invertible matrices we obtain
\[
m^{*} = (A^{-1} \Gamma_0(A^{T})^{-1} ) A^{T} \Gamma_0^{-1 } y = A^{-1 }y.
\]
Therefore, $\pi^y(u) \Rightarrow \delta_{m^{*}} = \delta_{A^{-1}y}$.
\end{proof}

Note that here, as in the overdetermined case, the prior plays no role in the small noise limit. Moreover, it can be shown as above that posterior consistency holds. The proof is identical and therefore omitted.

\begin{theorem}[Posterior Consistency]
	Suppose that the assumptions of Theorem \ref{lec1:thm1.5} hold, and that the data satisfies
\begin{equation} \label{eq:datapostcons}
y=Au^\dagger+\gamma \eta_0^\dagger, \quad\quad {\rm for \,\, fixed} \quad u^\dagger, \eta_0^\dagger \in \mathbbm{R}^\Du.
\end{equation}
	Then for any sequence $M(\gamma)\rightarrow\infty$ as $\gamma\rightarrow0$,
	\begin{align}
		\operatorname{\mathbbm{P}} ^{\pi^y} \bigl\{|\vct{u}-\vct{u}^\dagger|^2>M(\gamma)\gamma^2 \bigr\}\rightarrow0.
		\label{eq: maincovergence}
	\end{align}
	\label{thm: maintheorem}
\end{theorem}

 \subsubsection{Underdetermined Case}
Finally we consider the underdetermined case $d > k$. We assume that $A \in \mathbbm{R}^{k \times d}$ with ${\rm Rank}(A) = k$ and write
 \begin{equation}\label{lec1:thm2_0}
 A = (A_0\  0) Q^{T} =(A_0 \ 0 )(Q_1\  Q_2)^{T}= A_0Q_1^{T},
 \end{equation}
 with $A_0 \in \mathbbm{R}^{k \times k} $ an invertible matrix, $Q = (Q_1 \ Q_2) \in \mathbbm{R}^{d \times d}$ an orthogonal matrix so that $Q^{T} Q= I$, $Q_1 \in \mathbbm{R}^{d \times k}, Q_2 \in \mathbbm{R}^{d \times (d-k)}$. We have the following result:
  \begin{theorem}[Small Noise Limit of Posterior Distribution - Underdetermined]\label{lec1:thm2}
Suppose that Assumption \ref{a:sz2} holds, that ${\rm Rank}(A) = k,$ and $d>k$. In the small noise limit $\gamma^2\to 0$,
$$\pi^y \Rightarrow \mathcal{N}(m^+, C^+),$$ where
\[
\begin{aligned}
m^+ &=\hat{C} Q_1 (Q_1^{T} \hat{C} Q_1)^{-1}  A_0^{-1}y + Q_2(Q_2^{T}\hat{C}^{-1}Q_2)^{-1}Q_2^{T} \hat{C}^{-1}\hat{m}, \\
C^+ &=Q_2(Q_2^{T}\hat{C}^{-1}Q_2)^{-1} Q_2^{T}. \\
\end{aligned}
\]
  \end{theorem}

Since ${\rm Rank}(C^+) = {\rm Rank}(Q_2) =d-k<d$ this theorem
demonstrates that, in the small observational noise limit, the posterior
retains uncertainty in a subspace of dimension $d-k$,
and has no uncertainty in a subspace of dimension $k$. As a consequence there is no posterior consistency in the underdetermined case.

\begin{example}  To help understand the result in Theorem \ref{lec1:thm2},
we consider a simple explicit example. Assume that $A = (A_0 \ 0) \in \mathbbm{R}^{k \times d}, \Gamma = \gamma^2 \Gamma_0 =\gamma^2  I_k, \hat{C} =I_d$, $\hat{m} = 0$. Let $u = \left(
\begin{array}{c}
u_1 \\
u_2 \\
\end{array}
\right) \sim \mathcal{N}(0, I_d), u_1 \in \mathbbm{R}^\Dy, u_2 \in \mathbbm{R}^{d - k}$.
The data then satisfies
\[
y = Au + \eta = A_0 u_1 +\eta, \  \eta \sim \mathcal{N}(0, \gamma^2 I_k).
\]
The posterior $u| y$ is $\pi_{\gamma}^y(u) = \frac{1}{Z_{\gamma}} \exp(-  {\mathsf{J}}_{\gamma}(u) )$, where
\begin{align}
 {\mathsf{J}}_{\gamma}(u) &= \frac{ 1 }{2\gamma^2} | y - A_0 u_1 |^2 + \frac{1}{2} | u |^2  \notag \\
 &=\left(  \frac{ 1 }{2\gamma^2} | y - A_0 u_1 |^2 + \frac{1}{2} | u_1 |^2\right)
 +\frac{1}{2} |u_2|^2.  \label{lec1:thm2_1}
\end{align}
It is clear that
\[
\pi_{\gamma}^{y}(u_1) \Rightarrow \delta_{A_0^{-1} y} (u_1).
\]
Once $u_1$ is fixed as $A_0^{-1}y$, the first term in \eqref{lec1:thm2_1} is a constant $\frac{1}{2} | A_0^{-1}y |^2$. Since $u_1$ and $u_2$ are independent we can derive, formally, the limiting posterior as follows
\[
 \pi_{\gamma}^y(u) \Rightarrow \delta_{A_0^{-1} y} (u_1) \otimes \frac{1}{Z} \exp(- \frac{1}{2} |u_2|^2) = \delta_{A_0^{-1} y} (u_1) \otimes \mathcal{N}(0, I_{d-k}),
\]
where $Z = \int_{\mathbbm{R}^{d-k}} \exp(-\frac{1}{2}|u_2|^2 ) du_2$. In fact, this is exactly the limiting posterior measure given in Theorem \ref{lec1:thm2}.
\end{example}

To prove Theorem \ref{lec1:thm2}, we use the following decomposition of the identity $I_d.$
\begin{lemma}
Let $\hat{C} \in \mathbbm{R}^{d \times d}$ be invertible and $Q = (Q_1 \ Q_2)$ be an orthonormal matrix with $Q_1 \in \mathbbm{R}^{d \times k}, Q_2 \in \mathbbm{R}^{d \times (d-k)}$. We have the following decomposition of $I_d$
\begin{equation}\label{lec1:lem}
I_d =  \hat{C} Q_1 (Q_1^{T} \hat{C} Q_1)^{-1}   Q_1^{T} +Q_2(Q_2^{T}\hat{C}^{-1}Q_2)^{-1} Q_2^{T}\hat{C}^{-1}.
\end{equation}
\end{lemma}
\begin{proof}
Denote by $R$ the right-hand side of \eqref{lec1:lem}. Since $Q$ is orthonormal, we have $Q_1^{T} Q_2 = 0, Q_2^{T}Q_1 = 0$ and thus
\[
Q_1^{T}(R- I) = 0, \quad Q^{T}_2\hat{C}^{-1}  (R-I) = 0 .
\]
If $B :=(Q_1 \ \hat{C}^{-1} Q_2)$ is full rank, the above identities imply that $B^{T}(R-I) = 0$ and thus $R= I$. Note that
\[
Q^{T} B = \left(
\begin{array}{c}
Q_1^{T} \\
Q_2^{T} \\
\end{array}
\right)
( Q_1 \ \hat{C}^{-1} Q_2)  =
 \left(
\begin{array}{cc}
I_k  & Q_1^{T}\hat{C}^{-1} Q_2 \\
0 &  Q_2^{T}\hat{C}^{-1} Q_2 \\
\end{array}
\right).
\]
Since the last matrix is invertible, $B$ is invertible and the proof is complete.
\end{proof}

\begin{proof}[Proof of Theorem \ref{lec1:thm2}]
Using \eqref{lec1:lem}, we can decompose $u$ as follows
\[
\begin{aligned}
u &=\underbrace{  \hat{C} Q_1 (Q_1^{T} \hat{C} Q_1)^{-1} }_{S} \underbrace{Q_1^{T} u}_{u_1} + \underbrace{Q_2(Q_2^{T}\hat{C}^{-1}Q_2)^{-1}}_{T} \underbrace{Q_2^{T}\hat{C}^{-1}u}_{u_2}  = Su_1 + Tu_2 . \\
\end{aligned}
\]
Here $u_1$ and $u_2$ are Gaussian with $u_2 \sim \mathcal{N}(Q_2^{T}\hat{C}^{-1} \hat{m} , Q_2^{T}\hat{C}^{-1} Q_2  )$. The identity
\[
{\rm Cov}(u_1,u_2) = Q_1^{T} {\rm Cov}(u,u) \hat{C}^{-1} Q_2 = Q_1^{T} Q_2 = 0
\]
shows that $u_1$ and $u_2$ are independent, written $u_1 \perp u_2$. From \eqref{lec1:thm2_0}, we have
\begin{equation}\label{lec1:thm2_2}
y = Au + \eta  = A_0 Q_1^{T} u+ \eta= A_0 u_1 + \eta.
\end{equation}
Since $u\perp \eta$ and $u_1 \perp u_2$, we have that $u_2 \perp y, u_1$. We apply conditional probability to yield
\[
\pi^y(u_1, u_2)  :=\operatorname{\mathbbm{P}}(u_1, u_2 | y) =\operatorname{\mathbbm{P}}(u_2) \operatorname{\mathbbm{P}}(u_1 | y)  .
\]
Equation \eqref{lec1:thm2_2} suggests that $(y, u_1,\eta)$ with $A_0 \in \mathbbm{R}^{k \times k}$ invertible is Gaussian distributed and its posterior is exactly $\operatorname{\mathbbm{P}}(u_1 | y)$. Theorem \ref{lec1:thm1.5} shows that $\operatorname{\mathbbm{P}}(u_1 | y ) \Rightarrow \delta_{A_0^{-1}y} (u_1)$ as the noise vanishes, that is, as  $\gamma^2 \to 0$. Note that $u_2 \perp u_1$ and $u_2 \perp y$. The limiting posterior measure  $(u_1,u_2) | y$ is
\begin{equation}\label{lec1:thm2_3}
\pi^y(u_1, u_2) \Rightarrow \operatorname{\mathbbm{P}}(u_2) \otimes \delta_{A_0^{-1}y}(u_1)
\end{equation}
as $\gamma^2 \to 0$. Recall $u = Su_1 + Tu_2$ and $u_2 \sim \mathcal{N}(Q_2^{T}\hat{C}^{-1} \hat{m} , Q_2^{T}\hat{C}^{-1} Q_2  )$. The mean and variance of the limiting posterior measure $u|y$ is
\[
\begin{aligned}
m^+ &= \operatorname{\mathbb{E}}( Su_1 + Tu_2 | y  ) =  SA_0^{-1}y +T\operatorname{\mathbb{E}}( u_2 ) =SA_0^{-1}y + TQ_2^{T}\hat{C}^{-1}\hat{m}, \\
C^+ & = {\rm Var}(Su_1 + Tu_2 | y) = {\rm Var}( Tu_2) = T   Q_2^{T}\hat{C}^{-1} Q_2 T^{T}  =Q_2(Q_2^{T}\hat{C}^{-1}Q_2)^{-1} Q_2^{T}. \\
\end{aligned}
\]
We have thus completed the proof.
\end{proof}

\begin{remark}
Equation \eqref{lec1:thm2_3} shows that in the limit of zero  observational noise, the uncertainty is only in the variable $u_2$. Since ${\rm Span}(T) = {\rm Span}(Q_2)$ and $u = SA_0^{-1} y + Tu_2$, the uncertainty we observed is in ${\rm Span}(Q_2)$. The prior plays a role in the posterior measure,
in the limit of zero observational noise, but only in the
variables $u_2.$
\end{remark}

\subsection{Discussion and Bibliography}

The linear Gaussian setting plays, for several reasons, a central role in the study of
inverse problems. One is that it allows explicit
solutions which can be used to give insight into the subject area more
generally. The second is that in the large data limit many Bayesian
posteriors are approximately Gaussian. The paper \cite{franklin1970well},
which is in the linear-Gaussian setting, plays an important role in the
history of Bayesian inversion as it was arguably the first to formulate
Bayesian inversion in function space.

We have also employed the Gaussian setting to present a basic form of posterior consistency. For a treatment in infinite dimensions see
\cite{knapik2011bayesian,agapiou2013posterior,nickl2017bernstein}.
For the consistency problem in the classical statistical setting,
see the books \cite{gine2015mathematical,van1998asymptotic}.
The book \cite{van1998asymptotic} also contains definition and properties of
convergence in probability as used here.
For the non-statistical approach to inverse
problems, and consistency results, see \cite{engl1996regularization} and
the references therein.

\section{\Large{\sffamily{Optimal Particle Filter }}}
\label{sec:Lecture12}

This chapter is devoted to the optimal particle filter. Like the
bootstrap filter from the previous chapter, the optimal particle filter
approximates the filtering distribution by a sum of Dirac measures.
The setting will initially be the same
(nonlinear stochastic dynamics and nonlinear observations), but we will see that
the optimal particle filter cannot be implemented, in general, in the fully nonlinear case. For this reason, we will specify to the case of  linear observation operator, where the optimal particle filter can be  implemented in a straightforward fashion and may be characterized as a set of interacting
3DVAR filters. We conclude this chapter by
summarizing and comparing all of the filtering methods introduced in this and
preceding chapters, highlighting the setting in which they may be practically applied.

\subsection{Introduction}
The bootstrap particle filter from the previous chapter is based on
approximating the two components of a specific factorization of the
 filtering step. The factorization is
\begin{align*}
                                \operatorname{\mathbbm{P}}(v_{j+1}|Y_{j}) &= {\mathcal{P}} \operatorname{\mathbbm{P}}(v_{j}|Y_{j}),\\
                                \operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1}) &= {\mathcal{A}}_j \operatorname{\mathbbm{P}}(v_{j+1}|Y_{j}).
                        \end{align*}
This gives the factorization
		\[		\operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1})={\mathcal{A}}_j {\mathcal{P}} \operatorname{\mathbbm{P}}(v_{j}|Y_{j}),  \]
which is the basis for the bootstrap particle filter.
It is natural to ask if there are other factorizations of the
filtering update and whether they might lead to improved
particle filters. In this lecture we derive the Optimal Particle Filter (OPF)
which does just that.
We demonstrate a connection with 3DVAR, and we discuss the sense in which the
OPF has desirable properties in comparison with the BPF.
As usual we denote by $\pi_j$ the filtering distribution $\operatorname{\mathbbm{P}}(v_{j}|Y_{j}).$

\subsection{The Bootstrap and Optimal Particle Filters Compared}
The fundamental filtering problem that we are interested in is
determination of $\operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1})$ from $\operatorname{\mathbbm{P}}(v_{j}|Y_{j}).$
In the BPF, we are approximating the following manipulation:
\begin{align*}
	\operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1}) & = \operatorname{\mathbbm{P}}(v_{j+1}|y_{j+1}, Y_j)\\
	& = \frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1}, Y_j)\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)}{\operatorname{\mathbbm{P}}(y_{j+1}|Y_j)}\\
	& =\frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1}, Y_j)}{\operatorname{\mathbbm{P}}(y_{j+1}|Y_j)}     \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j,Y_j)\operatorname{\mathbbm{P}}(v_j|Y_j)dv_j\\
	&=\frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})}{\operatorname{\mathbbm{P}}(y_{j+1}|Y_j)}       \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\operatorname{\mathbbm{P}}(v_j|Y_j)dv_j\\
	&= {\mathcal{A}}_j {\mathcal{P}} \operatorname{\mathbbm{P}}(v_j|Y_j).
\end{align*}
The Markov kernel ${\mathcal{P}} ^{BPF}$ acts on arbitrary density $\pi$ by
\begin{equation*}
	{\mathcal{P}} ^{BPF}\pi(v_{j+1}) = \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\pi(v_j)dv_j,
\end{equation*}
and ${\mathcal{A}}_j$ acts on an arbitrary density $\pi$ by application of Bayes theorem, taking into account the likelihood of the data
\begin{equation*}
	{\mathcal{A}}_j\pi(v_{j+1}) = \frac{1}{Z}\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\pi(v_{j+1}),
\end{equation*}
with $Z$ normalization to a probability density.
The above manipulations are summarized by the relationship
\begin{equation}
\label{eq:pf}
\pi_{j+1}={\mathcal{A}}_j {\mathcal{P}}\pi_j.
\end{equation}
Note that in this factorization we apply a Markov kernel and then
Bayes theorem.  In contrast, to derive the OPF we perform the following manipulation:
\begin{align*}
	\operatorname{\mathbbm{P}}(v_{j+1}|Y_{j+1}) & = \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1},v_j|Y_{j+1})  \, dv_j \\
	& = \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j,Y_{j+1})\operatorname{\mathbbm{P}}(v_j|Y_{j+1})dv_j  \, \\
& = \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1},Y_j)\operatorname{\mathbbm{P}}(v_j|y_{j+1},Y_j) \, dv_j\\
	& = \int_{\mathbbm{R}^\Du} \operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1})\operatorname{\mathbbm{P}}(v_j|y_{j+1},Y_{j}) \, dv_j\\
	& = \int_{\mathbbm{R}^\Du}  \operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1})  \frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_j,Y_{j}) }{\operatorname{\mathbbm{P}}(y_{j+1}|Y_{j})}  \operatorname{\mathbbm{P}}(v_j|Y_{j}) \,dv_j\\
& = \int_{\mathbbm{R}^\Du} \operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1})  \frac{  \operatorname{\mathbbm{P}}(y_{j+1}|v_j) }{\operatorname{\mathbbm{P}}(y_{j+1}|Y_{j})}  \operatorname{\mathbbm{P}}(v_j|Y_{j}) \, dv_j \\
	& = {\mathcal{P}}^{OPF}_j {\mathcal{A}}_j^{OPF}\operatorname{\mathbbm{P}}(v_j|Y_{j}),
\end{align*}
with Markov kernel for particle update
\begin{equation*}
	{\mathcal{P}}_j^{OPF}\pi(v_{j+1}) = \int_{\mathbbm{R}^\Du}\operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1})\pi(v_j)dv_j
\end{equation*}
and application of Bayes theorem to include the likelihood
\begin{equation*}
	{\mathcal{A}}_j^{OPF}\pi(v_j) = \frac{1}{Z}\operatorname{\mathbbm{P}}(y_{j+1}|v_j)\pi(v_j).
\end{equation*}
Thus we have
\begin{equation}
\label{eq:opf}
\pi_{j+1}={\mathcal{P}}_j^{OPF} {\mathcal{A}}_j^{OPF}\pi_j.
\end{equation}
Note that in the factorization given by OPF we apply Bayes theorem and then a Markov kernel, the opposite order to the BPF. Moreover the propagation mechanism is different--it sees the data through
the Markov kernel ${\mathcal{P}}_j^{OPF}$-- and hence the weighting
of the particles is also different:  the OPF weights are proportional to the likelihood $\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})$ and the BPF weights are proportional to  $\operatorname{\mathbbm{P}}(y_{j+1}|v_{j})$ which may be, in general, not available in closed form.  We will see that the particle updates use a 3DVAR procedure.
In the BPF, the evolution of the particles and the observation of the data are kept separate from each other -- the Markov kernel ${\mathcal{P}}$ depends only on the dynamics and not the observed data and is thus independent of $j$.

\subsection{Implementation of Optimal Particle Filter: Linear Observation Setting}
In general it is not possible to implement the OPF in the fully nonlinear setting because of two computational bottlenecks:
\begin{itemize}
\item There may not be a closed formula for evaluating the likelihood $\operatorname{\mathbbm{P}}(y_{j+1}|v_{j}),$ making unfeasible the computation of the particle weights.
\item It may not be possible to sample from the Markov kernel $\operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1}),$ making unfeasible the propagation of particles.
\end{itemize}
However, when the observation function $h(\cdot)$ is linear, i.e. $h(\cdot) = H\cdot$ for some $H \in R^{k \times d}$ both bottlenecks are overcome.
We thus consider the following setting, which arises in many applications:
\begin{align*}
	v_{j+1} & = \Psi(v_j) + \xi_j, && \xi_j \sim \mathcal{N}(0,\Sigma)\text{ i.i.d.}, \\
	y_{j+1} & = Hv_{j+1} +\eta_{j+1}, && \eta_{j+1} \sim \mathcal{N}(0,\Gamma)\text{ i.i.d.},
\end{align*}
with $v_0 \sim \mathcal{N}(m_0,C_0)$ and $v_0, \{\xi_j\}, \{\eta_j\}$ independent. We will now show that in this linear observation both bottlenecks are overcome.
First, note that combining the dynamics and data models we may write
\begin{equation*}
	y_{j+1} = H\Psi(v_j) + H\xi_j + \eta_{j+1},
\end{equation*}
which shows that the conditional distribution for $y_{j+1}$ given $v_j$ is
\begin{equation*}
	\operatorname{\mathbbm{P}}(y_{j+1} | v_j) = \mathcal{N}(H\Psi(v_j),S),
\end{equation*}
where $S= H\Sigma H^T + \Gamma.$
We will use this formula to compute the weights, thus overcoming the first computational bottleneck.

We now show that under the linear observation assumption ${\mathcal{P}}^{OPF}_j$ is a Gaussian kernel, and hence the second computational bottleneck is overcome too.
We have
\begin{align*}
	\operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1}) &\propto \operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1}, v_j)\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\\
	&= \operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v_j)\\
	&\propto \exp \left(-\frac{1}{2}|y_{j+1}-Hv_{j+1}|^2_\Gamma -\frac{1}{2}|v_{j+1}-\Psi(v_{j})|^2_\Sigma \right)\\
	&=\exp\bigl(-{\mathsf{J}}_{\textsc{opt}}(v_{j+1})\bigr).
\end{align*}
This is a Gaussian distribution for $v_{j+1}$ as ${\mathsf{J}}_{\textsc{opt}}(v_{j+1})$ is quadratic with respect to $v_{j+1}$.\footnote{${\mathsf{J}}_{\textsc{opt}}$ is identical to
${\mathsf{J}}$ on the right-hand side of Table \ref{3DVAR:t1}, with $\hat{C}$ replaced by $\Sigma.$} Consequently, we can compute the mean $m_{j+1}$ and covariance $C$
(which, note, is independent of $j$) of this Gaussian by matching the mean
and quadratic terms in the relevant quadratic forms:
\begin{align*}
	C^{-1} &= H^T\Gamma^{-1}H + \Sigma^{-1},\\
	C^{-1}m_{j+1} &= \Sigma^{-1}\Psi(v_j) + H^T\Gamma^{-1}y_{j+1}.
\end{align*}
Then $\operatorname{\mathbbm{P}}(v_{j+1}|y_{j+1}, v_j)=\mathcal{N}(m_{j+1},C).$
This is hence a special case of 3DVAR in which the
analysis covariance is fixed at $C$; note that when we derived 3DVAR we
fixed the predictive covariance $\hat{C}$ which, here, is fixed at $\Sigma.$
As with the Kalman filter, and with 3DVAR, it is possible to implement
the prediction step through the following mean and covariance formulae
which avoid inversion in state space, and require inversion only in data
space:
\begin{align*}
	m_{j+1} & = (I-KH)\Psi(v_j) + Ky_{j+1},\\
	C & = (I-KH)\Sigma,\\
	K & = \Sigma H^T S^{-1},\\
	S & = H \Sigma H^T + \Gamma.
\end{align*}
Furthermore, as for 3DVAR, the inversion of $S$ need only be performed once
in a pre-processing step before the algorithm is run.
Since the expression for $\operatorname{\mathbbm{P}}(v_{j+1}|v_j,y_{j+1})$ is Gaussian we now
have the ability to sample directly from ${\mathcal{P}}_j^{OPF}$.
The OPF is thus given by the following update algorithm for approximations
$\pi_j^N \approx \pi_j$ in which we generalize the notational conventions
used in the previous chapter to formulate particle filters as random
dynamical systems:

\FloatBarrier
\begin{algorithm}
\caption{Algorithm for the Optimal Particle Filter with linear observation map}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Initial distribution $\operatorname{\mathbbm{P}}(v_0) = \pi_0$, observations $Y_{J}$, number of particles $N.$
\STATE {\bf Initial Sampling}: Draw $N$ particles $v_0^{(n)} \sim \pi_0$ so that $\pi_0^N=S^N\pi_0.$
\vspace{0.04in}
\STATE{\bf Subsequent Sampling} For $j = 0,1,\dots,J-1$, perform:
\begin{enumerate}
\item Set $\hat{v}_{j+1}^{(n)} = (I-KH)\Psi(v_{j}^{(n)}) + Ky_{j+1} + 		\zeta_{j+1}^{(n)}$ with $\zeta_{j+1}^{(n)}$ i.i.d.  $\mathcal{N}(0,C).$
\item Set $\bar{w}_{j+1}^{(n)} = \exp\left(-\frac{1}{2}|y_{j+1}-H\Psi(v_j^{(n)})|_S^2\right).$
\item Set $w_{j+1}^{(n)} = \bar{w}_{j+1}^{(n)} / \sum_{n=1}^N\bar{w}_{j+1}^{(n)}.$
\item Set $v_{j+1}^{(n)}=\sum_{m=1}^{N} {\mathbbm{1}}_{I_{j+1}^{(m)}}(r_{j+1}^{(n)})\hat{v}_{j+1}^{(m)}.$
\item Set $\pi_{j+1}^{N}(v_{j+1}) = \frac{1}{N}\sum_{n=1}^N \delta\left(v_{j+1}-{v}_{j+1}^{(n)}\right).$
\end{enumerate}
\vspace{0.04in}
\STATE {\bf Output}: $N$ particles $v_{J}^{(1)},v_{J}^{(2)}, \dots, v_{J}^{(N)}.$
\end{algorithmic}
\label{alg:opf}
\end{algorithm}
\FloatBarrier

It would be desirable to interpret this algorithm as an approximation of the
filter update \eqref{eq:opf} in the form
$$\pi_{j+1}^N =S^N {\mathcal{P}}_j^{OPF}{\mathcal{A}}_j^{OPF} \pi_j^N,  \quad \pi_j^0=S^N \pi_0.$$
However the order in which the resampling and the particle propagation
occurs implies that this is not possible. The following slight modification
of the OPF, however, may indeed be thought of as an approximation of
this form; we simply reorder the resampling and the propagation.
We refer to the resulting algorithm as the Gaussianized Optimal
Particle filter (GOPF).
We may write the resulting algorithm as follows:

\FloatBarrier
\begin{algorithm}
\caption{Algorithm for the Gaussianized Optimal Particle Filter}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Initial distribution $\operatorname{\mathbbm{P}}(v_0) = \pi_0$, observations $Y_{J}$, number of particles $N.$
\STATE {\bf Initial Sampling}: Draw $N$ particles $v_0^{(n)} \sim \pi_0$ so that $\pi_0^N=S^N\pi_0.$
\vspace{0.04in}
\STATE{\bf Subsequent Sampling} For $j = 0,1,\dots,J-1$, perform:
\begin{enumerate}
\item Set $\bar{w}_{j+1}^{(n)} = \exp\left(-\frac{1}{2}|y_{j+1}-H\Psi(v_j^{(n)})|_S^2\right).$
\item Set $w_{j+1}^{(n)} = \bar{w}_{j+1}^{(n)} / \sum_{n=1}^N\bar{w}_{j+1}^{(n)}.$
\item Set $\hat{v}_{j}^{(n)}=\sum_{m=1}^{N} {\mathbbm{1}}_{I_{j+1}^{(m)}}(r_{j+1}^{(n)}){v}_{j}^{(m)}.$
\item Set ${v}_{j+1}^{(n)} = (I-KH)\Psi(\hat{v}_{j}^{(n)}) + Ky_{j+1} +          \zeta_{j+1}^{(n)}$ with $\zeta_{j+1}^{(n)}$ i.i.d  $\mathcal{N}(0,C).$
\item Set $\pi_{j+1}^{N}(v_{j+1}) = \frac{1}{N}\sum_{n=1}^N \delta\left(v_{j+1}-{v}_{j+1}^{(n)}\right).$
\end{enumerate}
\vspace{0.04in}
\STATE {\bf Output}: $N$ particles $v_{J}^{(1)},v_{J}^{(2)}, \dots, v_{J}^{(N)}.$
\end{algorithmic}
\label{alg:opf}
\end{algorithm}
\FloatBarrier

\subsection{``Optimality'' of the Optimal Particle Filter}
Particle filter methods rely on approximating the distribution for the model by a swarm of point Dirac functions; it is clear that the distribution will not be well approximated by only a small number of particles in most cases. Consequently, a performance requirement for particle filter methods is that they do not lead to degeneracy of the particles. Resampling leads to degeneracy if
a small number of particles have all the weights. Conversely, non-degeneracy
may be promoted by ensuring that the weights $w_j^{(n)}$ are similar in magnitude, so that a small number of particles are not overly favoured during the resampling step. This condition can be formulated as a requirement that the variance of the weights be minimized; doing this results in the OPF.

To understand this perspective
we consider an arbitrary particle update kernel of the form
$\pi(v_{j+1}|v_j^{(n)},Y_{j+1})$
and we study the resulting particle filter without resampling.
It is then the case that the particle weights are updated according
to the formula
			\begin{equation*}
				\bar{w}^{(n)}_{j+1} = \bar{w}^{(n)}_j \frac{\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v^{(n)}_{j})}{\pi(v_{j+1}|v_j^{(n)},Y_{j+1})}.
			\end{equation*}

\begin{theorem}[Meaning of Optimality]
	The choice of $\operatorname{\mathbbm{P}}(v_{j+1}|v_j^{(n)}, y_{j+1})$ as the particle update kernel $\pi(v_{j+1}|v_j^{(n)},Y_{j+1})$ results in the minimal
variance of the weight $w^{(n)}_{j+1}$ with respect to all possible
choices of the particle
update kernel $\pi(v_{j+1}|v_j^{(n)},Y_{j+1}).$
\end{theorem}

\begin{proof}
We calculate the variance of the unnormalized weights (treated as random variables) $\bar{w}^{(n)}_{j+1}$ with respect to the transition density $\pi(v_{j+1}|v_j^{(n)},Y_{j+1})$ and obtain
\begin{align*}
	{\rm Var}_{\pi(v_{j+1}|v_j^{(n)},Y_{j+1})} &[\bar{w}^{(n)}_{j+1}]  = \int_{\mathbbm{R}^\Du} \left(\bar{w}^{(n)}_{j+1}\right)^2\pi(v_{j+1}|v_j^{(n)},Y_{j+1}) dv_{j+1}\\
	 &- \left[\int_{\mathbbm{R}^\Du} \bar{w}^{(n)}_{j+1}\pi(v_{j+1}|v_j^{(n)},Y_{j+1}) dv_{j+1} \right]^2\\
	& = \left(\bar{w}^{(n)}_{j}\right)^2\int_{\mathbbm{R}^\Du} \frac{\left(\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v^{(n)}_{j})\right)^2}{\pi(v_{j+1}|v_j^{(n)},Y_{j+1})} dv_{j+1}\\
	& \quad\quad\quad\quad\quad\quad\quad - \left(\bar{w}^{(n)}_{j}\right)^2\left[\int_{\mathbbm{R}^\Du} \operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v^{(n)}_{j}) dv_{j+1}\right]^2\\
	& = \left(\bar{w}^{(n)}_{j}\right)^2\left[\int_{\mathbbm{R}^\Du}\frac{\left(\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v^{(n)}_{j})\right)^2}{\pi(v_{j+1}|v_j^{(n)},Y_{j+1})} dv_{j+1} - \operatorname{\mathbbm{P}}(y_{j+1}|v^{(n)}_{j})^2\right].
\end{align*}
Choosing $\pi(v_{j+1}|v_j^{(n)},Y_{j+1}) = \operatorname{\mathbbm{P}}(v_{j+1}|v_j^{(n)}, y_{j+1})$, as in the OPF, we obtain
\begin{align*}
	{\rm Var}_{\operatorname{\mathbbm{P}}(v_{j+1}|v_j^{(n)}, Y_{j+1})}[\bar{w}^{(n)}_{j+1}]
	& = \left(\bar{w}^{(n)}_{j}\right)^2\left[\int_{\mathbbm{R}^\Du}\frac{\left(\operatorname{\mathbbm{P}}(y_{j+1}|v_{j+1})\operatorname{\mathbbm{P}}(v_{j+1}|v^{(n)}_{j})\right)^2}{\operatorname{\mathbbm{P}}(v_{j+1}|v_j^{(n)}, y_{j+1})} dv_{j+1} - \operatorname{\mathbbm{P}}(y_{j+1}|v^{(n)}_{j})^2\right]\\
	& = \left(\bar{w}^{(n)}_{j}\right)^2\left[\operatorname{\mathbbm{P}}(y_{j+1}|v^{(n)}_{j})^2 - \operatorname{\mathbbm{P}}(y_{j+1}|v^{(n)}_{j})^2\right]\\
	& = 0.
\end{align*}
\end{proof}

\begin{remark}
The optimal particle filter is optimal in the very precise sense of the
theorem. Note in particular that no optimality criterion is asserted by
this theorem with respect to iterating the particle updates, and in particular
when resampling is included. The nomenclature ``optimal'' should thus be
treated with caution.

\end{remark}

\subsection{Particle Filters for High Dimensions}
Particle filters often perform poorly for high-dimensional systems due to a collapse of the particle weights: only a few particles carry most of the weight; the optimal particle filter can ameliorate this issue because the proposal uses the data, meaning that particle predictions are more likely to be weighted highly. There have been attempts to formulate update steps that help to mitigate this weight collapse. Essentially, these methods aim to push the particles towards the region of high likelihood, such that all the particles will be representative of the distribution.
As will show in the next lecture, there are interesting particle based
methods which, whilst not statistically consistent, do perform well as
signal estimators. This perspective on particle filter type methods, namely
to use them for smart signal estimation rather than as estimators of the
filtering distribution, may become increasingly useful in high dimensional
systems.

\subsection{Filtering Overview}
\label{sec:overview}

Having introduced a range of filtering methods in this and the preceding
four chapters, it is helpful to summarize what these methods achieve,
in a comparitive fashion.

\subsubsection{Dynamical Model}
The stochastic dynamics model is defined by
\begin{equation*}
v_{j+1} = \Psi(v_j) + \xi_j, \quad j \ge 0,
\end{equation*}
where $\xi_j \sim \mathcal{N}(0,\Sigma)$ are independent and identically distributed random variables, also independent of the initial condition $v_0 \sim \mathcal{N}(m_0,C_0)$.
\FloatBarrier
\begin{figure}[htp]
  \begin{center}
    \includegraphics[width=0.5\textwidth]{figures/model2.png}
  \end{center}
  \caption{Prediction step.} \label{fig:model}
\end{figure}
\FloatBarrier
The filtering distribution of $v_j$ is denoted by
$\pi_j(v_{j}) = \operatorname{\mathbbm{P}}(v_j|Y_j).$
This is propagated by the stochastic dynamics model according to the
formula $\hat{\pi}_{j+1}={\mathcal{P}}\pi_j$ where ${\mathcal{P}}$ is defined in \eqref{eq:markov}.
This is shown schematically in Figure \ref{fig:model}. Note that ${\mathcal{P}}$ is independent of step $j$ because the Markov chain defined by the stochastic
dynamics model is time-homogeneous. In the absence of data the probability
distribution simply evolves through repeated application of ${\mathcal{P}}$.

\subsubsection{Data Model}
The data model is given by
\begin{equation*}
y_{j+1} = h(v_{j+1}) + \eta_{j+1},
\end{equation*}
where $\eta_{j+1} \sim {N}(0,\Gamma)$ are independent and identically distributed random variables, independent of both $v_0$ and the i.i.d. sequence
$\{\xi_j\}.$
Given data, we now view the prediction from the dynamical model
as a prior which we condition on the data. We write this prior at
time $j+1$ as $\hat{\pi}_{j+1}$, and prediction using the
stochastic dynamics model gives $\hat{\pi}_{j+1}={\mathcal{P}}\pi_j.$

Using the data to update this prior by Bayes theorem gives an improved
estimate of the distribution $\pi_{j+1}(v_{j+1})$ via the formula
\begin{equation*}
\pi_{j+1} = {\mathcal{A}}_j \widehat{\pi}_{j+1}
\end{equation*}
where ${\mathcal{A}}_j$ is given by \eqref{eq:bayes-again}.
This is shown schematically in Figure \ref{fig:data}. Note that ${\mathcal{A}}_j$ is nonlinear because $\widehat{\pi}_{j+1}$ appears in both the numerator and the denominator. It depends on $j$ because the data $y_{j+1}$ appears in the equation and this will change with each set of measurements.

\begin{figure}[h]
  \begin{center}
    \includegraphics[width=0.5\textwidth]{figures/data.png}
  \end{center}
  \caption{Update step.} \label{fig:data}
\end{figure}

\subsubsection{Interaction of the Dynamics and Data}

Recall the notation $Y_{j}=\{y_1, \dots, y_j\}$
for the collection of all data until time $j+1$.
The dynamics and data models described in the two previous subsections
can be combined as follows. Using the dynamics,
the probability distribution is propagated forward in time by the prediction step. The data is then used to update the probability distribution at that time in the analysis step.
The two can be combined to obtain
\begin{equation*}
{\pi}_{j+1} = {\mathcal{A}}_j {\mathcal{P}} \pi_j.
\end{equation*}
The combination of the two steps is shown
schematically in Figure \ref{fig:interaction}.

\begin{figure}[h]
  \begin{center}
    \includegraphics[width=0.8\textwidth]{figures/interaction.png}
  \end{center}
  \caption{Prediction and update step combined.} \label{fig:interaction}
\end{figure}

\subsubsection{Summary of Applicability of Discrete Filtering Methods}

There are several filtering methods for performing the prediction and analysis steps. Some methods, such as the bootstrap particle filter, can be applied generally to nonlinear problems. However, others require a linear model $( \Psi(\cdot) = M\cdot)$ and/or linear observations $(h(\cdot) = H\cdot)$.
Some of the methods we have described provably approximate the
probability distribution updates. Some just estimate the state and
simply use covariance information to weight the relative importance
of the predictions from the model and of the data.

The applicability of the methods introduced is summarized in the
table below, with respect to linearity/nonlinearity of the dynamics
and the observation model. Furthermore  ${\bf P}$ is used to denote
 methods which provably approximate the filtering distributions
$\pi_j$ in the large particle limit; ${\bf S}$ denotes methods
which only attempt to estimate the state, using the data.

\begin{description}[leftmargin=!,labelwidth=\widthof{\bfseries }]\setlength{\itemsep}{4pt}
\item[Kalman Filter] $ \Psi(\cdot) = M\cdot, h(\cdot) = H\cdot$. ${\bf P}.$
\item[3DVAR] $\text{general}\, \Psi, h(\cdot) = H\cdot $. ${\bf S}.$
\item[Bootstrap Particle Filter] $ \text{general}\, \Psi, h$. ${\bf P}.$
\item[Optimal Particle Filter] $\text{general}\, \Psi, h(\cdot) = H\cdot$. ${\bf P}$.
\item[Extended Kalman Filter] $\text{general}\, \Psi, h(\cdot) = H\cdot $. ${\bf S}.$
\item[Ensemble Kalman Filter] $\text{general}\, \Psi, h(\cdot) = H\cdot $. ${\bf S}.$
\end{description}

Some of these constraints can be relaxed on the setting in which they apply
can be relaxed, but the list above describes
the methods as we present them in these notes.
Furthermore the last two methods can accurately predict probability distributions
in situations where approximate Gaussianity holds; this may be induced
by small noise and by large data. However it is important to appreciate
that the raison d'etre of the EnKF is to facilitate the solution of
problems with high dimensional state space; typically small ensemble sizes are
used and the algorithm is employed far from the regime in which it is
able to provably approximate the distributions, even if they are close
to Gaussian. It is for this reason that we prefer to think of the EnKF
as an ensemble state estimator.

\subsection{Discussion and Bibliography}

Particle filters often perform poorly for high-dimensional systems due to
the fact that the particle weight typically concentrates on one, or
a small number, of particles --see the work of Bickel and Snyder in
\cite{bickel2008sharp, snyder2008obstacles, Snyder2011}. This is the
issue that the optimal particle filter tries to ameliorate; the
paper \cite{agapiou2017importance} shows calculations which demonstrate
the extent to which this amelioration is manifest in theory.
The optimal particle filter is discussed, and further references given,
in the very clear paper \cite{doucet2000sequential}; see section IID.
Throughout much of this chapter we considered the case of Gaussian additive noise
and linear observation operator
in which case the prediction step is tractable; the paper \cite{doucet2000sequential} discusses the more general setting.
The order in which the prediction and resampling is performed can
be commuted in this case and a discussion of this fact may be found in
\cite{pitt1999filtering}; this leads to the distinction between
what we term the GOPF and the OPF.
The convergence of the optimal particle
filter is studied in \cite{johansen2008note}.
The formulation of the bootstrap and optimal particle filters as random
dyamical systems may be found in \cite{kelly2016ergodicity}. An attempt to alleviate weight collapse by introducing alternative update steps can be found in \cite{Snyder2011}.

\section{\Large{\sffamily{Filtering Approach to the Inverse Problem}}}

In this final chapter we demonstrate how the two separate themes that
underpin this course, inverse problems and data assimilation, may be
linked. This opens up the possibility of transferring ideas from filtering
into the setting of quite general inverse problems.
In the first section we describe the general, abstract, connection
and introduce the revolutionary idea of sequential Monte Carlo
methods (SMC).
In the second section we analyze the concrete case of applying the EnKF
to solve an inverse problem, leading to ensemble Kalman inversion (EKI).
In the final section we link the EKI methodology to SMC.

\subsection{General Formulation}

Recall the inverse problem of finding $u \in \mathbbm{R}^\Du$ from $y \in \mathbbm{R}^\Dy$ where
\begin{align}
\label{eq:need99}
y=G(u)+\eta,\; \eta\sim \mathcal{N}(0,\Gamma_0)
\end{align}
and the related loss function
\begin{equation*}
\mathsf{L}_0(u)=\frac{1}{2}|y-G(u)|^2_{\Gamma_0}.
\end{equation*}
The reason for writing $\Gamma_0$, rather than $\Gamma$, will become apparent
below and will also be exploited in the third section of this chapter where
we study EKI.

If we put a prior $\rho$ on the unknown $u$ then the posterior takes
the form
$$\pi(u)=\frac{1}{Z}\exp\bigl(-\mathsf{L}_0(u)\bigr)\rho(u).$$
Let $J\in\mathbbm{N}$ and choose $h$ so that $Jh=1$. Then
define the family of pdfs $\{\pi_j\}_{j=0}^J$ by
$$\pi_j(u)=\frac{1}{Z_j}\exp\bigl(-jh\mathsf{L}_0(u)\bigr)\rho(u).$$
It follows that $\pi_0 = \rho$ and $\pi_J=\pi$ and, furthermore, we may update the
sequence $\{\pi_j\}_{j=0}^J$ sequentially using the formula
$$\pi_{j+1}(u)=\frac{Z_j}{Z_{j+1}}\exp\bigl(-h\mathsf{L}_0(u)\bigr)\pi_j(u).$$
This simply corresponds to application of Bayes theorem to the
inverse problem
\begin{align}
\label{eq:need00}
y=G(u)+\eta,\; \eta\sim \mathcal{N}(0,\Gamma),
\end{align}
with $\Gamma=\frac{1}{h}\Gamma_0$
and with prior $\pi_j.$ With this choice of $\Gamma$ the identity \eqref{eq:need00}
gives the likelihood proportional to $\exp(-h\mathsf{L}_0(u))$. The update
may be written
$$\pi_{j+1}(u)={\mathcal{A}}\pi_j,$$
noting that the analysis operator ${\mathcal{A}}$ is nonlinear, corresponding
to multiplication by $\exp(-h\mathsf{L}_0(u))$ and then normalization to
a pdf.

If we let ${\mathcal{P}}_j$ denote any Markov kernel for which $\pi_j$ is
invariant then we obtain
$$\pi_{j+1}={\mathcal{A}}{\mathcal{P}}_j \pi_j.$$
This update formula should be compared with the filtering update formula
\begin{align*}
\pi_{j+1} = {\mathcal{A}}_j {\mathcal{P}}\pi_j
\end{align*}
introduced and used in earlier chapters.

Whilst the $j-$dependence in Bayes rule and the Markov kernel
is interchanged between the inverse problem and the filtering problem,
this makes little material difference to implementation.
Firstly, once a Markov kernel ${\mathcal{P}}$ is identified under which $\pi=\pi_J$
is invariant, it can be easily adapted to find a family of kernels
${\mathcal{P}}_j$ under which $\pi_j$ is invariant, simply by rescaling the
observation covariance. Secondly the fact that the data $y$ is fixed,
rather than changing at each step, makes little difference to
the implementation.
This perspective opens up an entire field, known as {\em sequential
Monte Carlo}, in which ideas from filtering
may be transferred to other quite different problems, including Bayesian
inversion, as explained here.

Note however (see the discussion at the end of section \ref{sec:overview})
that of the filtering methods we have introduced so far only the bootstrap filter
applies directly to the case of {\em nonlinear} observation operators; since
$G$ is in general nonlinear this means that extra ideas are required to
implement the optimal particle filter, 3DVAR, ExKF and EnKF. One approach is
to replace the sequential optimization principles by the non-quadratic
optimization problem required for nonlinear $G$; we do not discuss this idea
in any detail but it is a viable option. Another approach is to use the linearization
technique which we now outline in the context of EKI.

\subsection{Ensemble Kalman Inversion}

We now make a detour into the subject of how to use
filters to estimate parameters $u$ from data $y$ satisfying
\eqref{eq:need00}. The approach we study,
and which we will relate to sequential Monte Carlo in
the next section, is to introduce an artificial
time dynamic. This can be done quite simply as follows: we write
\begin{align*}
u_{j+1} &=   u_j,\\
y_{j+1} & =   G(u_{j+1}) + \eta_{j+1},
\end{align*}
and we can think of finding the filtering distribution on $u_j|Y_j.$
We discuss how to relate the $y_j$ to the one data point $y$ below.
For now we take $\eta_j \sim \mathcal{N}(0,\Gamma)$ but we will revisit this
choice in the next section, linking the problem to solution of \eqref{eq:need99}.
Because the observation operator $G$ is, in general, nonlinear, this
does not render our system in a form where we can readily apply the
EnKF. To this end we introduce a new variable $w_{j}$ and rewrite the
filter as:
\begin{align*}
u_{j+1} & =   u_j,\\
w_{j+1} & =   G(u_{j}),\\
y_{j+1} & =   w_{j+1} + \eta_{j+1}.
\end{align*}

We introduce the new variables $v=(u,w)^T$, nonlinear map $\Psi(v)=\bigl(u, G(u)\bigr)^T$
and linear operators $H =  [0, I], H^{\perp} = [I,0].$  Then if we
write $v_j=(u_j,w_j)^T$ we may write the dynamical system in the form
\begin{subequations}
\label{eq:new}
\begin{align}
v_{j+1} &=  \Psi(v_j),\\
y_{j+1} &= Hv_{j+1} + \eta_{j+1}.
\end{align}
\end{subequations}
We note that
$$Hv= w, \quad H^{\perp}v=u.$$

\begin{remark}
Typically either of the following  are used to construct artificial
data $\{y_j\}$ for the filtering algorithm, given a single instance of data $y$:
 $$y_{j+1}=
  \begin{cases}
  & y \text{ (unperturbed obervations).}\\
  & y+\bar{\eta}_{j+1}, \bar{\eta}_{j+1}\sim \mathcal{N}(0,\Gamma)\text{ (perturbed observations).}
  \end{cases}$$
The first choice is natural if viewing the algorithm as a sequential
optimizer; the latter, in the linear case, is natural when seeking to draw
samples from the posterior.\qed
\end{remark}

We now apply the EnKF to the dynamics/data model \eqref{eq:new}.
We obtain, for $n = 1,\dots,N$,
\begin{align}
\label{eq:eq1}
\hat{v}_{j+1}^{(n)}&=\Psi(v_j^{(n)}),\\
\bar{v}_{j+1}&= \frac{1}{N}\sum_{n=1}^N\hat{v}_{j+1}^{(n)},\\
\widehat{C}_{j+1}&=\frac{1}{N}\sum_{n=1}^N \bigl(\hat{v}_{j+1}^{(n)}-\bar{v}_{j+1}\bigr)\otimes\bigl(\hat{v}_{j+1}^{(n)}-\bar{v}_{j+1} \bigr),\\
v_{j+1}^{(n)}&=
(I-K_{j+1}H)\hat{v}_{j+1}^{(n)}+ K_{j+1}y_{j+1}^{(n)},
\end{align}
with the Kalman gain
\begin{align*}
K_{j+1}& =\widehat{C}_{j+1}H^T S_{j+1},\\
S_{j+1}& =(HC_{j+1}H^T+\Gamma)^{-1}.
\end{align*}
Now we may simplify these expressions by using the specific $\Psi$, $v$, $H$ arising in the inverse problem. Writing
\begin{align*}
\widehat{C}_{j+1}=
\begin{bmatrix}
C_{j+1}^{uu} & C_{j+1}^{uw}\\
(C_{j+1}^{uw})^T & C_{j+1}^{ww}
\end{bmatrix}, \quad \quad
\;\;
\bar{v}_{j+1}=
\begin{pmatrix}
\bar{u}_{j+1}\\
\bar{w}_{j+1}
\end{pmatrix},
\end{align*}
we have
$$\bar{u}_{j+1} = \frac{1}{N}\sum_{n=1}^N u_{j}^{(n)},\; \quad \quad \bar{w}_{j+1} = \frac{1}{N}\sum_{n=1}^N G(u_{j}^{(n)}):=\bar{G}_j,$$
and
\begin{align*}
C_{j+1}^{uw} &= \frac{1}{N}\sum_{n=1}^N \bigl(u_{j}^{(n)}-\bar{u}_{j+1}\bigr)\otimes \bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr),  \\
 C_{j+1}^{ww} &= \frac{1}{N}\sum_{n=1}^N \bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr)\otimes\bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr).
\end{align*}
There is a similar expression for $C_{j+1}^{uu}$, but as we will show in what
follows it is not needed for the unknown parameter $u$ update formula.
Noting that, because of the structure of $H$, $S_{j+1}=(C_{j+1}^{ww}+\Gamma)^{-1}$
we obtain
\begin{align}
K_{j+1}=
\begin{pmatrix}\label{eq:2}
C_{j+1}^{uw}(C_{j+1}^{ww}+\Gamma)^{-1}\\
(C_{j+1}^{ww})(C_{j+1}^{ww}+\Gamma)^{-1}
\end{pmatrix}.
\end{align}
Combining equation \eqref{eq:2} with the update equation within \eqref{eq:eq1}
it follows that
\begin{align*}
\bigl\{v_j^{(n)}\bigr\}_{n=1}^N \rightarrow \bigl\{v_{j+1}^{(n)} \bigr\}_{n= 1}^N
\end{align*}
and
\begin{align*}
\bigl\{H^{\perp}v_j^{(n)} \bigr\}_{n= 1}^N \rightarrow \bigl\{H^{\perp}v_{j+1}^{(n)} \bigr\}_{n= 1}^N,
\end{align*}
and hence that
\begin{align*}
u_{j+1}^{(n)} = H^{\perp}v_{j+1}^{(n)} = u_j^{(n)}+ C_{j+1}^{uw}(C_{j+1}^{ww}+\Gamma)^{-1} \Bigl(y_{j+1}^{(n)}-G(u_j^{(n)})\Bigr).
\end{align*}
Thus we have derived the EKI step \begin{equation}
\label{eq:EnKF}
u_{j+1}^{(n)}  = u_j^{(n)}+ C_{j+1}^{uw} (C_{j+1}^{ww}+\Gamma)^{-1}\Bigl(y_{j+1}^{(n)}-G(u_j^{(n)})\Bigr).
\end{equation}
The full algorithm is described below:
\FloatBarrier
\begin{algorithm}
\caption{Algorithm for Ensemble Kalman Inversion}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Initial distribution $\pi_0 = \rho$, observations $Y_{J}$, number of particles $N.$
\STATE {\bf Initial Sampling}: Draw $N$ particles $u_0^{(n)} \sim \rho$ so that $\pi_0^N=S^N\rho.$
\vspace{0.04in}
\STATE{\bf Subsequent Sampling} For $j = 0,1,\dots,J-1$, perform
\begin{enumerate}
\item Set $\bar{u}_{j+1} = \frac{1}{N}\sum_{n=1}^N u_{j}^{(n)}.$
\item Set $\bar{G}_j= \frac{1}{N}\sum_{n=1}^N G(u_{j}^{(n)}).$
\item Set $C_{j+1}^{uw} = \frac{1}{N}\sum_{n=1}^N \bigl(u_{j}^{(n)}-\bar{u}_{j+1}\bigr) \otimes \bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr).$
\item Set $C_{j+1}^{ww} = \frac{1}{N}\sum_{n=1}^N \bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr)\otimes\bigl(G(u_{j}^{(n)})-\bar{G}_{j}\bigr).$
\item $u_{j+1}^{(n)}  = u_j^{(n)}+ C_{j+1}^{uw}\bigl(C_{j+1}^{ww}+\Gamma)^{-1}(y_{j+1}^{(n)}-G(u_j^{(n)})\bigr).$
\end{enumerate}
\vspace{0.04in}
\STATE {\bf Output}: $N$ particles $u_{J}^{(1)},u_{J}^{(2)}, \dots, u_{J}^{(N)}.$
\end{algorithmic}
\label{alg:EKI}
\end{algorithm}
\FloatBarrier

\begin{remark}
This algorithm may be viewed as a derivative-free optimization method,
within the broad class that contains genetic, swarm or ant colony
optimization. \qed
\end{remark}

The algorithm has the following invariant subspace property.

  \begin{theorem}(Space of Ensemble)
\label{t:cite}
  Define $\mathcal{A} = {\text{Span}} \bigl\{u^{(n)}_{0}: n = 1,\ldots, N \bigr\}$, then
for all $0\le j \le J$ and all  $1\le n \le N$,
$u^{(n)}_{j}$ defined by the iteration \eqref{eq:EnKF} lie
in $\mathcal{A}$.
  \end{theorem}
  \begin{proof}
  The proof proceeds by induction. We let $\mathfrak{u}_j \in ({\mathbbm{R}^\Du})^N$
denote the collection of $\{u^{(n)}_{j}\}_{n= 1}^N.$
  At first, let $0\le j \le J$ and let  $1\le m \le N$
and define the following two quantities:
  \begin{align*}
  d^{(n)}_{j}&:=(C^{ww}_{j+1}+\Gamma)^{-1} \Bigl(y^{(n)}_{j+1}-G(u^{(n)}_{j})\Bigr), \\
  D_{mn}(\mathfrak{u}_j)&:=-\langle d^{(n)}_{j},G(u^{(m)}_{j})-\bar{G}_j \rangle.
  \end{align*}
  Notice that the $d^{(n)}_{j}$ are elements of the data space ${\mathbb R}^J$,
whereas the $D_{mn}(\mathfrak{u}_j)$ are scalar quantities. Furthermore notice that
\begin{align*}
u_{j+1}^{(n)}  & = u_j^{(n)}+ C_{j+1}^{uw} (C_{j+1}^{ww}+\Gamma)^{-1}\Bigl(y_{j+1}^{(n)}-G(u_j^{(n)})\Bigr)\\
& = u_j^{(n)} + C_{j+1}^{uw} d^{(n)}_{j} \\
& = u_j^{(n)} + \frac{1}{N}\sum_{m=1}^N \bigl(u_{j}^{(m)}-\bar{u}_{j+1}\bigr) \otimes \bigl(G(u_{j}^{(m)})-\bar{G}_{j}\bigr) d^{(n)}_{j} \\
& = u^{(n)}_{j} - \frac{1}{N} \sum\limits_{m=1}^N D_{mn}(\mathfrak{u}_j)(u^{(m)}_{j}-\bar{u}_{j+1}).
\end{align*}
From the definition of $\bar{G}_j$ and the bilinearity of the inner product
it follows that $\sum\limits_{m=1}^N D_{mn}(\mathfrak{u}_j) = 0$. Therefore the
update expression may be rewritten, for $0\le j \le J$ and $1\le n \le N,$ as
$$u^{(n)}_{j+1}=u^{(n)}_{j}-\frac{1}{N} \sum\limits_{m=1}^N D_{mn}(\mathfrak{u}_j)u^{(m)}_{j}.$$
Hence if the property holds for all the particles of the time step $j$, it will clearly be the case for all the particles at time step $j+1$.
  \end{proof}

\begin{remark}
The choice of the initial ensemble of particles
$\{u^{(n)}_{0}\}_{n= 1}^N$ is thus crucial to the
performance of the EnKF,
since the algorithm remains in the initial
space of the initial ensemble.
In the setting of Bayesian inverse problems the initial ensemble
is frequently created by drawing from the prior.
Alternatively, if the prior is Gaussian, the first $N$ covariance
eigenvectors may be used, ordered by decreasing variance contribution.
More generally any truncated basis for the space $\mathbbm{R}^\Du$ is
a natural initial ensemble. However the question of how to adaptively
learn a good choice of ensemble subspace, in
response to observed data, is unexplored and potentially fruitful. \qed
\end{remark}

\begin{remark}
We describe an alternative way to approach the derivation of the EKI update
formulae. We apply Theorem \ref{eq:extra1} with the specific structure
arising from the dynamical system used in EKI. To this end we define
\begin{equation}
\label{eq:z2}
{\mathsf{I}}_{n}(b) := \frac{1}{2} \Big\| y_{j+1}^{(n)} - G({u}_{j}^{(n)})
-\frac{1}{N}\sum_{n=1}^N b_n \bigl(G({u}_{j}^{(n)})-\bar{G}_j\bigr)\Big\| ^{2}_{\Gamma}+\frac{1}{2N}\sum_{n=1}^N b_n^2.
\end{equation}
Once this quadratic form has been minimized with respect to $b$ then the
upate formula  \eqref{eq:deenz3} gives
\begin{align*}
u_{j+1}^{(n)}&={u}_{j}^{(n)}+\frac{1}{N}\sum_{n=1}^N b_n\bigl({u}^{(n)}_{j}-\bar{u}_{j}\bigr),\\
w_{j+1}^{(n)}&=G({u}_{j}^{(n)})+\frac{1}{N}\sum_{n=1}^N b_n\bigl(G({u}^{(n)}_{j})-\bar{G}_{j}\bigr).
\end{align*}
(Note that the vector $b$ depends on $n$; we have supressed this dependence
for notational convenience.)
Theorem \ref{t:cite} is an immediate consequence of this structure.
\qed
\end{remark}

\subsection{Linking Ensemble Kalman Inversion and SMC}
Here we link the two preceding sections.
In the first subsection
we describe how EKI may be linked with
SMC through a particular scaling of $\Gamma$ with $h$. We then
take the limit $h \to 0$ and obtain a system or ordinary
differential equations.
In the second subsection we study the resulting algorithm in
the case in which $G$ is linear, leading to insight into the
EKI algorithm in the iterated context.

\subsubsection{Continuous Time Limit}

Although we have emphasized the optimization perspective on ensemble methods,
we may think of one-step of ensemble Kalman inversion as approximating
the filtering mapping $\pi_j=\operatorname{\mathbbm{P}}(u_j|Y_j) \mapsto  \pi_{j+1}=
\operatorname{\mathbbm{P}}(u_{j+1}|Y_{j+1});$
this will be a good approximation when the measures are close to Gaussian and
 the number of ensemble members $N$ is large.
In order to link this to the mapping $\pi_{j+1}={\mathcal{A}}\pi_j$ in the
presentation of SMC in the first section of this chapter,
we set $\Gamma=\frac{1}{h}\Gamma_0,$ and we assume unperturbed observations $y_j^{(n)} = y$ for all $j.$
Let ${\mathfrak{u}}_j \in (\mathbbm{R}^\Du)^N$ denote, as in the previous section, the
collection of particles  $\{u^{(n)}_j\}_{n= 1}^N$ at discrete time $j.$
Now define
$$D^0_{mn}(\mathfrak{u}_j):=\Bigl\langle G(u^{(n)}_j)-y,G(u^{(m)}_j)-\bar{G}_j \Bigr\rangle_{\Gamma_0}.$$
Note that then, to leading order in $h \ll 1$,
\begin{align*}
D_{mn}(\mathfrak{u}_j) &= -\Bigl\langle (C^{ww}_{j+1}+\Gamma)^{-1}(y -G(u^{(n)}_{j})),G(u^{(m)}_{j})-\bar{G}_j \Bigr\rangle \\
&\approx h\langle G(u_j^{(n)})-y,G(u^{(m)})-\bar{G}_j \rangle_{\Gamma_0} \\
&= hD^0_{mn}(\mathfrak{u}_j).
\end{align*}
It follows that, also to leading order in $h$,
\begin{equation*}
u^{(n)}_{j+1}\approx u^{(n)}_{j}-\frac{h}{N} \sum\limits_{m=1}^N D^0_{mn}(\mathfrak{u}_j)u^{(m)}_{j}.
\end{equation*}
Then letting $h\to 0$ yields
\begin{equation}
\label{eq:has}
\frac{du^{(n)}}{dt}=-\frac{1}{N}\sum\limits_{m=1}^N D^0_{mn}(\mathfrak{u})u^{(m)},
\end{equation}
where, similarly as above, $\mathfrak{u}$ represents the collection of $N$ particles.

\begin{remark}
Notice that equation \eqref{eq:has}
has families of fixed points where: (i) either the particles fit
the data exactly (which corresponds to the left-hand side in the inner product
defining $D^0_{mn}(u)$ being zero);  or (ii) all particles collapse on their
mean value (which corresponds to the right-hand side of the same inner product).
This suggests that the system of ordinary differential equations which
describes the behaviour of ensemble Kalman inversion is driven by
two desirable attributes: matching the data and achieving consensus. \qed
\end{remark}

\subsubsection{Linear Setting}
Now suppose $G(\cdot)$ is a linear map denoted $A\cdot$,
and define
$$\bar{u}=\frac{1}{N}\sum\limits_{n=1}^N u^{(n)}.$$
In this setting we have
\begin{align*}
\frac{du^{(n)}}{dt}&=-\frac{1}{N}\sum\limits_{m=1}^N \bigl\langle Au^{(n)}-y,A(u^{(m)}-\bar{u}) \bigr\rangle_{\Gamma_0}u^{(m)} \\
&=-\frac{1}{N}\sum\limits_{m=1}^N \bigl\langle Au^{(n)}-y,A(u^{(m)}-\bar{u}) \bigr\rangle_{\Gamma_0}(u^{(m)}-\bar{u}) \\
&= -\frac{1}{N}\sum\limits_{m=1}^N \bigl\langle A^T\Gamma_0^{-1}(Au^{(n)}-y),u^{(m)}-\bar{u} \bigr\rangle (u^{(m)}-\bar{u}) \\
&= -\frac{1}{N}\sum\limits_{m=1}^N \bigl(u^{(m)}-\bar{u} \bigr)\otimes \bigl(u^{(m)}-\bar{u}\bigr) \bigl(A^T\Gamma_0^{-1}(Au^{(n)}-y)\bigr).
\end{align*}
If we define
\begin{equation}
C(\mathfrak{u})=\frac{1}{N}\sum\limits_{n=1}^N \bigl(u^{(n)}-\bar{u}\bigr) \otimes \bigl(u^{(n)}-\bar{u}\bigr)
\end{equation}
then we find that, for $n=1, \dots, N$,
\begin{align*}
\frac{du^{(n)}}{dt}&=-C(\mathfrak{u}) A^T\Gamma_0^{-1}(Au^{(n)}-y)\\
&=-C(\mathfrak{u})\nabla \mathsf{L}_0 (u^{(n)}),
\end{align*}
where
$$\mathsf{L}_0(u)=\frac12|y-Au|_{\Gamma_{0}}^2.$$
This corresponds to a gradient descent in the subspace defined by the
initial ensemble. In particular
\begin{align*}
\frac{d}{dt}\mathsf{L} \bigl(u^{(n)}(t)\bigr)&=\Bigl\langle \nabla \mathsf{L} \bigl(u^{(n)}(t)\bigr),\frac{du^{(n)}}{dt}(t) \Bigr\rangle \\
&= - \Bigl|C(u)^{1/2}\nabla \mathsf{L} \bigl(u^{(n)}(t)\bigr) \Bigr|^{2} \leq 0,
\end{align*}
demonstrating that the loss function $\mathsf{L}$ is decreasing along the trajectory
associated to each ensemble member.

We note also that, if we define $e^{(n)}=u^{(n)}-\bar{u}$, then
$$\frac{de^{(n)}}{dt}=-C(\mathfrak{u}) A^T\Gamma_0^{-1}Ae^{(n)}.$$
Because
$$C:=C(\mathfrak{u})=\frac{1}{N}\sum_{n=1}^N e^{(n)} \otimes e^{(n)}$$
it follows that
$$\frac{dC}{dt}=-2CA^T\Gamma_0^{-1}AC.$$
Note further that if $C$ is invertible (requires $N  \ge d$) then the inverse $P$ satisfies
$$\frac{dP}{dt}=2A^T\Gamma_0^{-1}A$$
so that $\|P\| \to \infty$ as $t \to \infty.$ This fact, which can be suitably interpreted
when $C$ is only invertible on a subspace, suggests that the covariance shrinks with time, causing ensemble collapse -- consensus -- while at the same time driving minimization of the loss $\mathsf{L}$ over an appropriate subspace.

\subsection{Discussion and Bibliography}

The idea of using particle filters to sample general distributions,
including those arising in Bayesian inversion, maybe be found in
\cite{del2006sequential}; a recent application to a Bayesian
inverse problem, which demonstrated the potential of the
methodology in that context, is \cite{kantas2014sequential}.
A simple proof of convergence of the method may be found in
\cite{beskos2015sequential}; it is based on the proof described in
\cite{rebeschini2015can} for the standard bootstrap particle filter.

The use of the ensemble Kalman filter for parameter
estimation was introduced in the papers \cite{lorentzen2001underbalanced,anderson2014kalman} in which a physical dynamical model was
appended with trivial dynamics for the parameters in order to estimate them;
the idea was extended to learn an entire field of parameter values
in \cite{naevdal2002near}.
The paper \cite{skjervheim2011ensemble} was the first to do what we do
here, namely to consider all the data at once and a single mapping
of unknown parameters to data. In that paper only one iteration is
used; the papers \cite{chen2012ensemble,emerick2013ensemble}
demonstrated how iteration could be useful.
See also the book \cite{oliver2008inverse} for the use of ensemble
Kalman methods in oil reservoir simulation.

Development of the method
for general inverse problems was undertaken in \cite{iglesias2013ensemble},
and further development of iterative methods is described in
\cite{iglesias2015iterative,iglesias2016regularizing}.
Ensemble inversion was analyzed in
\cite{schillings2017analysis}, including the continuous-time limit
and gradient flow structure described here. The potential for
ensemble inversion in the context of hierarchical Bayesian
methods is demonstrated in \cite{chada2017parameterizations}.
The idea that we explain here, of obtaining an evolution equation for
the covariance which is satisfied exactly by ensemble methods, appears in
the remarkable papers
\cite{reich2011dynamical, bergemann2012ensemble}, and in the other
papers of Bergemann and Reich referenced therein. Their work is at the heart of
the analysis in \cite{schillings2017analysis} which demonstrates
the ensemble collapse and approximation properties of the ensemble
method when applied to linear inverse problems.

\section{\Large{\sffamily{Optimization Perspective }}}

In this chapter we explore the properties of Bayesian inversion
from the perspective of an optimization problem which corresponds
to maximizing the posterior probability, in a sense which we will make
precise. We demonstrate the properties of the point estimator resulting
from this optimization problem, showing its positive and negative
attributes, the latter motivating our work in the following chapter.

\subsection{The Setting}

Once again we work in the inverse problem setting of finding $u\in \mathbbm{R}^\Du$
from $y\in \mathbbm{R}^\Dy$ given by
$$ y = G(u) + \eta$$
 with noise $\eta\sim\nu(\cdot)$ and prior $u\sim\rho,$ as in Assumption \ref{a:jc1}. The posterior $\pi^y(u)$ on $u|y$ is given by Theorem \ref{t:bayes} and has the form
\[
\pi^y (\vct u) = \frac{1}{Z} \nu (\vct y- \vct G(\vct u)) \rho (\vct u).
\]
We may define a loss:
\[
\mathsf{L} (\vct u) = - \log \nu \bigl(\vct y-\vct G(\vct u)\bigr),
\]
and a regularizer
\[
\mathsf{R}(\vct u) = - \log \rho(\vct u).
\]
When added together these two functions of $u$ comprise
an objective function of the form
\[
{\mathsf{J}}(\vct u) = \mathsf{L}(\vct u) + \mathsf{R}(\vct u).
\]
Furthermore
\[
\pi^y (\vct u) = \frac{1}{Z} \nu \bigl(\vct y-\vct G(\vct u)\bigr) \rho (\vct u) \propto e^{-{\mathsf{J}}(\vct u)}.
\]
We see that minimizing the objective function ${\mathsf{J}}(\cdot)$
is equivalent to maximizing the posterior $\pi^y(\cdot)$.
Therefore, recalling Definition \ref{def:map}, the MAP estimator can be rewritten in terms of ${\mathsf{J}}$ as follows:
\begin{align*}
u_{\mbox {\tiny{\rm MAP}}} &= \arg\max_{\vct{u}\in\mathbbm{R}^\Du}\pi^{\vct{y}}(\vct{u}) \\
&= \arg \min_{u \in \mathbbm{R}^\Du} {\mathsf{J}}(u).
\end{align*}

We will provide conditions under which the MAP estimator is attained in Theorem \ref{thm:achievable-obj} and we will give an interpretation of MAP estimators in terms of maximizing the probability of infinitesimal balls in Theorem
\ref{thm:ball-density}. This interpretation allows to generalize the definition of MAP estimators to measures that do not possess a Lebesgue density.

\begin{example}
Consider the Gaussian setting of Assumption \ref{a:sz1}.
Then, since the posterior is Gaussian, its mode agrees with its mean, which is given by $m$ as defined in Theorem \ref{t:sz1}.
\end{example}

\begin{example}
	\label{ex:loss-l2}
	If $\eta = \mathcal{N}(\zerovct ,\mtx \Gamma)$, then
	\(\nu\bigl(\vct y-\vct G(\vct u)\bigr) \propto \exp(- \frac{1}{2} |\vct y-\vct G(\vct u)|_{\mtx \Gamma} ^2)\). So the loss in this case is \(\mathsf{L}(\vct u) = \frac{1}{2} |\vct y-\vct G(\vct u)|_{\mtx \Gamma}^2\), a \(\mtx \Gamma\)-weighted \(\mathsf{L}_2\) loss.
\end{example}
\begin{example}
	\label{ex:reg-l2}
	If we have prior \(\rho(\vct u) = \mathcal{N}(\zerovct ,\mtx \hat{C})\), then
ignoring $u-$independent normalization factors, which appear as constant
shifts in ${\mathsf{J}}(\cdot)$, we may take the regularizer as \(\mathsf{R}(\vct u) = \frac{1}{2} |\vct u|_{\mtx \hat{C}}^2\). In particular, if \(\mtx \hat{C} = \lambda^{-1} \Id \), then \(\mathsf{R}(\vct u) = \frac{\lambda}{2} |\vct u|^2\), an \(\mathsf{L}_2\) regularizer.
\end{example}

If we combine Example \ref{ex:loss-l2} and Example \ref{ex:reg-l2}, we obtain a canonical objective function
$${\mathsf{J}}(\vct u) = \frac{1}{2} |\vct y-\vct G(\vct u)|_{\mtx \Gamma}^2 + \frac{\lambda}{2} |\vct  u|^2.$$
To connect with future discussions, here \(\lambda\) corresponds to prior precision, and may be learned from data, such as in hierarchical methods.

\begin{example}
	\label{ex:reg-l1}
	As an alternative to the \(\mathsf{L}_2\) regularizer,
consider \(\vct u=(u_1,\ldots,u_d)\) with \(u_i\) having prior
distribution i.i.d. Laplace. Then \(\rho(\vct u) \propto \exp(-\lambda \sum_{i=1}^d |u_i|) = \exp(-\lambda |\vct u|_1)\). In this case
\(\mathsf{R}( \vct u ) = \lambda |\vct u|_1\), an \(\mathsf{L}_1\) regularizer.
If we combine this prior with the weighted \(\mathsf{L}_2\) loss above
then we have objective function $${\mathsf{J}}(\vct u) = \frac{1}{2} |\vct y - \vct G ( \vct u ) |_{\mtx \Gamma}^2 + \lambda |\vct u |_1.$$
Even though this objective function promotes sparse solutions, samples
from the underlying posterior distribution are not typically sparse.
\end{example}

\subsection{Theory}

For any optimization problem for an objective function with a finite
infimum, it is of interest to determine whether the infinimum is attained. We have the following result.

		\begin{theorem}[Attainable MAP Estimator]
			\label{thm:achievable-obj}
			Assume that ${\mathsf{J}}$ is non-negative, continuous and that ${\mathsf{J}}(u) \to \infty$ as $|u| \to \infty. $
Then ${\mathsf{J}}$ attains it infimum. Therefore,
the MAP estimator of $u$ based on the posterior $\pi^y(u) \propto \exp\bigl(-{\mathsf{J}}(u)\bigr)$ is attained.
		\end{theorem}
		\begin{proof}
			By the assumed growth and non-negativity of ${\mathsf{J}}$, there is $R$ such that $\inf_{u\in \mathbbm{R}^\Du} {\mathsf{J}}(u) = \inf_{u\in B(0,R)}$ where $B(0,R)$ denotes the closed ball of radius $R$ around the origin. Since ${\mathsf{J}}$ is assumed to be continuous, its infimum over $B(0,R)$ is attained and the proof is complete.
		\end{proof}

\begin{remark}
Suppose that
\begin{enumerate}
				\item \(\vct G \in C(\mathbbm{R}^\Du, \mathbbm{R}^\Dy)\), i.e. \(\vct G\) is a continuous function;
				\item the objective function \({\mathsf{J}}(\vct u)\) has \(\mathsf{L}_2\) loss as defined in Example \ref{ex:loss-l2} and \(\mathsf{L}_p\) regularizer \(\mathsf{R}(\vct u) = \frac{\lambda}{p} |\vct u|_p^p\), \(p \in (0,\infty)\).
			\end{enumerate}
Then the assumptions on ${\mathsf{J}}$ in Theorem \ref{thm:achievable-obj} are satisfied.  This shows that if \(\vct G\) is continuous, the infinimum of ${\mathsf{J}}$ defined with \(\mathsf{L}_2\) loss and \(\mathsf{L}_p\) regularizer is attained at the MAP estimator of the  corresponding Bayesian problem with posterior pdf proportional to $\exp\bigl(-{\mathsf{J}}(u)\bigr)$.
\end{remark}

\begin{remark}
Notice that the assumption that ${\mathsf{J}}(u) \to \infty$ is not restrictive: this condition needs to hold in order to be able to normalize $\pi^y (u)\propto \exp\bigl(-{\mathsf{J}}(u)\bigr)$ into a pdf, which is implicitly assumed in the second part of the theorem statement.
\end{remark}

Intuitively the MAP estimator maximizes posterior probability. We make this
precise in the following theorem which links the objective function ${\mathsf{J}}(\cdot)$
to small ball probabilities.

		\begin{theorem}[Objective Function and Posterior Probability]
			\label{thm:ball-density}
			Making the same assumptions as in Theorem \ref{thm:achievable-obj}, let
			\[
			\alpha(\vct u,\delta) := \int_{\vct v \in B(\vct u, \delta)} \pi^y(\vct v) d\vct v = \operatorname{\mathbbm{P}}^{\pi^y}\bigl(B(\vct u,\delta)\bigr),
			\]
			be the posterior probability of a ball with radius \(\delta\)
centered at \(u\). Then, for all \(\vct u_1, \vct u_2 \in \mathbbm{R}^\Du\), we have
			\[
			\lim_{\delta \rightarrow 0} \frac{\alpha(\vct u_1, \delta)}{\alpha(\vct u_2, \delta)} = e^{{\mathsf{J}}(\vct u_2) - {\mathsf{J}}(\vct u_1)}.
			\]
		\end{theorem}
		\begin{proof}
Let $u_1, u_2 \in \mathbbm{R}^\Du$ and let $\epsilon>0.$ By continuity of ${\mathsf{J}}$ we have that, for all $\delta$ sufficiently small,
\begin{align*}
e^{-{\mathsf{J}}(u_1) - \epsilon} \le e^{-{\mathsf{J}}(v)} &\le e^{-{\mathsf{J}}(u_1) + \epsilon} \quad \text{for all} \,\, v \in B(u_1, \delta), \\
e^{-{\mathsf{J}}(u_2) - \epsilon} \le e^{-{\mathsf{J}}(v)} &\le e^{-{\mathsf{J}}(u_2) + \epsilon} \quad \text{for all}  \,\, v \in B(u_2, \delta).
\end{align*}
Therefore, for all $\delta$ sufficiently small,
\begin{align*}
&B_\delta e^{-{\mathsf{J}}(\vct u_1) - \epsilon}
			\leq
			\int_{v \in B(u_1,\delta)} e^{-{\mathsf{J}}(\vct v)} d\vct v \leq
			 B_\delta e^{-{\mathsf{J}}(\vct u_1) + \epsilon}, \\
&B_\delta e^{-{\mathsf{J}}(\vct u_2) - \epsilon}
			\leq
			\int_{v \in B(u_2,\delta)} e^{-{\mathsf{J}}(\vct v)} d\vct v \leq
			 B_\delta e^{-{\mathsf{J}}(\vct u_2) + \epsilon},
\end{align*}
		where \(B_\delta \) is the Lebesgue measure of a ball with radius \(\delta\). Taking the ratio of $\alpha$'s and using the above bounds we obtain that, for all $\delta$ sufficiently small,
		\[
			e^{{\mathsf{J}}(\vct u_2) - {\mathsf{J}}(\vct u_1) -2\epsilon} \leq \frac{\alpha(\vct u_1, \delta) }{\alpha(\vct u_2, \delta)} \leq  e^{{\mathsf{J}}(\vct u_2) - {\mathsf{J}} ( \vct u_1) + 2\epsilon}.
			\]
			Since $\epsilon$ was arbitrary the desired result follows.

		\end{proof}

\begin{remark}
This theorem shows that maximizing the probability of an
infinitesimally small ball is the same as minimizing the objective
function ${\mathsf{J}}(\cdot).$ This is obvious in finite dimensions, but
the proof above generalizes beyond measures
which possess a Lebesgue density, and may be used in infinite
dimensions.
\end{remark}

\subsection{Examples}

By means of examples we now probe the question of whether or not the MAP
estimator captures useful information about the posterior distribution.

\begin{example}
	If the posterior is single-peaked, such as a Gaussian or a Laplace distribution, as shown in Figure \ref{fig:gaussian-laplace}, the MAP estimator, i.e. minimizer of the objective functions, reasonably summarizes the most likely value of the unknown parameter.
\end{example}

\begin{figure}
  \centering
  \includegraphics[width=0.7\columnwidth]{figures/lecture4_gaussian-laplace.png}
  \caption{Posterior (left) and objective function (right) for \(\mathcal{N}(0,1)\) posterior (orange) and \(\mathrm{Laplace}(0,1)\) posterior (blue). }
  \label{fig:gaussian-laplace}
\end{figure}

We next consider several examples where a point estimator ---or a \(\delta\)-radius ball with \(\delta \rightarrow 0\)--- fails to summarize adequately the posterior distribution.
\begin{example}
	If the posterior is rather unevenly distributed, such as a slab-and-spike distribution, as shown in Figure~\ref{fig:slab-n-spike}, then it is
less clear that the MAP estimator usefully summarizes the posterior. For example, for the case in Figure \ref{fig:slab-n-spike} we may want the solution output of our Bayesian problem to be a weighted average of two Gaussian distributions, or two point estimators each with a separate
mean located at one of the two minima of the objective functions,
and weight describing the probability mass associated with each of
those two points.
\end{example}

\begin{figure}
	\centering
	\includegraphics[width=0.9\columnwidth]{figures/lecture4_slab-n-spike.png}
	\caption{Posterior (left) and objective function (right) for a posterior that is a sum of two Gaussian distributions, \(\mathcal{N}(0,0.1^2)\) with probability \(0.1\) and \(\mathcal{N}(10,3^2)\) with probability \(0.9\).}
	\label{fig:slab-n-spike}
\end{figure}

\begin{example}
	In addition to a multiple-peak posterior, there are cases where the objective function and the associated posterior density are simply very rough. In these cases, the small-scale roughness should be ignored, while the large-scale variation should be captured. For example in Figure \ref{fig:rough}, the objective function is very rough, and has a unique minimizer at a point far from \(0\). However, it also has a larger pattern: it tends to be smaller around \(0\), while larger away from \(0\). The MAP estimator cannot capture this large scale pattern, as it is found by minimizing the objective function. It is arguably the
case that \(u=0\) is a better point estimate. An alternative way to interpret this phenomena is that there is a natural ``temperature'' to this problem, in the sense that variations lower than this temperature could be viewed as random noise that do not capture meaningful information.
\end{example}
\begin{figure}
	\centering
	\includegraphics[width=0.9\columnwidth]{figures/lecture4_rough.png}
	\caption{Posterior (left) and objective function (right) from an objective function that is very rough in the small scale, but contains a regular patter on the larger scale. This specific example is generated by white noise summed with a quadratic function for the objective function, and the posterior is computed from the objective function.}
	\label{fig:rough}
\end{figure}

The preceding examples suggest that multi-peak distributions, or multi-minimum
objective functions, can cause problems for MAP estimation as a point estimator.
Next we illustrate that if the dimension \(d\) of the parameter \(\vct u \in \mathbbm{R}^\Du\) is high, then a single point estimator, even if a MAP estimator, is typically not a good summary of the posterior.
\begin{figure}[ht!]
	\centering
	\includegraphics[width=0.7\columnwidth]{figures/lecture4_highdim.png}
	\caption{Empirical density of \(\ell_2\) norm of \(\mathcal{N}(\zerovct,\Id)\) random vectors for various dimension: \(d=1\) (blue), \(d=5\) (orange), \(d=10\) (green), \(d=50\) (red), and \(d=100\) (purple). The empirical density is obtained from \(10000\) samples for each distribution.}
	\label{fig:highdim}
\end{figure}
\begin{example}
	We consider what is the ``typical size'' of a vector \(\vct u\) drawn
from the standard Gaussian distribution \(\mathcal{N}(\zerovct, \Id)\), as the dimension
increases. In Figure \ref{fig:highdim} we display the empirical density of the norm of such random vectors. We can see that at low dimensions, such as when \(d=1\), obtaining a value close to the mode \(\vct u=0\) is highly likely. In higher dimensions, however,
the probability for a vector from this distribution to have a small $\ell_2-$norm becomes increasingly small as $d$ grows.
 For example, let us consider the probability for the norm to be less than \(5\). Then \(\operatorname{\mathbbm{P}} (|\vct u| <5)\) is \(0.99999943\) when \(d=1\), \(0.99986\) when \(d=5\), \(0.99465\) when \(d=10\), \(0.001192\) when \(d=50\), and \(1.135 \times 10^{-15}\) when \(d=100\). So we see that, as the dimension increases, with probability close to \(1\) a sample from the posterior would
have a norm far from \(0\). Indeed, for \(d=1000\), the \(5\)th and \(95\)th percentiles are respectively \(30.3464\) and \(32.7823\). This means when \(d=1000\), we most likely will find a vector with size around \(31\), not \(0\).
Another way to see this is that, since the components $u_i$ of $u$
are i.i.d. standard unit Gaussians we have that, by the strong law of large
numbers,
$$\frac{1}{d}\sum_{i=1}^d u_i^2 \to 1$$
as $d \to \infty$ almost surely. Thus, with high probability, the
$\ell_2-$norm is of size $\sqrt{d}.$
This example suggests that in high dimension, a point estimatormay not capture enough information about the density.
\end{example}

The preceding examples demonstrate that MAP estimators should be treated
with caution as they may not capture the desired posterior information in
many cases. This motivates the study of alternative ways ---beyond MAP estimators--- to capture information from the posterior distribution. One such approach is to fit one or several Gaussian distributions to the posterior by minimizing an appropriate distance-like measure between distributions. This will be discussed in the next chapter.

\subsection{Discussion and Bibliography}

The optimization perspective on inversion predates the development of
the Bayesian approach as a computational tool, because it is typically
far cheaper to implement. The subject of classical regularization
techniques for inversion is discussed in \cite{engl1996regularization}.
The concept of MAP estimators, which links probability to
optimization, is discussed in the books
\cite{kaipio2006statistical,tarantola2005inverse}
in the finite dimensional setting.
The paper \cite{dashti2013map} studies this connection precisely: it
defines the MAP estimator for infinite dimensional Bayesian inverse problems, and the corresponding variational formulation, in a Gaussian setting.
The paper \cite{helin2015maximum} studies related ideas, but in the
non-Gaussian setting, and \cite{agapiou2017sparsity} generalizes
the variational formulation of MAP estimators to non-Gaussian priors that
are sparsity promoting.
The paper \cite{worthen2014towards} shows an application of optimization
based inversion to a large-scale geophysical application.

\section{\Large{\sffamily{Bayesian Inverse Problems and Well-Posedness}}}
In this chapter we introduce the Bayesian approach to inverse problems.  We will show that the Bayesian formulation leads to a form of well-posedness: small perturbations of the forward model or the observed data translate into small perturbations of the Bayesian solution. Well-posedness will be established in Hellinger distance. The total variation and Hellinger distances between probability densities play an important role in the analysis of Bayesian methodology. We introduce both distances in this chapter, as well as some characterizations and bounds between them that will be used throughout this book.

\subsection{Formulation of Bayesian Inverse Problems}
We consider the following setting. We let $G:  \mathbbm{R}^\Du \to \mathbbm{R}^\Dy$ be a forward model and aim to recover an unknown parameter $u \in \mathbbm{R}^\Du$ from data $y \in \mathbbm{R}^\Dy$ given by
\begin{equation}
 y = G(u) + \eta,
\label{eq:jc0}
\end{equation}
where $\eta \in \mathbbm{R}^\Dy$ represents observation noise.
We view $(u, y ) \in \mathbbm{R}^\Du \times \mathbbm{R}^\Dy$ as a random variable, whose distribution is specified by means of the following assumption.
\begin{assumption}\label{a:jc1}
We make the following probabilistic assumptions:
 \begin{itemize}
 \item $u \sim \rho(u), u \in \mathbbm{R}^\Du $.
 \item $\eta \sim \nu(\eta), \eta \in \mathbbm{R}^\Dy$.
 \item $ u$ and $\eta$ are independent, written $u \perp \eta.$
 \end{itemize}
\end{assumption}
Here $\rho$ and $\nu$ describe the (Lebesgue) probability density
functions (pdfs) of the random variables $u$ and $\eta$ respectively.
Then $\rho(u)$ is called the \emph{prior} pdf, $y |  u \sim \nu\bigl(y - G(u)\bigr)$ for each fixed $u\in \mathbbm{R}^\Du$ determines the \emph{likelihood} function.  In this
probabilistic perspective, the solution to the inverse problem is the conditional distribution of $u$ given $y$, which is called the \emph{posterior} distribution and will be denoted by $u | y\sim \pi^y(u).$  From the posterior pdf one can infer parameter values that are consistent with both the data and the prior pdf. The posterior also contains information on the uncertainty remaining in the parameter recovery: for instance, large posterior covariance may indicate that the data contains insufficient information to recover the input parameter.

 \subsection{Formula for Posterior pdf: Bayes Theorem}
Bayes theorem is a bridge connecting the prior, the likelihood and the posterior.
 \begin{theorem}[Bayes theorem]
\label{t:bayes}
 Let Assumption \ref{a:jc1} hold and assume that
\[
 Z = Z(y) :=\int_{\mathbbm{R}^\Du} \nu\bigl( y - G(u)\bigr)\rho(u) du  >0.\]
 Then $u | y\sim \pi^y(u),$ where
\begin{equation}\label{eq:bayesformula}
 \pi^y(u) = \frac{1}{Z} \nu\bigl(y - G(u)\bigr)\rho(u).
\end{equation}
 \end{theorem}
 \begin{proof}
 Denote by $\operatorname{\mathbbm{P}} (\cdot)$ the pdf of a random variable and by $\operatorname{\mathbbm{P}}(\cdot | \cdot )$ its conditional conditional pdf.
We have
 \[
 \begin{aligned}
 \operatorname{\mathbbm{P}}(u , y) &=  \operatorname{\mathbbm{P}} (u | y) \operatorname{\mathbbm{P}}(y),  \textrm{ if } \operatorname{\mathbbm{P}}(y) > 0 ,\\
 \operatorname{\mathbbm{P}}(u, y) &=  \operatorname{\mathbbm{P}} (y | u ) \operatorname{\mathbbm{P}}(u),  \textrm{ if } \operatorname{\mathbbm{P}}(u) > 0 .\\
\end{aligned}
 \]
 Note that the marginal pdf on $y$ is given by
 \[
 \operatorname{\mathbbm{P}}(y) = \int_{\mathbbm{R}^\Du} \operatorname{\mathbbm{P}}( u, y) du,
 \]
and similarly for $\operatorname{\mathbbm{P}}(u).$
 Assume $\operatorname{\mathbbm{P}}(y) >0 $. Then
 \begin{equation}
\label{eq:jc1}
 \operatorname{\mathbbm{P}}(u | y) = \frac{1}{\operatorname{\mathbbm{P}}(y) } \operatorname{\mathbbm{P}}(y | u) \operatorname{\mathbbm{P}}(u)   = \frac{1}{Z} \nu\bigl(y - G(u )\bigr) \rho(u)
 \end{equation}
 for both $\operatorname{\mathbbm{P}}(u) > 0$ and $\operatorname{\mathbbm{P}}(u) = 0$. Here we remark that
 \[
\operatorname{\mathbbm{P}}(y)=Z=\int_{\mathbbm{R}^\Du} Z \operatorname{\mathbbm{P}}(u | y) du = \int_{\mathbbm{R}^\Du} \nu\bigl(y - G(u)\bigr) \rho(u) du>0.
 \]
Thus the assumption that $\operatorname{\mathbbm{P}}(y) >0 $ is justified and the desired result
follows from equation \eqref{eq:jc1}.
\end{proof}

We will often denote the likelihood function by $g(u) := \nu \bigl(y - G(u)\bigr)$; we then write
$$ \pi(u) = \frac{1}{Z} g(u) \rho(u),$$
omitting the data $y$ in both the likelihood function and the posterior pdf.

\begin{remark}
The proof of Theorem \ref{t:bayes} shows that in order to apply Bayes formula \eqref{eq:bayesformula} one needs to guarantee that the normalizing constant $\operatorname{\mathbbm{P}}(y) = Z$ is positive. In other words, the marginal density of the observed data $y$ needs to be positive, i.e. the observed data needs to be consistent with the probabilistic assumptions \eqref{a:jc1}. From now on it will be assumed that $\operatorname{\mathbbm{P}}(y) = Z >0$ without further notice.
\end{remark}

The posterior distribution $\pi^y(u)$ contains all the knowledge on the parameter $u$ available in the prior and the data. In applications it is often useful, however, to summarize the posterior distribution through a few numerical values or through parameter regions of prescribed posterior probability (known as credible intervals). Summarizing the posterior is particularly important if the parameter is high dimensional, since then visualizing the posterior or detecting regions of high posterior probability is nontrivial. Two natural numerical summaries are the posterior mean and the posterior mode.
\begin{definition}\label{def:map}
The {\em posterior mean estimator} of $u$ given data $y$ is the mean of the posterior distribution:
$$u_{\mbox {\tiny{\rm PM}}} = \int_{\mathbbm{R}^\Du} u \pi^y(u) \, du.$$
	The {\em maximum a posteriori (MAP) estimator} of $u$ given data $y$ is the mode of the
posterior distribution $\pi^{\vct{y}}(\vct{u})$,  defined as
	$$u_{\mbox {\tiny{\rm MAP}}} =\arg\max_{\vct{u}\in\mathbbm{R}^\Du}\pi^{\vct{y}}(\vct{u}).$$
\end{definition}

The posterior mean and the MAP estimators already suggest the importance of computing maxima and integrals in the practical implementation of Bayesian formulations of inverse problems and data assimilation. For this reason, optimization and sampling will play an important role in this book.  An alternative way to make Bayesian formulations tractable is to approximate the posterior by a simple distribution, often a Gaussian or a combination of Dirac masses. An optimization perspective for inverse problems and data assimilation will be studied in Chapters 3 and 7, respectively, and the use of Gaussian approximations will be discussed in Chapters 4 and 10; Dirac approximations constructed via sampling will be studied in Chapters 5 and 6 (inverse problems) and in Chapters 11 and 12 (data assimilation).

We next consider two simple examples of a direct application of Bayes formula.

\begin{example}
Let $d=k=1$, $\eta \sim \nu = \mathcal{N}(0, \gamma^2)$ and
\[
\rho(u) = \begin{cases}
\frac{1}{2} ,  &  u\in (-1,1), \\
0 ,  &  u\in (-1,1 )^c. \\
\end{cases}
\]
Suppose that the observation is generated by $y = u + \eta$. Using Bayes Theorem \ref{t:bayes}, we derive the posterior
\[
\pi^y(u) =
 \begin{cases}
 \frac{1}{2Z} \exp(  -\frac{1}{2\gamma^2}|y- u |^2   ),  &  u \in (-1,1) , \\
 0 ,  & u \in (-1,1)^c , \\
 \end{cases}
\]
where $Z$ is a normalizing constant ensuring $\int_{\mathbbm{R}} \pi^y(u) du = 1$. The support of $\pi^y$, i.e. $(-1,1)$,  agrees with the support of the prior $\rho$. Now we find
the MAP estimator.
From the explicit formula for $\pi^y$, we have
\[
u_{\mbox {\tiny{\rm MAP}}}  = \arg\max_{u \in \mathbbm{R}} \pi^y(u) = \begin{cases}
y &  \textrm{if } y \in (-1, 1),  \\
-1&  \textrm{if } y  \leq - 1 , \\
1 & \textrm{if } y\geq 1 . \\
\end{cases}
\]
In this example, the prior on $u$ is supported
on $(-1,1)$ and the posterior on $u|y$ is supported on $(-1,1)$.
If the data lies in $(-1,1)$ then the MAP estimator is the data itself; otherwise it is the extremal point of
the prior support which matches the sign of the data. The posterior mean is
$$u_{\mbox {\tiny{\rm PM}}} = \frac{1}{2Z} \int_{-1}^1 u \exp\Bigl( -\frac{1}{2\gamma^2} |y - u|^2 \Bigr) \, du,$$
which may be  approximated for instance by using the sampling methods described in Chapters 5 and 6.

\end{example}

The following example illustrates once again the application of Bayes formula, and shows that the posterior may be concentrated near a low-dimensional manifold of the input parameter space $\mathbbm{R}^d$. In such a case it is important to understand the geometry of the support of the posterior density, which cannot be captured by point estimation or Gaussian approximations.

\begin{example}
Let $d =2, k=1, \rho \in C(\mathbbm{R}^2, \mathbbm{R}^+), 0 < \rho(u) \leq \rho_{\max} < +\infty$, for all  $u \in \mathbbm{R}^2$ and
 \[
 y = G(u) + \eta = u_1^2 + u_2^2 + \eta , \quad \eta \sim \nu = \mathcal{N}(0, \gamma^2),
\quad 0< \gamma \ll 1.
 \]
Assume that $y > 0$. Using Bayes theorem, we obtain
the posterior distribution
\[
\pi^y(u) = \frac{1}{Z} \exp\left(  - \frac{1}{2 \gamma^2}| u_1^2 + u_2^2 - y|^2   \right) \rho(u).
\]
We now show that the posterior concentrates on a manifold: the circumference  $\{ u \in \mathbbm{R}^2: u_1^2 + u_2^2=y\}.$
Denote by $A^{\pm}:=\{ u \in \mathbbm{R}^2 :  | u_1^2 + u_2^2 - y |^2 \leq \gamma^{2 \pm \delta}   \}$, for some $\delta \in (0,2)$, and let $\rho_{\min} = \inf_{u\in B} \rho(u)$, where $B$ is the closed ball of radius $2\sqrt{y}$ centered at the origin. Since $\rho(u)$ is positive and continuous and $B$ is compact, $\rho_{\min} > 0$. Let $u^+ \in A^+ \subset B, u^- \in (A^-)^c$. Taking the small noise limit yields
 \[
 \frac{\pi^y(u^+)}{ \pi^y(u^-) } \geq \exp\left( -\frac{1}{2} \gamma^{\delta} + \frac{1}{2} \gamma^{-\delta} \right)  \frac{\rho_{\min}}{\rho_{\max}}\to \infty, \ \textrm{as } \gamma \to 0^+.
 \]
Therefore, conditional on $y>0$, the posterior $\pi^y$ concentrates, as $\gamma \to 0,$ on the circumference with radius $\sqrt{y}$.
  \begin{figure}[h]
   \centering
   \includegraphics[width =0.45\textwidth, height = 0.4\textwidth]{figures/lecture1_cartoon.png}
   \caption{The posterior measure concentrates on a circumference with radius $\sqrt{y}$. Here, the blue shadow area is $A^+$ and the green shadow area is $(A^-)^c$. }
 \end{figure}
\end{example}

\subsection{Well-posedness of Bayesian Inverse Problems}

In this section we show that the Bayesian formulation of inverse problems
leads to a form of well-posedness.
More precisely, we study the sensitivity of the posterior pdf to perturbations of the forward model $G.$  In many inverse problems the ideal forward model $G$ is not accessible but can be approximated by some computable $G_\delta;$ consequently $\pi^y$ is replaced by $\pi_\delta^y$. An example that is often found in applications, to which the theory contained herein may be generalized,
is when  $G$ is an operator acting on an infinite-dimensional space
which is approximated, for the purposes of computation, by some finite-dimensional operator $G_\delta$. We seek to prove that, under certain assumptions, the small difference between $G$ and $G_\delta$ (forward error) leads to
similarly small difference between $\pi^y$ and $\pi^y_\delta$
(inverse error):

\vspace{0.1in}
\noindent{\bf Meta Theorem: Well-posedness}\\
       \[|G-G_\delta|=O(\delta)\ \Longrightarrow \ d(\pi^y,\pi_\delta^y)=O(\delta),\]
{\em for small enough $\delta>0$ and some metric $d(\cdot,\cdot)$ on probability densities.}
\vspace{0.1in}

This result will be formalized in Theorem \ref{thm1} below, which  shows that the $O(\delta)$-convergence of $\pi_\delta^y$ with respect to some distance $d(\cdot,\cdot)$ can be guaranteed under certain assumptions on the likelihood. We will conclude the chapter by showing an example where these assumptions hold true. In order to discuss these issues we will need to
introduce metrics on probability densities.

\subsubsection{Metrics on Probability Densities}
Here we introduce the total variation and the Hellinger distance, both of which have been used to show well-posedness results. In this chapter we will use the Hellinger distance to establish well-posedness of Bayesian inverse problems, and in Chapter 7 we employ the total variation distance to establish well-posedness of Bayesian formulations of filtering and smoothing in data assimilation.

\begin{definition}
The \emph{total variation distance} between two probability densities $\pi,\pi'$ is defined by
\[d_{\mbox {\tiny{\rm TV}}} (\pi,\pi')\coloneqq\frac{1}{2}\int|\pi(u)-\pi'(u)|du=\frac{1}{2}\|\pi-\pi'\|_{L^1}.\]
The \emph{Hellinger distance} between two probability densities $\pi,\pi'$ is defined by
\[d_{\mbox {\tiny{\rm H}}}(\pi,\pi')\coloneqq\Big(\frac{1}{2}\int|\sqrt{\pi(u)}-\sqrt{\pi'(u)}|^2du\Big)^{1/2}=\frac{1}{\sqrt{2}}\|\sqrt{\pi}-\sqrt{\pi'}\|_{L^2}.\]
\end{definition}

In the rest of this subsection we will establish bounds between the Hellinger and total variation distance, and show how both distances can be used to bound the difference of expected values computed with two different densities; these results will be used in subsequent chapters.  Before doing so, the next lemma motivates our choice of normalization constant $1/2$ for total variation distance and $1/\sqrt{2}$ for Hellinger distance: they are chosen so that the maximum possible distance between two densities is one.

      \begin{lemma} \label{l:dh1}
For any probability densities $\pi,\pi'$,
       \[0\leq d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')\leq 1,\quad 0\leq d_{\mbox {\tiny{\rm H}}}(\pi,\pi')\leq 1.\]
      \end{lemma}
      \begin{proof} The lower bounds follow immediately from the
definitions. We only need to prove the upper bounds:
      \[d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')=\frac{1}{2}\int|\pi(u)-\pi'(u)|du\leq \frac{1}{2}\int\pi(u)du+\frac{1}{2}\int\pi'(u)du=1, \]
      \begin{align*}
      d_{\mbox {\tiny{\rm H}}}(\pi,\pi')=&\ \biggl(\frac{1}{2}\int \Bigl|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Bigr|^2du\biggr)^{1/2}\\
      =&\ \bigg(\frac{1}{2}\int\Big(\pi(u)+\pi'(u)-2\sqrt{\pi(u)\pi'(u)}\,\, \Big)du \bigg)^{1/2}\\
      \leq&\ \Big(\frac{1}{2}\int\big(\pi(u)+\pi'(u)\big)du\Big)^{1/2}\\
      =&\ 1.\end{align*}
      \end{proof}
   The proofs also show that $\pi$ and $\pi'$ have total variation and Hellinger distance equal to one if and only if they have disjoint supports, that is, if $\int \pi(u)\pi'(u) du=0.$

The following result gives bounds between total variation and Hellinger distance.

      \begin{lemma} \label{l:dh2} For any probability densities $\pi,\pi'$,
       \[\frac{1}{\sqrt{2}}d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')\leq d_{\mbox {\tiny{\rm H}}}(\pi,\pi')\leq \sqrt{d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')}.\]
      \end{lemma}
      \begin{proof} Using the Cauchy–Schwartz inequality
      \begin{align*}
      d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')=&\ \frac{1}{2}\int\Big|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Big|\Big|\sqrt{\pi(u)}+\sqrt{\pi'(u)}\Big|du\\
      \leq&\ \biggl(\frac{1}{2}\int\Big|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Big|^2du\biggr)^{1/2}\biggl(\frac{1}{2}\int\Big|\sqrt{\pi(u)}+\sqrt{\pi'(u)}\Big|^2du\biggr)^{1/2}\\
      \leq&\ d_{\mbox {\tiny{\rm H}}}(\pi,\pi')\biggl(\frac{1}{2}\int\big(2\pi(u)+2\pi'(u)\big)du\biggr)^{1/2}\\
      =&\ \sqrt{2}d_{\mbox {\tiny{\rm H}}}(\pi,\pi').
      \end{align*}
      Notice that $|\sqrt{\pi(u)}-\sqrt{\pi'(u)}|\leq |\sqrt{\pi(u)}+\sqrt{\pi'(u)}|$ since $\sqrt{\pi(u)},\sqrt{\pi'(u)}\geq0$. Thus we have
      \begin{align*}
      d_{\mbox {\tiny{\rm H}}}(\pi,\pi')=&\ \biggl(\frac{1}{2}\int\Big|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Big|^2du\biggr)^{1/2}\\
      \leq&\ \biggl(\frac{1}{2}\int\Big|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Big|\Big|\sqrt{\pi(u)}+\sqrt{\pi'(u)}\Big|du\biggr)^{1/2}\\
      \leq&\ \Big(\frac{1}{2}\int\big|\pi(u)-\pi'(u)\big|du\Big)^{1/2}\\
      =&\ \sqrt{d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')}.
      \end{align*}
      \end{proof}

The following two lemmas show that if two densities are close in total variation or in Hellinger distance, expectations computed with respect to both densities are also close. In addition, the following lemma also provides a useful characterization of the total variation distance which will be used repeatedly throughout this book.

      \begin{lemma} \label{lemmatv} Let $f$ be a function such that $\sup_{u\in \mathbbm{R}^\Du}|f(u)| =: |f|_\infty<\infty$, then
      \[\big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\big|\leq 2|f|_\infty d_{\mbox {\tiny{\rm TV}}}(\pi,\pi').\]
Moreover, the following variational characterization of the total variation distance holds:
\begin{equation}
d_{\mbox {\tiny{\rm TV}}}(\pi, \pi') = \frac12 \sup_{|f|_\infty<1} \big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\big|.
\end{equation}
      \end{lemma}
      \begin{proof}
      For the first part of the lemma, note that
     	\begin{align*}
     	\Big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\Big|=&\ \Big|\int_{\mathbbm{R}^\Du}f(u)\bigl(\pi(u)-\pi'(u)\bigr)du\Big|\\
     	\leq&\ 2|f|_\infty\cdot\frac{1}{2}\int_{\mathbbm{R}^\Du}|\pi(u)-\pi'(u)|du\\
     	=&\ 2|f|_\inftyd_{\mbox {\tiny{\rm TV}}}(\pi,\pi').
     	\end{align*}
     	This in particular shows that, for any $f$ with $|f|_\infty =1,$
     	 $$d_{\mbox {\tiny{\rm TV}}}(\pi, \pi') \ge \frac12 \big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\big|.$$
     Our goal now is to show a choice of $f$ with $|f|_\infty =1$ that achieves equality.	Define $f(u):= \text{sign} \Bigl( \pi(u)- \pi'(u) \Bigr)$, so that $f(u) \Bigl( \pi(u)- \pi'(u) \Bigr) = |\pi(u) - \pi'(u)|.$ Then, $|f|_\infty =1$ and
     	\begin{align*}
     	d_{\mbox {\tiny{\rm TV}}}(\pi, \pi') &= \frac12 \int_{\mathbbm{R}^\Du} |\pi(u) - \pi'(u)| \, du \\
     	&= \frac12 \int_{\mathbbm{R}^\Du} f(u) \Bigl( \pi(u) - \pi'(u) \Bigr) \, du \\
     	& = \frac12 \Big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\Big|.
     	\end{align*}
     	This completes the proof of the variational characterization.
      \end{proof}

      \begin{lemma} \label{Lemma: hellinger} Let $f$ be a function such that $\operatorname{\mathbb{E}}^\pi[|f|^2]+\operatorname{\mathbb{E}}^{\pi'}[|f|^2]=: f_2^2< \infty$, then
      \[\big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\big|\leq 2 f_2 d_{\mbox {\tiny{\rm H}}}(\pi,\pi').\]
      \end{lemma}
      \begin{proof}
     	\begin{align*}
     	\Big|\operatorname{\mathbb{E}}^\pi[f]-\operatorname{\mathbb{E}}^{\pi'}[f]\Big|=&\ \biggl|\int_{\mathbbm{R}^\Du}f(u)\Bigl(\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Bigr)\Bigl(\sqrt{\pi(u)}+\sqrt{\pi'(u)} \, \Bigr) \,du\biggr|\\
     	\leq&\ \biggl(\frac{1}{2}\int \Big|\sqrt{\pi(u)}-\sqrt{\pi'(u)}\Big|^2du\biggr)^{1/2} \biggl(2\int|f(u)|^2\Big|\sqrt{\pi(u)}+\sqrt{\pi'(u)}\Big|^2 \,du\biggr)^{1/2}\\
     	\le &\ d_{\mbox {\tiny{\rm H}}}(\pi,\pi')\Big(4\int|f(u)|^2\bigl(\pi(u)+\pi'(u)\bigr)du\Big)^{1/2}\\
     	= &\ 2f_2 \,d_{\mbox {\tiny{\rm H}}}(\pi,\pi').
     	\end{align*}
      \end{proof}

     Note  that the result for Hellinger only assumes that $f$ is square integrable with respect to $\pi$ and $\pi'$. In contrast, the result for total variation distance assumes that $f$ is bounded, which is a stronger condition.

\subsubsection{Approximation Theorem}
We denote by
\[g(u)=\nu\bigl(y-G(u)\bigr)\quad \text{and}\quad g_\delta(u)=\nu\bigl(y-G_\delta(u)\bigr)\]
the likelihoods associated with $G(u)$ and $G_\delta(u),$ so that
\[\pi^y(u)=\frac{1}{Z}g(u)\rho(u)\quad \text{and}\quad \pi_\delta^y(u)=\frac{1}{Z_\delta}g_\delta(u)\rho(u),\]
where $Z, Z_\delta>0$ are the corresponding normalizing constants. Before we proceed to our main result we make some assumptions: \\
\begin{assumption}\label{a:dh1}
There exist $\delta^+>0$ and $K_1,K_2<\infty$ such that, for all $\delta\in (0,\delta^+),$
	\begin{itemize}
	\item[(i)] $|\sqrt{g(u)} - \sqrt{g_\delta(u)}|\leq \phi(u)\delta$, for some $\phi(u)$
such that $\operatorname{\mathbb{E}}^{\rho} [\phi^2(u)] \le K_1$;
	\item[(ii)] $\sup_{u\in\mathbbm{R}^\Du}(|\sqrt{g(u)}|+|\sqrt{g_\delta(u)}|) \le K_2.$
	\end{itemize}
\end{assumption}

\begin{remark}
Assumption \ref{a:dh1} only involves conditions on the likelihood $g$ and the approximate likelihood $g_\delta.$ While our presentation in this chapter emphasizes that this approximation may arise due to the need of approximating the forward model $G,$ another important scenario that the theory covers is approximation due to perturbations of the data $y.$ Well-posedness results that guarantee the stability under data perturbations of Bayesian data assimilation will be established in Chapter 7.
\end{remark}

Now we state the main result of this section:
\begin{theorem}[Well-posedness of Posterior] Under Assumption \ref{a:dh1} we have
      \[d_{\mbox {\tiny{\rm H}}}(\pi^y,\pi_\delta^y)\leq c \delta,\quad \delta\in(0,\tilde{\delta}^+),\]
      for some $\tilde{\delta}^+>0$ and some $c\in(0,+\infty)$ independent of $\delta$.
      \label{thm1}\\
\end{theorem}

Notice that this theorem together with Lemma \ref{Lemma: hellinger} guarantee that expectations computed with respect to $\pi^y$ and $\pi_\delta^y$ are order $\delta$ apart.
To prove Theorem \ref{thm1}, we first show a lemma which characterizes
the normalization factor $Z_\delta$ in the small $\delta$ limit.

     \begin{lemma}\label{lem1} Under Assumption \ref{a:dh1} there exist $ \tilde{\delta}^+>0$, $c_1,c_2\in(0,+\infty)$ such that
      \[|Z-Z_\delta|\leq c_1\delta\quad \text{and}\quad Z,Z_\delta>c_2,\quad \text{for}\ \delta\in(0,\tilde{\delta}^+).\]
      \end{lemma}
      \begin{proof}
Since $Z=\int g(u)\rho(u)du$ and
$Z_\delta=\int g_\delta(u)\rho(u)du$
we have
      \begin{align*}
      |Z-Z_\delta|=&\ \int \bigl(g(u)-g_\delta(u)\bigr)\rho(u)du\\
      \leq&\ \Big(\int \Bigl|\sqrt{g(u)}- \sqrt{g_\delta(u)}\Bigr|^2\rho(u)du\Big)^{1/2}\Big(\int \Bigl|\sqrt{g(u)}+\sqrt{g_\delta(u)}\Bigr|^2\rho(u)du\Big)^{1/2}\\
      \leq&\ \Big(\int \delta^2 \phi(u)^2\rho(u)du\Big)^{1/2}\Big(\int K_2^2\rho(u)du\Big)^{1/2}\\
      \leq&\ \sqrt{K_1}K_2\delta,\quad \delta\in(0,\delta^+).
      \end{align*}
      And when $\delta\leq \tilde{\delta}^+\coloneqq\min\{\frac{Z}{2\sqrt{K_1}K_2},\delta^+\}$, we have
      \[Z_\delta\geq Z-|Z-Z_\delta|\geq \frac{1}{2}Z.\]
      The lemma follows by taking $c_1=\sqrt{K_1}K_2$ and $c_2=\frac{1}{2}Z$.\\
      \end{proof}

      \begin{proof}[Proof of Theorem \ref{thm1}]
      We break the distance into two error parts, one caused by the difference between $Z$ and $Z_\delta$, the other caused by the difference between $g$ and $g_\delta$:
      \begin{align*}
      d_{\mbox {\tiny{\rm H}}}(\pi^y,\pi_\delta^y)=&\ \frac{1}{\sqrt{2}}\Bigl\|\sqrt{\pi^y}-\sqrt{\pi_\delta^y}\Bigr\|_{L^2}\\
      =&\  \frac{1}{\sqrt{2}}\Big\|\sqrt{\frac{g\rho}{Z}}-\sqrt{\frac{g\rho}{Z_\delta}}+\sqrt{\frac{g\rho}{Z_\delta}}-\sqrt{\frac{g_\delta\rho}{Z_\delta}}\Big\|_{L^2}\\
      \leq&\ \frac{1}{\sqrt{2}}\Big\|\sqrt{\frac{g\rho}{Z}}-\sqrt{\frac{g\rho}{Z_\delta}}\Big\|_{L^2}+\frac{1}{\sqrt{2}}\Big\|\sqrt\frac{g\rho}{Z_\delta}-\sqrt{\frac{g_\delta\rho}{Z_\delta}}\Big\|_{L^2}.
      \end{align*}
      Using Lemma \ref{lem1}, for $\delta\in(0,\tilde{\delta}^+)$, we have
      \begin{align*}
      \Big\|\sqrt{\frac{g\rho}{Z}}-\sqrt{\frac{g\rho}{Z_\delta}}\Big\|_{L^2}=&\ \Big|\frac{1}{\sqrt{Z}}-\frac{1}{\sqrt{Z_\delta}}\Big|\Big(\int g(u)\rho(u)du\Big)^{1/2}\\
      =&\ \frac{|Z-Z_\delta|}{(\sqrt{Z}+\sqrt{Z_\delta})\sqrt{Z_\delta}}\\
      \leq&\ \frac{c_1}{2c_2}\delta,
      \end{align*}
      and
      \[\Big\|\sqrt{\frac{g\rho}{Z_\delta}} - \sqrt{\frac{g\rho}{Z_\delta}}\Big\|_{L^2}=\frac{1}{\sqrt{Z_\delta}}\Big(\int\Bigl|\sqrt{g(u)}- \sqrt{g_\delta(u)}\Bigr|^2\rho(u) du\Big)^{1/2} \leq \sqrt{\frac{K_1}{c_2}}\delta.\]
      Therefore
      \[d_{\mbox {\tiny{\rm H}}}(\pi^y,\pi_\delta^y)\leq \frac{1}{\sqrt{2}}\frac{c_1}{2c_2}\delta+\frac{1}{\sqrt{2}}\sqrt{\frac{K_1}{c_2}}\delta=c \delta,\]
      with $c=\frac{1}{\sqrt{2}}\frac{c_1}{2c_2}+\frac{1}{\sqrt{2}}\sqrt{\frac{K_1}{c_2}}$ independent of $\delta$.
      \end{proof}

\subsubsection{Example}
Many inverse problems arise from differential equations with unknown input parameters. Here we consider a simple but typical example where $G(u)$ comes from the solution of an ODE, which needs to be solved numerically. Let $x(t)$ be the solution to the initial value problem
\begin{equation}
\label{eq:dh1}
\frac{d x}{dt}=F(x;u),\quad x(0)=0,
\end{equation}
where $F:\mathbbm{R}^\Dy\times \mathbbm{R}^\Du\rightarrow \mathbbm{R}^\Dy$ is a function such that $F(x;u)$ and the partial Jacobian $D_xF(x;u)$ are uniformly bounded with respect to $(x,u)$, i.e.
\[|F(x;u)|,|D_xF(x;u)|< F_{{\rm max}}  ,\quad \text{ for all } (x,u)\in \mathbbm{R}^\Dy\times \mathbbm{R}^\Du,\]
for some constant $F_{{\rm max}}$, and thus $F(x;u)$ is Lipschitz in $x$ in that
\[|F(x;u)-F(x';u)|\leq F_{{\rm max}} |x-x'|,\quad \text{ for all } x,x'\in \mathbbm{R}^\Dy.\]

Now consider the inverse problem setting
$$y=G(u)+\eta,$$
where
$$G(u):=x(1)=x(t)|_{t=1},$$
and $\eta\sim \mathcal{N}(0,\gamma^2 I_k)$. We assume that the exact mapping $G(u)$ is replaced by some numerical approximation $G_\delta(u)$. In particular, $G_\delta(u)$ is given by using the forward Euler method to solve the ODE \eqref{eq:dh1}. Define $X_0=0$, and
\[X_{\ell+1}=X_\ell+\delta F(X_\ell;u),\quad \ell\geq0,\]
where $\delta=\frac{1}{L}$ for some large integer $L$. Finally define $G_\delta(u):=X_L$. \\

In what follows, we will prove that $G_\delta(u)$ is uniformly bounded and close to $G(u)$ when $\delta$ is small, and then we will use these results to show
that Assumption \ref{a:dh1} is satisfied. Therefore, we can apply Theorem~\ref{thm1} to this example and claim that the approximate posterior $\pi_\delta^y$ is close to the unperturbed one $\pi^y$.\\

Define $t_\ell=\ell\delta$, $x_\ell=x(t_\ell)$. The following lemma gives an estimate on the error generated from using the forward Euler method.

\begin{lemma} \label{l:maruyama} Let $E_\ell:=x_\ell-X_\ell$. Then there is $c<\infty$ independent of $\delta$ such that
\[|E_\ell|\leq c\delta,\quad 0\leq \ell\leq L. \]
In particular
\[|G(u)-G_\delta(u)|=|E_L|\leq c\delta.\]
      \end{lemma}
      \begin{proof}
For simplicity of exposition we consider the case $k=1$; the case
$k>1$ is almost identical, simply requiring the integral form for
the remainder term in the Taylor expansion.  Using Taylor expansion
in the case $k=1,$ there is $\xi_\ell\in[t_\ell,t_{\ell+1}]$ such that
     	\begin{align*}
     	x_{\ell+1}=&\ x_\ell+\delta \frac{dx}{dt}(t_\ell)+\frac{\delta^2}{2}\frac{d^2x}{dt^2}(\xi_\ell)\\
     	=&\ x_\ell+\delta F(x_\ell;u)+\frac{\delta^2}{2}D_xF\bigl(x(\xi_\ell);u\bigr)F\bigl(x(\xi_\ell);u\bigr).
     	\end{align*}
     	Thus we have
     	\begin{align*}
     	|E_{\ell+1}|=&\ |x_{\ell+1}-X_{\ell+1}|\\
     	=&\ \Big|x_\ell-X_\ell+\delta\Bigl(F(x_\ell;u)-F(X_\ell;u)\Bigr)+\frac{\delta^2}{2}D_xF\bigl(x(\xi_\ell);u\bigr)F\bigl(x(\xi_\ell);u\bigr)\Big|\\
     	\leq&\ |x_\ell-X_\ell|+\delta \bigl|F(x_\ell;u)-F(X_\ell;u)\bigr|+\frac{\delta^2}{2} \bigl|D_xF\bigl(x(\xi_\ell);u\bigr) \bigr| \bigl|F\bigl(x(\xi_\ell);u\bigr)\bigr|\\
     	\leq&\ |E_\ell|+\delta F_{{\rm max}}|E_\ell|+ \frac{\delta^2}{2}F_{{\rm max}}^2.
     	\end{align*}
     	Noticing that $|E_0|=0$, the discrete Gronwall inequality gives
     	\begin{align*}
     	|E_\ell|\leq&\ (1+\delta F_{{\rm max}})^\ell|E_0|+\frac{(1+\delta F_{{\rm max}})^\ell-1}{\delta F_{{\rm max}}}\cdot\frac{\delta^2}{2}F_{{\rm max}}^2\\
     	\leq&\ \bigg( \Big(1+\frac{F_{{\rm max}}}{L}\Big)^L-1\bigg)\cdot\frac{F_{{\rm max}}\delta}{2}\\
     	\leq&\ \frac{(e^{F_{{\rm max}}}-1)F_{{\rm max}}}{2} \delta.
     	\end{align*}
     	The lemma follows by taking $c=\frac{(e^{F_{{\rm max}}}-1)F_{{\rm max}}}{2}$.
      \end{proof}

\begin{lemma} For any $u \in \mathbbm{R}^\Du,$
\[|G(u)|,|G_\delta(u)|<F_{{\rm max}}.\]
      \end{lemma}
      \begin{proof} For $G(u)$ we use that $F(x;u)$ is uniformly bounded, so that
     	\[|G(u)|=|x(1)|=\Big|\int_0^1F(x(t);u)dt\Big|\leq \int_0^1\bigl|F(x(t);u)\bigr|dt\leq F_{{\rm max}}.\]
     	As for $G_\delta(u)$, we first notice that
     	$$|X_{\ell+1}| = |X_\ell +\delta F(X_\ell;u)| \le |X_\ell| + \delta |F(X_\ell;u)| \le |X_\ell| + \delta F_{{\rm max}},$$
     	and by induction
     	$$|X_{\ell}| \le |X_0| +  \ell \delta F_{{\rm max}}  = \ell \delta F_{{\rm max}}.$$
     	In particular,
     	$$|G_\delta(u) | = |X_L| \le L \delta F_{{\rm max}} = F_{{\rm max}}.$$
      \end{proof}

To conclude this chapter we show that in this example Assumption \ref{a:dh1} is satisfied.
Recall that $\eta\sim \mathcal{N}(0,\gamma^2I_k)$, and thus
\[\sqrt{g(u)}=\sqrt{\nu\bigl(y-G(u)\bigr)}=\frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\exp\Big(-\frac{1}{4\gamma^2}|y-G(u)|^2\Big),\]
\[\sqrt{g_\delta(u)}=\sqrt{\nu\bigl(y-G_\delta(u)\bigr)}=\frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\exp\Big(-\frac{1}{4\gamma^2}|y-G_\delta(u)|^2\Big).\]
\begin{itemize}
\item For Assumption \ref{a:dh1}(i) notice that the function $e^{-w}$  is Lipschitz for $w>0$, with Lipschitz constant $1$. Therefore we have
\begin{align*}
\Big|\sqrt{g(u)}-\sqrt{g_\delta(u)}\Big|\leq&\ \frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\cdot \frac{1}{4\gamma^2}\cdot \big||y-G(u)|^2-|y-G_\delta(u)|^2\big|\\
= &\ \frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\cdot \frac{1}{4\gamma^2} \cdot |2y-G(u)-G_\delta(u)||G(u)-G_\delta(u)|\\
\leq &\ \frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\cdot \frac{1}{4\gamma^2} \cdot (2|y|+2F_{{\rm max}}) c\delta\\
=&\ \tilde{c}\delta.
\end{align*}
That is to say, Assumption \ref{a:dh1}(i) is satisfied with $\phi(u)=\tilde{c}$ and $\int_{\mathbbm{R}^\Du} \phi^2(u)\rho(u)du=\tilde{c}^2<\infty$.
\item Assumption \ref{a:dh1}(ii) is satisfied since
\[\sqrt{g(u)}=\frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\exp\Big(-\frac{1}{4\gamma^2}|y-G(u)|^2\Big)\leq \frac{1}{(2\pi)^{k/4}\gamma^{k/2}},\]
\[\sqrt{g_\delta(u)}=\frac{1}{(2\pi)^{k/4}\gamma^{k/2}}\exp\Big(-\frac{1}{4\gamma^2}|y-G_\delta(u)|^2\Big)\leq \frac{1}{(2\pi)^{k/4}\gamma^{k/2}}.\]
\end{itemize}

\subsection{Discussion and Bibliography}

The book by Kaipio and Somersalo \cite{kaipio2006statistical} provides an introduction to the Bayesian approach to inverse problems, especially in the context of differential equations. An overview of the
subject of Bayesian inverse problems in differential equations, with
a perspective informed by the geophysical sciences,
is the book by Tarantola  \cite{tarantola2005inverse} (see, especially, Chapter 5).

In the paper \cite{stuart2010inverse} the Bayesian approach to regularization is reviewed,
developing a function space viewpoint on the subject. A well-posedness theory
and some algorithmic approaches which are used when adopting the Bayesian
approach to inverse problems are introduced.
The function space viewpoint on the subject is developed in more detail in the
chapter notes of Dashti and Stuart \cite{dashti2013bayesian}.
An application of this function space methodology to a large-scale
geophysical inverse problem is considered in
\cite{martin2012stochastic}.
The paper \cite{lieberman2010parameter} demonstrates the potential
for the use of dimension reduction techniques from control theory
within statistical inverse problems.

See \cite{gibbs2002choosing} for more detail on the subject of
metrics, and other distance-like functions, on probability measures.
See \cite{stuart2010inverse,dashti2013bayesian} for  more detailed
discussions on the well-posedness of Bayesian inverse problems with
respect to perturbations in the data; and see \cite{cotter2010approximation}
for applications concerning numerical approximation of
partial differential equations appearing in the forward model.
Related results, but using divergences rather than the Hellinger
metric, may be found in \cite{marzouk2009stochastic}.
The paper \cite{iglesias2014well} contains an interesting set of examples
where the Meta Theorem stated in this chapter fails
in the sense that, whilst well-posedness holds, the posterior is
H\"older with exponent less than one, rather than Lipschitz, with respect
to perturbations.

\section{\Large{\sffamily{Particle Filter}}}
\label{sec:Lecture11}

This chapter is devoted to the particle filter, a method that
approximates the filtering distribution by a sum of Dirac measures.
Particle filters provably converge to the filtering distribution as the number of Dirac measures
approaches infinity. We focus on the bootstrap particle filter, also known as sequential importance resampling;
it is linked to the material on Monte Carlo and importance sampling described in chapter 5.
We note that the Kalman filter completely characterizes the filtering
distribution, but only in the linear Gaussian setting; the Kalman-based methods introduced in the two previous chapters outside the linear-Gaussian assumption are built by approximating the predictive distribution via a Gaussian ansatz, and then using the Kalman formulae for the analysis step. Similarly, particle filters approximate the predictive distribution by a particle approximation measure and then solve exactly the analysis step. In this light both Kalman-based methods (with linear observations) and the bootstrap particle filter do exact application of Bayes formula with wrong priors.  However, particle filters have the potential of recovering an accurate approximation to the filtering distribution in nonlinear, non-Gaussian
settings provided that the number of particles is large enough; their caveat is that they tend to struggle in high dimensional problems.

\subsection{The Setting}
Let us return to the setting in which we introduced filtering
and smoothing, with nonlinear stochastic dynamical system and nonlinear
observation operator, namely the model
\begin{align*}
	v_{j+1} &= \Psi(v_j) + \xi_j, && \xi_j \sim \mathcal{N}(0, \Sigma) \text{ i.i.d.}, \\
	y_{j+1} &= h(v_{j+1}) + \eta_j, && \eta_j \sim \mathcal{N}(0, \Gamma) \text{ i.i.d.},
\end{align*}
with \(v_0 \sim \pi_0 := \mathcal{N}(m_0, C_0)\) independent of the i.i.d.
sequences \(\{\xi_j\}\) and  \(\{\eta_j\}\). Here \(\Psi(\cdot)\) drives the dynamical system and \(h(\cdot)\) is the observation operator.
Recall that we denote by \(Y_j = \{y_1,\hdots,y_j\}\) all the data up to time
\(j\) and by \(\pi_j\) the pdf of \(v_j | Y_j\), that is, \(\pi_j = \operatorname{\mathbbm{P}}(v_j|Y_j)\). The filtering problem is to determine \(\pi_{j+1}\) from \(\pi_j\). We may do so in two steps: first, we run forward the Markov chain generated by the stochastic dynamical system
(prediction), and second, we incorporate the data by an application of Bayes
Theorem (analysis).

For the prediction step, we define the operator \({\mathcal{P}}\) acting on a pdf \(\pi\) as an application of a Markov kernel defined by
\begin{equation}
\label{eq:markov}
	({\mathcal{P}} \pi)(v) = \int_{\mathbbm{R}^d} p(u,v) \pi(u) du,
\end{equation}
where \(p(u,v)\) is the associated pdf of the stochastic dynamics, so that
\begin{equation*}
	p(u,v) = \frac{1}{\sqrt{(2\pi)^d {\rm det}\Sigma}} \text{exp}\left( -\frac{1}{2}|v - \Psi(u)|_{\Sigma}^2 \right).
\end{equation*}
Thus we obtain
\begin{equation*}
	\operatorname{\mathbbm{P}}(v_{j+1}|Y_j)=\hat{\pi}_{j+1} = {\mathcal{P}} \pi_j.
\end{equation*}
We then define the analysis operator \({\mathcal{A}}_j\) acting on a pdf \(\pi\) to correspond to an application of Bayes theorem, namely
\begin{equation}
\label{eq:bayes-again}
	({\mathcal{A}}_j \pi)(v) = \frac{\text{exp}(-\frac{1}{2}|y_{j+1} - h(v)|_{\Gamma}^2)\pi(v)}{\int_{\mathbbm{R}^d} \text{exp}(-\frac{1}{2}|y_{j+1} - h(v)|_{\Gamma}^2)\pi(v) dv}
\end{equation}
and so finally we obtain
\begin{equation*}
	\pi_{j+1} = {\mathcal{A}}_j \hat{\pi}_{j+1} = {\mathcal{A}}_j {\mathcal{P}} \pi_j.
\end{equation*}
We now describe a way to numerically approximate, and update,
the pdfs $\pi_j.$

\subsection{The Bootstrap Particle Filter}
The bootstrap particle filter (BPF) can be thought of as
performing sequential importance resampling. Let \(S^N\) be an operator acting on a pdf \(\pi\) by producing an $N-$samples Dirac approximation of \(\pi\), that is
\begin{equation*}
	(S^N \pi)(u) = \sum_{n=1}^N w_n \delta(u - u^{(n)}),
\end{equation*}
where \(u^{(1)},\hdots,u^{(N)}\) are i.i.d samples from \(\pi\) that are weighted uniformly i.e. \(w_n = \frac{1}{N}\). Note that $S^N \pi=\pi^{N}_{MC}$, as introduced in Chapter 5, equation \eqref{eq:indeed}. We will use the operator $S^N$ to approximate
the measure produced by the Markov kernel step ${\mathcal{P}}$ within the overall
filtering map ${\mathcal{A}}_j {\mathcal{P}}$.
Note that $S^N$ is a {\em random} map taking pdfs into pdfs
if we interpret weighted sums of Diracs as a pdf.

Let \(\pi_0^N = \pi_0 = \mathcal{N}(m_0, C_0)\) and let $\pi^N_j$ denote
a particle approximation of the pdf $\pi_j$
that we will determine in what
follows. We define
\begin{equation*}
	\hat{\pi}^N_{j+1} = S^N {\mathcal{P}} \pi^N_j;
\end{equation*}
this is an approximation of \(\hat{\pi}_{j+1}\) from the previous section. We then apply the operator \({\mathcal{A}}_j\) to act on \(\hat{\pi}^N_{j+1}\) by appropriately reconfiguring the weights \(w_j\) according to the data.

To understand this reconfiguration of the weights
we use the fact that, if
\begin{equation*}
	({\mathcal{A}} \pi)(v) = \frac{g(v) \pi(v)}{\int_{\mathbbm{R}^\Du} g(v) \pi(v) dv}
\end{equation*}
and if
$$\pi(v)=\frac{1}{N} \sum_{n=1}^N \delta(v-v^{(n)}),$$
then
$$({\mathcal{A}} \pi)(v)= \sum_{n=1}^N w^{(n)} \delta(v-v^{(n)}),$$
where
$$\bar{w}^{(n)}=g(v^{(n)})$$
and the ${w}^{(n)}$ are found from the $\bar{w}^{(n)}$ by renormalizing
them to sum to one.
We use this calculation concerning the application of Bayes formula
to sums of Diracs within the following desired approximation of the
filtering update formula:
\begin{equation*}
	\pi_{j+1} \approx \pi_{j+1}^N = {\mathcal{A}}_j \hat{\pi}_{j+1}^N = {\mathcal{A}}_j S^N {\mathcal{P}} \pi_j^N.
\end{equation*}
The steps for the method are summarized in Algorithm \ref{alg:bpf}.

\FloatBarrier
\begin{algorithm}
\caption{Bootstrap Particle Filter}
\begin{algorithmic}[1]
\vspace{0.1in}
\STATE {\bf Input}: Initial distribution $\pi_0^N = \pi_0$, observations $Y_J$, number of particles $N.$  \\
\vspace{.04in}
\STATE {\bf Particle Generation}: For $j = 0,1,\dots,J-1$, perform
\begin{enumerate}
\item Draw $v_j^{(n)} \sim \pi_j^N$ for $n=1,\ldots,N$ i.i.d.
\item Set $\hat{v}^{(n)}_{j+1} = \Psi(v_j^{(n)}) + \xi_j^{(n)}$ with $\xi_{j}^{(n)}$ i.i.d.  $\mathcal{N}(0,\Sigma).$
\item Set $\bar{w}_{j+1}^{(n)} = \text{exp}(-\frac{1}{2}|y_{j+1} - h(\hat{v}^{(n)}_{j+1})|_{\Gamma}^2).$
\item Set $w_{j+1}^{(n)} = \bar{w}_{j+1}^{(n)} / \sum_{n=1}^N \bar{w}_{j+1}^{(n)}.$
\item Set $\pi_{j+1}^N (u) = \sum_{n=1}^N w_{j+1}^{(n)} \delta(u - \hat{v}^{(n)}_{j+1}).$
\end{enumerate}
\vspace{.04in}
\STATE{\bf Output:} pdf $\pi_{J}^N$ that approximates the distribution $\operatorname{\mathbbm{P}}(v_J|Y_J).$
\end{algorithmic}
\label{alg:bpf}
\end{algorithm}
\FloatBarrier

\subsection{Bootstrap Particle Filter Convergence}

We will now show that under certain conditions, the BPF
converges to the true particle filter distribution in
the limit \(N \rightarrow \infty\). The proof is similar to that of the
Lax-Equivalence Theorem from the numerical approximation of
evolution equations, part of which is the statement that
consistency and stability together imply convergence.
For the BPF consistency refers to a Monte Carlo error estimate, similar to that
derived in the chapter on importance sampling, and stability
manifests in bounds on the
Lipschitz constants for the operators ${\mathcal{P}}$ and ${\mathcal{A}}_j.$

Our first step is to define what we mean by convergence, that is, we need
a metric on probability measures. Notice that the operators \({\mathcal{P}}\) and \({\mathcal{A}}_j\) are deterministic, but the operator \(S^N\) is random since it requires sampling.
As a consequence the approximate pdfs \(\pi_j\) are also random.
Thus, in fact, we need a metric on random probability measures. To this
end, for random probability measures $\pi$ and $\pi'$, we define
\begin{equation*}
	d(\pi, \pi') = \sup_{|f|_\infty \leq 1} \Bigl( \operatorname{\mathbb{E}}\Bigl[\bigl(\pi(f) - \pi'(f)\bigr)^2\Bigr] \Bigr)^{1/2},
\end{equation*}
where the expectation is taken over the random variable, in our case,
the randomness from sampling with \(S^N\), and the supremum is taken over
all functions $f: \mathbbm{R}^d \to [-1,1].$
Here we have used the notation defined in equation \eqref{eq:meas2}.
The following lemma is straightforward to prove, and provides some useful
intuition about the metric.

\begin{lemma}
$d(\cdot,\cdot)$ as defined above does indeed define a metric on random
probability measures. Furthermore, when \(\pi,\pi'\) are deterministic,
then we have \(d(\pi,\pi') = 2d_{\mbox {\tiny{\rm TV}}}(\pi,\pi')\).
\end{lemma}

We now prove three lemmas which together will enable us to prove convergence
of the BPF.  The first shows consistency; the second
and third show stability estimates for ${\mathcal{P}}$ and ${\mathcal{A}}_j$ respectively.

\begin{lemma}
	Let \({\mathscr{P}}\) be the set of probability densities on \(\mathbbm{R}^\Du\) then
	\[\sup_{\pi \in {\mathscr{P}}} d(\pi, S^N \pi) \leq \frac{1}{\sqrt{N}}.\]
\end{lemma}
\begin{proof}
Note that $S^N\pi$ agrees with $\pi_{MC}^N$ as defined in Chapter 5. Therefore Theorem \ref{t:MC} shows that for any $f$ with $|f|_\infty \le 1$ and for any probability density $\pi$ it holds that
$$\operatorname{\mathbb{E}}\left[ \bigl(S^N \pi(f) - \pi(f)\bigr)^2\right] \le \frac{1}{N}.$$
Taking the square root and the supremum over $f$ and $\pi$ gives the desired result.
\end{proof}

Now we prove a stability bound for the operator ${\mathcal{P}}$ defined in equation \eqref{eq:markov}.

\begin{lemma}
	\[d({\mathcal{P}} \pi, {\mathcal{P}}\pi') \leq d(\pi,\pi').\]
\end{lemma}

\begin{proof}  For $|f|_\infty \le 1$  define a function $q$ on $\mathbbm{R}^d$ by
\[q(v') = \int_{\mathbbm{R}^d}p(v',v)f(v)dv,  \]
where, recall, $p$ denotes the transition pdf associated to the stochastic dynamics.
Note that
\[|q(v')| \leq \int_{\mathbbm{R}^d}p(v',v)dv = 1 ,\]
and so
\begin{align*}
	\pi(q) &= \int_{\mathbbm{R}^d}q(v')\pi(v')dv' = \int_{\mathbbm{R}^d} \Bigg[ \int_{\mathbbm{R}^d}p(v',v)f(v)dv \Bigg] \pi(v')dv' \\
	&= \int_{\mathbbm{R}^d} \Bigg[ \int_{\mathbbm{R}^d}p(v',v)\pi(v')dv' \Bigg] f(v)dv \\
	&= \int_{\mathbbm{R}^d} ({\mathcal{P}}\pi)(v)f(v)dv
\end{align*}
by exchanging the order of integration. Consequently, we have
\[\pi(q) = ({\mathcal{P}}\pi)(f).\]
Finally,
\begin{align*}
	d({\mathcal{P}}\pi,{\mathcal{P}}\pi') &= \sup_{|f|_\infty \leq 1} \Bigl(\operatorname{\mathbb{E}} \Bigl[\bigl(({\mathcal{P}}\pi)(f) - ({\mathcal{P}}\pi')(f)\bigr)^2\Bigr]\Bigr)^{1/2} \\
	&\leq \sup_{|q|_\infty \leq 1} \Bigr( \operatorname{\mathbb{E}}  \Bigl[\bigl(\pi(q) - \pi'(q)\bigr)^2\Bigr]\Bigr)^{1/2} \\
	&= d(\pi,\pi').
\end{align*}
\end{proof}

To prove the next lemma and the main convergence
theorem of the BPF below,
we will make the following assumption which encodes the idea of a bound
on the observation operator:

\begin{assumption}
\label{a:LE}
There exists \(\kappa \in (0,1)\) such that, for all  $v\in\mathbbm{R}^d,$ and $j \in \{1,\hdots,J\},$
\begin{equation*}
	\kappa \leq g_j(v) \leq \kappa^{-1}.
\end{equation*}
\end{assumption}

It may initially appear strange to use the same constant $\kappa$ in the
upper and lower bounds, but recall that $g$ is undefined up to a multiplicative
constant. Consequently, given any upper and lower bounds, $g$ can be
scaled to achieve the bound as stated. Relatedly it is $\kappa^{-2}$
which appears in the stability constant in the next lemma;
if $g$ is not scaled to produce the same constant $\kappa$ in the
upper and lower bounds in
Assumption \ref{a:LE}, then it is the ratio of the upper and lower
bounds which would appear in the stability bound.

\begin{lemma} \label{lemma:analysis}
Let Assumption \ref{a:LE} hold. Then
	\[d({\mathcal{A}}_j \pi,{\mathcal{A}}_j \pi') \leq \frac{2}{\kappa^2} d(\pi,\pi').\]
\end{lemma}

\begin{proof}
\begin{align*}
	({\mathcal{A}}\pi)(f) - ({\mathcal{A}}\pi')(f) &= \frac{\pi(fg)}{\pi(g)} - \frac{\pi'(fg)}{\pi'(g)} \\
	&= \frac{\pi(fg)}{\pi(g)} - \frac{\pi'(fg)}{\pi(g)} + \frac{\pi'(fg)}{\pi(g)} - \frac{\pi'(fg)}{\pi'(g)} \\
	&= \frac{1}{\kappa} \bigg( \frac{\pi(\kappa fg) - \pi'(\kappa fg)}{\pi(g)} + \frac{\pi'(fg)}{\pi'(g)} \frac{\pi'(\kappa g) - \pi(\kappa g)}{\pi(g)} \bigg).
\end{align*}
Applying Bayes Theorem we obtain
\[\Bigl|\frac{\pi'(fg)}{\pi'(g)}\Bigr| = |({\mathcal{A}}\pi')(f)| \leq 1. \]
Therefore,
\[\bigl|({\mathcal{A}}\pi)(f) - ({\mathcal{A}}\pi')(f)\bigr| \leq \frac{1}{\kappa^2}\Bigl( \bigl|\pi(\kappa fg) - \pi'(\kappa fg)\bigr| + \bigl|\pi'(\kappa g) - \pi(\kappa g)\bigr|   \Bigr).  \]
It follows that
\[\operatorname{\mathbb{E}} \Bigl[\bigl( ({\mathcal{A}}\pi)(f) - ({\mathcal{A}}\pi')(f)\bigr)^2\Bigr] \leq \frac{2}{\kappa^2}\Bigl(\operatorname{\mathbb{E}} \Bigl[(\pi(\kappa fg) - \pi'(\kappa fg))^2\Bigr] + \operatorname{\mathbb{E}}\Bigl[\bigl(\pi'(\kappa g) - \pi(\kappa g)\bigr)^2\Bigr] \Bigr).  \]
Since $|\kappa g| \le 1$ we find that
\[\sup_{|f|_\infty \leq 1} \operatorname{\mathbb{E}}\Bigl[\bigl(({\mathcal{A}}\pi)(f) - ({\mathcal{A}}\pi')(f)\bigr)^2\Bigr] \leq \frac{4}{\kappa^4} \sup_{|f|_\infty \leq 1} \operatorname{\mathbb{E}}\Bigl[\bigl(\pi(f) - \pi'(f)\bigr)^2\Bigr],  \]
and hence
\[d({\mathcal{A}}_j\pi,{\mathcal{A}}_j\pi') \leq \frac{2}{\kappa^2} d(\pi,\pi'). \]
\end{proof}

\begin{theorem}[Convergence of the BPF]
Let Assumption \ref{a:LE} hold.
Then there exists a $c = c(J,\kappa)$ such that, for all $j = 1, \ldots, J,$
	\[d(\pi_j,\pi_j^N) \leq \frac{c}{\sqrt{N}}. \]
\end{theorem}

\begin{proof}
Let $e_j = d(\pi_j,\pi_j^N)$, then
\begin{align*}
	e_{j+1} &= d(\pi_{j+1},\pi_{j+1}^N) = d({\mathcal{A}}_j{\mathcal{P}}\pi_j,{\mathcal{A}}_jS^N {\mathcal{P}}\pi_j^N) \\
	&\leq d({\mathcal{A}}_j{\mathcal{P}}\pi_j  , {\mathcal{A}}_j{\mathcal{P}}\pi_j^N) + d({\mathcal{A}}_j{\mathcal{P}}\pi_j^N,{\mathcal{A}}_jS^N {\mathcal{P}}\pi_j^N)
\end{align*}
by the triangle inequality. Applying the stability bound for ${\mathcal{A}}_j$,
we have
\[e_{j+1} \leq \frac{2}{\kappa^2}\Bigl[d({\mathcal{P}}\pi_j   , {\mathcal{P}}\pi_j^N) + d(\hat{\pi}_j^N,S^N\hat{\pi}_j^N)\Bigr],\]
where $\hat{\pi}_j^N = {\mathcal{P}}\pi_j^N$. By the stability bound for ${\mathcal{P}}$,
\[d({\mathcal{P}}\pi_j  , {\mathcal{P}}\pi_j^N) \leq d(\pi_j, \pi_j^N) \]
and by the consistency bound for $S^N$
\[d(\hat{\pi}_j^N,S^N \hat{\pi}_j^N) \leq \frac{1}{\sqrt{N}}.\]
Therefore,
\begin{align*}
	e_{j+1} &\leq \frac{2}{\kappa^2}\Bigl(d(\pi_j,\pi_j^N) + \frac{1}{\sqrt{N}}\Bigr) \\
	&\leq \frac{2}{\kappa^2}\Bigl(e_j + \frac{1}{\sqrt{N}}\Bigr).
\end{align*}
We let $\lambda = 2/\kappa^2$ and note that \(\lambda \geq 2\) since \(\kappa \in (0,1]\). Then the discrete Gronwall inequality gives
\begin{equation*}
	e_{j} \leq \lambda^je_0 + \frac{\lambda}{\sqrt{N}}\frac{1-\lambda^j}{1-\lambda}.
\end{equation*}
Recall that \(\pi_0^N = \pi_0\) hence \(e_0 = 0\). Thus letting
\[c = \frac{\lambda(1 - \lambda^J)}{1 - \lambda}\]
completes the proof since \(\lambda(1 - \lambda^j)/(1 - \lambda)\) is increasing in \(j\).
\end{proof}

\subsection{The Bootstrap Particle Filter as a Random Dynamical System}

A nice interpretation of the BPF is to view it as a random dynamical system for a set of interacting
particles $\{v_j^{(n)}\}_{n=1}^N.$
To this end, a measure
\begin{equation*}
	\bar{\pi}_j^{N}(u) = \frac{1}{N}\sum_{n=1}^N \delta(u-v_j^{(n)}) \approx \pi_j^N(u) \approx \pi_j(u)
\end{equation*}
with equally weighted particles may be naturally defined after the resampling step from $\pi_j^N.$
It can then be seen that the BPF updates the particle positions
\begin{equation*}
\{v_j^{(n)} \}_{n = 1}^N  \mapsto \{v_{j+1}^{(n)} \}_{n = 1}^N
\end{equation*}
via the random map
\begin{align*}
	\hat{v}_{j+1}^{(n)} & = \Psi(v_j^{(n)}) + \xi_j^{(n)}, && \xi_j^{(n)} \sim \mathcal{N}(0,\Sigma) \text{ i.i.d.,}\\
	v_{j+1}^{(n)} & = \sum_{m=1}^N \mathbbm{1}_{I_{j+1}^{(m)}}\left(r_{j+1}^{(n)}\right) \hat{v}_{j+1}^{(m)}, && r_{j+1}^{(n)} \sim \text{Uniform}(0,1)\text{ i.i.d.}
\end{align*}
Here the supports $I_{j}^{(m)}$ of the indicator functions have widths
given by the weights appearing in  $ {\pi}_j^{N}(u).$ Specifically we have
\begin{align*}
	I_{j+1}^{(m)} & = \left[\alpha_{j+1}^{(m-1)},\alpha_{j+1}^{(m)}\right), &  \alpha_{j+1}^{(m+1)} = \alpha_{j+1}^{(m)}+w_{j+1}^{(m)}, \quad \alpha_{j+1}^{(0)} = 0.
\end{align*}
Note that, by construction, $\alpha_j^{(N)} = 1.$

Thus the underlying dynamical system on particles comprises $N$ particles
governed by two steps: (i) the underlying stochastic dynamics model, in
which the particles do not interact; (ii) a resampling of the resulting collection of particles, to reflect the different weights associated with them, in
which the particles do then interact. The interaction is driven by the
weights which see all the particle positions, and measure their goodness
of fit to the data.

\subsection{Discussion and Bibliography}

Particle filters are overviewed from an algorithmic viewpoint
in \cite{doucet2001introduction,doucet2000sequential},
and from a more mathematical perspective in \cite{del2004feynman}.
The convergence of particle filters was addressed
in \cite{crisan1998discrete}; the clean proof presented here
originates in \cite{rebeschini2015can} and may also
be found in \cite{law2015data}.
For problems in which the dynamics evolve in relatively low dimensional
spaces they have been an enormously successful. Generalizing
them so that they work for the high dimensional problems that arise,
for example, in geophysical applications, provides a major challenge
\cite{van2015nonlinear}.

\section{\Large{\sffamily{The Extended and Ensemble Kalman Filters }}}

In this chapter we describe the Extended Kalman Filter  (ExKF)\footnote{The extended
Kalman filter is often termed the EKF in the literature, a terminology
introduced before the existence of the EnKF;
we find it useful to write ExKF to uneqivocally distinguish it from the EnKF.}
and the Ensemble Kalman Filter (EnKF). The ExKF approximates the predictive covariance by linearization, while the EnKF approximates it by the empirical covariance of a collection of particles. The status of the two methods in relation to the true filtering distribution
is as follows. The ExKF is a provably accurate approximation of the
filtering distribution in situations in which small noise is present in both
signal and data, and the filtering distribution is well-approximated by a
Gaussian. The EnKF is also in principle a good approximation of the
filtering distribution in this situation, if a large number of particles
is used. However the EnKF is typically deployed for problems where the
use of a sufficiently large number of particles is impractical; it
is then better viewed as an online optimizer, in the spirit of 3DVAR,
but using multiple particles to better estimate the covariances
appearing in the quadratic objective function which is minimized
to find particle updates.

\subsection{The Setting}

Throughout this chapter we consider the setting in which 3DVAR was introduced
and may be applied: the dynamics model
is nonlinear, but the observation operator is linear.
For purposes of exposition we summarize it again here:
\begin{align*}
        v_{j+1} &= \Psi(v_j) + \xi_j, && \xi_j \sim \mathcal{N}(0, \Sigma) \text{ i.i.d.}, \\
        y_{j+1} &= Hv_{j+1} + \eta_{j+1}, && \eta_j \sim \mathcal{N}(0, \Gamma) \text{ i.i.d.},
\end{align*}
with \(v_0 \sim \mathcal{N}(m_0, C_0)\) independent of the independent i.i.d.
sequences \(\{\xi_j\}\) and  \(\{\eta_j\}\).
Throughout this chapter we assume that $v_j \in \mathbbm{R}^d, \,
y_j \in \mathbbm{R}^k.$

\subsection{The Extended Kalman Filter}

This method is derived by applying the Kalman methodology, using
linearization to propagate the covariance $C_j$ to the predictive
covariance $\hat{C}_{j+1}.$ Table \ref{3DVAR:t2} summarizes the idea,
after which we calculate the formulae required in full detail.

\FloatBarrier
\begin{table}
        \begin{tabular}{ | c | c |}
                \hline
                Kalman Filter & ExKF \\ \hline
                $m_{j+1} = \arg \min_v {\mathsf{J}}(v)$ & $m_{j+1} = \arg \min_v {\mathsf{J}}(v)$  \\ \hline
                ${\mathsf{J}}(v) = \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2$  &  ${\mathsf{J}}(v) = \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2$ \\
                \hline
                $\hat{m}_{j+1} = M m_j$ & $\hat{m}_{j+1} = \Psi(m_j)$ \\ \hline
$\hat{C}_{j+1}$ update exact & $\hat{C}_{j+1}$ update by linearization \\ \hline
                $m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}$ & $m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}$\\ \hline
        \end{tabular}
        \caption{Comparison of Kalman Filter and ExKF update formulae}
                \label{3DVAR:t2}
\end{table}
\FloatBarrier

We first recall the Kalman filter update formulae and their derivation. We
have
\begin{align}
\widehat{v}_{j+1} = Mv_{j}+\xi_{j}, \quad v_{j}  \sim \mathcal{N}(m_{j},C_{j}), \quad \xi_{j} \sim \mathcal{N}(0,\Sigma).
\end{align}
From this we deduce, by taking expectations, that
\begin{equation}
\widehat{m}_{j+1} = \operatorname{\mathbb{E}}[\widehat{v}_{j+1} \mid   Y_{j}]=\operatorname{\mathbb{E}}[Mv_{j}+\xi_{j} \mid  Y_{j}]= \operatorname{\mathbb{E}}[Mv_{j}\mid  Y_{j}] + \operatorname{\mathbb{E}}[\xi_{j}\mid  Y_{j}] =Mm_{j} .
\end{equation}
The covariance update is derived as follows.
\begin{align}
\widehat{C}_{j+1} &= \operatorname{\mathbb{E}}\Bigl[(\widehat{v}_{j+1}-\widehat{m}_{j+1})\otimes(\widehat{v}_{j+1}-\widehat{m}_{j+1})\mid  Y_{j} \Bigr] \nonumber \\
				  &= \operatorname{\mathbb{E}}\Bigl[(M(v_{j}-m_{j})+\xi_{j})\otimes(M(v_{j}-m_{j})+\xi_{j})\mid  Y_{j}\Bigr] \nonumber \\
                  &= \operatorname{\mathbb{E}} \Bigl[(M(v_{j}-m_{j}))\otimes(M(v_{j}-m_{j}))\mid  Y_{j} \Bigr]+\operatorname{\mathbb{E}}\Bigl[\xi_{j}\otimes\xi_{j}\mid  Y_{j}\Bigr]  \\
&\quad  +\operatorname{\mathbb{E}}\Bigl[(M(v_{j}-m_{j}))\otimes \xi_{j} \mid Y_{j}\Bigr]
+\operatorname{\mathbb{E}}\Bigl[\xi_{j}\otimes (M(v_{j}-m_{j})) \mid Y_{j}\Bigr] \nonumber \\
                  &=M\operatorname{\mathbb{E}}\Bigl[(v_{j}-m_{j})\otimes(v_{j}-m_{j}) \mid Y_{j}\Bigr]M^{T} + \Sigma \nonumber \\
                 &=MC_{j}M^{T}+\Sigma. \nonumber
\end{align}

For the ExKF, the prediction map $\Psi$ is no longer linear.
But since
$\xi_{j}$ is independent of $Y_{j}$ and $v_{j}$,
we obtain
\begin{equation*}
\widehat{m}_{j+1} = \operatorname{\mathbb{E}}\Bigl[\Psi(v_{j})+\xi_{j} \mid Y_{j}\Bigr] = \operatorname{\mathbb{E}}\Bigl[\Psi(v_{j})\mid Y_{j}\Bigr]+\operatorname{\mathbb{E}}\Bigl[\xi_{j} \mid Y_{j}\Bigr] = \operatorname{\mathbb{E}} \Bigl[\Psi(v_{j})\mid Y_{j}\Bigr].
\end{equation*}
If we assume that the fluctuations of $v_j$ around its mean $m_j$ (conditional
on data) are small then a reasonable approximation is to take
$\Psi(v_{j}) \approx \Psi(m_{j})$ so that
\begin{equation}
\label{eq:1a}
\widehat{m}_{j+1} = \Psi(m_{j}).
\end{equation}
For the predictive covariance we use linearization; we have
\begin{align*}
\widehat{C}_{j+1} \nonumber &= \operatorname{\mathbb{E}}\Bigl[(\widehat{v}_{j+1}-\widehat{m}_{j+1})
\otimes(\widehat{v}_{j+1}-\widehat{m}_{j+1})\mid  Y_{j}\Bigr]\nonumber \\
                 &= \operatorname{\mathbb{E}}\Bigl[(\Psi(v_{j})-\Psi(m_{j})+\xi_{j})\otimes(\Psi(v_{j})-\Psi(m_{j})+\xi_{j})\mid  Y_{j}\Bigr]\\
&=\operatorname{\mathbb{E}} \Bigl[(\Psi(v_{j})-\Psi(m_{j}))\otimes(\Psi(v_{j})-\Psi(m_{j}))\mid  Y_{j}\Bigr]+\Sigma\\
&\approx D\Psi(m_{j})\operatorname{\mathbb{E}}\Bigl[(v_{j}-m_{j})\otimes(v_{j}-m_{j})\mid  Y_{j}\Bigr]D\Psi(m_{j})^T+\Sigma,
\end{align*}
and so, again assuming that fluctuations of $v_j$ around its mean $m_j$
(conditional on data) are small, we invoke the approximation
\begin{equation}
\label{eq:1b}
\widehat{C}_{j+1}=D\Psi(m_{j})C_{j}D\Psi(m_{j})^{T}+\Sigma.
\end{equation}
To be self-consistent $\Sigma$ itself should be small.

The analysis step is the same as for the Kalman filter:

\begin{align}\label{eq:1c}
\begin{split}
S_{j+1}&=H\widehat{C}_{j+1}H^{T}+\Gamma,\\
K_{j+1}&= \widehat{C}_{j+1}H^{T}S_{j+1}^{-1}, \\
m_{j+1}&=(I-K_{j+1}H)\widehat{m}_{j+1}+K_{j+1}y_{j+1},\\
C_{j+1}&=(I-K_{j+1}H)\widehat{C}_{j+1}.\\
\end{split}
\end{align}

Thus the overall ExKF comprises equations \eqref{eq:1a}, \eqref{eq:1b}
and \eqref{eq:1c}.
Unlike the Kalman filter, for the extended Kalman filter
the maps $C_{j} \mapsto\widehat{C}_{j+1}\mapsto C_{j+1}$ depend
on the observed data, through the dependence of the predictive
covariance on the filter mean.
To be self-consistent with the ``small fluctuations around the mean''
assumptions made in the derivation of the ExKF, $\Sigma$ and $\Gamma$
should both be small.

The analysis step can also be defined by
\begin{align*}
C_{j+1}^{-1} &= \widehat{C}_{j+1}^{-1}+H^{T}\Gamma^{-1}H, \\
m_{j+1}&=\arg \min_v {\mathsf{J}}(v),
\end{align*}
where
\begin{equation}\label{eq:Ifilter}
{\mathsf{J}}(v) = \frac{1}{2} |y_{j+1}-Hv |_{\Gamma}^{2} + \frac{1}{2}|v-\widehat{m}_{j+1} |^{2}_{\widehat{C}_{j+1}}
\end{equation}
and $\widehat{m}_{j+1}, \widehat{C}_{j+1}$ are calculated as above
in the prediction steps \eqref{eq:1a}, \eqref{eq:1b}.
The constraint formulation of the minimization
problem, derived for the Kalman filter in section \ref{sec:okf}, may
also be used to derive the update formulae above.

\subsection{Ensemble Kalman Filter}

When the dynamical system is in high dimension, evaluation and storage of
the predicted covariance, and in particular the Jacobian required for
the update formula \eqref{eq:1b},
becomes computationally inefficient and expensive for the
ExKF. The EnKF was developed to overcome
this issue. The basic idea is to maintain an ensemble of particles,
and to use their empirical covariance
within a Kalman-type update. The method is summarized in
Table \ref{3DVAR:t3}. It may be thought of as an ensemble 3DVAR technique
in which a collection of particles are generated similarly to 3DVAR, but
interact through an ensemble estimate of their covariance.

\FloatBarrier
\begin{table}
        \begin{tabular}{ | c | c |}
                \hline
                Kalman Filter & EnKF \\ \hline
                $m_{j+1} = \arg \min_v {\mathsf{J}}(v)$ & $m_{j+1} = \arg \min_v {\mathsf{J}}(v)$  \\ \hline
                ${\mathsf{J}}(v) = \frac{1}{2} |y_{j+1} - H v|_{\Gamma}^2 + \frac{1}{2} |v - \hat{m}_{j+1}|_{\hat{C}_{j+1}}^2$  &  ${\mathsf{J}}_n(v) = \frac{1}{2} |y_{j+1}^{(n)} - H m|_{\Gamma}^2 + \frac{1}{2} |v - \hat{v}_{j+1}^{(n)}|_{\hat{C}_{j+1}}^2$ \\
                \hline
                $\hat{m}_{j+1} = M m_j$ & $\hat{v}_{j+1}^{(n)} = \Psi(v_j^{(n)})+\xi_j^{(n)}$ \\ \hline
$\hat{C}_{j+1}$ update exact & $\hat{C}_{j+1}$ update by ensemble estimate\\ \hline
                $m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}$ & $m_{j+1} = (I-K_{j+1}H)\hat{m}_{j+1} + K_{j+1} y_{j+1}$\\ \hline
        \end{tabular}
        \caption{Comparison of Kalman Filter and EnKF update formulae}
                \label{3DVAR:t3}
\end{table}
\FloatBarrier

\subsection{Derivation of EnKF}

In the basic form which we present here, the
EnKF is applied  when $\Psi$ is nonlinear, while the observation
operator $H$ is linear.
The $N$ particles used at step $j$ are denoted $\{v_{j}^{(n)}\}^{N}_{n=1}.$
They are all given equal weight so it is possible, in principle, to
make an approximation to the filtering distribution of the form
$$\pi_j^N (v_j) \approx \frac{1}{N}\sum_{n=1}^N \delta(v_j-v_{j}^{(n)}).$$
However the EnKF is typically used with a relatively small number $N$ of
particles and may be far from approximating the desired distribution in
$\mathbbm{R}^\Du.$ It is then better understood as a sequential optimization method,
similar in spirit to 3DVAR, as described above; this is our perspective.

The state of all the particles at time $j+1$ are predicted
to give $\{\widehat{v}_{j+1}^{(n)}\}^{N}_{n=1}$ using the dynamical
model. The resulting empirical covariance is then used
to define the objective function (\ref{eq:Ifilter}c) which is minimized
in order to perform the analysis step and obtain $\{{v}_{j+1}^{(n)}\}^{N}_{n=1}.$
The updates are denoted schematically by
\[\{v_{j}^{(n)}\}^{N}_{n=1} \xmapsto{\textcircled{p}} \{\widehat{v}_{j+1}^{(n)}\}^{N}_{n=1}\xmapsto{\textcircled{a}} \{v_{j+1}^{(n)}\}^{N}_{n=1}.\]
We now detail these two steps.

\textcircled{p} prediction:

\begin{subequations}
\label{eq:deenz}
\begin{align}
\widehat{v}_{j+1}^{(n)} &= \Psi(v_{j}^{(n)})+\xi^{(n)}_{j}, \quad n=1,\ldots,N, \\
\widehat{m}_{j+1} &= \frac{1}{N}\sum^{N}_{n=1} \widehat{v}_{j+1}^{(n)}, \\
\widehat{C}_{j+1} &= \frac{1}{N}\sum^{N}_{n=1}\bigl(\widehat{v}^{(n)}_{j+1}-\widehat{m}_{j+1}\bigr)\otimes \bigl(\widehat{v}^{(n)}_{j+1}-\widehat{m}_{j+1}\bigr).
\end{align}
\end{subequations}
Here we have
\begin{equation*}
\xi_{j}^{(n)} \sim \mathcal{N}(0,\Sigma), \quad  \text{i.i.d.}
\end{equation*}

\textcircled{a} analysis:
\begin{subequations}
\begin{align}
S_{j+1} &= H\widehat{C}_{j+1}H^{T} + \Gamma, \\
K_{j+1} &=\widehat{C}_{j+1}H^{T}S_{j+1}^{-1}, \\
y_{j+1}^{(n)}&=y_{j+1}+s\eta_{j+1}^{(n)},\quad n=1,\ldots,N,\\
v_{j+1}^{(n)} &= (I-K_{j+1}H)\widehat{v}_{j+1}^{(n)}+K_{j+1}y_{j+1}^{(n)},\quad n=1,\ldots,N. \\
\end{align}
\end{subequations}\\
Here we take
\begin{equation*}
\eta_{j}^{(n)} \sim \mathcal{N}(0,\Gamma), \quad \text{ i.i.d.}
\end{equation*}
The constant $s$ takes value $0$ or $1$.
When $s=1$ the $y_{j+1}^{(n)}$ are referred
to as {\em perturbed observations}.
The analysis step may be written as
\begin{equation}
v_{j+1}^{(n)}=\arg \min_v {\mathsf{J}}_{n}(v),
\end{equation}
where
\begin{equation}
\label{eq:deenz2}
{\mathsf{J}}_{n}(v) := \frac{1}{2} |y_{j+1}^{(n)} - Hv|^{2}_{\Gamma} + \frac{1}{2} |v-\widehat{v}_{j+1}^{(n)} |^{2}_{\widehat{C}_{j+1}}
\end{equation}
and the predicted mean and covariance are given by step \textcircled{p}.
Note that $\widehat{C}_{j+1}$ is typically not invertible as it is a rank
$N$ matrix and $N$ is usually less than the dimension $d$ of the space
on which $\widehat{C}_{j+1}$ acts; this is since the typical use
of ensemble methods is for high dimensional state space estimation,
with a small ensemble size.
The minimizing solution can be found
by regularizing $\widehat{C}_{j+1}$ by additions of $\epsilon I$ for
$\epsilon>0$, deriving the update equations as above for \textcircled{a},
and then letting $\epsilon \to 0.$ Alternatively, the constraint
formulation of the minimization problem, derived for the Kalman filter
in section \ref{sec:okf}, may also be used to derive the update formulae above.

\subsection{Subspace Property of EnKF}

We now give another way to think of, and exploit in algorithms, the
low rank property of $\widehat{C}_{j+1}.$
Note that ${\mathsf{J}}_{n}(v)$ is undefined unless
$$v-\widehat{v}_{j+1}^{(n)}=\widehat{C}_{j+1}a$$
for some $a \in \mathbbm{R}^\Du.$
From the structure of $\widehat{C}_{j+1}$ given in
(\ref{eq:deenz}c) it follows that
\begin{equation}
\label{eq:deenz3}
v=\widehat{v}_{j+1}^{(n)}+\frac{1}{N}\sum_{n=1}^N b_n\bigl(\widehat{v}^{(n)}_{j+1}-\widehat{m}_{j+1}\bigr)
\end{equation}
for some unknown parameters $\{b_n\}_{n=1}^N,$ to be determined.
(Note that the vector $\{b_n\}_{n=1}^N$ depends on enesemble index $n$; we have supressed this
dependence for notational convenience.)
This form for $v$ can be substituted into \eqref{eq:deenz2} to obtain
a functional ${\mathsf{I}}_{n}(b)$ to be minimized over $b \in \mathbbm{R}^N.$
We re-emphasize that $N$ will typically be much smaller than $d$, the
state-space dimension.
Once $b$ is determined it may be substituted back into
\eqref{eq:deenz3} to obtain the solution to the minimization problem.

To dig a little deeper into this calculation we define
$$e^{(n)}=\widehat{v}^{(n)}_{j+1}-\widehat{m}_{j+1}$$
and note that then
$$\widehat{C}_{j+1}=\frac{1}{N}\sum_{n=1}^N e^{(n)} \otimes e^{(n)}.$$
Since
$$\widehat{C}_{j+1} a=\frac{1}{N}\sum_{n=1}^N b_n e^{(n)}$$
we deduce that this is solved by taking
$$b_n= \langle e^{(n)}, a \rangle.$$
Now note that
$$\frac{1}{2} | v-\widehat{v}_{j+1}^{(n)} |^{2}_{\widehat{C}_{j+1}}
= \frac12 \langle a, \widehat{C}_{j+1} a \rangle=\frac{1}{2N}\sum_{n=1}^N b_n^2.$$
We define
\begin{equation}
\label{eq:z2}
{\mathsf{I}}_{n}(b) := \frac{1}{2} \Bigl|y_{j+1}^{(n)} - H\widehat{v}_{j+1}^{(n)} -\frac{1}{N}\sum_{n=1}^N b_n H(\widehat{v}_{j+1}^{(n)}-\widehat{m}_{j+1}) \Bigr|^{2}_{\Gamma}+\frac{1}{2N}\sum_{n=1}^N b_n^2.
\end{equation}
We have shown:

\begin{theorem}[Implementation of EnKF in $N$ Dimensional Subspace]
\label{eq:extra1}
Given the prediction \textcircled{p} defined by (\ref{eq:deenz}a), the Kalman
update formulae \textcircled{a} may be found
by minimizing ${\mathsf{I}}_{n}(b)$ with respect to $b$ and substituting
into \eqref{eq:deenz3}.
\end{theorem}

\subsection{Discussion and Bibliography}

The development and theory of the extended Kalman filter is
documented in the text \cite{jazwinski2007stochastic}.
A methodology for analyzing evolving probability distributions
with small variance, and establishing the validity of the
Gaussian approximation, is described in \cite{sanz2016gaussian}.
The use of the ExKF
for weather forecasting was proposed in \cite{ghilande}.
However the dimension of the state space in most geophysical
applications renders the extended Kalman filter impractical.
The ensemble Kalman filter provided an innovation with far
reaching consequences in geophysical applications, because
it allowed for the use of partial empirical correlation
information, without the computation of the full covariance.
An overview of ensemble Kalman methods may be found in the
book \cite{evensen2009data}, including a historical perspective
on the subject, originating from papers of Evensen and
Van Leeuwen in the mid
1990s \cite{evensen1994sequential,evensen1996assimilation};
a similar idea was also developed by Houtkamer
within the Canadian meteorological service, around the same time;
\cite{houtekamer1995methods,houtekamer1998data}.

The presentation of the ensemble
Kalman filter as a smart optimization tool
is also developed in \cite{law2015data}, but the derivation of
the update equations in a space whose dimension is that of
the ensemble is not described there.
The analysis of ensemble methods
is difficult and theory is only just starting to emerge.
In the linear case the method converges in the large ensemble
limit to the Kalman filter \cite{le2009large},
but in the nonlinear case the
limit does not reproduce the filtering distribution
\cite{ernst2015analysis}.
In any case the primary advantage of ensemble methods is that
they can provide good state estimation when the number of
particles is {\em not} large; this subject
is discussed in \cite{gottwald2013mechanism,kelly2014well,tong2015nonlinear,tong2016nonlinear}.

Inverse Problems and Data Assimilation