%% ----------------------------------------------------------------
%% Chapter 3: Span effect on the turbulence nature of flow past a circular cylinder
%% ----------------------------------------------------------------
%!TeX root = subfile
\label{chapter:jfm2019}
\documentclass[../main.tex]{subfiles}
% ----------------------------------------------------------------
\begin{document}

\vspace{0.25cm}
Turbulent flow evolution and energy cascades are significantly different in 2-D and 3-D flows.
Studies have investigated these differences in obstacle-free turbulent flows, but solid boundaries have an important impact on the cross-over from 3-D to 2-D turbulence dynamics.
In this chapter, the span effect on the turbulence nature of flow past a circular cylinder at $Re=10^4$ is investigated.
It is found that even for highly anisotropic geometries, 3-D small-scale structures detach from the walls.
Additionally, the natural large-scale rotation of the K\'{a}rm\'{a}n vortices rapidly two-dimensionalises those structures if the span is 50\% of the diameter or less.
This is linked to the span being shorter than the Mode B instability wavelength.
The conflicting 3-D small-scale structures and 2-D K\'{a}rm\'{a}n vortices result in 2-D and 3-D turbulence dynamics which can coexist at certain locations of the wake depending on the domain geometric anisotropy.

\section{Introduction and literature review}

Incompressible viscous flow past 2-D bluff bodies involves complex physics such as the von-K\'{a}rm\'{a}n street phenomenon as well as 3-D wake dynamics as the Reynolds number is increased \citep{Roshko1954, Williamson1996b, Williamson1996a}.
Due to the two-dimensionality of circular cylinders, some authors have used the 2-D Navier--Stokes equations on multiple planes located along the span of the cylinder as a simplified model of the three-dimensionality without increased computational cost (a.k.a. strip-theory method, see \sref{sec:strip-theory}).
Such strip-theory methods are used in offshore and civil engineering applications to model flow along slender structures where the computational cost of fully resolved simulations is prohibitive, such as marine risers, tow and mooring cable systems, and tall pillars.
However, the physics inherent in 2-D simulations lead to poor 3-D predictions \citep{Bao2016}, and a better understanding of the evolution from 3-D to 2-D turbulent wakes is needed to improve strip-theory methods.

The fluid mechanics of turbulent flows behave quite differently when the spanwise spatial dimension is much more constrained than the others.
Instead of having a direct cascade of the turbulence kinetic energy (TKE) from the integral scales down to the dissipative scales, there is a dual cascade of (direct) enstrophy and (inverse) energy.
This was first suggested by \cite{Kraichnan1967} and further developed by \cite{Leith1968} and \cite{Batchelor1969} (known as the KLB 2-D turbulence theory).
More recently, the dual cascade has been demonstrated both experimentally and computationally \citep[for a comprehensive review see][]{Boffetta2012}.
Studies such as \cite{Xiao2009} have shown that physical processes of vortex thinning and vortex merging dominate the dynamics of 2-D turbulence generating larger and more intense vortical structures the energy of which piles up at the integral scale for bounded domains.

Previous work has studied the transition between 2-D and 3-D dynamics in obstacle-free turbulent flows in detail.
The effect of the $L_z$ constriction on the TKE distribution across the scales (or wavenumbers $\kappa$) is sketched in \fref{fig:ek_sketch}.
By constricting the domain, the size of the 3-D energy-containing structures (integral-scale structures) is reduced.
Since the energy-containing structures feed the inertial subrange structures down to the 3-D small-scale dissipative structures, smaller integral-scale structures result in less energy fed into the inertial subrange structures and, consequently, to the dissipative structures.
In the 2-D limit, no 3-D dissipative structures are present and, because of the lack of a dissipation mechanism, the turbulent vortical structures can only merge.
This creates larger structures promoting an inverse energy cascade as shown in \cite{Kraichnan1967, Leith1968, Batchelor1969}.

\cite{Smith1996} reviewed the aspect ratio depth effect together with a rotation effect of forced turbulence on a $L_x \times L_y \times L_z$ periodic box.
It was found that the turbulence dimensionality of the flow depended not only on the geometry constriction ($L_z/L_x$), but also on the rotation intensity.
A critical ratio between the span and the turbulence forcing scale was revealed below which two-dimensionalisation occurred for non-rotating cases.
Furthermore, it was found that higher rotation rates induced a more significant 2-D turbulence behaviour and that direct and inverse energy cascades for small and large scales can coexist respectively.
\cite{Celani2010} found a similar splitting of the turbulence cascade for a critical value of the relative forcing scale on a depth-restricted periodic box.

\begin{figure}
\vspace*{0.2cm}
  \centerline{\includegraphics[width=0.4\textwidth]{chapter3/fig1}}
  \caption{Sketch of the TKE cascade distribution across the scales as the span is reduced.
By constricting the domain, the integral scale is reduced and the energy-containing scales can feed less energy to the inertial and dissipative scales.}
\label{fig:ek_sketch}
\end{figure}

These differences between 2-D and 3-D turbulence dynamics have a significant practical impact.
For example, the forces induced on a cylinder are larger in magnitude and variability in 2-D systems \citep{Mittal1995, Norberg2003}.
In an attempt to dissipate the energised vortical structures and to prevent the vortex-merging dynamics of 2-D turbulence, models based on the eddy-viscosity hypothesis (a.k.a. the Boussinesq hypothesis) are incorporated into 2-D strip-theory methods.
However, these models assume that the anisotropic Reynolds-stress tensor is proportional to the mean rate-of-strain tensor by the scalar eddy viscosity, a relation which has been proven inaccurate even for simple shear flows \citep[p. 94]{Pope2000} and can potentially yield incorrect forces induced to the cylinder.
A further complication for strip theory is that the presence of walls adds a production mechanism of 3-D turbulence which is able to generate very fine 3-D vortical structures even in highly anisotropic geometries.

A ``thick'' strip-theory method proposed by \cite{Bao2016} showed how strips with a certain thickness are able develop to 3-D turbulence when its span is larger than the wavelength of the Mode B instability of circular cylinders, $\lambda_z/D\approx1$ (see \aref{chapter:appendixA} for a summary of phenomena arising in flow past a circular cylinder at different Reynolds regimes).
In fact, this instability creates rib-like streamwise vortical structures along the main K\'{a}rm\'{a}n 2-D vortices \citep{Noack1999}.
Therefore, it can be argued that the two-dimensionalisation of the wake arises from the geometry constriction which prevents the rib-like vortices developing when there is not room enough for its natural wavelength.
However, the connection between wake and wall turbulence and the persistence of 3-D turbulent structures in constricted span flows has not been fully explored.

This work studies the geometry constriction effect on the turbulence nature of a flow past a circular cylinder at $Re=10^4$.
To do this, a series of simulations ranging from $L_z=10$ to pure 2-D planes have been considered.
As discussed above, the inclusion of a body boundary provides an important change to the turbulence production mechanisms compared to previous research as well as novel information on the transition and cross-over between 3-D and 2-D turbulence for very constricted domains in wall-generated turbulent shear flows.
Multiple turbulence statistics are presented for the wide range of constricted wakes, providing new data on the transition from 3-D to 2-D turbulence.

\section{Problem formulation}\label{sec:circular_cylinder_details}

This study considers flow past a circular cylinder with diameter $D$ aligned on the $z$ direction in a 3-D $\left(35\times20\times L_z\right)D$ rectangular domain, where $L_z$ is the non-dimensional span.
To study the span effect on the turbulence nature of the wake, the following cases have been considered: $L_z=10, \pi, 1, 0.5, 0.25, 0.1$ as well as a fully 2-D case.

\begin{figure}
	\vspace*{0.2cm}
  \centerline{\includegraphics[width=0.8\textwidth]{chapter3/fig2}}
  \caption{Computational domain sketch non-dimensionalised with the cylinder diameter $D$.
  A very fine Cartesian grid domain (depicted in discontinuous lines) surrounds the cylinder and the close- and mid-wake regions.
  A stretched rectilinear grid (depicted in solid lines) transitions from the Cartesian grid to the boundaries.}
\label{fig:computational_domain}
\end{figure}

A Reynolds number of $Re=10^4$ is selected.
We define the Reynolds number as
\begin{equation}
Re=\frac{UD}{\nu},
\end{equation}
where $U$ is the scaling velocity and $\nu$ is the kinematic viscosity of the fluid.
Using this Reynolds number we ensure that the flow is well within the turbulence regime encompassing very different spatial and temporal scales in addition to the availability of DNS data \citep{Dong2005} which is used to validate our results in \aref{chapter:appendixB}.

The incompressible viscous fluid motion is described by the continuity equation and the non-dimensional Navier--Stokes momentum equations
\begin{eqnarray}
&\nabla\cdot\vect{u}=0,\label{eq:mass}\\
&\partial_{t}\vect{u}+\pars{\vect{u}\cdot\nabla} \vect{u}=-\nabla p+Re^{-1}\nabla^2\vect{u}.
\end{eqnarray}
Velocity and pressure fields are coupled with the pressure-Poisson equation.
The initial condition is defined as $\vect{u}\left(\vect{x}, 0\right)=\left(1,0,0\right)$ in the fluid.
The boundary conditions are: a uniform velocity profile on the inlet boundary, a natural convection condition on the outlet boundary, a no-penetration slip condition on the upper and lower boundaries, a periodic condition on the spanwise direction boundaries and a no-slip velocity condition on the cylinder.
Periodic boundary conditions on the constricting planes are used as in previous studies on the cross-over of 2-D and 3-D turbulence for obstacle-free flows such as \cite{Smith1996}, \cite{Celani2010} and \cite{Biancofiore2014}.
This choice is made to avoid the artificial high intensity turbulence enhancements and the deterioration of the 2-D behaviour of the flow when a no-penetration condition is enforced on the constricting planes, as noted on \cite{Biancofiore2012} for obstacle-free turbulent wakes.
Other studies such as \cite{Bao2016} and \cite{Bao2019} also use a periodic spanwise condition for thick strips modelling long circular cylinders.

The numerical methods to solve the governing system of equations have been described in \cref{chapter:theoretical_background}.
Particularly for the circular cylinder case, the domain is composed of a sufficiently fine Cartesian grid for the close- and mid-wake regions defined as $\left(L_x\times L_y\times L_z\right)D$, where $L_x$ and $L_y$ are the non-dimensional horizontal and vertical lengths respectively.
A stretched grid is considered for the regions far from the cylinder (see \fref{fig:computational_domain}).
A resolution of 90 cells per diameter in all spatial directions is chosen for the Cartesian grid subdomain.
The resolution in all spatial directions is kept constant as the span is reduced.

The 3-D simulations are started from a three-dimensionalised 2-D flow snapshot.
A time length of $t^*_0=200$ $(t^* = tU/D)$ is simulated before starting to record the flow statistics in order to achieve a statistically stationary state of the wake.
The flow statistics are then recorded for a total of $\Delta t^*=t^*-t^*_0=500$ convective time units (around 100 wake cycles).
A verification and validation of the wake turbulence dynamics of the investigated test case is included in \aref{chapter:appendixB}.
Finally, the turbulence statistics of the $L_z=10$ and $L_z=\pi$ cases are very similar as displayed in \fref{fig:TKE}a.
Hence, only the $L_z=\pi$ results are displayed on the other figures for clarity.

\section{Results and discussion}\label{Results and discussion}

The flow field is displayed in \fref{fig:instantaneous_vorticity} in terms of the instantaneous vorticity component $\omega_z$ as the span is varied.
The most striking feature is how the coherence of the K\'{a}rm\'{a}n vortices increases as the span is reduced.
However, even in highly-anisotropic geometries such as $L_z=0.25$, small-scale 3-D structures are generated from the cylinder wall.

An important result is that the two-dimensionalisation of these structures is faster (in the sense that it occurs closer to the cylinder) as the domain is constricted because of the geometry constriction and the natural rotation of the K\'{a}rm\'{a}n vortices.
The combination of these two mechanisms as a two-dimensionalisation method is also found in \cite{Smith1996} and \cite{Xia2011}.
For the $L_z\geq1$ cases, the 3-D small-scales structures detaching from the cylinder wall are not two-dimensionalised as rapidly.
In fact, it can be appreciated that only the far wake region of the $L_z=1$ case displays a coherent K\'{a}rm\'{a}n vortex.
This means that less anisotropic geometries promote a direct TKE cascade on the wake so that the 3-D dissipative structures are still sustained far from the cylinder.
\begin{figure}
	\vspace*{0.3cm}
  \centerline{\includegraphics[width=1\textwidth]{chapter3/fig3}}
  \caption{Instantaneous vorticity $\omega_z$ (red is positive, blue is negative) at the $z=L_z/2$ plane for: $(a)$ $L_z=0$, $(b)$ $L_z=0.1$, $(c)$ $L_z=0.25$, $(d)$ $L_z=0.5$, $(e)$ $L_z=1$, $(f)$ $L_z=\pi$.}
\label{fig:instantaneous_vorticity}
\end{figure}

Whether the wake turbulence dynamics is 2-D or 3-D is better captured on the TKE spectra which can be directly compared to classic turbulence theory.
For this, the Taylor's hypothesis is considered and the temporal spectrum at different points of the wake is calculated.
\fref{fig:velocity_spectras} a,b,c shows the temporal power spectrum (PS) of the $v$ velocity component at $(x,y)=(2,0.8),(4,0.8),(8,0.8)$.
The Welch method (using a time signal of 500 units split in 6 parts with 75\% overlapping) has been employed to compute the spectra at 8 different points along the span for each $(x,y)$ point.
These spectra are then averaged resulting in a single spectrum for each case.

First, note that all of the spectra display a peak around the $0.2$ non-dimensional frequency corresponding to the non-dimensional Strouhal number, $St=f_s D/U$, where $f_s$ is the vortex-shedding frequency.
A smaller harmonic peak around $0.4$ non-dimensional frequency is also found for the spectra at $(x,y)=(2,0.8),(4,0.8)$.
Second, the spectrum at the closest analysed point to the cylinder (\fref{fig:velocity_spectras}a) displays a 3-D turbulence behaviour with a $-5/3$ decaying rate with the exception of the pure 2-D and the $L_z=0.1$ cases.
For the latter cases, a decaying slope around $-11/3$ is found.
These 2-D flow spectra are steeper than the $-3$ rate predicted by the classical 2-D turbulence theory \citep{Kraichnan1967} because of the interaction of the coherent large-scale K\'{a}rm\'{a}n vortices, in agreement with the finds of \cite{Dritschel2008} and \cite{Biancofiore2014}.
The filamentary vorticity (filaments of vorticity around the coherent vortices) is likely to be destroyed by the interaction of the large-scale vortices rather than viscous effect (specially for high $Re$ flows), thus limiting its range of scales.
The coherent vortices induce a spiralling effect which limits the range of scales of incoherent filamentary vorticity \citep{Gilbert1988}.
On the other hand, as a $-5/3$ decay rate is captured for the $L_z=0.25$ case, it can be argued that 3-D turbulence is being generated from the cylinder wall even for highly reduced spans.

\begin{figure}
	\vspace*{0.3cm}
  \centerline{\includegraphics[width=1\textwidth]{chapter3/fig4}}
  \caption{Left: vertical velocity component temporal PS at different $\left(x,y\right)$ locations on the wake: $\left(a\right)=\left(2,0.8\right)$, $\left(b\right)=\left(4,0.8\right)$, $\left(c\right)=\left(8,0.8\right)$.
The PS lines of each case are shifted a factor of 10 for clarity and the vertical axis ticks correspond to the $L_z=\pi$ case.
The dashed lines have a $-11/3$ slope and the dotted lines have a $-5/3$ slope.
Right: two-point correlations along $z$ at the same $(x,y)$ locations as the left figures respectively.
The correlation value $\chi_{12}$ for a given $d$ corresponds to the averaged value of the multiple correlations of pairs of points separated a distance $d$ along the span.}
\label{fig:velocity_spectras}
\end{figure}

The $L_z=0.25$ and $L_z=0.5$ cases feature a decay rate that transitions from $-5/3$ to $-11/3$ as the spectra are computed further downstream from the cylinder.
In particular, both cases have $-5/3$ slopes in \fref{fig:velocity_spectras}a and $-11/3$ in \fref{fig:velocity_spectras}c, quantifying the turbulent two-dimensionalisation in the rotating wake.
Also note the coexistence of both 2-D and 3-D turbulence features for the $L_z=0.25,0.5$ and $L_z=1$ cases in \fref{fig:velocity_spectras}b and \fref{fig:velocity_spectras}c respectively.
The low-frequency structures behave mostly two-dimensionally (decay rate between $-3$ and $-11/3$, resulting from the presence of less coherent 2-D structures than the pure 2-D case) up to a certain point where a $-5/3$ rate is briefly recovered.
Hence, high-frequency structures interact in a 3-D fashion while low-frequency structures interact two-dimensionally.
This finding is in agreement with \cite{Smith1996} and \cite{Celani2010}.

Additionally, two-point correlations along the span have been analysed at the same $(x,y)$ locations as the PS plots (\fref{fig:velocity_spectras} d,e,f).
Given a distance $d$ along $z$ ranging from 0 to $L_z/2$, the two-point correlation is calculated with the temporal signals of the vertical velocity component at multiple pairs of points (namely $v_1$ and $v_2$) as follows
\begin{equation}
\chi_{12}\left(v_1(\vect{x},t), v_2(\vect{x}+\vect{r},t) \right) = \frac{\mathrm{cov}(v_1, v_2)}{\sqrt{\mathrm{cov}(v_1, v_1)\mathrm{cov}(v_2, v_2)}},
\end{equation}
where the distance vector is defined as $\vect{r}=(0,0,d)$.
The multiple correlation coefficients for a given $d$ are then averaged corresponding to a data point in the plots.

Very close to the cylinder (\fref{fig:velocity_spectras}d), the correlation coefficient quickly decreases with increasing $d$ for $L_z>0.1$.
This indicates the presence of 3-D structures near the body as also noted on the velocity spectrum plot counterpart.
Also, the decrease is more pronounced as the span increases.
For $L_z=\pi$, it is worth noting a local correlation coefficient maximum around $d \approx 0.9$.
This distance approximately corresponds to the Mode B instability wavelength $(\lambda_z)$.
Since the rib-like vortices associated with the Mode B instability (streamwise and crossflow vorticity) are not very well defined at this $Re$ regime \citep{Chyu1996}, the correlation increase is not as significant as at lower $Re$.
Still, $L_z=\pi$ is the only case displaying such phenomena because of the spanwise boundary conditions periodicity, which only allow the instability to develop if $L_z>\lambda_z$ (the $L_z=1$ case might be too critical to display such phenomenon considering also its intermittent nature).
The correlation coefficient increases when calculated further downstream as shown in \fref{fig:velocity_spectras}e and \ref{fig:velocity_spectras}f, evidencing again the wake two-dimensionalisation.

\begin{figure}
	\vspace*{0.3cm}
  \centerline{\includegraphics[width=1\textwidth]{chapter3/fig5}}
  \caption{TKE spatial plots.
The TKE is computed from the normal Reynolds stresses and averaged on the vertical direction: $(a)$ total TKE, $(b)$ ratio of the spanwise component $\overline{w'w'}$ to the total TKE.}
\label{fig:TKE}
\end{figure}

In summary, the transition to a 2-D wake is found at a certain point along the wake on all the cases with $L_z\le1$.
The combination of the large-scale rotation from the K\'{a}rm\'{a}n vortices plus the geometry constriction are mainly responsible for this phenomenon.
The main physical mechanism differing among the compared cases is the ability of the flow to develop Mode B-like 3-D structures in the wake as a result of a sufficiently long span.
The current $Re$ regime is characterised by a transition to turbulent flow at the shear layer (i.e. the TrSL2 regime) as reported in \cite{Bloor1964} and \cite{Kourta1987}.
We argue that when the span is too short for the Mode B instability to develop and thereby sustain these 3-D turbulent structures, the stratification effect of the K\'{a}rm\'{a}n vortices leads to the more coherent and energised wake seen in \fref{fig:velocity_spectras} d,e,f.

Next, the TKE along the $x$ direction, the Lumley triangle of turbulence and the ratio between the vortex-stretching and vortex-advecting terms are examined to further support the observed phenomena.
The TKE is defined as
\begin{equation}
TKE = \frac{1}{2}\left(\overline{u'u'}+\overline{v'v'}+\overline{w'w'}\right),
\end{equation}
where $\overline{\cdot}$ denotes a time average plus a spanwise average and the superscript $(\cdot)'$ denotes a fluctuating quantity such as $a'=a-\overline{a}$.
The six components of the Reynolds-stress tensor $\overline{u_i' u_j'}$ have been computed using the following relation
\begin{equation}
\overline{a'b'} = \overline{ab} - \overline{a}\overline{b}.
\end{equation}

\fref{fig:TKE} shows the streamwise spatial distribution of the TKE averaged along the $y$ direction from $-L_y/2$ to $L_y/2$ (noted as $\cdot|_y$).
From a general point of view, it can be observed that the total TKE (\fref{fig:TKE}a) peaks right after the recirculation region noting that the latter increases slightly with the span.
Also, the total TKE increases as the span is reduced because of the 2-D vortex-merging processes that generate larger and more energised vortical structures.
The contribution of the spanwise normal stress to the total TKE increases with the span as shown in \fref{fig:TKE}b.
Also, a decay of $\overline{w'w'}$ right after being generated from the cylinder wall can be noted and it becomes faster as the span is constricted.

The effect of the span on the TKE compared to the lift coefficient root mean square (r.m.s.) value $(\overline{C}_L)$ displays a quasi-linear relation as shown in \fref{fig:TKE_CL}.
Note that the drop on both values is more sensitive to the span constriction at the range where both 2-D and 3-D turbulence dynamics coexist, i.e. $0.25\le L_z\le1$.
On the other hand, a very small change of both values can be appreciated when 3-D turbulence fully dominates the wake, i.e. $L_z\ge\pi$.
Furthermore, as shown in \fref{fig:TKE}a, highly constricted domains yield large values of TKE because of the energised 2-D large-scale vortical structures present at the wake.
Even when small-scale 3-D structures are present in the close wake region and coexist with the large 2-D structures, the values of both the TKE and the $\overline{C}_L$ spike.

\begin{figure}
 \vspace*{0.3cm}
  \centerline{\includegraphics[width=0.5\textwidth]{chapter3/fig6}}
  \caption{Effect of the span constriction on the TKE and the $\overline{C}_L$.
The latter is calculated as $\overline{C}_L=|2F_y/(\rho U_\infty^2 D L_z)|_{\mathrm{rms}}$, where $F_y$ is the vertical lift force, $U_\infty$ is the free-stream velocity, and $\rho$ is the constant fluid density.}
\label{fig:TKE_CL}
\end{figure}

\begin{figure}
	\vspace*{0.3cm}
  \centerline{\includegraphics[width=0.9\textwidth]{chapter3/fig7}}
  \caption{Top: $C_L$ and $C_D$ temporal signals.
The latter is calculated as $C_D=2F_x/(\rho U_\infty^2 D L_z)$ and the former as detailed in \fref{fig:TKE_CL} (without the r.m.s norm).
Middle: upper $\theta_u$ and lower $\theta_l$ separation angles temporal signal.
The separation angle is calculated from the front stagnation point of the cylinder to the free shear layer separation point (vorticity practically 0 at the wall).
Bottom: correlation between the lift coefficient and the difference between $\theta_u$ and $360-\theta_l$.
Left: $L_z=\pi$.
Right: $L_z=0.5$.}
\label{fig:separation}
\end{figure}

In \fref{fig:separation}, the temporal signals of the lift and drag coefficients as well as the upper and lower separation angles of the free-shear layer are displayed for the $L_z=0.5$ and the $L_z=\pi$ cases.
The oscillation of the separation points increases as the span is constricted inducing larger forces on the cylinder.
A very high correlation of the upper and lower separation points angle with the lift coefficient is shown as well.
Again, the lack of the Mode B instability on the constricted cases yields more coherent 2-D vortical structures in the near wake region.
Therefore, the small-scale 3-D structures which normally dissipate most of the kinetic energy in 3-D turbulence are not present leading to high-intensity vortices.
This can be quantified by the enstrophy ($Z$) of the spanwise-averaged spanwise vorticity in the near wake region, $\Omega$ ($x \in [0.55, 2.1], y \in [-0.8, 0.8]$), which is time averaged for the same temporal length as the body forces signals as follows
\begin{equation}
Z = \int_\Omega\frac{1}{t_2-t_1}\int^{t_2}_{t_1}\frac{1}{L_z}\int^{L_z}_{0} \omega_z^{2} \,\,\mathrm{d}z\,\mathrm{d}t\,\mathrm{d}\Omega.
\label{eq:enstrophy}
\end{equation}
It can be observed in \tref{tab:enstrophy} that the enstrophy increases as the span is reduced, which can be understood as an increase of the rotational energy of the flow.

\begin{table}
  \begin{center}
\def~{\hphantom{0}}
  \begin{tabular}{lrrrrr}
  \toprule
       $L_z$ & $\pi$ & $1$  & $0.5$ & $0.25$ & $0.1$ \\
       \midrule
       $Z/Z_\pi$ & 1 & 1.32 & 1.71  & 2.07   & 2.47  \\
      \bottomrule
  \end{tabular}
  \caption{Near body enstrophy as defined in equation \ref{eq:enstrophy} for the different span cases.
$Z_\pi$ is the enstrophy of the $L_z=\pi$ case.}
  \label{tab:enstrophy}
  \end{center}
\end{table}

The highly energised coherent vortices forming at the near wake region for the constricted cases induce a larger convective force on the free-shear layer.
This translates to the large oscillations observed in \fref{fig:separation} and, ultimately, to the forces induced on the cylinder.
With this, it can be argued that the coherent 2-D structures have a greater impact on the forces induced to the cylinder than the 3-D small-scale structures when both are present.

\cite{Lumley1977} proposed the Lumley triangle of turbulence which provides a way to classify the anisotropic state of turbulence.
The anisotropy property of the Reynolds-stress tensor can be extracted with
\begin{equation}
b_{ij} = \frac{\overline{u_i' u_j'}}{\overline{u_k' u_k'}} - \frac{1}{3}\delta_{ij},
\end{equation}
where $\delta_{ij}$ is the Kronecker delta tensor and $b_{ij}$ is the anisotropic Reynolds-stress tensor (which evidently vanishes for isotropic turbulence).
This dimensionless and traceless tensor has two non-zero invariants, $II=-b_{ij}b_{ji}/2$ and $III=b_{ij}b_{jk}b_{ki}/3$.
These invariants are often rewritten as $\eta^2=-II/3$ and $\xi^3=III/2$ to better appreciate the nonlinear behaviour of the trajectories of return to isotropy of homogeneous turbulence \citep{Choi2001}.
It has been shown that all the possible turbulence states are mapped within the triangle \citep{Lumley1977, Lumley1979}.

The different states of turbulence are classified in the triangle as follows: the top right elbow indicates a one-dimensional (or one component) state with a single non-zero eigenvalue (the eigenvalues can be understood in physical terms as the normal stresses in the principal axes of the anisotropic Reynolds-stress tensor).
The top left elbow indicates a 2-D isotropic state where one eigenvalue vanishes and the two remaining are equal.
The top curve connecting the elbows indicates a 2-D turbulence state where one eigenvalue vanishes and the addition of the two remaining eigenvalues is constant.
The left and right straight lines correspond to a negative or positive $\left( \xi \right)$ axisymmetric state since one eigenvalue is smaller than the other two (which are equal) or greater than the other two (which are equal) respectively.
Finally, the $(0,0)$ point indicates 3-D isotropic turbulence since all of the anisotropic tensor eigenvalues vanish.

\begin{figure}
	\vspace*{0.3cm}
  \centerline{\includegraphics[width=1\textwidth]{chapter3/fig8}}
  \caption{Lumley triangle constructed across the wake width at: $(a)$ $x=2$, $(b)$ $x=4$, $(c)$ $x=6$, $(d)$ $x=8$.}
\label{fig:lumleys_triangle}
\end{figure}

\fref{fig:lumleys_triangle} displays the Lumley triangle constructed across the wake width at different $x$ locations.
Hence, every point in the triangle corresponds to a point in the domain and the collection of points for each case represents the collection of vertically aligned points in the domain at different $x$ locations on the wake.
It can be appreciated that all of the points are located within the triangle, therefore all of the computed Reynolds stresses are realisable (i.e. have positive and real eigenvalues).
In general, most of the cases transition from one state to another state of turbulence across the wake width.
Only for the almost 2-D case $\left(L_z=0.1\right)$, the state of turbulence is always 2-D (or two-component), since $\overline{w'w'}$ is effectively negligible everywhere.

As the triangle is constructed further downstream on the wake, the trajectories (or collection of points) of the $L_z=0.25$ and $L_z=0.5$ cases move closer to the 2-D turbulence state location (upper curve).
This emphasises once again the wake two-dimensionalisation caused by the large-scale vortical structures on cases with critical span which contain a cross-over of 2-D and 3-D turbulence, as shown in previous results.
In contrast, the $L_z=1$ and $L_z=\pi$ cases remain approximately at the same $\eta$ region showing that the two-dimensionalisation is not as effective.

Additionally, the trajectories present a negative axisymmetric almost two-component state for the locations far from the wake centreline, i.e. the location close to $y=\pm L_y/2$, since one of the normal turbulent stresses $(\overline{w'w'})$ is smaller than the others.
On the centreline, $\overline{v'v'}$ is larger than the other stresses causing a shift to the positive axisymmetric state.

A comparison of the different cases at the same $x$ location also shows a more noticeable 2-D turbulence state as the span is constricted.
This difference is less noticeable close to the cylinder (\fref{fig:lumleys_triangle}a), where even the $L_z=0.5$ case trajectory resembles the $L_z =\pi$ case.
Again, this evidences that 3-D turbulence is present close to the wall even in considerably constricted cases.

Consider now the vorticity transport equation (VTE) which can be written as
\begin{equation}
\partial_{t}\boldsymbol\omega+\pars{\vect{u}\cdot\nabla} \boldsymbol\omega=\pars{\boldsymbol\omega\cdot\nabla}\vect{u}+Re^{-1}\nabla^2\boldsymbol\omega,
\end{equation}
where $\boldsymbol\omega\left(\vect{x}, t\right) = \left(\omega_x, \omega_y, \omega_z\right)$ is the vorticity vector field defined as $\boldsymbol\omega=\nabla\times\vect{u}$.
The vortex-stretching term, $\pars{\boldsymbol\omega\cdot\nabla}\vect{u}$, is often pointed to as the term responsible for the direct energy cascade of the TKE.
The stretching of a vortex tube causes a reduction on its diameter while increasing the rotation speed of the vortex by conservation of angular momentum.
This term vanishes in the 2-D formulation of the VTE since the stretch of the vortex tube is perpendicular to the plane of rotation.
From this mathematical and physical difference, different turbulence dynamics is captured in 2-D or 3-D computations and, therefore, 3-D turbulence can be directly linked to this term.

\fref{fig:vortex_stretching} displays the modulus of the mean vortex-stretching term and its ratio $R$ to the mean vortex-advecting term.
These quantities are averaged in the vertical direction.
On \fref{fig:vortex_stretching}a, it can be observed that the vortex-stretching term decays faster while moving downstream from the cylinder for cases with shorter span.
The vortex-stretching decay demonstrates again the two-dimensionalisation of the flow by the large-scale vortices.
The ratio $R$ in \fref{fig:vortex_stretching}b shows that the vortex-advecting term decays faster (because of the wake momentum deficit) than the vortex-stretching term along the streamwise direction up to $x=6$ where the ratio is kept constant.
It can also be observed that the vortex-stretching term becomes as important as the vortex-advecting term with increasing span.
\begin{figure}
	\vspace*{0.3cm}
  \centerline{\includegraphics[width=1\textwidth]{chapter3/fig9}}
  \caption{$(a)$ Modulus of the mean vortex-stretching term averaged along the vertical direction.
$(b)$ Ratio of the modulus of the mean vortex-stretching term to the modulus of the mean vortex-advecting term averaged along the vertical direction.}
\label{fig:vortex_stretching}
\end{figure}

\section{Conclusion}\label{Conclusion}

The span effect on the turbulence dynamics of a flow past a circular cylinder at $Re=10^4$ has been investigated using spectra  and two-point correlations at different locations in the domain, the TKE along the wake, the separation points, the Lumley triangle of turbulence and the mean vortex-stretching term of the VTE.
It has been shown that 3-D turbulence is present even for highly constricted cases (for example $L_z=0.25$) which is generated by the cylinder wall (\fref{fig:velocity_spectras}a).
However, the small-scale structures rapidly get two-dimensionalised by the large-scale K\'{a}rm\'{a}n vortices when the span is 50\% of the diameter or less.
This is linked to the Mode B instability wavelength being longer than the periodic span, in agreement with \cite{Bao2016}.
Since the Mode B instability helps sustaining the turbulent structures advected from the shear layer, the lack of it prevents large-scale 3-D structures to be created and less dissipative structures can be sustained.
In this scenario, 2-D turbulence takes over and dominates the wake dynamics creating larger, stronger and more coherent vortices.
Ultimately, the coherent and energised vortical structures induce a larger convective force on the free-shear layer.
This translates to larger oscillations and, finally, higher forces on the cylinder.

The flow turbulence transition from 3-D to 2-D caused by a geometry constriction found in this work is in agreement with the physical mechanisms described in the obstacle-free turbulence work of \cite{Smith1996}, \cite{Celani2010} and \cite{Biancofiore2014}.
In the present study with solid boundaries, the main difference is found in the presence of small-scale 3-D turbulence even in highly constricted geometries $\left(L_z=0.25\right)$ which leads to a coexistence of 2-D and 3-D turbulence close to the cylinder wall.
This is observed not only in the $L_z=0.25$ case, but also for the $L_z=0.5$ and $L_z=1$ cases as shown in \fref{fig:velocity_spectras}b and \fref{fig:velocity_spectras}c respectively.
Note that the cross-over between 2-D and 3-D turbulence dynamics arises at different points in the spatial domain depending on the span length.
The shorter the span, the closer to the cylinder it takes place evidencing that the wake two-dimensionalisation transitions at different locations in function of the domain geometric anisotropy.

On the other hand, a very rapid two-dimensionalisation is found in the present cases because of the natural large-scale rotation motion of the K\'{a}rm\'{a}n vortices.
A large-scale rotation as a mechanism of two-dimensionalisation has been also found in other works such as \cite{Smith1996} and \cite{Xia2011}.
These two mechanisms combined yield to a rapid transition from the 3-D to 2-D turbulence dynamics when the span is shorter than the Mode B instability wavelength of the wake.
% ----------------------------------------------------------------
\end{document}


\documentclass[a4paper]{article}
\usepackage[letterpaper, margin=1in]{geometry} % page format
\usepackage{listings} % this package is for including code
\usepackage{graphicx} % this package is for including figures
\usepackage{amsmath}  % this package is for math and matrices
\usepackage{amsfonts} % this package is for math fonts
\usepackage{tikz} % for drawings
\usepackage{hyperref} % for urls
\usepackage{amsmath} %for math
\usepackage[shortlabels]{enumitem} % for the cool itemize letters
\usepackage{graphicx}
\usepackage{calligra}

\DeclareMathOperator*{\argmin}{\arg\min}

\title{Homework 0}
\author{Amy Pitts}
\date{1/23/19}

\begin{document}
\lstset{language=Python}

\maketitle

\section{Instructions}
\textit{No deliverables}

\section{Course Setup}

\subsection{Python}
After downloading all the required packages running
this script shows that everything is install and
their versions.
%proof that the intalls worked
\lstinputlisting[language=Python,frame=single]{install_check.py}
Output of the script file  \\
\includegraphics{python_downloads}

\subsection{GitHub Class Repository}

- Github name and class repository: \\
\includegraphics{github_name}\\
\bigskip
- Link to class repository: \textit{https://github.com/amypitts01/data440/invitations} \\
\bigskip
- Proof that pablorp80 was asked to be a collaborator: \\
\includegraphics[width=1\textwidth]{proof_of_collab}

\subsection{Kaggle Account}
My Kaggle user name is: \textbf{Amypitts01} \\
\includegraphics[width=.9\textwidth]{kaggle_proof}

\section{Mathematics Requirements}
\textit{No deliverables}

\section{Problems}

%-------------------QUESTION 1----------------------
%---------------------------------------------------
1. For the function $g(x) = -3x^2+24x-30$, find the value for x
that maximizes $g(x)$.
\smallskip
\\
\indent \textbf{Solution:} To maximize this function the first step
is to take the derivative such that $g'(x) = -6x+24$. Setting
the derivative equal to 0 to find the critical values we have:
\begin{align*}
-6x+24&=0 \\
-6(x-4)&=0\\
x-4&=0\\
x=4.
\end{align*}
The maximun occurs when the derivative is positive left of the
critical value and negative right of the critical value. Since
there is only one critical value we do not need to compare to
other critical values to find the biggest $y$ value.
To show that $4$ is in fact a max we see that:\\
\indent Left: $g'(0)=-6(0)+24=24$ \textit{positive} \\
\indent Right: $g'(10)=-6(10)+24=-36$ \textit{negative} \\
Therefore, $x=4$ maximizes the function.
\\
\\
%-------------------QUESTION 2----------------------
%---------------------------------------------------
2. Consider the following function:
\begin{equation*}
f(x)=3x_0^3-2x_0x_1^2+4x_1-8
\end{equation*}
what are the partial derivatives of $f(x)$ with respect to $x_0$ and $x_1$.
\\
\textbf{Solution:}
\\
\begin{align*}
\frac{\partial f}{\partial x_0}&= 9x^2_0-2x_1^2 \\
\frac{\partial f}{\partial x_1}&= -4x_0x_1+4
\end{align*}
\\
\\
%-------------------QUESTION 3----------------------
%---------------------------------------------------
3. Consider the matrix
$ A=\left[\begin{matrix}
1 & 4 & -3 \\
2 & -1 & 3
\end{matrix}\right] $ and
$ B=\left[\begin{matrix}
-2 & 0 & 5 \\
0 & -1 & 4
\end{matrix}\right] $m then answer the following and verify your answers in Python:
\begin{enumerate}[(a)] % (a), (b), (c), ...
\item can you multiply the two matrices? ellaborate on your answer.
\begin{itemize}
  \item \textbf{Solution:} No you can not multiply the two
  becuase $A$ does not have the same number of collumns and
  $B$'s rows. They are both $(2,3)$ matrices and to
  be able to multiply $A$ would need to be $(2,3)$ and $B$ dims $(3,2)$
  or $A$ could be $(3,2)$ and $B$ $(2,3)$.
\end{itemize}
\item multiply $A^T$ and $B$ and give its \textit{rank}.
\begin{itemize}
  \item \textbf{Solution:} First we need to find the transopes
  of $A$. This results in $A^T = \left[\begin{matrix}
    1 & 2   \\
    4 & -1  \\
    -3 & 3
  \end{matrix}\right] $. Now we can multiply $A^TB$. This gives us
  $A^TB = \left[\begin{matrix} 1 & 2 \\ 4 & -1 \\-3 & 3 \end{matrix}\right]
  \left[\begin{matrix}  -2 & 0 & 5 \\ 0 & -1 & 4 \end{matrix}\right] \\=
    \left[\begin{matrix}
      (1)(-2)+(2)(0) & (1)(0)+(2)(-1) & (1)(5)+(2)(4) \\
      (4)(-2)+(-1)(0) & (4)(0)+(-1)(-1) & (4)(5)+(-1)(4) \\
      (-3)(-2)+(3)(0) & (-3)(0)+(3)(-1) & (-3)(5)+(3)(4)
    \end{matrix}\right] \\
    = \left[\begin{matrix}
      -2 & -2 & 13 \\
      -8 & 1 & 16 \\
      6 & -3 & -3
    \end{matrix}\right]
    $.
  \item \textbf{Solution:} Next to find the rank we need to
  row reduce until we reach row echelon form. Then we have
  $ \left[\begin{matrix}
    -2 & -2 & 13 \\
    -8 & 1 & 16 \\
    6 & -3 & -3
  \end{matrix}\right] \rightarrow
  \left[\begin{matrix}
    1 & 1 & \frac{-13}{2} \\
    -8 & 1 & 16 \\
    6 & -3 & -3
  \end{matrix}\right] \rightarrow
  \left[\begin{matrix}
    1 & 1 & \frac{-13}{2} \\
    0 & 9 & -36 \\
    0 & -9 & 36
  \end{matrix}\right] \rightarrow
  \left[\begin{matrix}
    1 & 1 & \frac{-13}{2} \\
    0 & 1 & -4 \\
    0 & 0 & 0
  \end{matrix}\right]
  $.
  The row eachelon of $A^TB$ shows two independent row vectors
  therefore the \textbf{rank is 2}.
\end{itemize}
\item (extra credit) let $c=\left[\begin{matrix} 1 & 0  \\ 0 & 2
  \end{matrix}\right] $ be a new matrix; what is the result of $AB^T+C^{-1}$?
  \begin{itemize}
    \item \textbf{Solution:} First we need to find $c^{-1}$. This gives us \\
    $c^{-1} = \frac{1}{ad-bc}
      \left[\begin{matrix} d & -b \\ -c & a \end{matrix}\right]
      = \frac{1}{2} \left[\begin{matrix} 2 & 0 \\ 0 & 1 \end{matrix}\right]
      =  \left[\begin{matrix} 1 & 0 \\0 & \frac{1}{2} \end{matrix}\right]$. \\
    Now we need to find $B^T$. Then we have
    $B^T = \left[\begin{matrix}-2 & 0 \\0 & -1 \\4 & 5 \end{matrix}\right]$.
    Now we can find $AB^T$. Thus we have
    $AB^T = \left[\begin{matrix}1 & 4 & -3 \\2 & -1 & 3\end{matrix}\right]
      \left[\begin{matrix} -2 & 0  \\0 & -1 \\ 4 & 5 \end{matrix}\right]
      = \left[\begin{matrix} -14 & -19  \\8 & 16 \end{matrix}\right]$.
    Then adding in $C^{-1}$ we get \\
    $AB^T+C^{-1} = \left[\begin{matrix} -14 & -19 \\8 & 16 \end{matrix}\right]
      + \left[\begin{matrix} 2 & 0  \\0 & \frac{1}{2} \end{matrix}\right]
      = \left[\begin{matrix} -13 & -19  \\ 8 & 16.5 \end{matrix}\right].$
  \end{itemize}
\end{enumerate}

\indent \textbf{Solution:}
Here is the scipt I ran to check the matrices.
\lstinputlisting[language=Python,frame=single]{matrix_checks.py}
Python answer for 3.a \\
\includegraphics[width=1\textwidth]{matrix_errors}
\smallskip \\
Python answer for 3.b \\
\includegraphics[width=.6\textwidth]{matrix_answers}
\\
%-------------------QUESTION 4----------------------
%---------------------------------------------------
4. Give the mathematical definitions of the simple Gaussian,
multivariate Gaussian, Bernoulli, binomial, and exponential distributions.
\\
\indent \textbf{Solutions:}
\smallskip
\\
\indent \textbf{Gaussian:} This is also known as the normal distibution
and has a pdf of $f_X(x) = \frac{1}{\sigma \sqrt{2\pi}}exp\{- \frac{(x-\mu)^2}{2\sigma^2} \}$
and it is notation is $N(\mu,\sigma^2)$.
\smallskip
\\
\indent \textbf{Multivariate Gaussian:} This distrabution has
a pdf of $f_X(x)=(2\pi)^\frac{k}{\Sigma} |\Sigma|^{\frac{-1}{2}} e^{\frac{-1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu)}$
and it is notation is MVN$(\mu,\Sigma)$.
\smallskip
\\
\indent \textbf{Bernoulli:} This distrabution has a pdf of
$f_X(x) = p^x(1-p)^{1-x}$ with notation Bern$(p)$.
\smallskip
\\
\indent \textbf{binomial:} This distrabution has a pdf of
$f_X(x)= \binom{n}{k} p^x(1-p)^{n-x}$ with notation Bin$(n,p)$.
\smallskip
\\
\indent \textbf{exponential distributions:} This distrabution has a pdf of
$f(x;\lambda)= \begin{cases} \lambda e^{-\lambda x} & x\geq0 \\ 0 & x<0 \end{cases}$.
\bigskip
\\
%-------------------QUESTION 5----------------------
%---------------------------------------------------
5. (extra credit) What is the relationship between the Bernoulli
and binomial.
\\
\indent \textbf{Solution:} The Bernoulli distributions is made up of Bernoulli
trails where an even occurs $p$ or the even does not occur $1-p$.
There is only two options success or failure. When you have $n$
independent Bernoulli trails those successes and failures are
represented in a Binomial Distribution.
%https://www.unf.edu/~cwinton/html/cop4300/s09/class.notes/DiscreteDist.pdf
\\
\\
%-------------------QUESTION 6----------------------
%---------------------------------------------------
6. Suppoe that random variable $X \sim N(2,3)$. What is its expected
value?
\\
\indent \textbf{Solution:} $ N(2,3)$ is a normal distrabution it has a
$E[x]=\mu$. Since $N(\mu,\sigma^2)$ we have that in our case $E[x]=2$.
\\
\\
%-------------------QUESTION 7----------------------
%---------------------------------------------------
7. An \textit{euclidean projection} of a $d$-dimensional point
$y\in \mathbb{R}^d$ to a set $\mathcal{Z}$ is given by the following
optimzation problem:
\begin{center}
$x^*=\argmin_x ||x-y||^2_2$, subject to $x\in \mathcal{Z}$
\end{center}
where $\mathcal{Z}$ is the feasible set, $||\cdot||_2$ is the
\textit{l}$_2$-norm (euclidean) of a vector, and $x^*\in\mathbb{R}^d$
is the projected vector.
\begin{enumerate}[(a)]
\item What is $x^*$ if $y=1.1$ and $\mathcal{Z}=\mathbb{N}$?
  \begin{itemize}
    \item \textbf{Solution:} the $\argmin_x ||x-y||^2_2$ is the
    value of $x$ for which $||x-y||^2_2$ attains it's min value.
    Since $x \in \mathcal{Z} = \mathbb{N}$ we can use trial and errors
    to find the min. \\
    First we looks at $x=1$ we have: $||1-1.1||^2_2 = \sqrt{(1-1.1)\cdot (1-1.1)}^2 =0.01$. \\
    Finally looking at $x=2$ we have $||2-1.1||^2_2 = \sqrt{(2-1.1)\cdot (2-1.1)}^2= .81$. \\
    Because the natural numbers only increase from here the smallest
    value is going to be made when $x=1$ obtaining a $x^*=0.01$.
  \end{itemize}
\item Locate $x^*$ in the following picture:\\
The red dot should be the closest point on the
perimeter of the $\mathcal{Z}$. \\
  \includegraphics[width=.5\textwidth]{pic_7_answer.png} \\

\end{enumerate}
\textbf{Solution:}
\\
\\
%-------------------QUESTION 8----------------------
%---------------------------------------------------
8. Suppose that random variable $Y$ has distribution:
\begin{center}
$p(Y=y)=\begin{cases}
e^{-y} & y\ge 0\\
0 & otherwise
\end{cases}$
\end{center}
\begin{enumerate}[(a)]
\item Verify that $\int_{y=-\infty}^{\infty} p(Y=y) dy=1 $
\item What is $\mu_y = E[Y]= \int_{y=-\infty}^{\infty} p(Y=y)ydy$; that is, the expected value of $Y$?
\item What is $\sigma^2=Var[Y]=\int_{y=-\infty}^{\infty} p(Y=y)(y-\mu_y)^2dy$; that is, the variance of $Y$?
\item What is $E[Y|Y\geq 10]$; that is, the expected value of $Y$, given that (or conditioned on) $Y \geq 10$?
\end{enumerate}

\textbf{8.a Solution:}
\begin{align*}
\int_{y=-\infty}^{\infty} p(Y=y) dy &= 1 \\
\int_{-\infty}^{0} 0 dy + \int_{0}^{\infty} e^{-y} dy &= 1 \\
0 - e^{-y} |^{\infty}_0  &= 1 \\
1  &= 1.
\end{align*}

\textbf{8.b Solution:}
\begin{align*}
\mu_y = E[Y] &= \int_{y=-\infty}^{\infty} p(Y=y)y dy \\
&= \int_{-\infty}^{0} 0 dy + \int_{0}^{\infty} ye^{-y} dy \\
&= 0+\int_{0}^{\infty} ye^{-y} dy
\end{align*}
Now we need to do integration by parts. Using $u=y$,
$du=1dy$, $v=-e^{-y}$ and $dv=e^{-y}$ we have:
\begin{align*}
 \mu_y&= \int_{0}^{\infty} ye^{-y} dy \\
 &= -ye^{-y}|_{0}^{\infty} + \int_{0}^{\infty} e^{-y} dy \\
 &= 0 -e^{-y}|_{0}^{\infty} \\
 &= 1.
\end{align*}

\textbf{8.c Solution:}
\begin{align*}
\sigma^2=Var[Y]&=\int_{y=-\infty}^{\infty} p(Y=y)(y-\mu_y)^2dy \\
&=\int_{-\infty}^{0} 0(y-\mu_y)^2dy + \int_{0}^{\infty} e^{-y}(y-\mu_y)^2dy\\
&= \int_{0}^{\infty} e^{-y}(y^2-2y\mu_y+\mu_y^2)dy\\
&= \int_{0}^{\infty} e^{-y}y^2 dy -\int_{0}^{\infty}2e^{-y}y\mu_y dy+\int_{0}^{\infty}e^{-y}\mu_y^2dy
\end{align*}
Now looking at each integral individually we have
\begin{align*}
\int_{0}^{\infty} e^{-y}y^2 dy &= -y^2e^{-y}|_{0}^{\infty} - 2\int_{0}^{\infty} e^{-y}y dy\\
&= 0 +2E[Y] \\
&=  2.
\end{align*}

\begin{align*}
-\int_{0}^{\infty}2e^{-y}y\mu_y dy &= -\int_{0}^{\infty}2e^{-y}y\mu_y dy\\
&= -2\mu_y\int_{0}^{\infty}e^{-y}y dy\\
&= -2\mu_yE[Y]\\
&= -2\mu_y.
\end{align*}

\begin{align*}
\int_{0}^{\infty}e^{-y}\mu_y^2dy = \mu_y^2\int_{0}^{\infty}e^{-y}dy = \mu_y^2.
\end{align*}
Putting this all together we have that
\begin{align*}
Var[Y]= \int_{0}^{\infty} e^{-y}y^2 dy -\int_{0}^{\infty}2e^{-y}y\mu_y dy+\int_{0}^{\infty}e^{-y}\mu_y^2dy
= 2 -2\mu_y+\mu_y^2.
\end{align*}
Using the $\mu_y$ we found from above we have $2 -2\mu_y+\mu_y^2= 2-2+1^2=1$.
Therefore, $Var[Y]=1$.
\medskip
\\
\textbf{8.d Solution:}
\begin{align*}
E[Y|Y\geq 10] &= \int_{10}^{\infty} p(Y=y)y dy \\
&= \int_{10}^{\infty} ye^{-y} dy
\end{align*}
Now we need to do integration by parts. Using $u=y$,
$du=1dy$, $v=-e^{-y}$ and $dv=e^{-y}$ we have:
\begin{align*}
\int_{10}^{\infty} ye^{-y} dy
 &= -ye^{-y}|_{10}^{\infty} + \int_{10}^{\infty} e^{-y} dy \\
 &= 0 +10e^{-10}-e^{-y}|_{10}^{\infty} \\
 &= 10e^{-10}+e^{-10} \\
 &= 11e^{-10}.
\end{align*}

\end{document}


\documentclass[aspectratio=169]{beamer}

\mode<presentation>
{
  \setbeamertemplate{background canvas}[square]
  \pgfdeclareimage[width=6em,interpolate=true]{dsailogo}{../dsai-logo}
  \pgfdeclareimage[width=6em,interpolate=true]{erasmuslogo}{../erasmus-logo}
  \titlegraphic{\pgfuseimage{dsailogo} \hspace{0.2in} \pgfuseimage{erasmuslogo}}
  %\usetheme{default}
  \usetheme{Madrid}
  \usecolortheme{rose}
  \usefonttheme[onlysmall]{structurebold}
}

\usepackage{pgf,pgfarrows,pgfnodes,pgfautomata,pgfheaps,pgfshade}
\usepackage{amsmath,amssymb}
\usepackage{graphics}
\usepackage{ragged2e}
\usepackage[latin1]{inputenc}
\usepackage{colortbl}
\usepackage[absolute,overlay]{textpos}
\setlength{\TPHorizModule}{30mm}
\setlength{\TPVertModule}{\TPHorizModule}
\textblockorigin{10mm}{10mm}
\usepackage[english]{babel}
\setbeamercovered{dynamic}

\AtBeginSection[]{
  \begin{frame}<beamer>
  \frametitle{Outline}
  \tableofcontents[currentsection]
  \end{frame}
}

\title[Computer Vision]{Computer Vision\\Estimation}
\author{dsai.asia}
\institute[]{Asia Data Science and Artificial Intelligence Master's Program}
\date{}

% My math definitions

\renewcommand{\vec}[1]{\boldsymbol{#1}}
\newcommand{\mat}[1]{\mathtt{#1}}
\newcommand{\ten}[1]{\mathcal{#1}}
\renewcommand{\null}[1]{{\cal N}(#1)}
\def\Rset{\mathbb{R}}
\def\Pset{\mathbb{P}}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\argmin}{argmin}
\def\norm{\mbox{$\cal{N}$}}

\newcommand{\stereotype}[1]{\guillemotleft{{#1}}\guillemotright}

\newcommand{\myfig}[3]{\centerline{\includegraphics[width={#1}]{{#2}}}
    \centerline{\scriptsize #3}}

\begin{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%             CONTENTS START HERE

%\setbeamertemplate{navigation symbols}{}

\frame{\titlepage}

%--------------------------------------------------------------------
%\part<presentation>{Part name}
%
%\frame{\partpage}

\begin{frame}
\frametitle{Readings}

Readings for these lecture notes:
\begin{itemize}
\item[-] Hartley, R., and Zisserman, A. {\em Multiple View Geometry in
    Computer Vision}, Cambridge University Press, 2004, Chapter 4.
\item[-] Tomasi, C. {\em Mathematical Modeling of Continuous Systems},
    online lecture notes from Duke University, 2004.
\end{itemize}

\medskip

If you are unfamiliar with the mathematics used in these lecture
notes, study Carlo Tomasi's beautiful lecture notes.  You'll find the
notes on the course Web site under ``Readings.''

\medskip

These notes contain material $\copyright$ Hartley and Zisserman
(2004) and Tomasi (2004).

\end{frame}

%--------------------------------------------------------------------
\section{Introduction}
%--------------------------------------------------------------------

\begin{frame}
\frametitle{Introduction}
\framesubtitle{Estimation Problems}

In vision, we are frequently confronted with \alert{estimation} problems
in which parameters of some function must be estimated from measurements.

\medskip

Some examples of important estimation problems:

\begin{itemize}
\item \alert{2D homography}: Given a set of points $\vec{x}_i$ in
  $\Pset^2$ and corresponding points $\vec{x}_i'$ in $\Pset^2$,
  find a homography taking each $\vec{x}_i$ to $\vec{x}_i'$.
\item \alert{3D to 2D camera projection}: Given a set of points
  $\vec{X}_i$ in 3D space and corresponding points $\vec{x}_i$
  in an image, find the 3D to 2D projective mapping taking each
  $\vec{X}_i$ to $\vec{x}_i$.
\item \alert{Fundamental matrix computation}: Given a set of
  points $\vec{x}_i$ in one image and a set of corresponding
  points $\vec{x}_i'$ in another image, find the fundamental
  matrix $\mat{F}$ relating the two images.
\item \alert{Trifocal tensor computation}: Given a set of
  point correspondences $\vec{x}_i \leftrightarrow \vec{x}_i'
  \leftrightarrow \vec{x}_i''$ across three images, compute the
  trifocal tensor $\ten{T}_i^{jk}$ relating points or lines in
  three views.
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{Introduction}
\framesubtitle{Homography estimation}

First we'll consider homography estimation.

\medskip

We have a set of points $\vec{x}_i$ and corresponding points
$\vec{x}_i'$.  We want to compute $\mat{H}$ such that $\forall i,
\mat{H} \vec{x}_i = \vec{x}_i'$.

\medskip

How many points do we need?
\begin{itemize}
\item We already saw that each point
correspondence gives us 2 constraints (equations), one for the $x$
component and one for the $y$ component.
\item Since $\mat{H}$ has 8
degrees of freedom we need at least 4 correspondences.
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{Introduction}
\framesubtitle{Cost functions}

We know that 4 correspondences yields an \alert{exact solution}.

\medskip

Due to \alert{measurement error} and \alert{correspondence error} we
should get \alert{more than 4 correspondences} then find the
homography $\mat{H}$ minimizing some \alert{cost function}.

\medskip

\begin{block}{Gold Standard algorithm (Hartley and Zisserman, 2004)}
An estimation algorithm minimizing the cost function that
is \alert{the best possible cost function} under certain assumptions.
\end{block}

\medskip

All algorithms should be evaluated with respect to the Gold Standard.

\end{frame}

%--------------------------------------------------------------------
\section{Direct Linear Transform (DLT)}
%--------------------------------------------------------------------

\begin{frame}
\frametitle{Direct Linear Transform (DLT)}
\framesubtitle{Another view of the homography estimation problem}

Now we'll look at another way to derive the exact linear estimate of
$\mat{H}$ from 4 points.

\medskip

For corresponding points $\vec{x}_i \leftrightarrow \vec{x}_i'$ we
want $\vec{x}_i' = k_i \mat{H} \vec{x}_i$ for some nonzero scaling
factor $k$.

\medskip

Thus we can say that $\vec{x}_i'$ and $\mat{H}\vec{x}_i$ must be collinear.

\medskip

Recall that the \alert{cross product} of two collinear vectors is the
0 vector.

\medskip

This means we can write this constraint in the form
$\vec{x}_i' \times \mat{H} \vec{x}_i = \vec{0}$.

\end{frame}

\begin{frame}
\frametitle{Direct Linear Transform (DLT)}
\framesubtitle{Deriving the linear system}

Let's use $\vec{h}^{j}$ to denote the $j$-th row of $\mat{H}$ written
as a vector.  Then we have
\begin{equation*}
\mat{H}\vec{x}_i = \begin{pmatrix} \vec{h}^{1T}\vec{x}_i \\
\vec{h}^{2T}\vec{x}_i \\ \vec{h}^{3T}\vec{x}_i \end{pmatrix}.
\end{equation*}

\medskip

Now if $\vec{x}_i' = (x_i',y_i',w_i')^T$, the cross product can be written
\begin{equation*}
\vec{x}_i' \times \mat{H} \vec{x}_i = \begin{pmatrix}
y_i' \vec{h}^{3T}\vec{x}_i - w_i' \vec{h}^{2T}\vec{x}_i \\
w_i' \vec{h}^{1T}\vec{x}_i - x_i' \vec{h}^{3T}\vec{x}_i \\
x_i' \vec{h}^{2T}\vec{x}_i - y_i' \vec{h}^{1T}\vec{x}_i \end{pmatrix}.
\end{equation*}

\end{frame}

\begin{frame}
\frametitle{Direct Linear Transform (DLT)}
\framesubtitle{Deriving the linear system}

Since we want the cross product to be the zero vector, we can write
the linear system
\begin{equation*}
\begin{bmatrix}
\vec{0}^T        & -w_i'\vec{x}_i^T & y_i'\vec{x}_i^T  \\
w_i'\vec{x}_i^T  & \vec{0}^T        & -x_i'\vec{x}_i^T \\
-y_i'\vec{x}_i^T & x_i'\vec{x}_i^T  & \vec{0}^T        \end{bmatrix}
\begin{pmatrix} \vec{h}^1 \\ \vec{h}^2 \\ \vec{h}^3 \end{pmatrix} =
\vec{0}.
\end{equation*}

\medskip

We have three linear equations in 9 unknowns.
The third equation is actually a linear combination of the first
two.\footnote{To convince yourself of this, use the factor $-x_i'/w_i'$
for the first equation and $-y_i'/w_i'$ for the second equation.}
So we can drop it (see next slide)...

\end{frame}

\begin{frame}
\frametitle{Direct Linear Transform (DLT)}
\framesubtitle{Deriving the linear system}

Dropping the redundant third equation in the parameters of $\mat{H}$, we have:
\begin{equation}
\label{dlt-eqn}
\mat{A}_i\vec{h} =
\begin{bmatrix}
\vec{0}^T        & -w_i'\vec{x}_i^T & y_i'\vec{x}_i^T  \\
w_i'\vec{x}_i^T  & \vec{0}^T        & -x_i'\vec{x}_i^T \end{bmatrix}
\begin{pmatrix} \vec{h}^1 \\ \vec{h}^2 \\ \vec{h}^3 \end{pmatrix} =
\vec{0}.
\end{equation}
However, if $w_i' = 0$ we have an ideal point, and the first two
equations become linearly dependent.  In this case we should use the
third equation and the first or second equation, or just use all three
equations all the time.

\end{frame}

\begin{frame}
\frametitle{Direct Linear Transform (DLT)}
\framesubtitle{Deriving the linear system}

With 8 equations in 9 unknowns, $\mat{A}$ is $8\times 9$ and has rank
8.  It has a one-dimensional null space and we obtain $\vec{h} =
\null{\mat{A}}$.

\medskip

If we have \alert{more than 4 correspondences} the system
$\mat{A}\vec{h}=\vec{0}$ will be \alert{over-determined}:
\begin{itemize}
\item With \alert{perfect measurement}, the rank of $\mat{A}$ would
  still be 8 and it would still have a one-dimensional null space.
\item But \alert{measurement noise} means no exact solution.
\item So we try to find the vector $\vec{h}$ minimizing some \alert{cost
  function}.
\end{itemize}

\medskip

Since we want $\mat{A}\vec{h}$ to be as close as possible to
$\vec{0}$, a sensible cost function is the \alert{norm} of
$\mat{A}\vec{h}$, i.e., $\|\mat{A}\vec{h}\|$.

\medskip

However we must avoid the \alert{trivial solution} $\vec{h}=\vec{0}$,
so we impose a constraint $\|\vec{h}\|=1$.  This is OK since $\mat{H}$
is homogeneous.

\end{frame}

\begin{frame}
\frametitle{Direct Linear Transform (DLT)}
\framesubtitle{Solving the linear system}

So now we have the minimization problem
\begin{equation*}
\hat{\vec{h}} = \argmin_{\vec{h}} \|\mat{A}\vec{h}\|, \text{subject to }
\|\vec{h}\|=1.
\end{equation*}
whose solution is to let $\vec{h}$ be the unit eigenvector of
$\mat{A}^T\mat{A}$ with the smallest eigenvalue, or the unit singular
vector of $\mat{A}$ corresponding to the smallest singular value of
$\mat{A}$.

\medskip

\alert{This is important to remember:} to solve an overconstrained
homogeneous linear system $\mat{A}\vec{x}=\vec{0}$ by minimizing
$\|\mat{A}\vec{h}\|$ subject to $\|\vec{h}\|=1$, we \alert{perform SVD
  on $\mat{A}$} (see next section) or \alert{compute the eigenvector
  corresponding to the smallest eigenvalue of $\mat{A}^T\mat{A}$}.

\end{frame}

\begin{frame}
\frametitle{Direct Linear Transform (DLT)}
\framesubtitle{The basic DLT for $\mat{H}$}

\begin{block}{DLT: Objective}
Given $n \ge 4$ 2D to 2D point correspondences $\{ \vec{x}_i
\leftrightarrow \vec{x}_i' \}$, determine the 2D homography matrix
$\mat{H}$ such that $\vec{x}_i' = \mat{H}\vec{x}_i$.
\end{block}

\begin{block}{DLT: Algorithm}
\begin{itemize}
\item[(i)] For each correspondence $\vec{x}_i \leftrightarrow
  \vec{x}_i'$, compute the matrix $\mat{A}_i$ as in equation
  (1).  When $w_i'=0$, use different rows.
\item[(ii)] Assemble the $n$ $2\times 9$ matrices $\mat{A}_i$ into a
  single $2n\times 9$ matrix $\mat{A}$.
\item[(iii)] Obtain the SVD $\mat{A}=\mat{U}\mat{D}\mat{V}^T$.  The
  unit singular vector (column of $\mat{V}$) corresponding to the
  smallest singular value (diagonal element of $\mat{D}$) is the
  desired $\vec{h}$.
\item[(iv)] Rearrange $\vec{h}$ to obtain $\mat{H}$.
\end{itemize}
\end{block}

\end{frame}

\begin{frame}
\frametitle{Direct Linear Transform (DLT)}
\framesubtitle{Similar approaches}

Other methods: we can select one element of $\vec{h}$ to be equal to 1
(or any other arbitrary value) to obtain an inhomogeneous linear
system which can be solved by the usual least squares methods (see
text).

\medskip

We can also compute $\mat{H}$ in the same way using line
correspondences or conic correspondences.  The derivations are quite
similar.

\end{frame}

\begin{frame}
\frametitle{Direct Linear Transform (DLT)}
\framesubtitle{Example code}

For example code for the DLT in Octave, see \texttt{dlt\_demo.m} on
the course Web site.

\end{frame}

%--------------------------------------------------------------------
\section{Singuar value decomposition (SVD)}
%--------------------------------------------------------------------

\begin{frame}
\frametitle{Singular value decomposition (SVD)}
\framesubtitle{Definition}

The SVD is an incredibly useful factorization, particularly for the
kinds of estimation problems that come up in computer vision.

\medskip

\begin{block}{Singular Value Decomposition}
Given an $m\times n$ matrix $\mat{A}$, the \alert{singular value
  decomposition} of $\mat{A}$ is
\begin{equation*}
\mat{A} = \mat{U}\mat{D}\mat{V}^T
\end{equation*}
where the columns of $\mat{U} \in \Rset^{m\times m}$ and $\mat{V} \in
\Rset^{n\times n}$ are orthogonal unit vectors and $\mat{D} \in
\Rset^{m\times n}$ is a diagonal matrix whose elements $\sigma_i$,
with $\sigma_1 \ge \cdots \ge \sigma_p \ge 0$, ($p=\min(m,n)$), are
called the \alert{singular values} of $\mat{A}$.
\end{block}

\end{frame}

\begin{frame}
\frametitle{Singular value decomposition (SVD)}
\framesubtitle{Geometric interpretation}

Writing $\mat{A}=\mat{U}\mat{D}\mat{V}^T$ models the transformation
$\vec{y}=\mat{A}\vec{x}$ as a rotation, a ``stretch'' of the unit
hypershpere into a hyperellipse, and a rotation of the hyperellipse.
Example:

\begin{columns}
\column{1.1in}
\begin{equation*}
\mat{A} = \frac{1}{\sqrt{2}}
\begin{bmatrix}
\sqrt{3} & \sqrt{3} \\
-3 & 3 \\
1 & 1
\end{bmatrix}
\end{equation*}

\column{2.6in}
\myfig{2.5in}{Tomasi-fig3-2}{}

\end{columns}

\vspace{-0.25in}

\centerline{\scriptsize Example from Tomasi (2004), Section 3.}

\end{frame}

\begin{frame}
\frametitle{Singular value decomposition (SVD)}
\framesubtitle{Properties of the SVD}

The SVD has many useful properties:
\begin{itemize}
\item If $m=n$ and $\sigma_i \not= 0, \forall i$, then $\mat{A}$ is
  \alert{invertible}.  The ratio $C=\sigma_1/\sigma_n$ is called the
  \alert{condition number} of $\mat{A}$ and tells us how close
  $\mat{A}$ is to singularity.  When $1/C$ is close to numerical
  precision, we say $\mat{A}$ is \alert{ill-conditioned} and should be
  considered singular.
\item The number of nonzero $\sigma_i$ is the \alert{rank} of
  $\mat{A}$.  Numerically, we must specify a tolerance, e.g.\
  $\epsilon=10^{-6}$, and say the number of singular values greater
  than $\epsilon$ is the rank of $\mat{A}$.
\item We can get the \alert{inverse} or \alert{pseudoinverse} of
  $\mat{A}$ using the SVD: $\mat{A}^{-1}=\mat{V}\mat{D}^{-1}\mat{U}^T$
  or $\mat{A}^+ = \mat{V}\mat{D}_0^{-1}\mat{U}^T$, where
  $\mat{D}_0^{-1}$ is $\mat{D}^{-1}$ for the nonzero singular values
  and 0 otherwise.
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{Singular value decomposition (SVD)}
\framesubtitle{Properties of the SVD}

More properties:
\begin{itemize}
\item The \alert{columns} of $\mat{U}$ corresponding to the
  \alert{nonzero} $\sigma_i$ span the \alert{range} of $\mat{A}$.  The
  columns of $\mat{V}$ corresponding to the zero singular values span
  the null space of $\mat{A}$.
\item The \alert{squares of the nonzero singular values} of $\mat{A}$
  are the \alert{nonzero eigenvalues} of $\mat{A}^T\mat{A}$ and
  $\mat{A}\mat{A}^T$.  The \alert{columns} of $\mat{U}$ are the
  \alert{eigenvectors} of $\mat{A}\mat{A}^T$ and the \alert{columns}
  of $\mat{V}$ are the \alert{eigenvectors} of $\mat{A}^T\mat{A}$.
  Additionally, we can write $\mat{A}\vec{u}_k = \sigma_k \vec{v}_k$
  and $\mat{A}^T\vec{v}_k=\sigma_k\vec{u}_k$ where $\vec{v}_k$ is the
  $k$th column of $\mat{V}$ and $\vec{u}_k$ is the $k$th column of
  $\mat{U}$.
\item The singular values of $\mat{A}$ are related to the
  \alert{Frobenius norm} of $\mat{A}$:
\begin{equation*}
\|\mat{A}\|^2_F = \sum_{i,j}a_{i,j}^2 = \sum_i\sigma_i^2.
\end{equation*}
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{Singular value decomposition (SVD)}
\framesubtitle{Applications}

Here are some of the SVD's many uses:
\begin{itemize}
\item In \alert{inhomogeneous} linear least squares problems
  ($\mat{A}\vec{x}=\vec{b}$), we use the SVD to obtain the
  pseudoinverse of A and let $\vec{x}=\mat{A}^+\vec{b}$.
\item In \alert{homogeneous} least squares problems, when we want to minimize
  $\|\mat{A}\vec{x}\|$ subject to $\|\vec{x}\|=1$, we obtain the SVD
  and let $\vec{x}$ be the last column of $\mat{V}$ (since it is also
  the least eigenvector of $\mat{A}^T\mat{A}$).
\item In some cases we can use the SVD to \alert{enforce constraints
    on an estimated matrix}.  For example, if we obtain an estimate
  $\mat{R}$ of a rotation matrix that is not quite orthogonal, we can
  compute the orthogonal matrix
  $\hat{\mat{R}}=\mat{U}\mat{I}\mat{V}^T$ that is closest to $\mat{R}$
  measured by the Frobenius norm.  We can use a similar approach for
  rank constraints.
\end{itemize}

\end{frame}

%--------------------------------------------------------------------
\section{Cost functions}
%--------------------------------------------------------------------

\begin{frame}
\frametitle{Cost functions}
\framesubtitle{Algebraic distance}

DLT minimizes $\|\mat{A}\vec{h}\|$.  The vector $\vec{\epsilon} =
\mat{A}\vec{h}$ is called the \alert{residual vector}.  Each of the
components of $\vec{\epsilon}$ comes from one of the individual
correspondences generating a row of $\mat{A}$.

\medskip

The part of the vector $\vec{\epsilon}_i$ contributed by one
correspondence $\vec{x}_i \leftrightarrow \vec{x}_i'$ is called the
\alert{algebraic error} for correspondence $i$ and homography
$\mat{H}$.  The norm of $\vec{\epsilon}_i$ is called the
\alert{algebraic distance} between $\vec{x}_i'$ and
$\mat{H}\vec{x}_i$.

\medskip

The algebraic distance is a convenient cost function because it leads
to a straightforward linear solution, but \alert{it is not
geometrically or statistically meaningful}!

\medskip

We will find that \alert{normalization} is crucial to obtaining good
results from algorithms minimizing algebraic error.

\medskip

We will also look at algorithms that use DLT and similar linear
algebraic error-minimizing routines to obtain an initial solution,
then minimize a statistical or geometrical cost function from there.

\end{frame}

\begin{frame}
\frametitle{Cost functions}
\framesubtitle{Geometric error}

Here we'll use $\vec{x}$ to represent a \alert{measured} image point,
$\hat{\vec{x}}$ to denote an \alert{estimated} point, and
$\bar{\vec{x}}$ to represent the \alert{true value} of a point.

\medskip

As a starting point, imagine we have perfect measurements in the first
image and error in the second image.

\medskip

Let $d(\vec{x},\vec{y})$ be the \alert{Euclidean} distance between the
\alert{inhomogeneous} representations of points $\vec{x}$ and
$\vec{y}$.

\medskip

We call the \alert{transfer error} for the set of correspondences
\begin{equation*}
\sum_i d(\vec{x}_i',\mat{H}\bar{\vec{x}}_i)^2.
\end{equation*}
We can estimate the homography $\hat{\mat{H}}$ minimizing the
transfer error.

\end{frame}

\begin{frame}
\frametitle{Cost functions}
\framesubtitle{Geometric error}

In most practical situations, we don't actually know the true position
$\bar{\vec{x}}_i$ of the point $\vec{x}_i$.  Then it makes sense to
measure the \alert{symmetric transfer error}, i.e., the transfer error
in \alert{both directions}:
\begin{equation*}
\sum_i d(\vec{x}_i,\mat{H}^{-1}\vec{x}_i')^2 +
d(\vec{x}_i',\mat{H}\vec{x}_i)^2.
\end{equation*}
We can estimate the homography $\hat{\mat{H}}$ minimizing the symmetric
transfer error.

\end{frame}

\begin{frame}
\frametitle{Cost functions}
\framesubtitle{Reprojection error}

Another approach is to come up with not only an estimate
$\hat{\mat{H}}$, but also \alert{estimates} $\hat{\vec{x}}_i$ and
$\hat{\vec{x}}_i'$ of the \alert{true points} $\bar{\vec{x}}_i$ and
$\bar{\vec{x}}_i'$, ensuring that
$\hat{\mat{H}}\hat{\vec{x}}_i=\hat{\vec{x}}_i'$.

\medskip

In this case we want to minimize the \alert{reprojection error}
\begin{equation*}
\sum_i d(\vec{x}_i,\hat{\vec{x}}_i)^2 +
d(\vec{x}_i',\hat{\vec{x}}_i')^2, \text{\ subject\ to\ }
\hat{\vec{x}}_i' = \hat{\mat{H}}\hat{\vec{x}}_i, \forall i.
\end{equation*}

\medskip

Reprojection error will be the natural cost function when we are estimating
3D world points $\vec{X}_i$ projecting to $\vec{x}_i$ and
$\vec{x}_i'$.

\end{frame}

\begin{frame}
\frametitle{Cost functions}
\framesubtitle{Comparison of transfer error and reprojection error}

Here is a comparison of symmetric transfer error and reprojection
error:

\medskip

\myfig{4in}{HZ-fig3-2}{Hartley and Zisserman (2004), Fig.\ 4.2}

\end{frame}

\begin{frame}
\frametitle{Cost functions}
\framesubtitle{Comparison of algebraic and geometric distance}

The algebraic and geometric methods turn out to be \alert{equivalent}
whenever $\hat{w}_i' = w_i' = 1, \forall i$.

\medskip

This is always true in the case that $\mat{H}$ is an affinity, and the
DLT specializes to affinities without any problem (just set
$h_7=h_8=0$ in Equation (1)).

\end{frame}

\begin{frame}
\frametitle{Cost functions}
\framesubtitle{Other cost functions}

There are other cost functions that attempt to model the simplicity of
algebraic error while approximating geometric error as closely as
possible.

\medskip

One is the \alert{Sampson error}.  See text for details.

\end{frame}

\begin{frame}
\frametitle{Cost functions}
\framesubtitle{Maximum likelihood estimation}

If we assume spherical Gaussian measurement errors in \alert{one} image, the
\alert{maximum likelihood estimate} of $\mat{H}$ is the one minimizing
the \alert{transfer error}
\begin{equation*}
\sum_i d(\vec{x}_i',\mat{H}\bar{\vec{x}}_i)^2.
\end{equation*}
If we assume spherical Gaussian measurement errors in \alert{both} images, the
maximum likelihood estimate of $\mat{H}$ turns out to be the one
minimizing the \alert{reprojection error}
\begin{equation*}
\sum_i
d(\vec{x}_i,\hat{\vec{x}}_i)^2+d(\vec{x}_i',\hat{\vec{x}}_i')^2.
\end{equation*}

\medskip

See text for the derivations.  Notice that maximum likelihood with a
Gaussian noise model often leads to least-squares methods.

\end{frame}

\begin{frame}
\frametitle{Cost functions}
\framesubtitle{Maximum likelihood estimation}

That geometric error cost functions arise from maximum likelihood
means they have good theoretical justification: they are
\alert{statistically optimal} under certain assumptions.

\medskip

Is the assumption of \alert{Gaussian measurement noise} reasonable?

\medskip

It is reasonable if care is taken to \alert{eliminate outliers} prior
to performing the estimation.

\end{frame}

%--------------------------------------------------------------------
\section{Normalization}
%--------------------------------------------------------------------

\begin{frame}
\frametitle{Normalization}
\framesubtitle{Problem with the DLT}

What happens with the DLT when we replace $\vec{x}_i$ by
$\mat{T}\vec{x}_i$ and replace $\vec{x}_i'$ by $\mat{T}'\vec{x}_i'$
for arbitrary homographies $\mat{T}$ and $\mat{T}'$?
\begin{itemize}
\item We would prefer the DLT to give us a transformed homography
  $\tilde{\mat{H}} = \mat{T}'\mat{H}\mat{T}^{-1}$, where $\mat{H}$ is
  the homography we would obtain from the DLT using the untransformed
  points.
\item If this were the case, we would get the same estimated
  homography regardless of the image coordinate system, origin, etc.
\item However, \alert{it doesn't turn out that way}!  The DLT is
  \alert{not transformation invariant}.
\end{itemize}

The DLT's lack of transformation invariance is a big problem but can
be minimized through \alert{data normalization}.

\medskip

\alert{Geometric error minimization}, on the other hand, \alert{is}
invariant to \alert{similarity transformations}.

\end{frame}

\begin{frame}
\frametitle{Normalization}
\framesubtitle{Fixing the problem with the DLT}

Hartley and Zisserman find it is possible to \alert{pre-normalize}
$\vec{x}_i$ and $\vec{x}_i'$ with \alert{isotropic scaling} to obtain
reasonable solutions:

\begin{block}{Isotropic scaling}
\begin{itemize}
\item Transform the coordinates in each image so their centroid is at
  the origin.
\item Then scale the coordinates so that the average distance from the
  origin along each dimension is 1. In 2D, this means the
  average magnitude of $(x_i,y_i)^T$ becomes $\sqrt{2}$.
\end{itemize}
\end{block}

\medskip

With isotropic scaling, the DLT becomes invariant to similarity
transformations.

\begin{block}{}
Data normalization is an {\em essential} step in the DLT algorithm.
It must {\em not} be considered optional (Hartley and Zisserman, 2004,
p.\ 108).
\end{block}

\end{frame}

\begin{frame}
\frametitle{Normalization}
\framesubtitle{The normalized DLT for $\mat{H}$}

\begin{block}{Normalized DLT: Objective}
Given $n \ge 4$ 2D to 2D point correspondences $\{ \vec{x}_i
\leftrightarrow \vec{x}_i' \}$, determine the 2D homography matrix
$\mat{H}$ such that $\vec{x}_i' = \mat{H}\vec{x}_i$.
\end{block}

\begin{block}{Normalized DLT: Algorithm}
\begin{itemize}
\item[(i)] Compute a similarity transform $\mat{T}$ consisting of a
  translation and a scale that takes $\vec{x}_i$ to
  $\tilde{\vec{x}}_i$ such that the centroid is $(0,0)^T$ and the
  average distance from the origin is $\sqrt{2}$.
\item[(ii)] Do the same for $\vec{x}_i'$, estimating a similarity
  $\mat{T}'$ taking $\vec{x}_i'$ to $\tilde{\vec{x}}_i'$.
\item[(iii)] Apply the \alert{basic DLT} to obtain $\tilde{\mat{H}}$
  from $\tilde{\vec{x}}_i$ and $\tilde{\vec{x}}_i'$.
\item[(iv)] Return $\mat{H}=\mat{T}^{\prime -1} \tilde{\mat{H}} \mat{T}$.
\end{itemize}
\end{block}

\end{frame}

%--------------------------------------------------------------------
\section{Iterative minimization}
%--------------------------------------------------------------------

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Motivation}

We saw that the DLT is a simple algorithm that minimizes the
\alert{algebraic error}.

\medskip

However, we would prefer to minimize the \alert{geometric error}, not
the algebraic error.

\medskip

For some problems, geometric error can be minimized analytically, but
more often minimizing geometric error requires \alert{iterative
methods} such as Gauss-Newton.

\medskip

The advantage of iterative minimization methods are their power.  But
they have many disadvantages compared to linear minimization
techniques like the DLT:
\begin{itemize}
\item They are \alert{slower}.
\item They generally need an \alert{initial estimate}.
\item The \alert{might not always converge}.
\item Finding the right \alert{stopping criterion} can be difficult.
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Formulating the problem}

Here are Hartley and Zisserman's steps for implementing an iterative
minimization algorithm:
\begin{itemize}
\item Decide on a \alert{cost function} such as reprojection error.
\item Find a suitable \alert{parameterization} of the entity to be
  estimated.  Over-parameterization is usually OK and even beneficial
  (see text).
\item Write a \alert{function specification} expressing the cost as a
  function of the parameters.
\item Use a linear method such as the DLT for \alert{initialization}
  of the parameters.
\item Starting from the initial solution, perform \alert{iteration} to
  minimize the cost function.
\end{itemize}

We will consider as an example the problem of minimizing
reprojection error for the homography estimation problem.

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Function specification}

A large class of cost functions can be formulated in terms of
\alert{Mahalanobis distance} between a \alert{measurement vector}
$\vec{X \in \Rset^N}$ and points on a \alert{model} submanifold $S
\subset \Rset^N$.

\medskip

We formulate the problem in terms of a \alert{parameter vector}
$\vec{P} \in \Rset^M$ and write a function $\vec{f} : \Rset^M \mapsto
\Rset^N$ mapping $\vec{P}$ to an element of the measurement space.

\medskip

Then the cost function is the squared Mahalanobis distance
\begin{equation*}
\| \vec{X} - \vec{f}(\vec{P})\|^2_{\Sigma} =
(\vec{X}-\vec{f}(\vec{P}))^T \Sigma^{-1} (\vec{X}-\vec{f}(\vec{P})).
\end{equation*}
\begin{itemize}
\item If $\vec{f}(\vec{P})$ is \alert{linear}, we obtain a solution using a
  generalized pseudoinverse.
\item If $\vec{f}(\vec{P})$ is \alert{nonlinear} (it usually is in vision
  problems), we have a \alert{nonlinear least squares} problem that
  must be solved iteratively.  The most common algorithm is
  \alert{Levenberg-Marquardt}.
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Nonlinear least squares}

Here is the basic iterative scheme:
\begin{itemize}
\item Pick some initial solution $\vec{P}_0$ (hopefully close to the
  actual solution).
\item Let $k=0$.
\item While $\vec{P}_k$ is not a minimum of $\|\vec{X}-\vec{f}(\vec{P})\|^2_{\Sigma}$, do:
  \begin{itemize}
  \item Compute a step $\delta\vec{P}$.
  \item Let $\vec{P}_{k+1} = \vec{P}_{k} + \delta\vec{P}$.
  \item Let $k=k+1$.
  \end{itemize}
\end{itemize}

To find the best step $\delta\vec{P}$, we try to jump to the new point
$\vec{P}_k+\delta{\vec{P}}$ that would be optimal under some
simplified assumptions about $\vec{f}$.  One technique is to
\alert{linearize} $\vec{f}$ around $\vec{P}_k$.

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Linearizing $f$}

For vector-valued function $\vec{f}(\vec{P}) =
(f_1(\vec{P}),\ldots,f_N(\vec{P}))^T$, in the region of $\vec{P}$, we
can approximate $\vec{f}$ by the \alert{first-order Taylor expansion}
\begin{equation*}
\vec{f}(\vec{P}) =
\vec{f}\left(\vec{P}_0+(\vec{P}-\vec{P}_0)\right) =
\vec{f}(\vec{P}_0+\delta\vec{P}) \approx
\vec{f}(\vec{P}_0) + \mat{J}_{\vec{f}}(\vec{P}_0)\delta\vec{P},
\end{equation*}
where $\mat{J}_{\vec{f}}(\vec{P}_0)$ is the \alert{Jacobian} of $\vec{f}$
evaluated at $\vec{P}_0$:
\begin{equation*}
\mat{J}_{\vec{f}}(\vec{P}_0) =
\begin{pmatrix}
\nabla f_1^T(\vec{P}_0) \\
\vdots \\
\nabla f_N^T(\vec{P}_0)
\end{pmatrix}
\end{equation*}
and $\nabla f_i(\vec{P}_0)$ is the \alert{gradient} of $f_i$ evaluated
at $\vec{P}_0$, i.e.,
\begin{equation*}
\nabla f_i(\vec{P}_0) =
\begin{pmatrix}
\frac{\partial f_i}{\partial P_1}(\vec{P}_0),\ldots,
\frac{\partial f_i}{\partial P_M}(\vec{P}_0)
\end{pmatrix}^T.
\end{equation*}

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Minimizing the linearized $\vec{f}$}

We've approximated $\vec{f}(\vec{P})$ by the linear function
$\vec{f}(\vec{P}_0) + \mat{J}_{\vec{f}}(\vec{P}_0)(\vec{P}-\vec{P}_0)$.

\medskip

Let's assume for the moment that we use the $L_2$ norm
($\mat{\Sigma}=\mat{I}$) and that
$\vec{X}=\vec{0}$.  Then minimizing
$\|\vec{X}-\vec{f}(\vec{P})\|^2_{\Sigma}$ just means minimizing
\begin{equation*}
E(\vec{P}) = \sum_{i=1}^N f_i^2(\vec{P}).
\end{equation*}

\medskip

If $\vec{P}$ is a minimum of $E(\vec{P})$, we know that
\begin{equation*}
\vec{F}(\vec{P})=\frac{1}{2} \nabla E(\vec{P}) = \vec{0}.
\end{equation*}
Substituting the definition for $E(\vec{P})$ and writing as a matrix
equation, we can obtain
\begin{equation*}
\vec{F}(\vec{P})=\mat{J}_{\vec{f}}^T(\vec{P})\vec{f}(\vec{P}) = \vec{0}.
\end{equation*}

\medskip

Now we'll use \alert{Newton's method} to find the zero of $\vec{F}$.

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Minimizing the linearized $\vec{f}$}

Newton's method for a vector valued fuction $\vec{F}$ is to solve
\begin{equation*}
\mat{J}_{\vec{F}}(\vec{P}_k)\delta\vec{P} = -\vec{F}(\vec{P}_k)
\end{equation*}
for $\delta\vec{P}$ then jump to $\vec{P}_{k+1} = \vec{P}_k +
\delta\vec{P}$.

\medskip

In our case it turns out that
\begin{equation*}
\mat{J}_{\vec{F}}(\vec{P}) =
\mat{J}_{\vec{f}}^T(\vec{P})\mat{J}_{\vec{f}}(\vec{P}) + \sum_{i=1}^N
f_i(\vec{P}) \mat{H}_{f_i}(\vec{P}).
\end{equation*}
where $\mat{H}_{f_i}(\vec{P})$ is the \alert{Hessian} (matrix of
second derivatives of $f_i$) evaluated at $\vec{P}$.

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Minimizing the linearized $\vec{f}$}

Finally, to apply Newton's method we solve the linear system
\begin{equation*}
\left[
\mat{J}_{\vec{f}}^T(\vec{P})\mat{J}_{\vec{f}}(\vec{P}) + \sum_{i=1}^N
f_i(\vec{P}) \mat{H}_{f_i}(\vec{P}) \right] \delta{\vec{P}} =
-\mat{J}^T_{\vec{f}}(\vec{P})\vec{f}(\vec{P}).
\end{equation*}

\medskip

Crazy!  Let's do an example in one dimension to get the idea.

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Minimizing the linearized $\vec{f}$ (example)}

\textbf{Example in one dimension.}

\medskip

Let $f(p) = p^2-4p+5$.  We know the minimum is at $p=2$.

\medskip

The ``Jacobian'' in one dimension is actually just the derivative:
$$ \mat{J}_f(p) = \begin{bmatrix} f'(p) \end{bmatrix} =
\begin{bmatrix} 2p-4 \end{bmatrix}. $$

The error function is
$$ E(p) = f^2(p), $$

so the function we want to find the zero of using Newton's method is
$$ F(p) = \frac{1}{2}\nabla E(p) = \mat{J}_f^T(p)f(p) = (2p-4)(p^2-4p+5) .$$

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Minimizing the linearized $\vec{f}$ (example)}

\textbf{Example in one dimension, continued.}

\medskip

To find the zero of $F(p)$, we need
\begin{align*}
\mat{J}_F(p) & = \mat{J}^T_f(p)\mat{J}_f(p) +
                     \sum_{i=0}^N f_i(p)\mat{H}_{f_i}(p) \\
             & = (2p-4)^2 + (p^2-4p+5)\cdot 2 \\
             & = 6p^2-24p+26 .
\end{align*}

then we solve the linear system
$$ \mat{J}_F(p) \delta p = -F(p) .$$

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Minimizing the linearized $\vec{f}$}

Suppose our initial guess for the parameter is $p = 4$.

\bigskip

\centerline{\includegraphics[width=2.8in]{fig1}}

\bigskip

Verify that $\delta p = -10/13 \approx -0.76923$.

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Minimizing the linearized $\vec{f}$}

Problem with Newton's method: the Hessians
\begin{equation*}
\begin{bmatrix}
\frac{\partial^2 f_i}{\partial p_1^2} &
\frac{\partial^2 f_i}{\partial p_1 \partial p_2} & \cdots \\
\vdots & \vdots & \vdots \\
\frac{\partial^2 f_i}{\partial p_M \partial p_1} &
\frac{\partial^2 f_i}{\partial p_M \partial p_2} & \cdots
\end{bmatrix}
\end{equation*}
are tedious and expensive to calculate, especially if $M$, the
dimensionality of the parameter vector, is large.

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Minimizing the linearized $\vec{f}$}

Because of the difficulty of calculating Hessians we typically use
quick-and-dirty approximations rather than explicitly calculate them.

\medskip

The \alert{Gauss-Newton} algorithm drops the second-order terms
entirely.

\medskip

The \alert{Levenberg-Marquardt} algorithm approximates the second
order terms by a scaled identity matrix:
\begin{equation*}
\left[ \mat{J}_{\vec{f}}^T(\vec{P})\mat{J}_{\vec{f}}(\vec{P}) +
\mu\mat{I} \right] \delta{\vec{P}} =
-\mat{J}^T_{\vec{f}}(\vec{P})\vec{f}(\vec{P}).
\end{equation*}
where the parameter $\mu$ is adapted during optimization.

\medskip

Lucky for us, Levenberg-Marquardt is implemented in many packages:
Matlab {\tt lsqnonlin()}, Octave {\tt leasqr()}, and {\em Numerical
Recipes in C} {\tt mrqmin()}.

\medskip

See the text, Appendix 6, for information on adapting
Levenberg-Marquardt to very large problems.

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Iterative minimization applied to homography estimation}

In the case of \alert{homography estimation},
we have a set of 2D coordinates
of corresponding points $\vec{x}_i$ and $\vec{x}_i'$, so the
dimensionality of the \alert{measurement space} is $N=4n$ and the
measurement is a vector $\vec{X} \in \Rset^N$.

\medskip

Suppose our \alert{parameterization} was to choose a set of points
$\hat{\vec{x}}_i$ in the first image, then choose a homography
$\mat{H}$.
\begin{itemize}
\item The corresponding points
  $\hat{\vec{x}}_i'=\hat{\mat{H}}\hat{\vec{x}}_i$ would then be
  \alert{fixed}.
\item The \alert{parameter vector} $\vec{P} \in \Rset^M$, then, would
  contain the $2n$ parameters of $\hat{\vec{x}}_i$ and the 9
  parameters of $\hat{\mat{H}}$, so $M=2n+9$.
\item The resulting \alert{model} (the set of measurements in
  $\Rset^N$ that can be be generated with our parameterization) is a
  $2n+8$ dimensional submanifold $S \subset \Rset^N$.
\end{itemize}

\medskip

[Analogy: think about a circle as a 1D submanifold of $\Rset^2$.]

\end{frame}

\begin{frame}
\frametitle{Iterative minimization}
\framesubtitle{Iterative minimization applied to homography estimation}

Given the model in the previous page, it is straightforward to write
the mapping
\begin{equation*}
  \vec{f} : (\vec{h},\hat{\vec{x}}_1,\ldots,\hat{\vec{x}}_n) \mapsto
  (\hat{\vec{x}}_1,\hat{\vec{x}}_1',\ldots,\hat{\vec{x}}_n,\hat{\vec{x}}_n')
\end{equation*}
where $\hat{\vec{x}}_i' = \mat{H}\hat{\vec{x}}_i$.

\medskip

The reprojection error cost function becomes
$\|\vec{X}-\vec{f}(\vec{P})\|^2$, which is just the Mahalanobis cost
function with $\mat{\Sigma}=\mat{I}$.

\medskip

This means Levenberg-Marquardt applies.  The resulting algorithm is
Hartley and Zisserman's \alert{Gold Standard} MLE algorithm for
estimating $\mat{H}$.

\end{frame}

%--------------------------------------------------------------------
\section{Gold Standard algorithm}
%--------------------------------------------------------------------

\begin{frame}
\frametitle{Gold Standard algorithm}
\framesubtitle{The idea}

The \alert{Gold Standard algorithm} for $\mat{H}$ tries to find
\begin{equation*}
(\hat{\mat{H}},\hat{\vec{x}}_1,\ldots,\hat{\vec{x}}_n) =
\argmax_{\hat{\mat{H}},\hat{\vec{x}}_1,\ldots,\hat{\vec{x}}_n}
P(\vec{x}_1,\vec{x}_1',\ldots,\vec{x}_n,\vec{x}_n' \mid
\mat{H},\hat{\vec{x}}_1,\ldots,\hat{\vec{x}}_n)
\end{equation*}

\medskip

We know that maximizing the likelihood in the equation, assuming
Gaussian measurement error, means minimizing the reprojection error
\begin{equation*}
\sum_i d(\vec{x}_i, \hat{\vec{x}}_i)^2 +
       d(\vec{x}_i', \hat{\mat{H}}\hat{\vec{x}}_i)^2
\end{equation*}

\medskip

This is a nonlinear least squares problem, which Levenberg-Marquardt
can solve, but we need an \alert{initial solution}, which we can
obtain using the DLT.

\end{frame}

\begin{frame}
\frametitle{Gold Standard algorithm}
\framesubtitle{The algorithm}

\begin{block}{Gold Standard for $\mat{H}$: Objective}
  Given $n>4$ image point correspondences $\{\vec{x}_i \leftrightarrow
  \vec{x}_i' \}$, determine the maximum likelihood estimate
  $\hat{\mat{H}}$.
\end{block}

\begin{block}{Gold Standard }%for $\mat{H}$: Algorithm}
\begin{itemize}
\item[(i)] Compute an initial estimate of $\hat{\mat{H}}$ using
  the normalized DLT.
\item[(ii)] Compute an initial estimate of the subsidiary variables
  $\{\hat{\vec{x}}_i\}$ using $\{\vec{x}_i\}$ (see text for a better
  way).
\item[(iii)] Minimize the cost
\begin{equation*}
\sum_i d(\vec{x}_i, \hat{\vec{x}}_i)^2 +
       d(\vec{x}_i', \hat{\mat{H}}\hat{\vec{x}}_i)^2
\end{equation*}
  over $\hat{\mat{H}},\hat{\vec{x}}_1,\ldots,\hat{\vec{x}}_n$, using
  Levenberg-Marquardt over $2n+9$ variables: $2n$ for the points
  $\{\hat{\vec{x}}_i\}$ and 9 for the homography matrix
  $\hat{\mat{H}}$.
\end{itemize}
\end{block}

\end{frame}

\begin{frame}
\frametitle{Gold Standard algorithm}
\framesubtitle{Example code}

For example code for the Gold Standard
algorithm in Octave, see \texttt{gs\_demo.m} on
the course Web site.

\medskip

Note that if you're using Matlab you can use the Matlab port of the
Octave \texttt{leasqr()} function or use the Matlab Optimization Toolbox
function \texttt{lsqnonlin()}.  But be careful as the two functions work
a bit differently.

\end{frame}

%--------------------------------------------------------------------
\section{Robust estimation}
%--------------------------------------------------------------------

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{Introduction}

The Gold Standard algorithm is optimal if the \alert{measurement
  error} for the corresponding points $\vec{x}_i \leftrightarrow
\vec{x}_i'$ is actually \alert{Gaussian}.

\medskip

In practice, though, the points and their correspondences are obtained
through an \alert{automatic} procedure which makes \alert{mistakes}.

\medskip

These mistakes, or \alert{outliers}, will severely disrupt our
estimates, so they should be removed.

\medskip

We seek to obtain a set of \alert{inliers} that will be used for
estimation and a set of \alert{outliers} that will be ignored.

\medskip

This task is called \alert{robust estimation} because we want the
estimation method to be \alert{robust} to \alert{outliers} following
an unmodeled error distribution.

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{RANSAC motivation}

Example: fitting a line $x'=ax+b$ to a set of points.

\medskip

\begin{columns}
\column{2.2in}
\myfig{2.1in}{HZ-fig3-7a}{Least squares fit is skewed by outliers.}
\column{2.2in}
\myfig{2.1in}{HZ-fig3-7b}{RANSAC support for two candidate lines.}
\end{columns}

\medskip

The \alert{RANSAC} (Random Sample Consensus, Fischler and Bolles,
1981) algorithm is one of the many robust methods to solve this
problem.

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{RANSAC idea}

The idea of RANSAC is that we only need two points to determine a
line.

\medskip

So to begin, we pick two points \alert{at random} to define a line.

\medskip

The \alert{support} for this line is the number of points that lie
within a distance threshold $t$ of that line.

\medskip

We repeat for a while, and the \alert{line with the most support} is
deemed best.

\medskip

The points within the threshold are called \alert{inliers} and they
are said to make up the \alert{consensus set}.

\medskip

In the figure, we see that the line $\overline{\vec{a}\vec{b}}$ has a
support of 10 but the line $\overline{\vec{d}\vec{e}}$ has a support
of only 2.  We would select $\overline{\vec{a}\vec{b}}$ as a better
fit to the data than $\overline{\vec{d}\vec{e}}$.

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{RANSAC algorithm}

Hartley and Zisserman's (2004) adaptation of Fischler and Bolles'
(1981) RANSAC:

\begin{block}{RANSAC: Objective}
Robust fit of a model to a data set $S$ which contains outliers
\end{block}

\begin{block}{RANSAC: Algorithm}
\begin{itemize}
\item[(i)] Randomly select a sample $s$ from $S$ and instantiate the
  model from $s$.
\item[(ii)] Find the consensus set (inliers) $S_i$ within distance
  threshold $t$ of the model.
\item[(iii)] If $|S_i|\ge T$ re-estimate the model using all of the
  points in $S_i$ and terminate; otherwise repeat from (i).
\item [(iv)] After $N$ trials, select the largest consensus set $S_i$
  and re-estimate the model using all of the points in $S_i$.
\end{itemize}
\end{block}

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{RANSAC parameters}

RANSAC has three \alert{free parameters}:
\begin{itemize}
\item $t$: the \alert{distance threshold},
\item $T$: the \alert{minimum number of inliers} for early termination,
\item $N$: the \alert{number of samples}.
\end{itemize}

\medskip

$t$ can be determined empirically, or, if the \alert{error
  distribution} is \alert{known} to be Gaussian with standard
deviation $\sigma$, a 95\% or similar confidence interval can be
calculated.

\medskip

See Table 4.2 in the text for reasonable values of $t$ for various
vision problems.

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{RANSAC parameters}

$N$ can be determined empirically, or if the proportion $w$ of inliers
is approximately known, we can choose $N$ giving (e.g.) a 99\%
probability that \alert{on some iteration} we will choose a sample
containing \alert{inliers only}.

\medskip

Example: in homography estimation our sample size would be 4.  If we
assume a 50\% outlier rate, we should select $N=72$ samples to assure
a 99\% probability of sampling at least one set with 4 inliers.

\medskip

See Table 4.3 in the text for some example values and how to calculate
$N$ in general.

\medskip

$T$, the acceptable consensus set size, should be approximately the
\alert{number of inliers} thought to be in the data.

\medskip

For example, in the case of homography estimation, if we have 100
correspondences and a 50\% outlier rate, we should let $T=50$.

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{Adaptive version of RANSAC for unknown $w$}

One variant, when the percentage of inliers $w$ is unknown, is
\alert{adpative RANSAC}:
\begin{itemize}
\item Initialize $N=\infty$.
\item While running RANSAC, decrease $N$ whenever you obtain a sample
  with a bigger consensus set than previously seen.
\item Terminate after $N$ iterations.
\end{itemize}

\medskip

A bigger consensus set means $w$ is bigger
than previously thought, so $N$ need not be so large.

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{Robust maximum likelihood estimation}

Note that step (iv) of RANSAC was to re-estimate the model from
\alert{all of the points} in $S_i$.  We should use maximum likelihood
in this case.

\medskip

\alert{Problem}: the set of inliers could \alert{change} after we
compute the new maximum likelihood model.

\medskip

We could just \alert{accept the estimate anyway}.

\medskip

Some approaches \alert{recompute the inliers} after obtaining the
maximum likelihood model then \alert{repeat} maximum likelihood model
estimation until the consensus set converges.

\medskip

See text for detailed discussion and other alternative approaches.

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{Using RANSAC to estimate $\mat{H}$}

\begin{block}{Automatic $\mat{H}$ estimation: Objective}
  Given two images, compute the homography.
\end{block}

\begin{block}{Automatic $\mat{H}$ estimation: Algorithm}
\begin{itemize}
\item[(i)] Compute a set of \alert{interest points} in each image.
\item[(ii)] Compute \alert{putative correspondences} between the point
  sets.
\item[(iii)] \alert{RANSAC robust estimation}: Repeat for $N$ samples,
  where $N$ is determined adaptively as previously described:
  \begin{itemize}
  \item[(a)] Select a random sample of 4 correspondences and compute
    $\mat{H}$.
  \item[(b)] Calculate the distance $d_{\perp}$ for each putative
    correspondence.
  \item[(c)] Find the number of inliers for which $d_{\perp} < t =
    \sqrt{5.99}\sigma$ pixels.
  \end{itemize}
\item[(iv)] Re-estimate $\mat{H}$ using the inliers and the Gold
  Standard algorithm
\item[(v)] (Optional) use the new $\mat{H}$ to recompute the matching set of
  interest points and repeat from (iv) until convergence.
\end{itemize}
\end{block}

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{Using RANSAC to estimate $\mat{H}$}

There are two implementation details that need consideration: how to
measure the distance and how to select the samples.
\begin{itemize}
\item For distance, the \alert{symmetric transfer error}
  $d^2_{\text{transfer}} = d(\vec{x},\mat{H}^{-1}\vec{x}')^2 +
  d(\vec{x}',\mat{H}\vec{x})^2$ is appropriate since it is easy to
  compute.  Reprojection error is better but expensive.
\item Widespread distribution of the samples is good, to ensure good
  interpolation in the rest of the image.  The sampler can be biased
  to pick points in different regions of the image rather than
  uniformly.
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{Example results}

Example initial images (Hartley and Zisserman, 2004, Fig.\ 4.9):

\medskip

\begin{columns}[T]
\column{2.25in}
\myfig{2.2in}{HZ-fig3-9a}{Image 1}
\column{2.25in}
\myfig{2.2in}{HZ-fig3-9b}{\parbox{2in}{Image 2, related by rotation
    around camera center}}
\end{columns}

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{Example results}

Detected corners, about 500 on each image.

\begin{columns}[T]
\column{2.25in}
\myfig{2.2in}{HZ-fig3-9c}{}
\column{2.25in}
\myfig{2.2in}{HZ-fig3-9d}{}
\end{columns}

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{Example results}

Initial set of 268 correspondences obtained by SSD of image patches
around the corners:

\medskip

\begin{columns}[T]
  \column{2.25in} \myfig{2.2in}{HZ-fig3-9e}{\parbox{2in}{268 putative
      correspondences, Hartley and Zisserman (2004), Fig.\ 4.9(e)}}
  \column{2.25in} \myfig{2.2in}{HZ-fig3-9f}{\parbox{2in}{117/268
      outliers, Hartley and Zisserman (2004), Fig.\ 4.9(f)}}
\end{columns}

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{Example results}

Final set of 262 correspondences after RANSAC, guided matching, and MLE.

\medskip

\begin{columns}[T]
  \column{2.25in} \myfig{2.2in}{HZ-fig3-9g}{\parbox{2in}{151 inliers
      consistent with $\mat{H}$ found by RANSAC, Hartley and Zisserman
      (2004), Fig.\ 4.9(g).}}  \column{2.25in}
  \myfig{2.2in}{HZ-fig3-9h}{\parbox{2in}{Final set of 262
      correspondences after guided matching and MLE beginning from the
      RANSAC solution, Hartley and Zisserman (2004), Fig.\ 4.9(h).}}
\end{columns}

\end{frame}

\begin{frame}
\frametitle{Robust estimation}
\framesubtitle{Implementation}

See the course Web site for a test using OpenCV, its built-in
\alert{Shi-Tomasi feature detector} and its built-in RANSAC $\mat{H}$
estimation function.

\medskip

See the Torr toolbox at {\small
  \url{http://cms.brookes.ac.uk/staff/PhilipTorr/Code/master_code.htm}}
for a Matlab implementation using the \alert{Harris corner detector}.

\medskip

When there is significant rotation and/or scale in the image plane,
the \alert{SIFT} (Scale Invariant Feature Transform) feature detector is much
better than Harris or Shi-Tomasi.

\medskip

There is a large literature on feature detectors enabling
correspondence estimation over multiple views.

\medskip

Transform invariance matching will be very important when we move to
multiple views of a general scene and estimate the fundamental matrix
$\mat{F}$ rather than a homography $\mat{H}$.

\end{frame}

\end{document}