\documentclass[main.tex]{subfiles}
\begin{document}

% \section*{Thu Oct 31 2019}
\paragraph{Effective degrees of freedom}

% \marginpar{Friday\\ 2019-10-31, \\ compiled \\ \today}

% Neutrinos have very low mass: therefore they become relativistic very quickly.

% We saw last time the case of either bosons' or fermions' number density, energy density and pressure.

% In the nonrelativistic case instead we must consider the second order in \(q / \sqrt{m} \), and we get the Boltzmann suppression factor out of the integral: \(\exp(-m/T) \).

% In the nonrelativistic case there is no distinction between bosons and fermions.

% What happens in the part of the history of the universe where all the particles were in thermal equilibrium?

We have been discussing the behavior of a single particle species with \(g\) degrees of freedom; however we know that there were many types of particles in the early universe, so we need a way to generalize these results.
We do so by defining the number of effective degrees of freedom:
%
\begin{align}
g_*(T) =
\sum _{i \in \text{BE}} g_i \qty(\frac{T_i}{T})^{4}
+
\frac{7}{8}
\sum _{i \in \text{FD}} g_i \qty(\frac{T_i}{T})^{4}
\,,
\end{align}
%
where the index \(i\) labels all the particle species in our model, running over all of those which are relativistic at temperature \(T\) (that is, as a first approximation, we only count those with masses \(m_i < T\)).
The equilibrium temperature is \(T\), while \(T_i\) are the temperatures of the various particle species, which we allow to be different from \(T\) --- we will elaborate on this point in a moment.
We distinguish two different terms in the sum, depending on whether the particle species obey Bose-Einstein or Fermi-Dirac statistics, since as we have seen the latter have a prefactor of \(7/8\) in the expression for the energy density.

This definition is constructed so that we can write the compact relation
%
\begin{align} \label{eq:radiation-energy-density-effective-dof}
\rho (T) = g_*(T) \frac{\pi^2}{30} T^{4}
\,.
\end{align}

Of course, considering particles completely when \(m_i < T\) and not at all when \(m_i > T\) is a simplification: in the region in which the temperature is of the order of the mass of the particle there will be a transition, which can be calculated properly by doing the integrals numerically. The results are shown in figure 3 of a paper by \textcite[]{husdalEffectiveDegreesFreedom2016}, which can also be referred to for many more details on effective degrees of freedom.
Figure 1 of the same paper shows how \(g_*\) decreases while the temperature of the universe decreases and more and more particle species become nonrelativistic.

Why do we consider the possibility of the temperature of a particle species being different from the equilibrium temperature?

Each process involving particles, be it decay or scattering, is characterized by a certain timescale.
If the timescale of a certain interaction is larger than the cosmological timescale (the age of the universe), then that interaction statistically will not happen.
Particles which cannot reach thermal equilibrium because of this are called \emph{decoupled}, ones for which this is not the case are called \emph{coupled}.

Although they may not interact, as long as they are relativistic decoupled particles can still affect the energy density of the universe, so we need to count them.

\paragraph{The time-temperature relation}

We want to find a relation between time and temperature in the early universe.
Let us consider ultrarelativistic particles which are coupled, in the radiation-dominated early universe (here ``radiation'' refers to all kinds of ultrarelativistic particles).

% In this case, from the ``conservation of the stress-energy tensor'' we know that \(\rho \propto a^{-4}\).

We start from the first Friedmann equation
%
\begin{equation}
  H^2= \frac{8 \pi G}{3} \rho - \frac{k}{a^2}
\,,
\end{equation}
%
neglect the curvature term\footnote{We can do so since we know that right now the contribution to the global \(\Omega \) of curvature is small (we have not been able to distinguish it from zero) and while this term scales as \(a^{-2}\) the matter term scales as \(a^{-3}\). Since the matter term is dominant over the curvature term now, it was even more so earlier.} and use the facts that for a radiation-dominated universe \(\rho \propto a^{-4}\) while \(a \propto t^{1/2}\), meaning that \(H = \dot{a} / a =  1/ (2t)\).

% The equation becomes
% %
% \begin{subequations}
% \begin{align}
%   \qty(\frac{\dot{a} }{a})^2 &= \frac{8 \pi G }{3} \rho_{*} \qty(\frac{a}{a_{*}})^{-4} - \frac{k}{a^2}
% \,,
% \end{align}
% \end{subequations}
% %
% but it is not enough to look at the slopes: we can, however, have information about the normalization as well from present-day observations.

% If the second term is much smaller than the first today, then it was even more so in the far past. We can rewrite the equation as
% %
% \begin{equation}
%   1 = \Omega_{\text{tot}} - \frac{k}{a^2 H^2} \equiv \Omega _{\text{tot}} + \Omega_{\text{curvature}}
% \,,
% \end{equation}
% %
% where \(\Omega _{\text{curvature}} \equiv -k / (a^2 H^2)\).

% So we neglect the second term: approximately we then have
% %
% \begin{equation}
%   \qty(\frac{\dot{a } }{a})^2 = \frac{8 \pi  G }{3} \rho _{\text{rad}}
% \,.
% \end{equation}
% %
% We know that \(a(t) \propto t^{1/2}\), therefore \(\dot{a} / a = H = 1/2t\).

% \todo[inline]{Wait, how does this work?}

Substituting these, as well as the expression we have found for the energy density in terms of the effective degrees of freedom, we get
%
\begin{equation}
  \frac{1}{4 t^2} = \frac{8 \pi G}{3} g_{*} (T) \frac{\pi^2}{30} T^{4}
\,.
\end{equation}
%
% where
% %
% \begin{equation}
%   g_{*} = g_{*}(T) = \sum _{i \in BE } g_i \qty(\frac{T_{i}}{T})^{4} + \frac{7}{8} \sum _{i \in FD } g_i \qty(\frac{T_{i}}{T})^{4}
% \,,
% \end{equation}
% %
% where we insert a correction factor in order to not consider particles which are nonrelativistic (?).
% \(BE\) and \(FD\) denote the bosonic and fermionic degrees of freedom respectively.

% The \(i\) spans the different types of particles in the SM.
% The \(T_i\) are the thermalization temperatures of the various species.

% The plot of \(g_{*}\) in terms of the temperature goes ``down in steps'', as more and more species become nonrelativistic.

Then we have a formula for temperature in terms of time:
%
\begin{align}
  \frac{1}{2t} &= \qty(\frac{8 \pi G}{3})^{1/2} g_{*}^{1/2} \qty(\frac{\pi^2}{30})^{1/2} T^2 \\
  t &\approx \underbrace{\frac{1}{2 \sqrt{\frac{8 \pi \pi^2}{3 \times 30}}}}_{\approx\num{.301}} g_{*}^{-1/2} \frac{m_P}{T^2} \approx \qty(\frac{T}{\SI{}{MeV}})^{-2} \SI{}{s}
  \label{eq:time-temperature-relation}
\,,
\end{align}
%
where \(m_P  = G^{-1/2} \approx \SI{1.2e19}{GeV}\) is the Planck mass. Beware: there are different conventions for this, sometimes the definition is chosen as \(m_P = (8 \pi G)^{-1/2}\), which simplifies the Friedmann and Einstein equations somewhat.
This mass corresponds to the energy scale at which quantum gravitational effects cannot be neglected.

The last approximation in \eqref{eq:time-temperature-relation} is quite rough, as it neglects the variation of the effective number of degrees of freedom completely: however, the factor \(g_{*}^{-1/2}\) is of order 1 around \(T \approx \SI{1}{MeV}\), which is the region in which we will apply our formula, so this is fine for our purposes.

% When the universe is 1 second old, weak interactions decouple.

% Why are we allowing different thermalization temperatures?

% A hypothesis is that the particles exit the Planck epoch, \(t \approx m_P\), thermalized.
% When decoupling occurs, they can stop being thermalized.

% Another hypothesis is that they stop being thermalized at a certain temperature during inflation.

% Some time ago we discussed entropy conservation.

\paragraph{Entropy effective degrees of freedom}

Entropy density is defined as entropy per unit volume, \(s = S / V = (P+\rho ) / T\). Since the total entropy in a comoving region is conserved (if there is thermal equilibrium) the quantity \(s a^3\), proportional to \(S\), is conserved.

If we only have relativistic particles (which satisfy \(P = \rho / 3\)), the entropy density can be expressed as
%
\begin{equation}
s = (P + \rho ) / T = \frac{4}{3} \frac{\rho}{T} = (2 \pi^2 / 45) g_{* s} T^{3}
\,;
\end{equation}
%
where we defined a new number of effective degrees of freedom, \(g_{*s}\), whose definition is slightly different from that of the one used for the energy, \(s \propto T^3\) as opposed to \(\rho \propto T^{4}\):
%
\begin{equation}
  g_{* s} \equiv \sum _{i \in BE} g_i \qty(\frac{T_i}{T})^3
  + \frac{7}{8} \sum _{i \in FD} g_i \qty(\frac{T_i}{T})^3
\,.
\end{equation}

The expression for \(s \propto g_{*s} T^3\) is more general than simply \(s \propto T^3\), and in fact with this new one we can \textbf{update Tolman's law}: taking the cube root of the conserved quantity \(s a^3\) we find \(T a g_{* s }^{1/3} = \const\).

% When the neutrinos decouple, their temperature keeps scaling like \(1/a\): they keep being thermalized with the photons even though they do not react.

\subsection{Decoupling and radiation temperature}

The temperature \SI{1}{MeV} occurs when the age of the universe is approximately \SI{1}{s}, and this is the point at which the weak interactions involving neutrinos stop occurring.\footnote{This point is also relevant for another process: the weak interaction mediated processes are also what allows there to be an equilibrium between protons and neutrons, so when those reactions stop they become independent, and they evolve differently, since protons are stable while neutrons are not. This is crucial when discussing nucleosynthesis, the formation of nuclei.}

% When we reach \(T \approx \SI{0.5}{MeV}\) electrons and positrons become thermalized: they decay into photons: the photon temperature \(T\) increases with respect to what would happen without this process (it actually just decreases slower).

When this happens, neutrinos decouple, so they stop interacting: they start evolving as any relativistic particle species would (they are relativistic since their mass is much lower than the \SI{}{MeV}).

At this point, the temperatures of neutrinos and photons are ``disconnected'', there is no mechanism to equalize them.
However, while the neutrinos are freely floating by, their energy density scaling like \(\rho \propto a^{-4}\), the photons will be active for a few more seconds.

The mass of electrons and positrons is around \SI{0.5}{MeV}, so until about \SI{4}{s} into the life of the universe, \SI{3}{s} from the decoupling of the neutrinos, the reaction \(e^{+} + e^{-} \leftrightarrow 2 \gamma \) is still in equilibrium.
After this, the populations of electrons and positrons annihilate, and since they have less effective degrees of freedom in which to deposit their energy than before (since the neutrinos are not coupled anymore) they dump it all into photons, thus increasing their temperature.

% The neutrinos are not updated when this process happens: their temperature then becomes consistently less than the one of the photons, but they keep scaling the same: one of their temperatures is a constant multiple of the other after this event.

After this occurs, the photons keep evolving like any other relativistic particle species, with \(\rho \propto a^{-4}\) but their temperature is higher than that of the neutrinos. Since they evolve in the same way, the ratio of the temperatures is constant.
Now we will calculate this ratio.

% First, at \SI{1}{MeV}, neutrinos decouple. Then, at \SI{0.5}{MeV}, electrons and positrons decouple.
We impose continuity of the entropy in a comoving volume across the transition which happens at \SI{0.5}{MeV} between the stage in which electrons, positrons and photons are in equilibrium and the stage in which they decouple, since the electrons and positrons are not relativistic anymore and thus annihilate.

This transition only affects the temperature of the photons, while the neutrinos decoupled three seconds earlier; so, before the transition the temperatures of photons and neutrinos are equal, after it the photons' temperature increases.

% We require continuity at this transition: \(T_{>}\) and \(T_{<}\) must both be \(T_{e}\).
% %
% \begin{equation}
%   s a^3 =
%   \frac{2 \pi^2}{45} g_{*s>}T^{3}_{>} a^{3}_{>} =
%   \frac{2 \pi^2}{45} g_{*s<}T^{3}_{<} a^{3}_{<}
% \,,
% \end{equation}
% %
% but we can drop all the factors we know to be continuous on the boundary.
Let us denote with an index \(>\) the quantities pertaining to an earlier time, \(t _{\text{transition}} > t\), while an index \(<\) will denote the quantities pertaining to a later time, \(t _{\text{transition}} < t\).

The ``updated'' version of Tolman's law reads \(T a g_{*s}^{1/3} = \const\),  and if the transition is fast enough the scale factor can be taken to be equal on both sides of it.
Therefore, we have
%
\begin{equation}
  T_{<} = \qty(\frac{g_{*s>}}{g_{*s<}})^{1/3} T_{>}
\,,
\end{equation}
%
so we can compute the temperature after the transition, \(T_<\), if we find the effective degrees of freedom before and after: before the transition we have photons, electrons and positrons. Photons have two polarization, and so do both electrons and positrons; also, the latter are fermions, so we find
%
\begin{equation}
  g_{* s>} = 2 + \frac{7}{8}\qty(4) = \frac{11}{2}
\,,
\end{equation}
%
while after the transition only photons are relativistic, so we have
%
\begin{equation}
  g_{*s<} = 2
\,.
\end{equation}

This means that the temperature of the photons increases by a factor
%
\begin{equation}
  T_{<} = \qty(\frac{11}{4})^{1/3} T_{>} \approx \num{1.4} T_>
\,,
\end{equation}
%
which allows us to compute the neutrino temperature at any time, since \(T_< / T_> = T_\gamma / T_\nu \), and because they scale in the same way:
%
\begin{equation}
  T_{\nu } = T_{\gamma } \qty(\frac{4}{11})^{1/3}
\,.
\end{equation}

Right now, the temperature will be around \(T_{0 \nu } \approx (4/11)^{1/3} T_{0 \gamma } \approx \SI{1.94}{K} \), where \(T_{0 \gamma }\) is the current CMB temperature.

As an exercise, let us compute the number of effective degrees of freedom some time after the decoupling of electrons, say at \(T = \SI{0.1}{MeV}\).
The global temperature \(T\) we are referring to is the one of the photons: so, applying the definition we find
%
\begin{subequations}
\begin{align}
  g_{*} &=  \sum _{i \in BE} g_i \qty(\frac{T_i}{T})^{4} +
  \frac{7}{8}\sum _{i \in FD} g_i \qty(\frac{T_i}{T})^{4}  \\
  &= 2 + \frac{7}{8} \qty(3 \times 2 \qty(\frac{T_{\nu }}{T_{\gamma }})^{4})  \\
  &= 2 + \frac{21}{4} \qty(\frac{4}{11})^{4/3} \approx 3.36
\,,
\end{align}
\end{subequations}
%
since we need to consider neutrinos (of which there are three flavors, each having two polarization states), which contribute to the total energy density, but not electrons which are not relativistic anymore.

With this result, we can find the energy density of radiation at that time according to \eqref{eq:radiation-energy-density-effective-dof}: we get
%
\begin{align}
\rho_r (\SI{.1}{MeV}) \approx \SI{1.1e-4}{MeV^4} = \SI{25.7}{g cm^{-3}}
\marginnote{Multiplied by \(\hbar^{-3}c^{-5}\) to get the CGS units.}
\,.
\end{align}
%

% \medskip
% \begin{center}
%     ***
% \end{center}
% \medskip

% Last hour we discussed the Planck mass \(m_P \approx \SI{1.2e19}{GeV}\): it also defines a wavelength, \(\lambda_{P} \approx \SI{e-33}{cm}\) and a timescale \(t_{P} \approx \SI{e-43}{s}\) (both of these are \(1/m_P\) in natural units).

% When the age of the universe is of this order, our theories are not guaranteed to work.

\section{Problems with the Hot Big Bang model, inflation}

Around the 1960s, cosmologies were trying to piece together a description of the early universe in terms of particle physics, as we discussed in this chapter up to here.
However, soon it became apparent that the standard cosmological model in use has some inconsistencies.
Let us now explore these.

\subsection{The cosmological horizon problem}

This was noticed as early as 1956.
Let us consider radial null geodesics in a universe described by a FLRW metric. These are the worldlines of photons we can detect with telescopes.
Imposing \(\dd{s}^2 = 0 \) we find:
%
\begin{align}
  c^2 \dd{t^2} = a^2(t) \frac{1}{1-kr^2} \dd{r^2}
\,,
\end{align}
%
which we can integrate (taking one of the two solutions for simplicity, choosing one over the other just amounts to parametrizing time in the opposite direction) to find
%
\begin{align}
  \int_0^{t} \frac{c \dd{t}}{a(t)} = \int_0^r \frac{ \dd{\widetilde{r}} }{\sqrt{1-k \widetilde{r}^2}} = f(r)
\,.
\end{align}

The function \(f(r)\) gives us the proper distance between emission and detection of a photon, but it does so in terms of the adimensional coordinate \(r\): in order to get something which has the dimensions of a length we need to multiply by \(a\) calculated at a certain time,\footnote{Note that this choice is arbitrary: we are computing the comoving distance \emph{as measured at the cosmic time of detection}.}
%
\begin{align}
  d _{\text{hor}} (t) = a(t) \int_0^{t} \frac{c \dd{\widetilde{t}}}{a(\widetilde{t})}
\,.
\end{align}

\textbf{If this integral is convergent, we should be worried}: let us see why.

If we integrate from the beginning of time to now, we get the spatial (current) comoving distance elapsed by a photon which started moving at the start of time.
This is the radius of the largest region we could in principle observe. It is of the order of \SI{3}{\giga\parsec}.
Since the integral is convergent this is finite, and it is increasing as time passes. So, ever-further regions are ``coming into view'' (at least in principle).
Roughly speaking, the issue is that the regions which we start seeing at the edges should be causally disconnected from the ones already in view, so we would not expect them to exhibit the same properties --- but \emph{they do}.
This is the basic idea, let us formalize it slightly and connect it to observations.

We cannot actually see light coming from the very edge of the in-principle-observable universe, since for redshifts larger than \(z _{LS} \approx 1100\) the universe was opaque to electromagnetic radiation. The surface of points at this redshift is called the \emph{Last Scattering} surface. So, we refer our expectations to the CMB, which was emitted as the primordial plasma became transparent.

% This is not the case, for example, in Minkowski spacetime.

% If there is an end of time, there is also a future horizon.

% The quantity \(d _{\text{Hor}} (t) \) is increasing in time: this causes the Cosmological Horizon problem.

% \todo[inline]{What is the problem?}

% An important moment is the \emph{recombination of hydrogen}: the formation of the first Hydrogen atoms, so the first moment at which Compton scattering can occur.
% This causes the decoupling of radiation and baryionic matter.
% This is the moment at which the radiation in the CMB was emitted.

The CMB was emitted at a cosmic time of \(t \approx \SI{3.8e5}{yr}\) after the Big Bang.
It is observed to be very close to being uniform, with \(\Delta T / T \sim \num{e-5}\) after correcting for the Doppler dipole modulation:
it looks like a distribution emitted by matter in thermal equilibrium.
Crucially, this holds at any angular scale we choose: the equilibrium is there across the whole sphere.

Recall that the \emph{angular diameter distance} \(d_A\) is defined so that if an object with linear size at emission \(\Delta x\) spans an angle \(\Delta \theta \) then we have (in the small-angle approximation)
%
\begin{align}
d_A = \frac{\Delta x}{\Delta \theta }
\,.
\end{align}

The angular diameter distance to the last scattering surface  is approximately \(d_A (z_{LS}) \approx \SI{12.8}{\mega\parsec}\).
On the other hand, the scale of the particle horizon at that redshift can be calculated by taking the difference of comoving distances to us, \( d_C(z_\infty) - d_C(z_{LS}) \approx \SI{281}{\mega\parsec}\) and multiplying it by the scale factor, \(a(z_{LS}) \approx \num{9.2e-4}\), which yields a horizon scale of approximately \(r_H \approx \SI{260}{\kilo\parsec}\) at that time.\cprotect\footnote{All the calculations were made automatically using the \texttt{astropy} package, using a flat \(\Lambda \)CDM model with parameters obtained from the Planck mission \cite[]{adePlanck2015Results2016}.
\begin{lstlisting}[language=Python]
from astropy.cosmology import Planck15 as cosmo
import numpy as np
import astropy.units as u
z_LS = 1089
dx = (cosmo.comoving_distance(np.inf) - cosmo.comoving_distance(z_LS)) * cosmo.scale_factor(z_LS)
dA = cosmo.angular_diameter_distance(z_LS)
(dx / dA).to(u.degree, equivalencies=u.dimensionless_angles())
\end{lstlisting}}

We can then say that \(\Delta x \sim r_H\), therefore the angular scale at which we expect to be able to observe correlations since there can be causal connections is around
%
\begin{align}
\Delta \theta \approx \frac{ \Delta x}{d_A} \approx \SI{0.02}{rad} \approx \SI{1.2}{\degree}
\,.
\end{align}

\todo[inline]{A similar calculation \cite[eqs.\ 8--12]{tojeiroUnderstandingCosmicMicrowave} yields \(\Delta \theta \sim (1+z_{LS})^{-1/2} \approx \SI{1.7}{\degree}\), using the (reasonable) assumption of matter dominance in the epoch of recombination. Some steps there are not really clear to me, so I'm not sure whether my line of reasoning is equivalent (and valid) besides the assumption.}

This is in stark opposition with the scale of observed correlations, which span the whole sky!

% Let us consider the past light cones from two points diametrically opposite with respect to us, at this point in time: they do not overlap, so the CMB cannot be causally correlated. However, it seems like it is!

% \todo[inline]{What is the angular scale at which the light cones overlap?}

% We have a classification by Bianchi of non-isotropic universes (in 9 classes).

\todo[inline]{Mention in the lecture of the Mixmaster Universe by Misner (and the Bianchi classification of Lie Algebras for context) as an alternative to inflation --- is this relevant here?}

% There is the Mixed Master Universe.

% Let us consider
% %
% \begin{align}
%   R _{\text{Hor}} = \int_0^{t} \frac{c \dd{\widetilde{t}} }{a(\widetilde{t})}
% \,.
% \end{align}

\paragraph{Cosmic inflation}

Now, if the quantity \(d _{\text{Hor}}(t)\) were to diverge this would mean that we could have a causal connection with any point in the universe, provided we went far enough back in time.

We can approximate
%
\begin{align}
  d _{\text{Hor}} (t) = a(t) \int_0^{t} \frac{c \dd{\widetilde{t}} }{a(\widetilde{t})} \approx ct \sim \frac{c}{H} \equiv
  d _{\text{H}}
\,,
\end{align}
%
where we defined the new \emph{Hubble distance}, \(d_H = c/H\).
This is a physical distance, but we can also define the corresponding
dimensionless comoving Hubble radius: \(r_H = c/(Ha) = c / \dot{a} \), which satisfies \(d_H = a r_H\).

We can hypothesize that there was a period in the early universe when the comoving radius \(r_H\) was decreasing with time: if this is the case, the regions we are observing today as ``coming into view'' could actually have been in causal contact in the early universe.

For the comoving radius to be decreasing (\(\dot{r}_H\) < 0), the condition is (neglecting factors of \(c\), or working in natural units):
%
\begin{align}
  \dot{r}_H = - \frac{ \ddot{a} }{\dot{a}^2} <0
\,,
\end{align}
%
therefore we need \(\ddot{a} >0\) for at least some time.

The second Friedmann equation (in natural units) tells us that
%
\begin{align}
  \ddot{a} = -\frac{4 \pi G }{3} \qty(\rho + 3P)
\,,
\end{align}
%
therefore the condition we need to have is \(\rho + 3P <0\).
So, since the energy density is positive, the condition is \(P < - \rho /3\).

\paragraph{Types of inflation}

Another way to express the parameter \(\ddot{a}\) is as the derivative of \(\dot{a} = Ha\):
%
\begin{align}
  \ddot{a} = \dot{a}H + a \dot{H} = a (H^2 + \dot{H})>0
\,,
\end{align}
%
so the condition can also be expressed as \(H^2+ \dot{H} >0\).

We have shown earlier that, neglecting curvature, the time-dependence of the scale factor looks like
%
\begin{align}
  a(t) = a_{*} \qty(1 + \frac{3}{2}\qty((1+w)H_{*}\qty(t-t_{*})))^{2/ (3(1+w))}
\,,
\end{align}
%
and it is reasonable to use this result since, as we will discuss in this section, inflation makes curvature negligible.

We can characterize the solutions based on the sign of \(\dot{H}\), which determines whether the equation of state parameter \(P / \rho = w\) is larger or smaller than \(-1\): the possibilities are
\begin{enumerate}
    \item \(\dot{H} < 0\) while \(\dot{H} + H^2 >0\): this corresponds to \(-1 < w < -1/3\), which in General Relativity-speak is called a \emph{violation of the weak energy condition}, and inserting it in our general solution gives \(a(t) \propto t^{\alpha }\) for some \(\alpha > 0\), a \textbf{power-law inflation};
    \item \(\dot{H} =0\): this is a De Sitter, dark-energy dominated universe, whose scale factor evolves like \(a(t) = \exp(Ht) \), corresponding to \(w = -1\);
    \item \(\dot{H} >0\) (and so also \(\dot{H} + H^2 > 0\)),
    which corresponds to \(w<-1\) and which gives us \(a(t) \propto (t - t _{\text{bounce}})^{-\alpha }\) with \(\alpha > 0\), a singularity in the future.
\end{enumerate}

% Big Rip singularity: with \(a(t) = \abs{t - t_{as}}^{- \alpha } \) with \(\alpha >0\). This is called poli-inflation.
The boundary at \(w = -1\) is called the \emph{phantom divide}.

\paragraph{An estimate of the inflation \(e\)-foldings.}

By how much does the early universe need to inflate?
The condition we need to impose is that the comoving radius of the universe at some early time, \(r_H (t _{\text{in}})\), should be larger than the current one, \(r_H (t_0 )\). Since \(r_H = d_H / a\), we can write this inequality as
%
\begin{align}
\frac{d_H (t _{\text{in}})}{a(t _{\text{in}})} a(t_f)
\geq
\frac{d_H (t _0)}{a(t _0)} a(t_f)
\,,
\end{align}
%
where we multiplied both sides by the scale factor calculated at \(t_f\), a time corresponding to the end of inflation, a minimum for the scale factor. Let us define \(Z _{\text{min}} = a(t_f) / a(t _{\text{in}})\).
This will be \(\gg 1\), and it will describe by how much the universe inflated.

In our rough approximation \(d_H \sim H^{-1}\), so we can say that the boundary of the inequality, the minimum inflationary expansion, will be
%
\begin{align}
Z _{\text{min}} &= \frac{d_H (t_0 )}{d_H (t _{\text{in}})}
\frac{a(t_f)}{a(t_0 )}  \\
&= \frac{H(t _{\text{in}})}{H(t_0 )} \frac{a(t_f)}{a(t_0 )}  \\
&= \frac{H(t _{\text{in}})}{H(t_f )}
\frac{H(t_f)}{H(t_0 )} \frac{a(t_f)}{a(t_0 )}  \\
Z _{\text{min}} \frac{H_f}{H _{\text{in}}} &= \frac{H_f}{H_0} \frac{a_f}{a_0} \marginnote{Denoting \(H(t_i) \equiv H_i\) and similarly for \(a\).}
\,.
\end{align}

Now, we want to put some numbers into this expression: we know from the first Friedmann equation that (as long as there is no spatial curvature) \(H^2 \propto \rho \), while the third tells us that \(\rho \propto a^{-3 (1+ w)}\): therefore, \(H \propto a^{- 3 \frac{1 + w}{2}}\). Since we are working with ratios, proportionality is all we need.
Now, what \(w\) should we use? At any stage in the evolution of the universe there are several fluids, but in order to simplify the calculation we will only consider the dominant one and neglect dark energy in the current phase of the evolution of the universe.
In the inflationary phase we will have an undetermined \(w = w _{\text{inf}}\), so that
%
\begin{align}
\frac{H_f}{H _{\text{in}}} = \qty(\frac{a_f}{a _{\text{in}}})^{- 3\frac{1+w}{2}}
\,,
\end{align}
%
so the left-hand side of the equation reads
%
\begin{align}
Z _{\text{min}} \frac{H_f}{H _{\text{in}}}
= Z _{\text{min}}^{1 - 3 \frac{1+w _{\text{inf}}}{2}}
= Z _{\text{min}}^{\frac{-1 - 3w _{\text{inf}}}{2}}
= Z _{\text{min}}^{\abs{\frac{1 + 3w _{\text{inf}}}{2}}}
\,,
\end{align}
%
since \(w _{\text{inf}} > - 1/3 \) means \(1 + 3w _{\text{inf}} > 0\).
The right-hand side has precisely the same form, so we can express it as
%
\begin{align}
\frac{H_f}{H_0 } \frac{a_f}{a_0 } = \qty(\frac{a_f}{a_0 })^{- \frac{1 + 3 w}{2}}
\,,
\end{align}
%
where \(w\) is that of the dominant fluid from the end of inflation to now. The issue is, there is not a single one! In the early stages radiation was dominant, then matter started dominating (now dark energy is dominant, but we shall not worry about it).
So, we split the term in two, with the radiation-matter equality being the breaking point. The earlier radiation-dominated phase is characterized by \(w = 1/3\), while the latter matter-dominated phase is characterized by \(w = 0\), so we can compactly write the term as
%
\begin{align}
\frac{H_f}{H_0 } \frac{a_f}{a_0 } = \qty(\frac{a_f}{a _{\text{eq}}})^{- \frac{1 + 1}{2}} \qty(\frac{a _{\text{eq}}}{a_0 })^{- \frac{1 + 0}{2}}
= \qty(\frac{a _{\text{eq}}}{a_f}) \qty(\frac{a_0}{a _{\text{eq}}})^{1/2}
\,.
\end{align}
%

With this result, we now have an almost explicit expression for \(Z _{\text{min}}\):
%
\begin{align}
Z _{\text{min}}
= \qty[\qty(\frac{a _{\text{eq}}}{a_f}) \qty(\frac{a_0}{a _{\text{eq}}})^{1/2}]^{\abs{ \frac{2}{1 + 3 w _{\text{inf}}}}}
= \qty[\qty(\frac{a _0}{a_f}) \qty(\frac{a_0}{a _{\text{eq}}})^{-1/2}]^{\abs{ \frac{2}{1 + 3 w _{\text{inf}}}}}
\,.
\end{align}

The ratio \(a_0 / a _{\text{eq}}\) can be written as \(1 + z _{\text{eq}} \approx \num{2.3e4} \Omega h^2 \approx \num{e4}\) (very roughly).
The other rough estimate we make is to apply Tolman's law, so that
%
\begin{align}
\frac{a_0 }{a_f} \approx \frac{T_f}{T_0 } = \frac{T_f}{m_p} \underbrace{\frac{m_p}{T_0 }}_{\sim \num{e32}}
\,;
\end{align}
%
properly speaking this only holds when there is radiation dominance so it does not apply for the whole range in which we are applying it (up to today), but we are estimating the order of magnitude \emph{of an exponent}, so even an order-of-magnitude error is not an issue.
We normalized by the Planck mass (or temperature, since we are using natural units) since the temperature \(T_f\) at the end of inflation probably was of that order of magnitude.
The final estimate we get is
%
\begin{align}
Z _{\text{min}} \approx \qty[\num{e30} \frac{T_f}{m_p}]^{\abs{ \frac{2}{1 + 3 w _{\text{inf}}}}}
\,.
\end{align}

We do not know what \(w _{\text{inf}}\) is besides it being smaller than \(- 1/3\); let us say that it is of the order of \(-1\) like the current dark-energy dominated phase.
If we further assume that \(T_f / m_p \sim 1\),\footnote{The constraints on \(T_f\) are the parameters of baryogenesis (we can observe it through the isotope distribution early galaxies) and the fact we have not detected gravitational waves from this early time. These tell us that \(T_f / m_p\) cannot be larger than \num{e-3}.} we find \(Z _{\text{min}} \sim \num{e30} \approx \exp(70)\).

This is often written as ``70 \(e\)-foldings'', meaning 70 \(e\)-fold increases in size.

% \todo[inline]{What is this about?}

% The third condition is very hard to achieve.

\subsection{The flatness problem}

Now, let us consider the \emph{flatness problem}, which was first proposed by Dicke and Peebles in 1986.

\todo[inline]{Cannot seem to find the paper or article\dots}

% The parameter \(\Omega = \frac{8 \pi G \rho(z) }{3 H^2(z)} \) diverges from 1 as time increases.
% Measurements of \(\Omega _{\text{tot}}\) gave approximately 0.1.

% The first Friedmann equation gives \(H^2\) in terms of \(\rho \) and \(k/a^2\); the first term scales like \(\rho \propto a^{-3(1+w)}\), while the second scales like \(a^{-2}\), so as long as \(w > - 1/3\) (which is the case since radiation dominance) the density term is dominant over the spatial curvature term.
% Therefore, as \(a\) increases the
The first Friedmann equation can be rearranged as
%
\begin{align}
\underbrace{\frac{3 a^2 H^2}{8 \pi G}}_{= a^2 \rho_C} &= \rho a^2 - \frac{3k}{8 \pi G}  \\
(\rho_C - \rho ) a^2 &=
\qty( \frac{1}{\Omega } - 1) \rho a^2
= - \frac{3k}{8 \pi G} = \const
\,,
\end{align}
%
where \(\rho _C = 3 H^2 / 8 \pi G\) is the critical density. The right-hand side only contains constants, therefore the left-hand side must be constant as well.
As the universe evolves \(\rho \) decreases while \(a\) increases; however \(\rho \propto a^{-3 (1+w)}\) by the third Friedmann equation, meaning that the term scales like \(\rho a^2 \propto a^{-1 - 3 w}\): as long as \(w > -1/3\), which is the case for most of the universe's evolution (with radiation and matter dominance) the term is decreasing, meaning that the other term must be increasing.

If \(k = 0\) we also have \(\Omega \equiv 1\), so the point is moot, however as we have already mentioned there are reasons to think this is unlikely.

So, let us consider the case \(k = \pm 1\): then, the term \(\Omega^{-1}- 1\) must scale like \(a^{1 + 3 w}\) in order to balance the other.

% Let us assume that \(w\) is constant. Then, \(\rho (z) = \rho_{0} (1+z)^{3(1+w)}\). Recall that
% %
% \begin{subequations}
% \begin{align}
%   H^2(z) &= \frac{8 \pi G }{3} \rho (z) - \frac{k}{a^2}  \\
%   H_0^2 &= \frac{8 \pi G}{3}\rho_0 - \frac{k}{a_0^2}
% \,,
% \end{align}
% \end{subequations}
% %
% the latter of which implies \(1 = \Omega - k / (a_0^2 H_0^2)\). So,
% %
% \begin{subequations}
% \begin{align}
%   H^2(z) &= H_0^2 \qty(\frac{8 \pi G }{3 H_0^2} \rho (z) - \frac{k}{a^2 H_0^2}) \\
%   &= H_0^2 \qty(\frac{\rho (t)}{\rho_0 }\Omega_0 \frac{a_0^2}{a} (1-\Omega_0 ))  \\
%   &= H^2(z) = H_0^2 (1+z)^2 \qty(\Omega_0 (1+z)^{1+3w} + (1-\Omega_0 ))
% \,,
% \end{align}
% \end{subequations}
% %
% so in the end
% %
% \begin{subequations}
% \begin{align}
%   \Omega (z) &= \frac{8 \pi G \rho_0 }{3 H_0^2 } \frac{\rho (z)}{\rho_0 } \frac{H_0^2}{H^2(z)} \\
%   &=\Omega_0 (1+z)^{1+3w} \qty(1-\Omega_0 + \Omega_0 (1+z)^{1+3w})
% \,,
% \end{align}
% \end{subequations}
% %
% and since \(\rho (z) = \rho_0 (1+z)^{3(1+w)}\) we get
% %
% \begin{subequations}
% \begin{align}
%   \Omega^{-1} (z) -1 &= \frac{\Omega_0 \qty((1-\Omega_0 ) + \Omega_0 (1+z)^{1+3w}) - (1+z)^{1+3w} }{(1+z)^{1+3w}}  \\
%   &= (\Omega_0^{-1} -1) (1+z)^{-(1+3w)}
% \,.
% \end{align}
% \end{subequations}

If we assume \(w = 1/3\) for all times we get (using the fact that \(a \propto (1+z)^{-1}\) and Tolman's law) that the term calculated at a redshift \(z\) is given by:
%
\begin{align}
  \Omega ^{-1} (z) -1
  = (\Omega_0 ^{-1} -1) \qty(1+z)^{-2}
  = (\Omega_0 ^{-1} -1) \qty(\frac{T_0 }{T(z)})^{2}
\,.
\end{align}

\todo[inline]{The reasoning in \cite[]{paccianiAppuntiCorsoPhysical2018} looks way more complicated, but this way seems just as valid\dots}
The assumption \(w \equiv 1/3\) is not really correct, but the result would only change by a few orders of magnitude if we did the calculation properly, and as before we are giving only a rough estimate \emph{of an exponent}.

Let us extend this line of reasoning back to the Planck epoch, since beyond that our theory of gravity might behave differently so it is not justified to apply Friedmann's equations.

If we compute \(T _{\text{Pl}} / T_0 \) we get approximately \num{e32}.
% Then when squaring we get \num{e-64}, without the approximation \(w=1/3\) we get \num{e-60}.
This means that
%
\begin{align}
\Omega^{-1}(z_{\text{Pl}}) - 1 \approx (\Omega_0^{-1} - 1) \num{e-64}
\,.
\end{align}

The correct calculation, keeping track of the dominant fluid in each phase, gives \(\num{e-60}\) instead of \(\num{e-64}\): not that significant a difference when discussing these kinds of numbers.

The current estimate for \(\Omega_0 \), as discussed in section \ref{sec:global-omega-contributions}, is \emph{at most} of the order of \(\num{e-3}\) away from 1 --- therefore, in the Planck epoch the parameter would have needed to be different from 1 by a part in \num{e-63}.

% Then, we see that there is something deeply unnatural in the Friedmann model.
% How is it possible that the universe is still so flat, even when the universe is so old? Oldness and flatness seem incompatible.
This is a type of \emph{fine-tuning} problem: the initial conditions seem to require an ``unnatural'' number like \(\Omega \sim 1 \pm \num{e-63}\).
Typically, a fine-tuning problem is interpreted as a signal that we should improve our theory.\footnote{This approach has also received criticism \cite[sec.\ 3.2]{hossenfelderScreamsExplanationFinetuning2019}: we do not know what distribution the initial conditions are drawn from, so how can we say whether a certain number is more likely (or ``natural'') than another?}

\end{document}


\documentclass[11pt]{extarticle}

\usepackage[utf8]{inputenc}
\usepackage{extsizes} % Allow for more sizes such as 14pt or 17pt on document class.
\usepackage{mathtools} % Allows conditional math expressions, etc.

% Don't output references in case they're empty - http://tex.stackexchange.com/questions/74476/how-to-avoid-empty-thebibliography-environment-bibtex-if-there-are-no-refere

\let\myBib\thebibliography
\let\endmyBib\endthebibliography

\renewcommand\thebibliography[1]{\ifx\relax#1\relax\else\myBib{#1}\fi}

\begin{document}

%% Front page.
\title{06.02 - Machine Learning System Design}

    \date{}

    \maketitle

\newpage
%% Abstract page.

%% Table of contents page.

%% Body start.
\section{Prioritizing what to work on: Spam classification
example}\label{prioritizing-what-to-work-on-spam-classification-example}

If we want to build a spam filter, we first need to know what is the
\emph{x} and the \emph{y} in our problem.

\emph{y} is easy: 1 if spam, 0 if ham.

One way of choosing our \emph{x} features would be to find 100 words
indicative of spam/ham. Then, our feature vector would be full of
boolean features each indicating whether the word appeared in the email
or not.

\subsection{How to spend your time to make it have low
error?}\label{how-to-spend-your-time-to-make-it-have-low-error}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  Collect data. Eg ``honeypot'' project.
\item
  Sophisticated features based on email header.
\item
  Sophisticated features based on email body. Eg should ``discount'' and
  ``discounts'' be treated as the same word.
\item
  Develop algorithm to detect misspellings (med1cine, w4tches).
\end{itemize}

\section{Error Analysis}\label{error-analysis}

\subsection{Recommended approach}\label{recommended-approach}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  Start with a simple algorithm that you can implement quickly (avoid
  premature optimization).
\item
  Plot learning curves to decide if more data, more features are likely
  to help.
\item
  Error analysis: Manually examnie the examples that your algorithm made
  errors on. See if you spot any trend in the type of examples it is
  making errors on.
\end{itemize}

In the case of the emails, if we have a certain amount of missclassified
emails, we can check the type of them (eg most common are pharma,
replica, steal passwords, \ldots{}) and what features you thjink would
have helped the algorithm to classify them correctly.

Manually doing this can serve the purpose of finding the thing where
spending time is more worth it.

\subsection{The importance of numerical
evaluation}\label{the-importance-of-numerical-evaluation}

It's important to have a reference number (such as accuracy) in order to
know whether doing a change is an improvement or not.

Eg, should discount/discounts/discounted/discounting be treated as the
same word? Check score doing it / not doing it and decide based on that.

\section{Error metrics for skewed
classes}\label{error-metrics-for-skewed-classes}

\subsection{Cancer classification
example}\label{cancer-classification-example}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  Logistic regression model.
\item
  1\% Error on test set.
\end{itemize}

Problem: 0.5\% of patients have cancer. An algorithm that simply says
that y = 0 (no cancer) has a 0.5\% of error!!

\textbf{Skewed classes} = when we have a lot more data from a class than
the other one.

\subsection{Different error metrics}\label{different-error-metrics}

In this cases, we need to use different error metrics (confusion
matrix):

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  Precision: Of all patients where we predicted y = 1, what fraction
  actually has cancer? Equation \ref{eq:precision}.
\item
  Recall: Of all patients that actually have cancer, waht fraction did
  we correctly detect as having cancer? Equation \ref{eq:recall}.
\end{itemize}

\begin{equation} \label{eq:precision}
\text{precision} = \frac{\text{True positives}}{\text{\# predicted positives}}
\end{equation}

\begin{equation} \label{eq:recall}
\text{recall} = \frac{\text{True positives}}{\text{\# actual positives}}
\end{equation}

So, in the case of the model that simply says y = 0, we could use
recall. The result would be a recall of zero, which we will then try to
improve.

\textbf{y = 1} in presence of the most \textbf{rare class}.

\section{Trading Off Precision and
Recall}\label{trading-off-precision-and-recall}

In the example of the cancer algorithm, we want to be very confident as
it's very critical to fail.

Suppose we want to predict y = 1 only if very confident. If we're using
logistic regression, we can simply cut the threshold in which we decide
whether someone has cancer at a higher \%.

Doing that, we get higher precision and lower recall. \bigskip

On the other hand, if we want to avoid mising cases of cancer (avoid
false negatives), we can just cut the threshold lower so more people are
classified as having cancer.

We will have lower precision and higher recall. \bigskip

We can see this in figure \ref{fig:precision_recall_threshold}.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{img/precision_recall_threshold.eps}
\caption{Precision and recall depending on the threshold we cut at.}
\label{fig:precision_recall_threshold}
\end{figure}

\subsection{F score}\label{f-score}

A way to compare precision and recall, as given in equation
\ref{eq:f_score}.

\begin{equation} \label{eq:f_score}
\text{f score} = 2 \frac{PR}{P+R}
\end{equation}

\section{Data For Machine Learning}\label{data-for-machine-learning}

\begin{quote}
It's not the best algorithm that wins, but the one with more data.
\end{quote}

We must make sure the features we have have sufficient info to predict y
accurately, though. A good way to tell that is thinking about whether a
human expert could predict y with the data we have.

\subsection{Large data rationale}\label{large-data-rationale}

Pick an algorithm with many parameters so that it can learn difficult
functions.

Also use a large training set (unlikely to overfit).

Doing this we will have a small train error and the train error will be
more or less equal to the test error. That means that the train error
will be small. That is assuming we have a loooot of data.

%% Body end.

%% Bibliography.

    \nocite{*}

\bibliographystyle{plain}
\bibliography{references}

\end{document}

theory/06.02 - Machine Learning System Design/06.02 - Machine Learning System Design.tex


%!TEX root = main.tex
\chapter{Physical Modeling}
\label{Karplus}

\begin{figure}[h]
  \begin{center}
    \includegraphics[width = 14cm]{img/strohgeige.jpg}
    \caption{Die Strohgeige, ein eigentuemliches Saiteninstrument}
    \label{fig:metering}
  \end{center}
\end{figure}

\section{References}

\begin{itemize}
  \item \cite{bilbao_numerical_2009}
  \item \cite{smith_physical_2010}
  \item \cite{cook_real_2002}
\end{itemize}

% Karplus strong und PM,\\
% schwingungssysteme,\\
% simulation..

% Ableton Live: donner,\\
% rechtfertigung: Computerspiele, etc.\\
% (Wellen-)Experiment.\\
% Geschichte: Bernoulli und D'Alembert\\
% Wellengleichung
% Karplus Strong (bauen, geschichte erzählen)
% (Pause)
% Waveguide, Scattering Junctions erwähnen undje nach interesse behandeln. Kelly und Lochbaum modell.
% Kaffeehäferl synthetisieren. (Praat, reson)

\section{Further Information}

\begin{itemize}
  \item \link{https://www.youtube.com/watch?v=Li6OEMCtQ9k}{Wave Equation}
  \item \link{https://www.youtube.com/watch?v=dUcNzPhZdwk}{Julius O. Smith about Physical Modeling in Stanford}
  \item \link{http://www.fon.hum.uva.nl/praat/}{Praat audio analysis}
\end{itemize}

\section{Overview}

\begin{enumerate}
	\item motivation: games, and efficient realistic sound generation
	\item PM: erklärung
	\item Karplus-Strong
	\item Schulübung: sounddesign challenge in pd.

\end{enumerate}

\section{Theory: Oscillating Systems}

Wave equation, infinite one-dimensional string:
\begin{equation}
	c^2 \cdot \frac{\delta^2 f}{\delta x^2} = \frac{\delta ^2 f}{\delta t ^2}
\end{equation}

Visualization:

\begin{figure}[H]
  \begin{center}
    \includegraphics[width = 14cm]{img/wellengleichungVis.png}
    \caption{Visualisierung der Wellengleichung}
    \label{fig:wellengleichung}
  \end{center}
\end{figure}

Two possible solutions for the differential equation:
\begin{itemize}
	\item Bernoulli = ca. fourier, spectral, summe von sinus schwingungen.
	\item D'Alembert = ca. Physical modeling, zwei funktionen die entlang der saite reisen (in entgegengesetzte richtung).
\end{itemize}

Bernoulli:
\begin{equation}
  y(t,x) = \sum^{\infty}_{k=0} A_k sin(k\pi x/L)cos(k\pi \nu t)
\end{equation}
(Displacement length $L$, vibrating string at time $t$ at position $x$, frequency $\nu$)

\begin{equation}
y(x, t) = y ^+ \Bigg(t - \frac{x}{c}\Bigg) + y^- \Bigg(t + \frac{x}{c} \Bigg)
\end{equation}

\section{Karplus Strong Synthesis}
% D'Alembert sagt also, es gäbe zwei wellen, die in zwei richtungen entlang der saite reisen, die Lässt sich einfach simulieren, mittels zwei in einander geschalteten delays.
So D'Alembert says, there are two waves, one traveling to the right and one traveling to the left. Their superposition is what makes up the actual wave along a string. \\
Observing a string more closely, we find that a wave gets inverted at its borders and travels back.
All this can easily be simulated in a computer, see Figure~\ref{fig:string}.

  \begin{figure}[htb]
  \centering
  \label{fig:string}

  % \resizebox{10cm}{!}{%
  \begin{tikzpicture}[auto, thick, node distance=2.3cm, >=triangle 45]

  \draw node at (5,0) [block] (del1) {\Large$\ \ \ \ \ \ \ \ z^{-m}\ \ \ \ \ \ \ \ $};
  \draw node at (5,-3) [block] (del2) {\Large$\ \ \ \ \ \ \ \ z^{-m}\ \ \ \ \ \ \ \ $};
  \draw node at (0, -1.5) [mult] (m1) {\Large$-1$};
  \draw node at (10, -1.5) [mult] (m2) {\Large$-1$};

  \draw[->] (del1) -| node {}(m2);
  \draw[->] (m2) |- node {}(del2);
  \draw[->] (del2) -| node {}(m1);
  \draw[->] (m1) |- node {}(del1);

  \end{tikzpicture}
  \caption{general idea of Waveguide synthesis/Karplus-Strong Model.}
\end{figure}

The model in Figure~\ref{fig:string} is a rather theoretical one\footnote{You might notice that there are no inputs and no outputs. This structure is theoretical for us, since we do not talk about something called the \textit{Wave Digital Filter}(WDF). When talking about WDFs we find similar structures since they are strongly inspired by physics. Since in physics we always have a force and another force counteracting the first one we always have left and right traveling energy.}. But it leads us to a more useful implementation. Since all elements in Figure~\ref{fig:string} are LTI, we can rearrange them in any order. Multiplying a signal times $-1$ two times is the same as doing nothing. And two delays of length $m$ are equivalent to one delay of length $2m$.\\
So we arrive at a simple delay with feedback. To simulate energy loss we need to attenuate the feedback signal and to simulate frequency dependent loss, we can use a lowpass filter inside the feedback loop. After all this, we arrive at Figure~\ref{fig:string2}.

\begin{figure}[h!]
  \centering

  \label{fig:string2}

  \begin{tikzpicture}[auto, thick, node distance=2.3cm, >=triangle 45]

  \draw node at (0,0) [input] (excitation) {};
  \draw node [sum, right of=excitation, node distance = 2.cm] (sum1) {\Large$+$};
  \draw node [block, right of=sum1, node distance = 2cm] (delay) {\Large$z^{-m}$};
  \draw node [block, below of=delay, node distance = 2cm] (loss) {\Large$G(z)$};
  \draw node at (6.,-0.001) {\textbullet};

  \draw node [output, right of=delay,node distance = 3cm] (out) {};
  \draw[->] (excitation) -- node {\Large$x[n]$}(sum1);
  \draw[->] (sum1) -- node {}(delay);
  \draw[->] (loss.west) -| node {}(sum1.south);

  \draw[->] (6., -0.001) |- node {}(loss.east);

  \draw[->] (delay) -- node {\Large$y[n]$}(out);

  \end{tikzpicture}
  \caption{single string, $G(z)$ a lowpass, \glqq{}Loss Filter\grqq{}}
\end{figure}

\section{Practical Karplus-Strong}
\begin{figure}[H]
	\begin{center}
		\includegraphics[scale = 1.]{img/karplusSimple.png}
		\caption{Basic setup, a delay with feedback.}
		\label{fig:karplusBasic}
	\end{center}
\end{figure}

\begin{figure}[H]
  \begin{center}
    \includegraphics[scale = 0.3]{img/karplusComplete.png}
    \caption{Complete Algorithm, mind the lowpass filter in the feedback loop}
    \label{fig:karplusComplete}
  \end{center}
\end{figure}

\section{Modal Synthesis with Bandpass filters}

\section{Schulübung, Sounddesign challenge}

zB:
\begin{itemize}
	\item Vokal
	\item E-gitarre
	\item beliebiges Physikalisches object

\end{itemize}

% \section{Key Points}
% \begin{itemize}
%   \item Make sure you understood how the Karplus-Strong algorithm works and know its structure.
%   \item Make sure you understand what the elements of the karplus strong algorithm might mean (physical analogies).
% \end{itemize}


\documentclass[]{article}

%%%%%%%%%%%%%%%%%%%
% Packages/Macros %
%%%%%%%%%%%%%%%%%%%
\usepackage{amssymb,latexsym,amsmath}     % Standard packages

%%%%%%%%%%%
% Margins %
%%%%%%%%%%%
\addtolength{\textwidth}{1.0in}
\addtolength{\textheight}{1.00in}
\addtolength{\evensidemargin}{-0.75in}
\addtolength{\oddsidemargin}{-0.75in}
\addtolength{\topmargin}{-.50in}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Theorem/Proof Environments %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newtheorem{theorem}{Theorem}
\newenvironment{proof}{\noindent{\bf Proof:}}{$\hfill \Box$ \vspace{10pt}}

%%%%%%%%%%%%
% Document %
%%%%%%%%%%%%
\begin{document}

\title{Sample \LaTeX ~File}
\author{David P. Little}
\maketitle

\begin{abstract}
This document represents the output from the file ``sample.tex" once compiled using your favorite \LaTeX compiler.  This file should serve as a good example of the basic structure of a ``.tex" file as well as many of the most basic commands needed for typesetting documents involving mathematical symbols and expressions.  For more of a description on how each command works, please consult the links found on our course webpage.
\end{abstract}

\section{Lists}
%%%%%%%%%%%%%%%
\begin{enumerate}
\item {\bf First Point (Bold Face)}
\item {\em Second Point (Italic)}
\item {\Large Third Point (Large Font)}
    \begin{enumerate}
        \item {\small First Subpoint (Small Font)}
        \item {\tiny Second Subpoint (Tiny Font)}
        \item {\Huge Third Subpoint (Huge Font)}
    \end{enumerate}
\item[$\bullet$] {\sf Bullet Point (Sans Serif)}
\item[$\circ$] {\sc Circle Point (Small Caps)}
\end{enumerate}

\section{Equations}
%%%%%%%%%%%%%%%%%%%

\subsection{Binomial Theorem}
\begin{theorem}[Binomial Theorem]
For any nonnegative integer $n$, we have
$$(1+x)^n = \sum_{i=0}^n {n \choose i} x^i$$
\end{theorem}

\subsection{Taylor Series}
The Taylor series expansion for the function $e^x$ is given by
\begin{equation}
e^x = 1 + x + \frac{x^2}{2} + \frac{x^3}{6} + \cdots = \sum_{n\geq 0} \frac{x^n}{n!}
\end{equation}

\subsection{Sets}

\begin{theorem}
For any sets $A$, $B$ and $C$, we have
$$ (A\cup B)-(C-A) = A \cup (B-C)$$
\end{theorem}

\begin{proof}
\begin{eqnarray*}
(A\cup B)-(C-A) &=& (A\cup B) \cap (C-A)^c\\
&=& (A\cup B) \cap (C \cap A^c)^c \\
&=& (A\cup B) \cap (C^c \cup A) \\
&=& A \cup (B\cap C^c) \\
&=& A \cup (B-C)
\end{eqnarray*}
\end{proof}

\section{Tables}
%%%%%%%%%%%%%%%%
\begin{center}
\begin{tabular}{l||c|r}
left justified & center & right justified \\ \hline
1 & 3.14159 & 5 \\
2.4678 & 3 &  1234 \\ \hline \hline
3.4678 & 6.14159 & 1239
\end{tabular}
\end{center}

\section{A Picture}
%%%%%%%%%%%%%%%%%%%
\begin{center}
\begin{picture}(100,100)(0,0)
\setlength{\unitlength}{1pt}
\put(20,70){\circle{30}}  \put(20,70){\circle*{10}}   % left eye
\put(80,70){\circle{30}}  \put(80,70){\circle*{10}}   % right eye
\put(40,40){\line(1,2){10}} \put(60,40){\line(-1,2){10}} \put(40,40){\line(1,0){20}} % nose
\put(50,20){\oval(80,10)[b]} % mouth
\multiput(0,90)(4,0){10}{\line(1,3){4}}  % left eyebrow
\multiput(100,90)(-4,0){10}{\line(-1,3){4}}  % right eyebrow
\end{picture}
\end{center}

\end{document}


\documentclass[conference]{IEEEtran}
\usepackage{cite}
\usepackage{graphicx}
% \usepackage{amsmath} %\usepackage{amssymb}
\usepackage{amsmath,amssymb,amsthm}
%\usepackage{algorithmic}
\usepackage{fixltx2e}
\usepackage{stfloats}
\usepackage{url}
\usepackage{hyperref}
\usepackage{xcolor}

\usepackage[justification=centering,font=small]{caption}
\usepackage{movie15}
\usepackage[normalem]{ulem}
\usepackage{bm}
\useunder{\uline}{\ul}{}
\newcommand{\norm}[1]{\left\lVert#1\right\rVert}
% for theorems &Co
\usepackage[utf8]{inputenc}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{corollary}{Corollary}[theorem]
\newtheorem{lemma}[theorem]{Lemma}

\begin{document}
%
% paper title
% Titles are generally capitalized except for words such as a, an, and, as,
% at, but, by, for, in, nor, of, on, or, the, to and up, which are usually
% not capitalized unless they are the first or last word of the title.
% Linebreaks \\ can be used within to get better formatting as desired.
% Do not put math or special symbols in the title.
\title{Darcy flow:\\ Pollution of a drinking water reservoir}

% author names and affiliations
% use a multiple column layout for up to three different
% affiliations
\author{
\IEEEauthorblockN{Zampieri Luca}
\IEEEauthorblockA{Computational Science \& Engineering\\
\href{mailto:luca.zampieri@epfl.ch}{Email: luca.zampieri@epfl.ch}}
}

\maketitle
\vspace{-10cm}

\begin{abstract}
This paper reports the solutions of a problem of drinking water pollution using Finite Elements methods. In particular, since it involves the Darcy Equations, we will explore the use of the Raviart Thomas elements. The corresponding code implemented in FreeFem++ as well as the source of this report can be found on github: \url{https://github.com/LucaZampieri/Darcy_Flow}. This project is part of the course MA-468: Numerical Methods for Saddle Point Problems given by Prof. Annalisa Buffa at EPFL.

\end{abstract}
\IEEEpeerreviewmaketitle
\section{Notations}
We will use standard notation, complemented by the following (even if some are quite informal):
\begin{itemize}
\item $\norm{\cdot}_0 = \norm{\cdot}_{[L^2(\Omega)]^d}$, $\forall d$ number of dimensions
\item $\Gamma_{ij} = \Gamma_i \cup \Gamma_j$    with $ i,j < 10$
\item $H^1_0(\Omega)= \{ x \in H^1(\Omega) : x{\restriction}_{\partial \Omega}=0 \}$
\item $H^1_{0,\Gamma}(\Omega)= \{ x \in H^1(\Omega) : x{\restriction}_{\Gamma}=0 \}$
\item $H(div,\Omega) = \{ x \in H^1(\Omega) : div(x) = 0\}$
\item $H_{0}(div,\Omega) = \{ x \in H(div,\Omega) : x\cdot n{\restriction}_{\partial \Omega} = 0\}$
\item $H_{0,\Gamma}(div,\Omega) = \{ x \in H(div,\Omega) : x\cdot n{\restriction}_\Gamma = 0\}$
\item $\cdot^{CS}$ = use of Cauchy–Schwarz inequality
\item $\cdot^{HD}$ = use of the Hodler inequality
\item $\cdot^{SB}$ = use of the Sobolev inequality
\end{itemize}
if the domain of the space is omitted, it is assumed to be $\Omega$.

\section{Problem (question 1)}
For the complete statement of the problem, refer to the sheet given by the assistants. Here we make a quick reminder of the problem.\\
Consider a drinking water reservoir with fluid flowing left to right of the domain $\Omega$. There is a pollution accident at $C_2$ and a pump is placed at the center of the domain in $C_1$. We are interested in knowing how much pollutant has been extracted by the pump. \\
Considering $\Omega = [0,1]^2$ with border $\Gamma_i , i=1:4$ denoted in anti-clockwise sense, $C_1 = [0.48,0.52]^2 \in \Omega$ and $C_2=[0.28,0.32]^2 \in \Omega$. We then have to solve the equations:

\begin{equation}
    \begin{cases}
      -k \nabla p = \textbf{u} , & \text{in $\Omega$}  \\
      \nabla \cdot \textbf{u}, & \text{in $\Omega$} \\
      p=p_{in} , & \text{on $\Gamma_4$} \\
      p=p_{out} , & \text{on $\Gamma_2$} \\
      \textbf{u} \cdot \textbf{n} = 0, & \text{on } \Gamma_1 \cup \Gamma_3
    \end{cases}
    \label{eq:darcy}
\end{equation}
\begin{equation}
    \begin{cases}
      -\nu \Delta c + \textbf{u} \cdot \nabla c = g ,& \text{in $\Omega$}  \\
      c = 0, & \text{on } \Gamma_1 \cup \Gamma_3 \cup \Gamma_4 \\
      \nu \nabla c \cdot \textbf{n} = 0, & \text{on } \Gamma_2
    \end{cases}
    \label{eq:advection}
\end{equation}
Where (\ref{eq:darcy}) is a darcy equation, (\ref{eq:advection}) is an advection-diffusion equation and $f,g$ are defined as:
\begin{equation}
	f(\textbf{x})=
    \begin{cases}
      -1000,   & \text{if } \textbf{x} \in C_1 \\
      0,       & \text{if } \textbf{x} \notin C_1 \\
    \end{cases}
\end{equation}
and
\begin{equation}
	g(\textbf{x})=
    \begin{cases}
      1000, & \text{if } \textbf{x} \in C_2 \\
      0,    & \text{if } \textbf{x} \notin C_2 \\
    \end{cases}
\end{equation}
as such $f,g \in L^{\infty}(\Omega)$.\\
We fix $k=I$, $p_{in} = 2$ and $p_{out} = 0$. We are then interested in the quantity $\phi(c) = \int_{C_1} c$.

%%%%%%%%%%%%%%%%%%%%%%%%%%%% Weak Form %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Weak forms (question 1)}
We want to prove the well-posedness of the system of equations. For this we rewrite the sytem of equations (\ref{eq:darcy},\ref{eq:advection} ) in the following form:\\
Find $\textbf{u} \in V$, $q \in Q$, $c \in W$ such that:
\begin{equation}
    \begin{cases}
      a(u,v)+b(p,v) = F_a(v) , &\forall \textbf{v} \in V \\
      b(u,q) = F_b(q) ,        &\forall q \in Q \\
      a_{ad}(c,w) = G(w) ,     &\forall w \in W
    \end{cases}
    \label{eq:weak}
\end{equation}

where $a,b,a_{ad}$ are bilinear forms and $F_a, F_b, G$ are linear operators.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%   Darcy Weak   %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Darcy equation}
For the Darcy equation we seek a mixed-form weak formulation. We set the spaces
$V=H(div,\Omega)$ and $Q=L^2(\Omega)$. Multiplying the first equation of (\ref{eq:darcy}) by a test function $v\in V$, the second equation by a test function $q\in Q$ and integrating we get:
\begin{equation}
    \begin{cases}
      \int_\Omega-k^{-1}\textbf{u}\cdot \textbf{v}
      -\int_\Omega \nabla p \cdot \textbf{v} = 0 , & \forall \textbf{v} \in V  \\
      \int_\Omega (\nabla \cdot \textbf{u})q = \int_\Omega fq, & \forall q \in Q \\
    \end{cases}
    \label{eq:darcy_tmp}
\end{equation}
Integrating by parts the second term of the first equation of (\ref{eq:darcy_tmp}) and applying boundary conditions we get the mixed-form weak formulation:\\
Find \textbf{u} $\in V$, q $\in Q$ such that  :
\begin{equation}
    \begin{cases}
      \int_\Omega -k^{-1}\textbf{u}\cdot \textbf{v} + \int_\Omega  p (\nabla\cdot \textbf{v})
      = \int_{\partial\Omega}  \tilde{p} (\textbf{n}\cdot \textbf{v})  , & \forall \textbf{v} \in V  \\
      \int_\Omega (\nabla \cdot \textbf{u})q = \int_\Omega fq, & \forall q \in Q
    \end{cases}
    \label{eq:darcy_weak}
\end{equation}
with $\tilde{p}$ is such that $\tilde{p}{\restriction}_{\Gamma_{13}} = 0 $, $\tilde{p}{\restriction}_{\Gamma_2} = p_{out} $ and $\tilde{p}{\restriction}_{\Gamma_4} = p_{in} $
Which is the form we desired (\ref{eq:weak}) since $a,b$ are clearly bilinear and $F_a,F_b$ are trivialy linear. \\
From the theory on mixed-forms, to prove well-posedness we additionally need:
\begin{itemize}
\item Coercivity on the Kernel i.e. $a(u_0,u_0) \geq \alpha \norm{u_0}_{H(div,\Omega)}^2$ , $\forall u_0 \in Ker(B)$ where $Ker(B)=\{v \in H_{0,\Gamma_{13}} : div(v)=0 \}$
\item Inf-Sup conditions i.e: $ inf_{q\in Q} sup_{v\in V} \frac{ b(v,q) }{ \norm{v}_V \norm{q}_Q } \geq \beta >0   $
\end{itemize}
We then proceed to prove these two properties:
%\subsubsection{Coercivity on the kernel}
\begin{proof}[proof: Coercivity on the kernel]
We have:
$$a(u_0,u_0)=\int k^{-1} |u_0|^2 \geq k_{min} \norm{u_0}^2_{L^2(\Omega)} = \norm{u_0}^2_{H(div,\Omega)}$$
where $k_{min}$ is a minimal eigenvalue of $k$ and where we used both the facts that $k=I$ and that $u_0 \in Ker(B)$ thus $\norm{div(v)}_{L^2(\Omega)}=0$, which concludes the proof.
\end{proof}

%\subsubsection{Inf-Sup Condition}
\begin{proof}[Proof of Inf-Sup condition]
Consider the separate problem on the same domain:\\
Let $f \in L^2(\Omega)$:
\begin{equation}
	\begin{cases}
		-\Delta z = f_L ,  \text{  in } \Omega \\
		\partial_n z = 0 , \text{  on } \Gamma_{13}
	\end{cases}
	\label{eq:laplacian}
\end{equation}
which finds solution $z \in H^1(\Omega)$ with $\norm{z}_{H^1(\Omega)} \leq C_L \norm{f_L}_0 $.\\
If we now consider $\nabla z \in L^2(\Omega)^2$ of the previous problem, we have $\nabla z \in H_{0,\Gamma_{13}}(div,\Omega)$ and from the solution of the previous problem we get:
\begin{equation}
	\norm{\nabla z}_{H_{0,\Gamma_{13}}(div)} \leq C_L \norm{f_L}_0
	\label{eq:laplacianContinuity}
\end{equation}
and shows that $B=div: H_{0,\Gamma_{13}}(\Omega) \longrightarrow L^2(\Omega)$ is surjective. \\
However, by the theorem of the equivalence of the Inf-Sup condition (\ref{th:equivalence}), this implies that the inf-sup condition holds.
\end{proof}
In parallel we could have done as below (\ref{proof:InfSupBis}).

\begin{proof}[Proof of Inf-Sup condition BIS]
\label{proof:InfSupBis}
We recall that $V=H_{0,\Gamma_{13}}(\Omega)$, $Q=L^2(\Omega)$. By integrating (\ref{eq:laplacian}) and multiplying by $q\in Q$
\begin{equation}
	\int_{\Omega} q \cdot div(\nabla z) \geq \norm{f_L}_0 \norm{q}_Q \geq
	\frac{1}{C_L} \norm{\nabla z}_V \norm{q}_Q
\end{equation}
Since $\nabla z, u \in V$ we can set $u = \nabla z$. Taking the supremum of the LHS we get:
\begin{equation}
	sup_{u\in V} \int_{\Omega} q \cdot div(u) \geq \norm{f_L}_0 \norm{q}_Q \geq
	\frac{1}{C_L} \norm{u}_V \norm{q}_Q
\end{equation}
which can be rewritten as:
\begin{equation}
	sup_{u\in V} \frac{ \int_{\Omega} q \cdot div(u)}{\norm{u}_V} \geq \frac{1}{C_L} \norm{q}_Q
\end{equation}
which is equivalent to the Inf-Sup condition.
\end{proof}

We can finally conclude on the well-posedness of equation (\ref{eq:darcy_weak}) by the main theorem of the course.

%%%%%%%%%%%%%%%%%%%%% advec diffusion  %%%%%%%%%%%%%%%%%%%5
\subsection{Advection Diffusion}
Due to the previous subsection, we have $\textbf{u} \in V$. Set $W= H^1_{0,\Gamma_{134}} $. Multiplying the equation in (\ref{eq:advection}) by a test function $w \in W$, integrating and using integration by part we can reach the weak formulation:\\
Find $c \in W$ such that:
\begin{equation}
      a_{ad}(c,w) = \int_\Omega \nu \nabla c \nabla w
      + \int_\Omega (\textbf{u} \cdot \nabla c) w
      = \int_\omega gw = G(w),  \forall w \in W  \\
    \label{eq:advection_weak}
\end{equation}
where we used the fact that $\int_{\partial\Omega} \nu (\nabla c \cdot \textbf{n}) w = 0$ since on $\Gamma_2$ we have homogeneous Neumann conditions and on the rest of $\partial\Omega$ we have $w=0$ since it belongs to $W$. We note that $a_{ad}(c,w)$ is a bilinear form and $G(w)$ a linear operator.\\
We want to conclude using the Lax-Milgram Lemma thus we desire:
\begin{itemize}
\item Continuity of G(w) which is implicit since $g\in L^{\infty}(\Omega)$
\item Continuity of the bilinear form $a_{ad}(c,w)$
\item Coercivity of the bilinear form $a_{ad}(c,w)$
\end{itemize}
Note that thanks to the Pointcare inequality (\ref{th:pointcare}), the norms $\norm{\cdot}_{H^1}$ and $\norm{\cdot}_{H^1_0}=|\cdot|_{H^1}=\norm{\nabla \cdot}_0$ are equivalent meaning there exist $m,M >0 $ s.t. $m\norm{\cdot}_{H^1} \leq \norm{\cdot}_{H^1_0} \leq M\norm{\cdot}_{H^1}$. \\
We can proceed:
\subsubsection{Continuity of the bilinear form} Rewriting
\begin{align}
	  |a_{ad}(c,w)|
      & \leq^{CS} \nu \norm{\nabla c}_0 \norm{\nabla w}_0 + | \textbf{u} \cdot \nabla c w | \\
      & \leq^{HD} \nu \norm{\nabla c}_0 \norm{\nabla w}_0 + \norm{\textbf{u}}_{L^4(\Omega)} \norm{\nabla c}_0 \norm{w}_{L^4(\Omega)} \\
      & \leq^{SB} (\nu + \norm{\textbf{u}}_{L^4(\Omega)}) \norm{c}_{H^1} \norm{w}_{H^1}
\end{align}
and since $\norm{\cdot}_{H^1}$ and $\norm{\cdot}_{H^1_0}$ are equivalent norms, we have continuity.

\subsubsection{Coercivity of the bilinear form}
\begin{equation}
      a_{ad}(c,c) = \nu \norm{\nabla c}_0^2 + \int_{\Omega} (\textbf{u}\cdot \nabla c) c
\end{equation}
To handle the last term of the equation, we need an assumption. The weakest we found is $\textbf{u} \cdot \textbf{n} \geq 0$. We can rewrite the last term as:
\begin{align}
      \int_{\Omega} (\textbf{u}\cdot \nabla c) c
      & = \frac{1}{2} \int_{\Omega} \textbf{u}\cdot \nabla (c^2) \\
      & = - \int_{\Omega} (\nabla \cdot \textbf{u}) c^2 + \int_{\partial \Omega} (\textbf{u} \cdot \textbf{n}) c^2 \\
\end{align}
The first term vanishes due to $u\in V$ which is divergence free, and the second term vanishes on $\Gamma_{134}$ since it has diriclet boundary conditions.
Finally, using $\textbf{u} \cdot \textbf{n} \geq 0$:
\begin{equation}
      a_{ad}(c,c) \geq \nu \norm{\nabla c}_0^2 + \int_{\Gamma_2} (\textbf{u}\cdot \textbf{n}) c^2
      \geq \nu \norm{\nabla c}_0^2
\end{equation}
Which proves coercivity of $a_{ad}(\cdot,\cdot)$.\\

Using Lax-Milgram Lemma (\ref{th:laxMilgram}) we conclude on the well-posedness of the advection-diffusion equation (\ref{eq:advection_weak}) thus concluding on the well-posedness of the system of equations.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%  Numerical discretization %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Numerical Discretization (Question 2)}
We should now find a suitable numerical discretization for the weak formulation of both the mixed-form of Darcy problem and the diffusion-advection equation for the pollutant agent, i.e. we want to find the spaces $V_h \subseteq V$, $Q_h \subseteq Q$ and  $W_h \subseteq W$ such that the discrete problem is well-posed. Finally, we will state the expected rate of convergence with these elements. \\

We will use results of \cite{Perotto}. To begin with let's discretize the domain. We consider $\tau_h$ as the triangulation og $\Omega$ (i.e. the cover with non-overlapping triangles of $\Omega$). We then define the discretized domain as:
\begin{equation}
	\Omega_h = int \bigg(\bigcup_{K\in \tau_h} K \bigg)
\end{equation}
and since $\Omega$ is a polygon the internal part of the union of the triangles of $\tau_h$ perfectly coincides with $\Omega$.\\

We define the space of polinomials of degree $r$ on $d$ dimensions
$$\mathbb{P}_r= \{p(\textbf{x}) = \sum\limits_{i_1+...+i_d \leq d} a_{i_1...i_d}\textbf{x}_1^{i_1}...\textbf{x}_d^{i_d}, a_{i_1...i_d} \in  \mathbb{R}\}$$
with $0 \leq i_1,...,i_d $.\\
Let us also define $\tilde{\mathbb{P}}_r$ as the space of polinomials of exact degree $r$:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%  Discrete Darcy %%%%%%%%%%%%%%%%%%%%%%%
\subsection{Discrete Darcy}
Lets begin with the Darcy equation. As seen during the lecture, a suitable family of finite elements space would be the Raviart-Thomas elements:
\begin{equation}
	\mathbb{RT}_r (K) = \mathbb{P}_r^d(K) \oplus \textbf{x}\cdot \tilde{\mathbb{P}}_r(K)
\end{equation}
where $ \tilde{\mathbb{P}}_r(K) = \mathbb{P}_r(K) \backslash \ \mathbb{P}_{r-1}(K) $. They are in fact (to some extension) the only appropiate elements to solve the Darcy Flow.\\
Setting $V_h=\{ \textbf{v} \in V : v|_k = \mathbb{RT}_r(K) , \forall K \in \tau_h \}$ and $Q_h= \{ q \in Q : d|_k \in  \mathbb{P}_r, \forall K \in \tau_h\}$. Note that $div(V_h) \subseteq Q_h$, which we need for consistency.\\
The discrete problem writes\\
Find $u_h \in V_h$, $p_h \in Q_h$ s.t.:
\begin{equation}
	\begin{cases}
		\int_{\Omega_h} k^{-1} u_h \cdot v_h + \int_{\Omega_h} div(v_h) p_h = ... ,&\forall v_h \in V_h \\
		\int_{\Omega_h} div(u_h) q_h = \int_{\Omega_h} f\ q_h, &\forall q_h \in Q_h
	\end{cases}
	\label{eq:DDarcy}
\end{equation}
and again, to prove well-posedness we need the two properties:
\begin{itemize}
\item $a(u_h,u_h)=\int_{\Omega_h} k^{-1} |u_h|^2 \geq \alpha \norm{u_h}^2_{V_h} , \forall u_h \in Ker(B_h)$
\item $sup_{u_h \in V_h} \frac{\int_{\Omega_h} div(u_h) p_h}{\norm{u_h}_{V_h}} \geq \beta \norm{p_h}_{Q_h}$
\end{itemize}
where $\norm{\cdot}_{V_h} = \norm{\cdot}_{V} $, $\norm{\cdot}_{Q_h} = \norm{\cdot}_{Q} $ and $Ker(B_h)=\{u_h \in V_h : \int_{\Omega_h} div(u_h) q_h = 0 \forall q_h \in Q_h \}$.\\
\subsubsection{Discrete coercivity on the kernel}
\begin{proof}
Take $q_h = div(u_h)$ (possible since $div(V_h)=Q_h$):
$$ \int_{\Omega_h} |div(u_h)|^2 = 0 \implies div(u_h) = 0 $$
and $Ker(B_h)\subset Ker(B)$.
Which implies $, \forall u_h \in Ker(B_h)$:
\begin{equation}
 a(u_h,u_h)=\int_{\tau_h} k^{-1} |u_h|^2 \geq \alpha (\norm{u_h}^2_0 + \norm{div(u_h)}^2_0)
\end{equation}
since here $\norm{u_h}^2_0 = \norm{u_h}^2_0 + \norm{div(u_h)}^2_0 = \norm{u_h}^2_{V_h}$ as $\norm{div(u_h)}^2_0 = 0$
\end{proof}
\subsubsection{Discrete Inf-Sup conditions}
\begin{proof}
From the continuous Inf-Sup condition, with $p_h \in L^2(\Omega)$:
\begin{equation*}
	\exists u \in H(div,\Omega) : \frac{ \int_{\Omega} div(u)\cdot p_h}{\norm{u}_V} \geq  \norm{p_h}_0
\end{equation*}
by the definition of the $L^2$ projection $\Pi^s$ we have
\begin{align*}
	\int_{\Omega_h} div(u) p_h &= \int_{\Omega_h}\Pi^s(div(u))p_h \\
	&=\int_{\Omega_h} div(I_{RT} u) p_h \\
	&=\int_{\Omega_h} div(u_h) p_h
\end{align*}
and by the continuity of $I_{RT}$
\begin{equation*}
	\norm{I_{RT}u}_{H(div,\Omega)} \leq C_I \cdot \norm{u}_{H(div,\Omega)}
\end{equation*}
we can then write:
\begin{align*}
	 \frac{ \int_{\Omega_h} div(u_h)\cdot p_h}{\norm{u_h}_{H(div,\Omega)}}
	 &= \frac{ \int_{\Omega_h} div(u)\cdot p_h}{\norm{u_h}_{H(div,\Omega)}}
	 & \geq \frac{1}{C_I} \frac{ \int_{\Omega_h} div(u)\cdot p_h}{\norm{u}_{H(div,\Omega)}}
	 & \geq \frac{\beta}{C_I} \norm{p_h}_0
\end{align*}
proving that the Inf-Sup condition holds
\end{proof}

Thus the Discrete Darcy problem (\ref{eq:DDarcy}) is well-posed and verifies optimal error estimates in the energy norms:
\begin{align*}
	\norm{u-u_h}_V + \norm{p-p_h}_Q &\leq \\
	 C_{I} (inf_{u_h \in V_h}&\norm{u-vh}_V + inf_{q_h \in Q_h}\norm{p-p_h}_Q)
\end{align*}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%  Discrete Advec-diff %%%%%%%%%%%%%%%%%%%%%%%
\subsection{Discrete Advection-Diffusion}
Setting $W_h=$, the discrete problem writes\\
Find $c_h \in W_h$ s.t.:
\begin{equation}
	  a_{ad}(c_h,w_h) = \int_{\Omega_h} \nu \nabla c_h \nabla w_h
      + \int_{\Omega_h} (u_h \cdot \nabla c_h) w_h
      = \int_{\Omega_h} g_h w_h = G(w_h)
\end{equation}
$\forall w_h \in W_h$.\\
Repeating the steps done for the continuous case, we use the Lax-Milgram Lemma to prove existence and uniqueness of the solution. Moreover, Cea's lemma gives us the error estimate:\\
$$ \norm{c-c_h}_W \leq \frac{M}{\alpha} inf_{w_h \in W_h} \norm{c-w_h}_W $$
where $M,\alpha$ are respectively the continuity and the coercivity contants of the bilinear form.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Rates of convergence %%%%%%%%%%%%%%%%%%%%%%%
\subsection{Rates of convergence}
Since we may need them for the convergence analysis, let's make few considerations on the grid. Setting $h_k=diam(K)$ where $diam(K)=max_{x,y\in K}|x-y|$, we will characterize the mesh size with $h=max_{K\in \tau_h} h_k$. For simplicity, in the following we may require that the grid satisfy a regularity condition:
\begin{equation}
	\frac{h_k}{\rho_k} \leq \delta , \forall K \in \tau_h
	\label{eq:regularMesh}
\end{equation}
for a suitable $\delta > 0$ and where $\rho_k$ is the sphericity of $K$ (i.e. the diameter of the circle inscribed in the triangle K). Note that regularity (\ref{eq:regularMesh}) usually excludes anisotropic grids. Since we will use anisotropic grids it is important to notice that the results we will present can be generalised to anisotropic grids with further work. Following \cite{Perotto} some of these generalisations can be found in \cite{anisotropic} and \cite{anisotropic2}.\\

By theorem 4.7 of \cite{Perotto} we have:
$$ \norm{c-c_h}_{H^1(\Omega)} \leq \frac{M}{\alpha} C_e h^r |c|_{H^{r+1}(\Omega)} $$
where $C_e$ is a constant independent of h and u, $M,\alpha$ are respectively the continuity and the coercivity constant of the bilinear form, and we have used the regularity of the mesh. \\
And from exercise session 13 we have:
\begin{align*}
	\norm{u-u_h}&_{H(div,\Omega)} + \norm{p-p_h}_{L^2(\Omega)} \leq  \\
	&C_d h^{r+1}
(|u|_{H^{r+1}(\Omega)}+|div(u)|_{H^{r+1}(\Omega)}+|p|_{H^{r+1}(\Omega)})
\end{align*}
where $C_d$ is a constant independent of $h$ and $u$.\\

Assuming that the solutions are regular enough, for RT1,P1,P1 we expect thus to have linear convergence for $c$ and quadratic convergence for $u$ and $p$. In the following section we will investigate if this rates are respected.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% RESULTS %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{figures}

\begin{figure}[!ht]
\centering
%\includegraphics[width=0.5\textwidth]{images/conv1D.jpg}
\caption{lala}
\label{fig:fig7}
\end{figure}

\begin{figure}[!ht]
\centering
\begin{minipage}[t]{4cm}
\centering
%\includegraphics[width=4cm]{images/dilation.png}
\caption{im2}
\label{fig:fig8}
\end{minipage}
\begin{minipage}[t]{4cm}
\centering
%\includegraphics[width=4cm]{images/stride.png}
\caption{im1}
\label{fig:fig9}
\end{minipage}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%  Appendix %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Appendix}\label{sec:appendix}
\subsection{Code}
code:
\subsection{Properties, Lemmas \& Theorems}
Some of these theorems are in the course, the others can be found in \cite{Salsa} and \cite{Perotto}. \\
\begin{theorem}[Sobolev Embeddings\cite{Salsa}]
\label{th:sob}
Let $ u \in H^1(\mathbb{R}^2)$ \\
Then $u \in L^p(\mathbb{R}^2)$ for $2 \leq p < \infty$, and
$$\norm{u}_{L^p(\mathbb{R}^2)} \leq c(p) \norm{u}_{H^1(\mathbb{R}^2)} $$
\end{theorem}
\begin{lemma}[Lax-Milgram]
\label{th:laxMilgram}
Let V be a Hilbert space and $a(.,.)$ a bilinear form. If
$$ a(u,v) \leq M \norm{u}_V \norm{v}_V \qquad \qquad a(u,u)\geq \alpha \norm{u}_V^2$$
then the problem \textbf{Find u $\in$ V: a(u,v)=(f,v) $\forall$ v $\in$ V} admits a unique solution.
\end{lemma}

\begin{lemma}[Pointcare]
\label{th:pointcare}
If one of the following is true:
\begin{itemize}
\item $u \in H^1_{0,\Gamma}(\Omega)$ with $\Gamma \neq \emptyset$ \\
\item $u \in H^1(\Omega) $ and $u=0$ a.e. on some subset $E\subset \Omega $ with $|E|=a>0$ \\
\end{itemize}

then $\exists C_p$ independent of $u$ such that:
\begin{equation}
	\norm{u}_{L^2(\Omega) \leq C_p \norm{\nabla u}_{L^2(\Omega,\mathbb{R}^n)}}
\end{equation}
\end{lemma}

%\begin{theorem}[Existence and uniqueness for mixed form]
%\label{th:mainTheorem}
%
%\end{theorem}

%\begin{theorem}[Equivalence for inf-sup Condition]
%\label{th:equivalence}
%\end{theorem}

\begin{thebibliography}{1}

\bibitem{Salsa}[PDE]
Partial Differential Equations in Action - From Modelling to Theory | Sandro Salsa | Springer.” Accessed June 7, 2018. https://www.springer.com/gp/book/9783319150932.

\bibitem{Perotto}[ANEDP]
“Numerical Models for Differential Problems | Alfio Quarteroni | Springer.” Accessed June 7, 2018. https://www.springer.com/gp/book/9788847058835.

\bibitem{anisotropic}
“Anisotropic Finite Elements: Local Estimates and Applications (Advances in Numerical Mathematics): Thomas Apel: 9783519027447: Amazon.Com: Books.” Accessed June 8, 2018. https://www.amazon.com/Anisotropic-Finite-Elements-Applications-Mathematics/dp/3519027445.

\bibitem{anisotropic2}
Formaggia, L., and S. Perotto. “New Anisotropic a Priori Error Estimates.” Numerische Mathematik 89, no. 4 (October 1, 2001): 641–67. https://doi.org/10.1007/s002110100273.

\end{thebibliography}

\end{document}


\documentclass[journal,onecolumn]{IEEEtran}
%\usepackage[left=2.2cm,right=2.2cm,top=2.5cm,bottom=2.5cm]{geometry}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{minted}
\usepackage{booktabs}
\usepackage{commath}
\usepackage{float}
\usepackage{mathtools}
\usepackage{color}
\usepackage{amsthm}
\usepackage{parskip}

\usepackage[binary-units=true]{siunitx}

\newcommand{\py}[1]{\mintinline{python}{#1}}

\title{Artificial Intelligence (\texttt{LINGI2261}) \\ Assignment 3 --- Group 13}
\author{Martin Braquet, Gilles Peiffer}
\date{November 27, 2019}

\begin{document}

\maketitle

\section{Alpha-Beta Search}
Figure~\ref{fig:minimax} can be obtained by applying the minimax algorithm to the tree given in the statement.
\subsection{MiniMax Algorithm}
\begin{figure}[H]
 \centering
 \includegraphics[width=\textwidth]{img/MiniMax.png}
 \caption{Minimax algorithm}
 \label{fig:minimax}
\end{figure}
One notices that nodes in the ``MAX'' nodes simply take on the highest value among their child nodes, whereas ``MIN'' nodes take on the lowest value among their child nodes.

\subsection{Alpha-Beta Algorithm (Left To Right)}
Using a technique called ``Alpha-Beta pruning'', the number of nodes which have to be explored can be reduced (though it stays exponential in the worst case).
This is shown in Figures~\ref{fig:alphabeta} and \ref{fig:alphabeta_reverse}.
\begin{figure}[H]
 \centering
 \includegraphics[width=\textwidth]{img/Alphabeta.png}
 \caption{Alpha-Beta algorithm (left to right)}
 \label{fig:alphabeta}
\end{figure}
Every red cross denotes a pruning operation.
For example, the two leftmost prunings are possible because no matter what values \(x\) and \(y\), they give, their parent node is going to take on value \(\min\{-2, x, y\} \le -2 < 2\), which is the value which is going to be conserved when moving up to the ``MAX'' node on the row above.
The values of \(\alpha\) and \(\beta\) are used to keep track of when a subtree can be pruned.

\subsection{Alpha-Beta Algorithm (Right To Left)}
\begin{figure}[H]
 \centering
 \includegraphics[width=\textwidth]{img/Alphabeta_reverse.png}
 \caption{Alpha-Beta algorithm (right to left)}
 \label{fig:alphabeta_reverse}
\end{figure}

\subsection{Alpha-Beta Algorithm (Ordered)}
Using this pruning algorithm, it is possible to find an ordering of the nodes which, when used to determine which nodes are explored first, minimizes the amount of nodes that need to be explored.
This optimal tree is shown in Figure~\ref{fig:alphabeta_ordered}, assuming a left-to-right exploration.
To obtain this tree, one needs to order nodes in ascending order on MAX levels and in descending order on MIN levels.
\begin{figure}[H]
 \centering
 \includegraphics[width=\textwidth]{img/Alphabeta_ordered.png}
 \caption{Alpha-Beta algorithm}
 \label{fig:alphabeta_ordered}
\end{figure}

\subsection{Alpha-Beta For More Than Two Players}
For games with more than two players, each node is associated with a tuple giving the value of each player for that state.

Tree pruning is possible if there is an upper bound on the sum of all components of this tuple, and if there is a lower bound on the values of each component.
We define the sum $S$ as the global upper bound on the sum of all components of the \(N\)-tuple, and all components are assumed to be nonnegative.

The conditions \py{v <= alpha} and \py{v >= beta} are now given by \py{Best[Player] >= Bound} where the bound equals the sum $S$ minus the last best node of the upper layer.
Indeed, let us take a player 1 from the upper layer which is ensured to get a value of at least $x$ for his layer: $(\ge x, \dots, \dots)$. Then, if player 2 from the lower layer unveils a child with more than $S - x$ for its own value, it will result in a score of $(\le x, \ge S - x, \dots)$. Thus, all the remaining children can be dropped since player 1 knows that these children will be chosen by player 2 only if the value of player 1 is less than $x$. \cite{multiplayer}

\begin{figure}[H]
 \centering
 \includegraphics[width=0.7\textwidth]{img/multiplayer.png}
 \caption{Shallow pruning in three-player game tree \cite{multiplayer}}
 \label{fig:multiplayer}
\end{figure}

\section{Squadro}

\subsection{Comparison Of Two Evaluation Functions}
One would expect that the new evaluation will work better since it takes into account the opponent, and will thus try to prevent them from winning, for example, by killing their pawns.
The old agent is not able to deliberately kill the opponent, consequently decreasing its chances of winning.

However, the yellow player still always wins in the four possible starting configurations because it begins with the best pawns at the center, progressing by steps of three.
The yellow player rapidly puts these pawns on the other side, which then form a barrier for the opponent's progress.
Thus, when the depth of the minimax algorithm is not sufficient, the new basic agent is not good enough to overcome the yellow player's innate advantage.

\subsection{Cut-Off Function}
The \emph{depth} in the \py{minimax.py} implementation is the number of steps along the path from the root node to a node in the layer where we stop the tree search and evaluate the node.

When the depth increases, the new basic agent is able to win even when playing as the red player because it can now predict and find a counter-attack against the yellow barrier.
The new basic agent wins for all starting configurations when the depth is higher than six.

Since the Squadro contest has a time limit, one needs to select the time allowed for each move.
There exist different methods to choose the maximum time allowed for each search, but an interesting option is to decrease the allotted time throughout the game in order to prevent exceeding the limit.
Indeed, one can split the time for each move $t_\textnormal{move}$ according to the next formula:
\[
    t_\textnormal{move} =
    \left\{
    \begin{array}{ll}
      0.2 \: t_\textnormal{left}^2, & \mathrm{if}\quad t_\textnormal{left}\leq \min\{\SI{100}{s}, t_\textnormal{total}/3\}, \\
      20, & \mathrm{otherwise.} \\
    \end{array}
    \right.
\]
where $t_\textnormal{left}$ is the remaining amount of time for the considered player and \(t_\textnormal{total}\) is the initial time limit.
One can notice that the allowed time goes to zero as the remaining time drops, as expected.

Using iterative deepening is a good choice to stop the algorithm during its search.
Iterative deepening depth-first search is a search method which iteratively performs a depth-limited depth-first search using the minimax algorithm, starting from a limit depth of one, up to some predefined value.
By recording the optimal path at each depth, the moves can be ordered in such a way as to minimize the total search time, coompensating for the (asymptotically negligible) cost of searching with lower depth limits.

The depth is chosen so that the algorithm gives good results, for instance eight.
However, when time runs out, the program returns the move selected by the deepest completed search.

The cut-off function, which reflects the moment when we need to stop the search, is activated when one of the following three conditions is true:
\begin{enumerate}
 \item \py{depth > self.current_depth}: the depth of the analyzed node is higher than the maximum depth,
 \item \py{state.game_over_check()}: the game is over,
 \item \py{time() - self.start_time > self.max_time}: the time elapsed during this search has exceeded the maximum allowed period for this search.
\end{enumerate}

\subsection{Evaluation Function}
In the figure presented in the instructions, the yellow basic agent will move its pawn number 3 since it will lead to the highest value of its evaluation function: the sum over all pawns of each's advancement.
However, it is clear that this is not the best possible move, since moving pawn number 4 would make it win the game.
A stronger heuristic thus has to take into account that a player wins if only four of their pawns return to their initial position.
In practice, we reduce this progress evaluation to only the four most advanced pawns, bringing a more cogent representation of the state.

The weak points of the evaluation function of the basic agents are
\begin{itemize}
	\item The first basic agent always tries to maximize its own total advancement, meaning that it does not take into account changes to the total advancement of the opponent.
	The second basic agent solves this by trying to maximize the difference between the total advancements.
	\item Both basic agents fail to recognize a potential game-winning move, which is solved in the smart agent by adding the requirement on the number of pawns which need to return to their initial position to win the game.
\end{itemize}

In addition to the sum of each pawn’s advancements, the following features are detailed below: the smart agent looks at
\begin{itemize}
 \item the total advancement of both players; and
 \item the number of pawns which have reached their initial position again.
\end{itemize}

The improved evaluation function thus takes into account these two features.
\begin{minted}{Python}
def evaluate(self, state):
    l1 = []
    l2 = []
    for pawn in [0, 1, 2, 3, 4]:
        l1.append(state.get_pawn_advancement(self.id, pawn))
        l2.append(state.get_pawn_advancement(1 - self.id, pawn))
    l1.sort()
    l2.sort()
    return sum(l1[1:]) - sum(l2[1:])
\end{minted}
It creates for every player, a list with the advancements of each pawn.
It sorts these lists and then sums over the four most advanced pawns for each, return the difference between these sums.

\subsection{Contest}
For the contest agent, we been interested in the AlphaGo Zero algorithm and how it managed to beat all other algorithms, and thus all humans, without even feeding it with strategies or human games. It has learned everything by itself while it continuously played against him. Its architecture is very insightul and led us to build a strong model detailed below.

\subsubsection*{Algorithm description}
Our algorithm uses a deep neural network $DNN(s)$ for which all the parameters need to be carefully updated. This neural network takes as an input the raw board representation $s$ of the 10 pawns' positions, and outputs both move probabilities and a value, $(p, v) = DNN(s)$.
The vector of move probabilities $p$ represents the probability of selecting each move $a$, $p_a = Pr(a|s)$. The value $v$ is a scalar evaluation, estimating the probability of the current player winning from position $s$. This neural network combines the roles of both policy network and value network into a single architecture \cite{alphago}.

\subsubsection*{Deep neural network}
The neural network consists of 3 residual fully connected blocks with batch normalization and rectifier nonlinearities (ReLU) (see Figure \ref{fig:DNN}).

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{img/dnn.png}
    \caption{Deep neural network}
    \label{fig:DNN}
\end{figure}

\subsubsection*{Monte Carlo Tree Search}

The neural network is trained from games of self-play by a reinforcement learning algorithm. In each position $s$, a Monte Carlo Tree Search is executed, guided by the neural network. The MCTS search outputs probabilities \(\pi\) of playing each move. These search probabilities usually select much stronger moves than the raw move probabilities $p$ of the neural network; MCTS may therefore be viewed as a powerful policy improvement operator. As shown in Figure \ref{fig:mcts}, this search algorithm consists to analyse the most promising moves, expanding the search tree based on random sampling of the search space. The application of Monte Carlo tree search in games is based on many playouts also called roll-outs. In each playout, the game is played out to the very end by selecting moves at random. The final game result of each playout is then used to weight the nodes in the game tree so that better nodes are more likely to be chosen in future playouts.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{img/mcts.png}
    \caption{Monte Carlo Tree Search}
    \label{fig:mcts}
\end{figure}

\subsubsection*{Simulations and results}

Although this algorithm is very powerful, it is strongly limited by the computation power available in the common machines. Indeed, training such a network has been very difficult due to the computation required to play each game. We thus received few samples of data, and these samples could not allow our agent to reach a top human player level.

The algorithm has been tested 100 times against the main previous agents. The results give 95\% of victory against the random agent, 55\% against the basic agent, and 25\% against the smart agent. Even if these results are not as good as expected, it is important to notice the very original method that we have used. It is thus still very convincing to see pretty good results with network a that has never been trained by ourselves and without any evaluation function specific to this game.

\begin{thebibliography}{9}
\bibitem{multiplayer}
Richard E. Korf, Multi-player alpha-beta pruning, Artificial Intelligence 48 (1991) 99--111, 1990.

\bibitem{alphago}
Silver, D., Schrittwieser, J., Simonyan, K. et al., Mastering the game of Go without human knowledge, Nature 550, 354–359, 2017.
\end{thebibliography}

\end{document}