\documentclass[]{article}
\usepackage{caption,subcaption,graphicx,float,url,amsmath,amssymb,amsthm,tocloft,cancel,thmtools}
\usepackage[toc,nonumberlist]{glossaries}
\usepackage{glossaries-extra}
\newcommand\numberthis{\addtocounter{equation}{1}\tag{\theequation}}
\newtheorem{defn}{Definition}
\newtheorem{thm}{Theorem}
\newtheorem{lemma}[thm]{Lemma}
\graphicspath{{figs/}}
\widowpenalty10000
\clubpenalty10000

\makeglossaries

%opening
\title{
	Theoretical Minimum\\
	Statistical Mechanics
}

\author{Simon Crase (compiler)\\simon@greenweaves.nz}

\begin{document}

\newglossaryentry{gls:adiabatic}{
	name={adiabatic},
	description={
		An adiabatic process is a process in which the entropy doesn’t
		change.}}

\newglossaryentry{gls:chemical_potential}{
	name={chemical potential},
	description={
		 chemical potential of a species is energy that can be absorbed or released due to a change of the particle number of the given species, e.g. in a chemical reaction or phase transition. The chemical potential of a species in a mixture is defined as the rate of change of free energy of a thermodynamic system with respect to the change in the number of atoms or molecules of the species that are added to the system.},
	 symbol=\ensuremath{\mu}}

\newglossaryentry{gls:control:parameter}{
	name={control parameter},
	description={a parameter that we, as experimenters, can easily change. A Control Parameter is a macroscopic parameter: we are not trying to change details of one molecule at a time.}}

\newglossaryentry{gls:exact}{
	name={exact},
	description={
		If $\frac{\partial G}{\partial y} = \frac{\partial H}{\partial x}$, we say that the differential form $dF = G dx + H dy$ is exact.}}

\newglossaryentry{gls:heat_capacity}{
	name={heat capacity},
	description={$C_V \triangleq \frac{\partial}{\partial T} \langle E \rangle$},
	symbol=\ensuremath{C_V}}

\newglossaryentry{gls:helmholtz}{
	name={Helmholtz Free Energy},
	description={Thermodynamic potential that measures the useful work obtainable from a closed thermodynamic system at a constant temperature and volume $A\triangleq E - TS$ },
	symbol=A}

\newglossaryentry{gls:phase:transition} {
	name={phase transition},
	description={
		The term phase transition (or phase change) is most commonly used to describe transitions between solid, liquid, and gaseous states of matter, as well as plasma in rare cases. A phase of a thermodynamic system and the states of matter have uniform physical properties. During a phase transition of a given medium, certain properties of the medium change, often discontinuously, as a result of the change of external conditions, such as temperature, pressure, or others. For example, a liquid may become gas upon heating to the boiling point, resulting in an abrupt change in volume.}}

\newglossaryentry{gls:closed:system}{
	name={closed systsm},
	description={A system which is \emph{either} everything \emph{or} it is sufficiently isolated from everything else that the other things don't influence it.}}
\maketitle

\begin{abstract}
	These are my notes for the \emph{Statistical Mechanics} lectures\cite{susskind2013statistical} from Leonard Susskind's \emph{Theoretical Minimum} series\cite{susskind2007theoretical}.

	Disclaimer: I have created these notes as an aide-m\'emoire for my own use; if you find them useful, you are welcome, but I'd appreciate hearing from you. They are not intended
	as a substitute for listening to the lectures. The intellectual property for all material derived from the lectures belongs, of course, to Professor Susskind; any mistakes, however, are my own.

	The notes were created using TexStudio\cite{TexStudio}, which I recommend for compiling them to a PDF, and the bibliography was created using JabRef\cite{Jabref}.
\end{abstract}

\tableofcontents

\listoffigures

\listoftheorems[ignoreall,onlynamed]

\section{Entropy and conservation of information}

Statistical Mechanics is pre-modern physics, modern physics, and post-modern physics. The 2nd law of thermodynamics will probably outlast anything. Statistical mechanics and thermodynamics may not be as sexy as the Higgs Boson, but they are deeper.

Classical and quantum mechanics are about perfect predictability. If you know initial point of a \gls{gls:closed:system} and the laws of motion, you know its future (in principle).

This lecture focuses on the law of conservation of energy, the $-1^{th}$ law of physics.

\begin{thm}[Liouville's theorem]
	The phase-space distribution function is constant along the trajectories of the system—i.e. the density of system points in the vicinity of a given system point travelling through phase-space is constant with time.
\end{thm}

\begin{align*}
S \triangleq \log(M) \numberthis \label{eq:entropy:simple}
\end{align*}

Proper laws of physics are reversible and therefore preserve the distinctions between states - i.e. information.  In this sense, the conservation of information is more fundamental than other physical quantities such as temperature or energy.
\begin{itemize}
	\item $S$ conserved as long as we can follow in detail, e.g. if we can follow the motion of each molecule that is disturbed by Friction.
	\item $S$ increases as our ignorance goes up;
	\item $S$ depends on the system and our state of knowledge.
\end{itemize}

\begin{align*}
S \triangleq - \sum_{i=1}^{N} p_i \log{p_i} \numberthis \label{eq:entropy}
\end{align*}

In (\ref{eq:entropy:simple}) and (\ref{eq:entropy}) we use the logarithm, so that entropy adds rather than multiplies.

The First law of Thermodynamics is energy conservation. For a closed system:
\begin{align*}
\frac{dE}{dt} =& 0, \text{}\\
\sum_i \frac{dE_i}{dt} =& 0
\end{align*}

\section{Temperature}

Temperature is really a measure of Energy. So the energy of one molecule of an Ideal Gas is given by:
\begin{align*}
E =& \frac{3}{2} k_B t\text{, where:}\\
k_B =& \text{ Boltzmann's constant}\\
T =& \text{ temperature in Kelvin}
\end{align*}
 In natural units we define a new unit of temperature, $T=k_Bt$, so $k_B=1$.

 We want to redefine temperature in terms of Entropy and Energy, and then show that this new definition agrees with our everyday idea of temperature. Temperature is not a fundamental quantity, but is derived as the amount of energy required to add an incremental amount of entropy to a system.  As the energy of a system increases, the number of possible states of a system increases, which means that the entropy increases.  This is the concept behind the second law of thermodynamics; it implies that temperature is always positive.

 Our definition reduces to:
\begin{align*}
\Delta E =& \underbrace{\frac{\partial E}{\partial S}}_\text{1 Degree of T} \Delta S\text{,or }\\
dE =& T dS \numberthis \label{eq:T}
\end{align*}

Consider two systems, $A$ and $B$, joined by a narrow pipe. Theorem \ref{thm:heat:flow} shows that T captures our intuition about temperature: heat flows from a higher temperature to lower.

\begin{thm}[Heat flows from higher temperature to lower]\label{thm:heat:flow}
	Given:
	\begin{align*}
	dE_A + dE_B =& 0 \text{, first law of thermodyamics}\numberthis \label{eq:thm:dE}\\
	dS_A + dS_B>& 0 \text{, second law of thermodynamics}\numberthis \label{eq:thm:eS} \\
	dE_i =& T_i dS_i \text{, from (\ref {eq:T})} \numberthis \label{eq:thm:T} \\
	T_A>&T_B \text{, Assumption.}\numberthis \label{eq:thm:Tgt}
	\end{align*}
	then energy flows from A to B, i.e.
	\begin{align*}
	dE_A<0\numberthis \label{eq:heat:flow:demonstrandum}
	\end{align*}
\end{thm}
\begin{proof}
	\begin{align*}
	T_A dS_A =& dE_A \text{, from (\ref{eq:thm:T})}\\
	=& - dE_B \text{, from (\ref{eq:thm:dE})}\\
	=& - T_B dS_B  \text{, from (\ref{eq:thm:T})}\\
	\implies&\\
	T_A dS_A + T_A dS_B =& T_A dS_B -T_B dS_B\\
	\implies&\\
	T_A (dS_A + dS_B) =& (T_A  -T_B) dS_B\\
	\implies& \text{, using (\ref{eq:thm:eS}) and (\ref{eq:thm:Tgt})}\\
	dS_B >& 0\text{, since temperature positive.}\\
	\implies& \text{using (\ref{eq:thm:T})}\\
	dE_B >& 0\\
	\implies& \text{, using (\ref{eq:thm:dE})}\\
	dE_A<&0\text{, which is (\ref{eq:heat:flow:demonstrandum})}
	\end{align*}
\end{proof}

\section{Maximizing entropy}

As the number of particles in a system grows, the distribution of states clumps more tightly around a mean. This is the Maxwell-Boltzmann distribution. The derivation requires Stirling's approximation and Lagrange multipliers.

Assume that we have N particles, and that there are $n_i$ particles in the $i^{th}$ energy level. Then the number of arrangements is given by
\begin{align*}
C\triangleq\frac{N!}{\prod_{i}n_i!} \numberthis \label{eq:number:combinations}
\end{align*}
 We want to maximize this subject to two constraints:
\begin{align*}
\sum_{i=1}^{N} n_i =& N \text{, total number of particles} \numberthis\label{eq:sum:probilities}\\
\sum_{i=1}^{N} E_i n_i =& E N \text{, average energy.}\numberthis\label{eq:total:energy}
\end{align*}

We will find Lemma \ref{lemma:Stirling} useful.
\begin{lemma}[Stirling's approximation]\label{lemma:Stirling}
	\begin{align*}
	N! \approxeq& N^N e^{-N}\numberthis\label{eq:stirling}
	\end{align*}
\end{lemma}

\begin{proof}
	\begin{align*}
	\log(N!) =& \sum_{i=1}^{N}\log (i)\\
	\approxeq &\int_{1}^{N} \log(x) dx\\
	\approxeq& \big[x \log(x) - x \big]_1^N\\
	\approxeq& N\log(N)-N\text{, whence} \\
	N! \approxeq& N^N e^{-N}
	\end{align*}
\end{proof}

\begin{thm}[Maximum entropy principle.]
	To maximize the number of arrangements we need to maximize the entropy.
\end{thm}

\begin{proof}
	\begin{align*}
	C=&\frac{N!}{\prod_{i}n_i!} \tag*{from (\ref{eq:number:combinations})}\\
	\approxeq& \frac{N^N e^{-N}}{\prod_{i}n_i^{n_i} e^{-\sum_{i=1}^{N}n_i}}\tag*{from Lemma \ref{lemma:Stirling}}\\
	\approxeq& \frac{N^N \bcancel{e^{-N}}}{\prod_{i}n_i^{n_i} \bcancel{e^{-N}}}\tag*{using (\ref{eq:sum:probilities})}\\
	\log(C) \approxeq & N \log(N) - \sum_{i=1}^{N} n_i \log(n_i)\\
	\approxeq& N \log(N) - \sum_{i=1}^{N} N p_i \log(N p_i)\\
	\approxeq& N \log(N) - \sum_{i=1}^{N} N p_i \big(\log(N) + \log(p_i))\big)\\
	\approxeq& N \log(N) -  N \log(N) \bcancel{\sum_{i=1}^{N} p_i}  - N \sum_{i=1}^{N}p_i \log(p_i)\tag*{using (\ref{eq:sum:probilities})}\\
	\approxeq& \bcancel{N \log(N)} - \bcancel{N \log(N)}  - N \sum_{i=1}^{N}p_i \log(p_i)\\
	\approxeq& NS \text{, using (\ref{eq:entropy})} \numberthis \label{eq:entropy:logC}
	\end{align*}
\end{proof}

\section{The Boltzmann distribution}

\subsection{The Boltzmann distribution}

To maximize S subject to constraints (\ref{eq:sum:probilities}) and (\ref{eq:total:energy}), we minimize -S, using Lagrange Multipliers, i.e., minimize
\begin{align*}
\sum_{i=1}^{N}p_i \log(p_i) + \alpha(\sum_{i=1}^{N} p_i-1) + \beta(\sum_{i=1}^{N} E_i p_i-E)\text{, where $\alpha$ and $\beta$ are parameters to be determined.}
\end{align*}

Differentiating with respect to $p_i$:
\begin{align*}
\log(p_i)+1 + \alpha +\beta E_i=&0\text{, whence}\\
p_i =& e^{-(1+\alpha)} e^{-\beta E_i}\\
=& \frac{1}{Z} e^{-\beta E_i}\text{, where $Z\triangleq e^{(1+\alpha)}$} \numberthis\label{eq:maximize:entropy}
\end{align*}

From (\ref{eq:sum:probilities}) and (\ref{eq:maximize:entropy})
\begin{align*}
1 =& \frac{1}{Z}  \sum_{i=1}^{N} e^{-\beta E_i}\\
Z(\beta) =& \sum_{i=1}^{N} e^{-\beta E_i} \text{. $Z$ is known as the Partition Function.} \numberthis \label{eq:partition:function}
\end{align*}

\begin{thm}[$\beta$ is the inverse temperature]\label{thm:inverseT}
	\begin{align*}
		T =& \frac{1}{\beta}\numberthis\label{eq:inverseT}
	\end{align*}
\end{thm}

\begin{proof}
	From (\ref{eq:total:energy}) and (\ref{eq:maximize:entropy})
	\begin{align*}
	E =& \frac{1}{Z}  \sum_{i=1}^{N} e^{-\beta E_i} E_i \text{\, and}\numberthis \label{eq:E}\\
	\frac{\partial Z}{\partial \beta} =& - \sum_{i=1}^{N} e^{-\beta E_i} E_i \text{, from (\ref{eq:partition:function}), whence} \numberthis \label{eq:dZ}\\
	E(\beta) =& - \frac{1}{Z} \frac{\partial Z}{\partial \beta} = - \frac{\partial \log(Z)}{\partial \beta} \numberthis \label{eq:E:beta}
	\end{align*}
	Now, from (\ref{eq:entropy})
	\begin{align*}
	S =& -\sum_{i=1}^{N} p_i \log(p_i) \text{, which, using  (\ref{eq:maximize:entropy}) gives}\\
	=& - \sum_{i=1}^{N} \frac{1}{Z} e^{- \beta E_i}\big[- \beta E_i -\log(Z)\big]\\
	=& \beta \sum_{i=1}^{N} \frac{1}{Z} e^{- \beta E_i}  E_i +  \frac{1}{Z} \log(Z) \underbrace{\sum_{i=1}^{N}  e^{- \beta E_i}}_{Z} \\
	=& \beta E +  \bcancel{\frac{1}{Z}} \log(Z)  \bcancel{Z}\\
	=& \beta E +   \log(Z)  \numberthis \label{eq:S_Z}
	\end{align*}
	Differentiating:
	\begin{align*}
	dS =& \beta dE + \cancel{E} d\beta + \underbrace{\frac{\partial \log(Z)}{\partial \beta}}_{=\cancel{-E(\beta)} \text{, using (\ref{eq:E:beta})}} d\beta\\
	=& \beta dE \numberthis \label{eq:ds:Beta}
	\end{align*}
	Now (\ref{eq:T}) defines Temperature as follows:
	\begin{align*}
	dS =& \frac{1}{T} dE \text{, so comparing with (\ref{eq:ds:Beta})} \\
	T =& \frac{1}{\beta}
	\end{align*}
\end{proof}

\subsection{Ideal Gas}.

We will find the following theorem useful.

\begin{thm}[Gaussian Integral]\label{thm:gaussian_integral}
	\begin{align*}
		\int_{-\infty}^{\infty} e^{-a^2 x^2} dx = \frac{\sqrt{\pi}}{a}
	\end{align*}
\end{thm}

\begin{proof}
	\begin{align*}
		\int_{-\infty}^{\infty} e^{-a^2 x^2} dx =& \frac{1}{a} \int_{-\infty}^{\infty} e^{-(ax)^2} d(ax) \\
		=& \frac{1}{a} \int_{-\infty}^{\infty} e^{-z^2} dz \numberthis \label{eq:gauss_a}\\
		\int_{-\infty}^{\infty} e^{-x^2} dx \int_{-\infty}^{\infty} e^{-y^2} dy =& \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} e^{-(x^2+y^2)}  dx dy\\
		=& \int_0^{2\pi} \int_{0}^{\infty}e^{-r^2} r dr d\theta\\
		=& \int_0^{2\pi}  d\theta  r dr\\
		=& 2\pi \int_{0}^{\infty}e^{-(r^2)} \frac{1}{2} d(r^2)\\
		=& \pi \int_{0}^{\infty} e^{-z} dz\\
		=& - \pi  e^{-z}\bigg\vert_{z=0}^{\infty}\\
		=& \pi\text{, whence}\\
		\int_{-\infty}^{\infty} e^{-x^2} dx =& \sqrt{\pi}\text{, from (\ref{eq:gauss_a})}\\
		\int_{-\infty}^{\infty} e^{-a^2 x^2} dx =& \frac{\sqrt{\pi}}{a}
	\end{align*}
\end{proof}

Assume molecules don't interact. State consists of 6N coordinates,  $\{X_1,...X_{3N}\}$ and momenta $\{P_1,...P_{3N}\}$. From (\ref{eq:partition:function})
\begin{align*}
Z =& \int d^{3N}x d^{3N}p . e^{- \frac{\beta}{2m} \sum_{n=1}^{3N} p_n^2}\\
=& \frac{V^N}{N!} \int d^{3N}p . e^{- \frac{\beta}{2m} \sum_{n=1}^{3N} p_n^2}\textit{--Factorial is controversial}\\
=& \frac{V^N}{N!} \big[\int dp. e^{-\frac{\beta}{2m}p^2}\big]^{3N}\\
=& \frac{V^N}{N!} \big[\frac{2 m \pi}{\beta}\big]^{\frac{3N}{2}}\numberthis\label{eq:Z_beta}\text{, from Theorem \ref{thm:gaussian_integral}}
\end{align*}
We now use Stirling's formula (\ref{eq:stirling}).
\begin{align*}
\frac{V^N}{N!} \approxeq& \big(\frac{eN}{N}\big)^N \\
=& \big(\frac{e}{\rho}\big)^N,\text{, and (\ref{eq:E:beta}) becomes}\\
Z \approxeq& \big(\frac{e}{\rho}\big)^N .  \big[\frac{2 m \pi}{\beta}\big]^{\frac{3N}{2}}
\end{align*}

\begin{align*}
\log(Z) =& - \frac{3N}{2} \log(\beta) + const\\
E =& - \frac{\partial \log(Z)}{\partial \beta}\numberthis \label{eq:E:Z}\\
=& \frac{3N}{2} T
\end{align*}

\section{Pressure of an ideal gas and fluctuations}

\subsection{Pressure of an ideal gas}

\begin{itemize}
	\item Motion is isotropic, even near wall. We know this from an honest use of statistical mechanics.
	\item Give up intuition because it might not be correct ''ride the mathematics''.
	\item Question: Once we have done the mathematics of a problem, can’t we check it with our intuition?
	\item Answer: Yes we can, but what we are really doing is checking our intuition not the mathematics.
	\item ''All very great physicists are master of thermodynamics.''
\end{itemize}

We want to determine the Equation of State.

From (\ref{eq:S_Z}) and Theorem \ref{thm:inverseT}.
\begin{align*}
S =& \frac{E}{T} + \log(Z)\\
A\triangleq &E - TS\text{. We call this the \emph{Helmholtz Free Energy}} \\
=& -T \log(Z) \numberthis \label{eq:helmholtz}
\end{align*}

Variables come in pairs, e.g. (V,P):
\begin{itemize}
	\item \Gls{gls:control:parameter} --\glsdesc{gls:control:parameter}
	\item Conjugate Thermodynamic Variable (P).
\end{itemize}

\begin{lemma}[Derivatives along contour]\label{thm:derivative:contour}
	Given two dependent variables, S and E, say, and two independent variables, T and V, say:
	\begin{align*}
		\frac{\partial E}{\partial V}\bigg|_S = \frac{\partial E}{\partial V}\bigg|_T - 	\frac{\partial E}{\partial S}\bigg|_V \frac{\partial S}{\partial V}\bigg|_T
	\end{align*}
\end{lemma}
\begin{proof}
	Along a contour of fixed S:
	\begin{figure}[H]
		\caption{$E(B)-E(A) = \big[E(C)-E(A)\big]+\big[E(B)-E(C)\big]$}\label{fig:contour:of:fixed:S}
		\includegraphics[width=0.9\textwidth]{partial-derivatives}
	\end{figure}
	\begin{align*}
		\Delta E =& \frac{\partial E}{\partial V} \bigg|_T \Delta V + \frac{\partial E}{\partial T} \bigg|_V \Delta T \text{, by inspecting Figure \ref{fig:contour:of:fixed:S}. The $2^{nd}$ term gives:}\\
		\frac{\partial E}{\partial T} \bigg|_V \Delta T=& \frac{\partial S}{\partial T} \bigg|_V \frac{\partial E}{\partial S} \bigg|_V \Delta T\text{, whence:}\\
		\frac{\partial E}{\partial V} \bigg|_S=& \frac{\partial E}{\partial V} \bigg|_T  + \frac{\partial S}{\partial T} \bigg|_V \frac{\partial E}{\partial S} \bigg|_V \frac{\Delta T}{\Delta V} \numberthis\label{eq:dEdV}\\
		\text{Now }\Delta S =& \frac{\partial S}{\partial V} \bigg|_T \Delta V + \frac{\partial S}{\partial T} \bigg|_V \Delta T\\
		=& 0\text{, along a contour of constant S, whence}\\
		\frac{\Delta T}{\Delta V} =& - \Bigg[\frac{\frac{\partial S}{\partial V}\big|_T}{\frac{\partial S}{\partial T}\big|_V}\Bigg]\text{, along a contour of constant S, so (\ref{eq:dEdV}) becomes:}\\
		\frac{\partial E}{\partial V} \bigg|_S=& \frac{\partial E}{\partial V} \bigg|_T  - \cancel{\frac{\partial S}{\partial T} \bigg|_V} \frac{\partial E}{\partial S} \bigg|_V \frac{\frac{\partial S}{\partial V}\big|_T}{\cancel{\frac{\partial S}{\partial T}\big|_V}}\\
		=& \frac{\partial E}{\partial V} \bigg|_T  -  \frac{\partial E}{\partial S} \bigg|_V \frac{\partial S}{\partial V}\bigg|_T
	\end{align*}
\end{proof}

NB. For most situations, S is a monotonic increasing function of E. As E increases, distribution gets broader, and S increases.

\begin{figure}[H]
	\begin{center}
		\caption{Imagine piston is vertical}\label{fig:vertical:piston}
		\includegraphics[width=0.5\textwidth]{vertical-piston}
	\end{center}
\end{figure}

Imagine piston is vertical.

\begin{itemize}
	\item Move piston out, gas does work, which is paid for from energy.
	\item Move piston in, we do work, which adds to energy.
\end{itemize}

Move piston slowly, and don't let energy in. Adiabatic: slowly, and no heat in or out. $dE = - P \mathcal{A}dx$, where $\mathcal{A}$ is the area of the piston in Figure \ref{fig:vertical:piston}. Since $\mathcal{A}dx = dV$, $dE = -P dV$.

\begin{align*}
\frac{\partial E}{\partial V}\bigg|_S =& - P\numberthis \label{eq:defP}
\end{align*}

\begin{figure}[H]
	\caption{Energy Levels as a function of volume}
	\includegraphics[width=0.9\textwidth]{energy-levels-volume}
\end{figure}

\begin{thm}[Adiabatic Theorem]\label{thm:adiabatic}
	Change volume \gls{gls:adiabatic}ally, the system will ride along the energy level: it will change, but not jump.
\end{thm}

\begin{proof}
	TBP
\end{proof}
\begin{itemize}
	\item One consequence of Theorem \ref{thm:adiabatic} is that the $p_i$ are fixed.

	\item Isentropic is also adiabatic.

	\item Easier to use T as it appears in Bolzmann.
\end{itemize}

\begin{align*}
P =& - \frac{\partial E}{\partial V}\bigg|_S \\
=& - \frac{\partial E}{\partial V}\bigg|_T + \frac{\partial S}{\partial V}\bigg|_T \frac{\partial E}{\partial S}\bigg|_V\text{, from Lemma \ref{thm:derivative:contour}}\\
=& - \frac{\partial E}{\partial V}\bigg|_T + \frac{\partial S}{\partial V}\bigg|_T T\text{, from (\ref{eq:T})}\\
=& - \frac{\partial(E-TS)}{\partial V}\bigg|_T \text{, from (\ref{eq:helmholtz})--\gls{gls:helmholtz}} \glssymbol{gls:helmholtz}\\
=& T \frac{\partial \log(Z)}{\partial V}\bigg|_T \numberthis \label{eq:P_Z_V}
\end{align*}

\begin{itemize}
	\item We haven't used assumption that we are dealing with a gas, so this is completely general.
	\item could have used different conjugate pair.
\end{itemize}

The partition function is given by:
\begin{align*}
Z(\beta) =& \int dx dp e^{- \beta \frac{p^2}{2m}}\\
=& \frac{V^N}{N!} f(\beta)\\
\log(Z) =& N \log(V) + \log(f(\beta))\text{, so (\ref{eq:P_Z_V})  becomes:}\\
P =& \frac{NT}{V}\text{, or}\\
P =& \rho T
\end{align*}

\subsection{Fluctuations}

\begin{align*}
	\langle E -  \langle E \rangle \rangle^2 =& \langle E^2 \rangle - \langle E \rangle^2\text{. Now, from (\ref{eq:E:beta})}\\
	\langle E \rangle=& - \frac{\partial \log(Z)}{\partial \beta}\text{, and from (\ref{eq:dZ})}\\
	 \langle E^2 \rangle =& \frac{1}{Z} \frac{\partial^2 Z}{\partial \beta^2} \text{, whence}\\
	 \langle E^2 \rangle =& \frac{1}{Z} \frac{\partial^2 Z}{\partial \beta^2} - \frac{1}{Z^2} \bigg(\frac{\partial Z}{\partial \beta}\bigg)^2 \numberthis \label{E2Z}
\end{align*}
All of statistical mechanics is about the power of the partition function.

\begin{align*}
	\frac{\partial^2 \log(Z)}{\partial \beta^2} =& \frac{\partial}{\partial \beta} \frac{1}{Z} \frac{\partial Z}{\partial \beta}\\
	=& \frac{1}{Z} \frac{\partial^2 Z}{\partial \beta^2} - \frac{1}{Z^2} \bigg(\frac{\partial Z}{\partial \beta}\bigg)^2\\
	(\Delta E)^2 =& \frac{\partial}{\partial \beta} \frac{1}{Z} \frac{\partial Z}{\partial \beta}\text{, from (\ref{E2Z})}\\
	=& - \frac{\partial}{\partial \beta} \langle E \rangle\\
	=&  \frac{\partial}{\partial T} \langle E \rangle T^2 \tag*{from (\ref{eq:inverseT})}\\
	=& C_V T^2\text{, where} \numberthis \label{eq:einstein:gibbs}\\
	C_V \triangleq& \frac{\partial}{\partial T} \langle E \rangle \text{, \Gls{gls:heat_capacity}}
\end{align*}

\begin{itemize}
	\item (\ref{eq:einstein:gibbs}) is due to Einstein and Gibbs.
	\item True for any system: we haven't assumed ideal gas.
\end{itemize}

\section{Weakly interacting gases, heat, and work}

\subsection{Weakly interacting gases}

We will assume:
\begin{itemize}
	\item dilute;
	\item molecules on average far from each other;
	\item range of forces small compared to distance between particles;
	\item potential energy or forces small.
\end{itemize}

Then we will ask where this breaks down; where does correction becomes as large as uncorrected.

We calculate partition function for a system where total energy not equal to sum of kinetic energies.

Assume energy has the following form:
\begin{align*}
E =& \sum_{n} \frac{p^2}{2M} + \underbrace{\sum_{n>m}u(\vert(x_m-x_n)\vert)}_{U(x)\text{, say}}
\end{align*}
, where $u$ is small compared to kinetic energy, and define:
\begin{align*}
U_0 \triangleq& \int d^3x x.u(\vert x \vert)\text{, then}\\
\int d^3x_1 d^3s_2 u(\vert x_1 - x_2\vert) = & V U_0\text{, ignoring surface expects!}
\end{align*}
NB: $U$ factors in collisions!

\begin{align*}
Z =& \int \frac{dp dx}{N!} e^{- \beta \frac{p^2}{2M}} e^{-\beta U(x)}\\
=&\underbrace{ \int \frac{dp}{N!} e^{- \beta \frac{p^2}{2M}}}_{\Big(\sqrt{\frac{2\pi m}{\beta}}\Big)^3} \int dx.e^{-\beta U(x)}\\
=&\underbrace{ \int \frac{dp.V^N}{N!} e^{- \beta \frac{p^2}{2M}}}_{\text{Partition function for Ideal Gas}} \int dx.\frac{e^{-\beta U(x)}}{V^N}\\
=&Z_0(\beta) \int dx.\frac{e^{-\beta U(x)}}{V^N}\text{ say.}\numberthis \label{eq:introducingZ0}\\
\int dx.\frac{e^{-\beta U(x)}}{V^N}\approxeq& \int \frac{dx}{V^N} \Big(1 - \beta U(x)\Big)\text{, assuming U small}\\
\approxeq & 1 - \beta \int \frac{dx}{V^N} U(x)\\
\approxeq & 1 - \beta \int \frac{dx}{V^N} \sum_{n>m} u(\vert x_n - x_m\vert)\\
\approxeq & 1 - \frac{N(N-1)\beta}{2} \int \frac{dx_1.dx_2}{V^N}  u(\vert x_1 - x_2\vert) V^{N-2}\\
\approxeq& 1 - \frac{N^2}{2}\beta \frac{U_0}{V}\\
Z \approxeq& Z_0(\beta) \Big[1 - \frac{\beta N^2}{2V} U_0\Big]
\end{align*}

\begin{align*}
\log(Z) =& \log(Z_0) + \log\Big[1 - \frac{\beta N^2}{2V} U_0\Big] \\
\approxeq & \log(Z_0) - \frac{\beta N^2}{2V} U_0\\
E =& - \frac{\partial \log Z}{\partial \beta}\\
=& \frac{3}{2}NT + \frac{N^2}{2V}U_0\\
=& \Big[\frac{3}{2}T + \frac{ \rho}{2}U_0\Big] N
\end{align*}

Since E is difficult to measure, we'll also calculate P. From (\ref{eq:P_Z_V}).

\begin{align*}
P =& T \frac{\partial \log(Z)}{\partial V}\bigg|_T\\
=& \rho T + \frac{\rho^2}{2} U_0\text{, provided $\rho U_0 \ll T$, i.e. potential energy much less than kinetic energy.}
\end{align*}

\subsection{Heat \& Work}

We shall find the following theorem useful.

\begin{thm}[Clairaut-Schwarz]\label{thm:2partial}
	If $\frac{\partial^2 F}{\partial x \partial y}$ and $\frac{\partial^2 F}{\partial y \partial x}$ both exist and are continuous at some point $(x_0,y_0)$, then there exists a neighbourhood of $(x_0,y_0)$ where:
	\begin{align*}
	\frac{\partial^2 F}{\partial x \partial y} = \frac{\partial^2 F}{\partial y \partial x}
	\end{align*}
\end{thm}

Given $F(x,y)$
\begin{align*}
dF =& \frac{\partial F}{\partial x}dx + \frac{\partial F}{\partial y}dy\\
=& G dx + H dy\text{, say}\numberthis \label{eq:dF}\\
\frac{\partial G}{\partial y} =& \frac{\partial H}{\partial x}\text{, using Theorem \ref{thm:2partial}}\numberthis \label{eq:exact}
\end{align*}

\glsdesc{gls:exact}

Consider an adiabatic change to system of cylinder and piston.

\begin{align*}
dE =& -P dV \text{, change V only}\\
=& T dS \text{, hold V, add heat}\\
=& \underbrace{-P dV}_{dW} + \underbrace{T dS}_{dQ}\text {, change both. First Law of Thermodynamics}
\end{align*}

Is $dQ$ an exact differential?

\begin{align*}
dQ =& dE + P.dV\text{. If exact}\\
\frac{\partial Q}{\partial E}=&1, \frac{\partial Q}{\partial V} = P\\
\frac{\partial^2 Q}{\partial V \partial E} \ne& \frac{\partial^2 Q}{\partial E \partial V}\text{, so $dQ$ is not exact.}
\end{align*}

\section{Entropy vs. reversibility}

\subsection{The speed of sound in an ideal gas}

Create a small overdense region. How fast does it move out? With the average velocity of molecules.

In thermal equilibrium in a dilute gas, every molecule has velocity $v$
\begin{align*}
\frac{3}{2} k_B T =& \frac{m}{2} v^2\text{ We'll use laboratory units, so $k_B$}\\
v^2 =& \frac{3 k_B T}{m}\text{. A more detailed analysis gives:} \numberthis \label{eq:c:rough}\\
c^2 =& \frac{1}{m}\frac{\partial P}{\partial \rho} \text{, where $c$ represents speed of sound (not light)}\\
P =& \rho k_B T\text{ Ideal gas law}\\
\implies&\\
c^2 =& \frac{k_B T}{m}\text{. (\ref{eq:c:rough}) is within order of magnitude!}
\end{align*}

\subsection{A single harmonic oscillator in a heat bath}

\begin{align*}
Energy =& \frac{m}{2} \dot{x}^ 2 + \frac{k}{2} x^2\tag{ $k$ is not Boltzmann!}\\
=& \frac{1}{2m} p^ 2 + \frac{k}{2} x^2\\
Z(\beta) =& \int e^{-\beta \frac{1}{2m} p^ 2 } e^{-\beta \frac{k}{2} x^2} dp dx\\
=& \int e^{-\beta \frac{1}{2m} p^ 2 } dp \int e^{-\beta \frac{k}{2} x^2} dx\\
=& \sqrt{\frac{2m\pi}{\beta}}\sqrt{\frac{2 \pi}{\beta k}} \text{, using Theorem \ref{thm:gaussian_integral}}\\
=& \frac{2 \pi}{\omega}\frac{1}{\beta}\text{, where} \numberthis\label{eq:Z:classical:q}\\
\omega =& \sqrt{\frac{k}{m}} \text{ represents the frequency. Now}\\
\log(Z) =& \log(2\pi) -\log(\omega) - \log(\beta)\text{, whence }\\
E =& - \frac{\partial \log(Z)}{\partial \beta}\text{, from (\ref{eq:E:Z})}\\
=& \frac{1}{\beta}\\
=& T \text{, from Theorem \ref{thm:inverseT}}
\end{align*}

\begin{itemize}
	\item Why no 3? Because 1 dimensional.
	\item Why not 2? Because 2 integrals.
\end{itemize}

Note:
\begin{itemize}
	\item Energy doesn't depend on $m$;
	\item Energy doesn't depend on $k$. This seems wrong (make $k$ very large). This was a big problem around the end of 19th century (what about diatomic molecules 3 becomes 5?) We have ignored quantum mechanics!
\end{itemize}

\begin{align*}
	Z(\beta) =& \sum_{n} e^{- \beta n \hbar \omega}\\
	 =& \sum_{n} \Big(e^{- \beta\hbar \omega}\Big)^n\\
	 =& \frac{1}{1 - e^{- \beta\hbar \omega}}\numberthis\label{Z_quantum_shm}\\
	 E =& - \frac{1}{Z} \frac{\partial Z}{\partial \beta}\\
	 \frac{\partial Z}{\partial \beta}=& \frac{- \hbar \omega e^{- \beta\hbar \omega}}{(1 - e^{- \beta\hbar \omega})^2}\\
	 E =& \frac{ \hbar \omega e^{- \beta\hbar \omega}}{(1 - e^{- \beta\hbar \omega})}
\end{align*}

High temperature: $T\rightarrow\infty \implies \beta \rightarrow 0$
\begin{align*}
	\frac{ \hbar \omega e^{- \beta\hbar \omega}}{(1 - e^{- \beta\hbar \omega})}\rightarrow& \frac{\hbar \omega}{1 -(1 - \beta \hbar \omega...)}\\
	\rightarrow& \frac{\hbar \omega}{\beta \hbar \omega}\\
	\rightarrow& \frac{1}{\beta}\\
	\rightarrow& T\text{, which agrees with classical physics.}
\end{align*}

Low temperature: $T\rightarrow 0 \implies \beta \rightarrow \infty$

\begin{align*}
\frac{ \hbar \omega e^{- \beta\hbar \omega}}{(1 - e^{- \beta\hbar \omega})}\rightarrow&\hbar \omega e^{- \beta\hbar \omega}\\
\rightarrow&0
\end{align*}

So for low temperature, oscillation is suppressed.	Where is the crossover between classical and quantum behaviour? This occurs when exponential goes from large to small (and vice versa), i.e. when $\beta\hbar \omega \approx 1$.

\begin{itemize}
	\item $\beta\hbar \omega < 1$ Classical.
	\item $\beta\hbar \omega > 1$ Quantum.
	\item $\beta\hbar \omega = 1\equiv\hbar \omega=T$ Crossover. At this temperature, system has one quantum of energy; it doesn't want to have less!
	\item For very low temperature, diatomic molecule behaves like monoatomic.
	\item For a stiff molecule, $\omega$ is high, so crossover temperature is high.
\end{itemize}

c.f. Einstein specific heat.

Can we derive classical partition function from quantum?

\begin{align*}
Z(\beta) =&  \frac{1}{1 - e^{- \beta\hbar \omega}} \text{, from (\ref {Z_quantum_shm})}\\
\approxeq & \frac{1}{1 - 1 + \beta\hbar \omega}\text{, for high T}\\
\approxeq & \frac{1}{\beta\hbar \omega}\text{, c.f. (\ref{eq:Z:classical:q})}
\end{align*}

\begin{itemize}
	\item So the classical and quantum partition functions agree within a multiplicative constant, which disappears when we differentiate $\log(Z)$ to do anything useful.
	\item Heat system: it unfreezes degrees of freedom and we discover complexity. Cool system: discover quantum behaviour.
	\item Violin string: freeze out higher frequency states, so energy finite.
\end{itemize}

\subsection{Contradiction between the 2nd and reversibility}
\begin{itemize}
	\item Newton's laws reversible
	\item 2nd law of thermodynamics is not reversible!
	\item Consider a small blob in phase space: according to Liouville's theorem, volume, and hence entropy, does not change! Correct view of microscopic system.
	\item We need to coarse grain. Resolution is limited, both by QM, and by our instruments.
	\item Imagine that blob turns into long thing snake (same volume), but we can't resolve, so we cover with blobs. We are now losing information.
	\item Chaos! Sensitive dependence on initial conditions. Imagine phase space filled with cotton!

	\item Phase space trajectory will eventually come back to starting point within specified tolerance: time depends on tolerance.

	\item Entropy probably increases -- Boltzmann.
\end{itemize}

\section{Entropy, reversibility, and magnetism}

\subsection{2nd Law and Poincar\'e Recurrences}

Imagine room with all particles of air in one half. In 6N dimensional phase space, momentum is bounded because we have finite energy. Phase point moves chaotically. Phase point $\frac{1}{2^N}$. Or, imagine configuration space has N dimensions, and is of volume $V^N$; what is probability of finding system in smaller volume $v^N$? $\frac{v^N}{V^N}$.

\begin{thm}[Poincar\'e Recurrences]
	Any dynamical system defined by an ordinary differential equation determines a flow map $f^t$ mapping phase space on itself. The system is said to be volume-preserving if the volume of a set in phase space is invariant under the flow. For instance, all Hamiltonian systems are volume-preserving because of Liouville's theorem. The theorem is then: If a flow preserves volume and has only bounded orbits, then for each open set there exist orbits that intersect the set infinitely often.
\end{thm}

\begin{proof}
	The proof, speaking qualitatively, hinges on two premises:

	\begin{itemize}
		\item 	A finite upper bound can be set on the total potentially accessible phase space volume. For a mechanical system, this bound can be provided by requiring that the system is contained in a bounded physical region of space (so that it cannot, for example, eject particles that never return) – combined with the conservation of energy, this locks the system into a finite region in phase space.
		\item The phase volume of a finite element under dynamics is conserved. (for a mechanical system, this is ensured by Liouville's theorem)
	\end{itemize}

	Imagine any finite starting volume of phase space and follow its path under dynamics of the system. The volume "sweeps" points of phase space as it evolves, and the "front" of this sweeping has a constant size. Over time the explored phase volume (known as a "phase tube") grows linearly, at least at first. But, because the accessible phase volume is finite, the phase tube volume must eventually saturate because it cannot grow larger than the accessible volume. This means that the phase tube must intersect itself. In order to intersect itself, however, it must do so by first passing through the starting volume. Therefore, at least a finite fraction of the starting volume is recurring.

	Now, consider the size of the non-returning portion of the starting phase volume – that portion that never returns to the starting volume. Using the principle just discussed in the last paragraph, we know that if the non-returning portion is finite, then a finite part of the non-returning portion must return. But that would be a contradiction, since any part of the non-returning portion that returns, also returns to the original starting volume. Thus, the non-returning portion of the starting volume cannot be finite and must be infinitely smaller than the starting volume itself.
\end{proof}
Boltzmann's Brain: most of the ways we could have gotten here are by random fluctuation, and they don't have coherent histories.

\subsection{Magnetism}

A normal magnet is made up of lots of little magnets, each of which can be as small as a single atom, or they could be little crystal grains. At a high enough temperature, e.g. room temperature, orientation is random, not just over whole object but also in each little region. If we cool it down, and if the energy stored in each pair of magnets is such that the magnets like to line up in the same direction, then, as you cool it down you find that pairs of magnets in each region tend to be in alignment, but a nearby region may all be aligned in a different direction. As you cool it down you get we get small domains, with the same direction within domain. As you suck more and more enrgy out of the system, domains will want to align more and more. Domains start to grow: you may hit a point where magnets are all aligned in the same direction. That is a phase transition, called a ferromagnetic transition. It is the simplest kind of phase transition. At zero temperature we are in the lowest possible energy state, so all the magnets line up.\url{https://youtu.be/3hh0lJZbUfo?t=2455}

$N$ simple magnets, either up or down. Assume magnetic field, no interaction. Spin up, $\sigma(1).$

\begin{align*}
	E=&(n-m)\mu H\text{, where n are up, m down, $m+n=N$}\\
	\text{number of states} =&\frac{N!}{n!m!}\\
	Z =&\sum_{mn\mid m+n=N} e^{- \beta \mu H(n-m)}\\
	=&\sum_{n}\frac{N!}{n!(N-n)}X^nY^{N-n}\text{, where $X=e^{-\beta\mu H},Y=e^{\beta\mu H}$}\\
	=&\big(X+Y\big)^N\tag*{from the Binomial Theorem}\\
	=&\big(e^{-\beta\mu H}+e^{\beta\mu H}\big)^N\\
	=&2^N \big(\cosh(\beta\mu H)\big)^N\tag*{Definition of cosh}
\end{align*}

NB: biased by magnetic field!

\begin{align*}
	M \triangleq& \big< \frac{n-m}{N}\big>\text{, Magnetization} \\
	E =& M N \mu H\\
	=& \frac{\partial \log(Z)}{\partial \beta} \text{, from (\ref{eq:E:beta})}\\
	=& - N \frac{\sinh(\beta\mu H)}{\cosh(\beta\mu H)} \mu H\\
	M=& \tanh(\beta\mu H)
\end{align*}

Figure \ref{fig:tanh} shows that $M$ is zero for $\beta=0$ (high temperature) and one for large $\beta$ (small temperature).

\begin{figure}[H]
	\caption{Graph of $\tanh$}\label{fig:tanh}
	\includegraphics[width=0.8\textwidth]{tanh}
\end{figure}

The Ising model
\begin{align*}
E =& - j \sum_{n} \sigma(n) \sigma(n+1)
\end{align*}

\begin{itemize}
	\item Two ground states
	\item tiniest external magnetic field will bias
	\item spontaneous symmetry breaking
\end{itemize}

\section{The Ising model}

\subsection{One-dimensional Ising model}

Consider one spin in a heat bath, so we can focus on one at a time

\begin{align*}
E =& -j \sigma \text{, where $j=\mu B$}\\
Z =& \sum_{\sigma=\pm 1} e^{-\beta j \sigma}\\
=& e^{\beta j}+ e^{- \beta j}\\
=& 2 \cosh(\beta j) \numberthis \label{eq:E:cosh:beta:j}\\
\big<E\big> =& - \frac{\partial \log(Z)}{\partial \beta}\text{, from (\ref{eq:E:beta})}\\
=& - j \tanh{\beta j}\\
\big<\sigma\big> =& - \tanh{\beta j}
\end{align*}

Figure \ref{fig:tanh} shows that spin points up for $\beta\rightarrow\infty$, so spin parallel to j for low temperature; spin all over the place at high temperature.

Now consider a line of spins. We will assume a specific spin is up, and ask about correlation with some other spin $\big<\sigma(i) \sigma(i+n)\big>$. Assume first spin is up, and focus on relationships, $\mu_i\triangleq\sigma_i\sigma_{i+1}$ (state of link).\footnote{NB: if we reverse the sign of $j$ and reverse the sign of every $2_{nd} \sigma_i$, we get the same problem. }

\begin{align*}
E =& -j \sum_i \sigma_i \sigma_{i+1}\\
=& -j \sum_{b} \mu_b\\
Z =& \sum_{\sigma} e^{- j \beta\sum_i  \sigma_i \sigma_{i+1}} \text{, Partition function}\\
=& 2 \sum_{\mu} e^{- j \beta  \sum_b \mu_b} \text{, allowing for $\sigma_1$}\\
=& \big(2 \cosh(\beta j)\big)^{N-1}\text{, by the same argument as (\ref{eq:E:cosh:beta:j})--}\numberthis \label{eq:Z:cosh}
\end{align*}

\begin{align*}
\big< \mu \big> =& \big< \sigma_i \sigma_{i+1} \big>\\
=& \tanh(\beta j) \textit{, prove this!}\text{ Correlation between neighbours}\numberthis \label{eg:Z_cosh}
\end{align*}

\begin{align*}
\big < \sigma_i \sigma_{i+n} \big> =&  \big < \sigma_i \sigma_{i+1} \sigma_{i+1}... \sigma_{i+n} \big>\text{, since $\sigma_i^2 = 1$}\\
=& \mu_1 \mu_2 ... \mu_{n-1}\\
=& \big( \tanh(\beta j)\big)^{n-1}\text{, from (\ref{eg:Z_cosh}), since $\mu_i$ independent}
\end{align*}

If $T=0(\beta=\inf)$, $\tanh(..)=\pm1$, otherwise $\lvert \tanh(..) \rvert<1$ and $\tanh(..)^n$ becomes arbitrarily small as $n$ increases.

\begin{itemize}
	\item Note the duality. We have found an equivalence between system of links and system of spins.
	\item We also see that there can be no magnetization in 1 dimensional system, except for $T=0$. C.f. "Chinese whispers". Likely to be long chains of agreement, then switch.
\end{itemize}

\subsection{Phase transitions in the Ising model}

There is no \gls{gls:phase:transition} in the 1D Ising model. For two or more dimensions, there are several paths along which information about one site can propagate, so "Chinese whispers" might not apply.

We will use a "mean field" approximation. In $d$ dimensions, where $d$ is very large, each site has $2d$ neighbours. We expect fluctuations to be much smaller than actual value. Let us assume bias; average spin is $\bar{\sigma}$.

\begin{align*}
E =& -j \sigma \sum_{neighbours} \sigma_{neighbour} \text{, for one spin.}\\
=& -j \sigma \sum_{neighbours} \bar{\sigma}\text{, where $\bar{\sigma}$ is average: mean field approximation.}\\
\approxeq &-2 d j \sigma \bar{\sigma}\text{. This gets more accurate as $d$ increases.} \numberthis \label{eq:E:sigma}\\
Z =& 2 \cosh(\beta 2 d j \bar{\sigma}) \text{--compare (\ref{eq:Z:cosh})}\\
\bar{\bar{\sigma}} =& \tanh\big(\underbrace{2 \beta d j  \bar{\sigma}}_\text{$=y$, say}\big) \text{, where $\bar{\bar{\sigma}}$ is expected $\sigma$} \\
\bar{\bar{\sigma}} =& \bar{\sigma} \text{, ''self-consistent'' approximation. We now solve:}\\
\frac{T y}{2 d j} =& \tanh(y)\numberthis \label{eq:tanh_mean_field}
\end{align*}

Figure \ref{fig:tanh_mean_field} exhibits a phase-transition at $T=2dj$; a non zero solution for $\bar{\sigma}$ appears.
\begin{itemize}
	\item If T is very large there is no solution other than $y=0\implies\bar{\bar{\sigma}}=0$
	\item Reduce $T$ until $\frac{T}{2 d j}= 1$, then straight line is a tangent to $\tanh(y)$
	\item If $\frac{T}{2 d j}< 1$, there a a non-zero solution to (\ref{eq:tanh_mean_field}): system has a global non-zero average spin. Zero is a solution, but tiniest outside magnetization will bias result.
\end{itemize}

\begin{figure}[H]
	\caption{Solutions to mean field equation (\ref{eq:tanh_mean_field})}\label{fig:tanh_mean_field}
	\includegraphics[width=0.8\textwidth]{tanh_mean_field}
\end{figure}

Why doesn't this work for $d=1$?

Start with T=0: everything aligned--Figure \ref{fig:T:zero:aligned}.

\begin{figure}[H]
	\centering
	\caption{Ground State, 1D}\label{fig:T:zero:aligned}
	\includegraphics[width=0.7\textwidth]{T_zero_aligned}
\end{figure}

 We will compute the partition function--(\ref{eq:partition:function}). Starting from $T=0$, Figure \ref{fig:T:zero:aligned} is the dominant configuration, or in 2D, Figure \ref{fig:2d:t0}.

 If we flip one spin in Figure \ref{fig:2d:t0} we increase energy by 4; we can do this at any point in lattice. If we flip two spins, energy increases by 6 if the spins are adjacent--Figure \ref{fig:2d:adjacent}--or 8 if they are apart; once again, there are many ways to do this.

 \begin{figure}[H]
 	\caption{Two dimensional Ising Model}
 	\begin{subfigure}[b]{0.45\textwidth}
 		\centering
 		\caption{Ground State}
 		\includegraphics[width=\textwidth]{ising2D0}\label{fig:2d:t0}
 	\end{subfigure}
 	\hfill
 	\begin{subfigure}[b]{0.45\textwidth}
 		\centering
 		\caption{Flip two adjacent spins}\label{fig:2d:adjacent}
 		\includegraphics[width=\textwidth]{ising2D2a}
 	\end{subfigure}
 \end{figure}

Figure \ref{fig:1D:flip_adjacent} shows that we break only two bonds when any number of adjacent spins are flipped. There are therefore many states that are close to Ground State in 1d case: this is the bases of the reason why the mean field approximation doesn't work for 1D.

\begin{figure}[H]
	\center
	\caption{1D flip adjacent spins}\label{fig:1D:flip_adjacent}
	\includegraphics[width=0.7\textwidth]{T_zero_aligned_flip}
\end{figure}

Returning to 2D, lets apply a small magnetic field $B$ favouring up-spin. Then (\ref{eq:E:sigma}) becomes:
\begin{align*}
	E =& -2 d j \sigma \bar{\sigma} - B\sigma\\
	=& 	\big[-2 d j  \bar{\sigma} - B\big] \sigma \text{, and (\ref{eq:tanh_mean_field}) becomes}\\
	\frac{T y}{2 d j} =& \tanh(y +B\beta)\numberthis\label{eq:tanh_mean_field_B}
\end{align*}

Figure \ref{fig:tanh_mean_field_B} illustrates the solution to (\ref{eq:tanh_mean_field_B}). There is no longer a solution $\sigma=0$ for finite $T$.
\begin{figure}[H]
	\caption[Solving Mean Field equation]{Solving $\frac{T y}{2 d j} = \tanh(y +B\beta)$}\label{fig:tanh_mean_field_B}
	\includegraphics[width=0.9\textwidth]{tanh_mean_field_B}
\end{figure}

\section{Liquid-gas phase transition}

\subsection{Ising model continued}

We will use  Ising to model the liquid-gas phase transition; this isn't the way that Van der Waals modelled it, but it gives equivalent results. NB mean field is not accurate for 1D: the more neighbours a site has, the more accurate is mean field.

Energy was stored in the links between sites. An unbroken bond(parallel) has low energy, broken bonds have high. Every broken bond costs energy $2J$.

\begin{align*}
E =& -J \sum_{links} \sigma(i) \sigma(j) + \sum_{i}h\sigma(i)
\end{align*}

2D lattice has 2 links per site.

Ground state energy not important; it is the difference between energy levels that matters.

Our mean field equation (\ref{eq:tanh_mean_field}) becomes
\begin{align*}
\frac{ y}{2 d j \beta} =& \tanh(y - \beta h)
\end{align*}

When $h=0$, Figure \ref{fig:tanh_mean_field} shows two solution for sufficiently small T.

\begin{itemize}
	\item If T is very large there is no solution other than $y=0\implies\bar{\bar{\sigma}}=0$
	\item Reduce $T$ until $\frac{T}{2 d j}= 1$, then straight line is a tangent to $\tanh(y)$. We define $T_{crit}= 2 d j$
	\item If $\frac{T}<T_{crit}$, there a a non-zero solution to (\ref{eq:tanh_mean_field}): system has a global non-zero average spin. Zero is a solution, but tiniest outside magnetization will bias result.
	\item $T_{crit} = 2 d J$ is poor for $d=2$, good ($\pm10\%$) for $d=3$
\end{itemize}

Figures \ref{fig:TgtTcrit} and \ref{fig:TltTcrit} shows the difference in behaviour at the critical point as $h$ crosses the horizontal axis. NB, we can avid this jump if we heat, and go around $T_{crit}$.

\begin{figure}[H]
	\caption{Phase transition at $T_{crit}$}\label{fig:Phase:transition:magnetic}
	\begin{subfigure}[b]{0.45\textwidth}
		\center
		\caption{$T>T_{crit}$: as $h$ crosses from -ve to +ve, $\sigma$ changes continuously}\label{fig:TgtTcrit}
		\includegraphics[width=\textwidth]{TgtTcrit}
	\end{subfigure}
	\hfill
	\begin{subfigure}[b]{0.45\textwidth}
		\center
		\caption{$T<T_{crit}$: as $h$ crosses from -ve to +ve, $\sigma$ changes from -1 to +1}\label{fig:TltTcrit}
		\includegraphics[width=\textwidth]{TltTcrit}
	\end{subfigure}
\end{figure}

\subsection{Liquid-gas phase transition}

We will apply Ising model to Liquid-gas phase transition; it won't work for solid-liquid. Imagine that a box, and that a particle inside the box has less energy than one outside--Figure \ref{fig:box1}; that energy is known as the \gls{gls:chemical_potential}--\glssymbol{gls:chemical_potential}. In Figure \ref{fig:box2}, particles inside box have lower energy, so the Boltzmann distribution will favour them.

\begin{figure}[H]
	\caption{Box of fluid}
	\begin{subfigure}[b]{0.45\textwidth}
		\center
		\caption{Imagine that a particle inside the box has less energy than one outside}\label{fig:box1}
		\includegraphics[width=\textwidth]{box1}
	\end{subfigure}
	\hfill
	\begin{subfigure}[b]{0.45\textwidth}
		\center
		\caption{Walls are permeable, so we can have particles inside \& outside.}\label{fig:box2}
		\includegraphics[width=\textwidth]{box2}
	\end{subfigure}
\end{figure}

What is density of fluid as a function of the depth of the potential? We start with the partition function.

\begin{align*}
	Z(\beta) = \sum_{P,X,N}e^{- \beta E + N \mu \beta} \text{, N is number of particles in well}
\end{align*}

Vary $\mu$. Phase transition is a discontinuity in energy.

What are the conditions required to have a liquid-gas phase transition?

\begin{itemize}
	\item There must be a hard core repulsion between molecules.
	In fact, molecules always have hard core repulsion.
	You cannot stick two molecules into the same site.
	\item  There must be a little bit
	of attraction between molecules when there are not
	quite touching.
\end{itemize}

In Figure \ref{fig:lattice} we model the fluid as a lattice. If there is no particle at a point, set $\sigma=-1$, otherwise $\sigma=+1$. Since the magnet forbids any other value of $\sigma$, it prevents two particles occupying the same cell.

We will use for the energy exactly the Ising model
of energy plus a little magnetic field. And we shall see how
this translates in the language of our particles of fluid which
are hard core and have short range attraction.

\begin{figure}[H]
	\caption{Fluid modelled as a lattice}\label{fig:lattice}
	\begin{subfigure}[b]{0.3\textwidth}
		\center
		\caption{Ground State}\label{fig:lattice0}
		\includegraphics[width=\textwidth]{lattice0}
	\end{subfigure}
	\hfill
	\begin{subfigure}[b]{0.3\textwidth}
		\center
		\caption{One molecule}
		\label{fig:lattice1}
		\includegraphics[width=\textwidth]{lattice1}
	\end{subfigure}
	\hfill
	\begin{subfigure}[b]{0.3\textwidth}
		\center
		\caption{Bonds.}
		\label{fig:lattice2}
		\includegraphics[width=\textwidth]{lattice2}
	\end{subfigure}
\end{figure}

For an empty lattice, Figure \ref{fig:lattice0} we find:
\begin{align*}
	E =& -J \sum_{links} \sigma_i \sigma_j - \underbrace{2 j N_{links}}_\text{Ground state energy}\text{, but we can ingore constant term, giving}\\
	=& -J \sum_{links} \sigma_i \sigma_j\numberthis \label{eq:empty_lattice}
\end{align*}

\begin{itemize}
	\item If we add one particle, Figure \ref{fig:lattice1} we have four bonds, Figure \ref{fig:lattice2}, so we add energy $8J$.
	\item Two particles that aren't adjacent give $16J$, even if they are diagonals
	\item Two adjacent gives $12J$.
\end{itemize} Note that there is a chemical potential, since on particle gives $8J$. But we want to be able to vary potential.

\begin{align*}
 	E =& -J \sum_{links} \sigma_i \sigma_j + \underbrace{ h \sum_{sites}\sigma_i}_\text{adds $2h$ each time we add one particle} \text{, modifying (\ref{eq:empty_lattice})} \numberthis\label{eq:lattice:chem:potential}
\end{align*}

So the energy to introduce an isolated particles becomes $8J+2h$.

Now we have everything we need to study the liquid-gas
transition in a fluid of molecules. From our Ising magnet
model we have built a system that is equivalent to a col-
lection of particles on a lattice. It has the following three
specifications:
\begin{enumerate}
	\item It has a variable chemical potential. Notice how we
	vary it. We do it by varying the magnetic field. The
	magnetic field gave us an energy per particle.
	\item It has the hard core billiard ball potential which is
	repulsive.
	\item And it has a short-range attraction, the coefficient of
	which is 4j (going from 16j when the two particles
	are far apart, to 12j when they are separated by only
	one link).
\end{enumerate}

We want to keep the molecular properties fixed, therefore
we are not going to play with j. That has to do with the
molecular properties. By design, it tells us what the potential energy between them is.

All we can vary is the chemical potential. And that is the
same thing in our model as varying h.
So the problem of the behaviour of the molecules of a fluid,
with the three specs listed above, has become exactly the
same as the previous problem of a magnet plunged into an
external field, which was summarized in Figure \ref{fig:Phase:transition:magnetic}. They
obey precisely the same model.

There is a temperature that we can vary. Particles are at
some temperature. And there is a chemical potential that
we can vary. Now $h$ is not exactly the chemical potential.
There is an offset by an amount $8j$.

What about the particle density? We haven’t discussed it
yet. So let’s do it. How many particles are there, on the
average, at a particular point? The answer is somewhere,
first of all, between 0 and 1. Of course the number of particle is either 0 or 1, but we are talking about the average.

The average number of particles at a site is $\frac{1+\bar{\sigma}}{2}$.

\begin{itemize}
	\item If $T>T_{crit}$, there is no jump as we vary chemical potential (by varying pressure);
		\item If $T<T_{crit}$, there is a jump as we vary chemical potential-- boiling or condensation.
\end{itemize}

\appendix
\printglossaries

\bibliographystyle{unsrt}
\addcontentsline{toc}{section}{Bibliography}
\bibliography{tm}

\end{document}


\section{Global Optimizations}
\setlength{\parindent}{0pt}

(Prepared by Aditya Senthilnathan)

\vspace{0.3cm}

Global optimisations go beyond the scope of a single basic block. It needs reasoning of the whole control flow graph which may be within the body of a method.

% Example 1
% Example 2

\subsection{Global Constant Propagation}

\begin{itemize}
    \item This is similar to local constant propagation but now we are reasoning in a global level (across several basic blocks within the body of a method)
    \item In this optimisation, we basically ask the question "When can we replace the use of a variable \textbf{x} with a constant \textbf{k}

    Ans: If on \textbf{every path} to the use of x, if the \textbf{last assignment} to x is \textbf{x:=k}, then we can do this transformation/optimisation.
\end{itemize}

% Example 3

The algorithm for global constant propagation requires a class of algorithms which are called "Global Dataflow Analysis". This kind of reasoning (with paths, etc.) is required for many different types of optimisation problems in compilers. And so, it helps to create a common framework in which we can implement the same logic but with different kinds of properties so that there is reuse of the same idea.

Let's identify some traits that global optimisations tasks share:

\begin{itemize}
    \item \underline{Optimisation depends on knowing property X at a certain program point P}

    Eg. In global constant propagation, we were interested in knowing if x:=k is the last assignment on all possible paths, at the program point of the last basic block (where x is used) in our example graph.

    \item \underline{Proving X at P requires knowledge of entire program}

    Eg. In our global constant propagation example, to be able to say x:=k is the last assignment to x on all possible paths that can reach program point P, we need to know the characteristics of the entire program. As opposed to local optimistaion where you only need info about the concerned basic block. Here, even though transformation is only made in a basic block, it relies on a property that requires knowledge of the entire program.

    \item \underline{It's OK to be conservative}

    We can either say,
    \begin{center}
        \textit{X is definitely true}
    \end{center}
    or say,
    \begin{center}
        \textit{We don't know if X is true}
    \end{center}

    We want to get "X is definitely true" as much as possible because only then can we apply the transformation but if we get the conservative answer "We don't know if X is true", then we don't apply the transformation to preserve program correctness. Knowing if X is definitely not true is useless because in this case we still don't apply the transformation and so this use case is clubbed with "We don't know if X is true".
\end{itemize}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%        REPORT OF THE PROJECT
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass[a4paper,11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[english]{babel}
\usepackage{amsmath,amssymb,amsthm,amsopn}
\usepackage{mathrsfs}
\usepackage{graphicx}
%\usepackage{tikz}
%\usepackage{array}
%\usepackage[top=1cm,bottom=1cm]{geometry}
%\usepackage{listings}
%\usepackage{xcolor}
\usepackage{hyperref}
\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    citecolor=red,
}

% fancy headers and footers

%\usepackage{fancyhdr}
%\pagestyle{fancy}
%\fancyhead[L]{BCPST 2 - Lycée Jacques Prévert}
%\fancyhead[R]{Rappels d'analyse}
%\pagenumbering{gobble} % no page numbering

% Création des labels Théorème, Lemme, etc...

\newtheoremstyle{break}%
{}{}%
{\itshape}{}%
{\bfseries}{}%  % Note that final punctuation is omitted.
{\newline}{}

\newtheoremstyle{sc}%
{}{}%
{}{}%
{\scshape}{}%  % Note that final punctuation is omitted.
{\newline}{}

\theoremstyle{break}
\newtheorem{thm}{Theorem}[section]
\newtheorem{lm}[thm]{Lemma}
\newtheorem{prop}[thm]{Proposition}
\newtheorem{cor}[thm]{Corollary}

\theoremstyle{sc}
\newtheorem{exo}{Exercice}

\theoremstyle{definition}
\newtheorem{defi}[thm]{Definition}
\newtheorem{ex}[thm]{Example}

\theoremstyle{remark}
\newtheorem{rem}[thm]{Remark}

% Raccourcis pour les opérateurs mathématiques (les espaces avant-après sont modifiés pour mieux rentrer dans les codes mathématiques usuels)
\DeclareMathOperator{\Ker}{Ker}
\DeclareMathOperator{\Id}{Id}
\DeclareMathOperator{\Img}{Im}
\DeclareMathOperator{\Card}{Card}
\DeclareMathOperator{\Vect}{Vect}
\DeclareMathOperator{\Tr}{Tr}
\DeclareMathOperator{\Mod}{mod}
\DeclareMathOperator{\Ord}{Ord}
\DeclareMathOperator{\lcm}{lcm}

% Nouvelles commandes
\newcommand{\ps}[2]{\left\langle#1,#2\right\rangle}
\newcommand{\ent}[2]{[\![#1,#2]\!]}
\newcommand{\diff}{\mathop{}\!\mathrm{d}}
\newcommand{\ie}{\emph{i.e. }}

% opening
\title{Normal bases generation in C}
\author{Édouard \textsc{Rousseau}\\Supervised by Michaël \textsc{Quisquater}}

\begin{document}

\maketitle

\begin{abstract}
  This is the report of a C project about the generation of normal elements in
  finite fields. Consider a field extension $\mathbb{F}_{p^d}/\mathbb{F}_p$, we
  say that $\alpha\in\mathbb{F}_{p^d}$ is normal if $\left\{
  \alpha,\alpha^p,\alpha^{p^2},\cdots,\alpha^{p^{d-1}} \right\}$ is a basis of
  $\mathbb{F}_{p^d}$ over $\mathbb{F}_p$. We first give some theory to
  characterize normal elements, then we describe three algorithms to compute
  normal elements : a randomized algorithm, Lüneburg's algorithm, and
  Lenstra's algorithm. Finally, we give experimental results about our
  implementation. All this work is based on Gao's PhD thesis~\cite{Ga93}.

\end{abstract}

\tableofcontents

\clearpage

\section{Introduction}
\subsection{Normal bases}
With the rise of electronics and computer science, areas like
cryptography and coding theory have been largely studied. In both these
domains, finite fields often have a fundamental role. Normal bases can be
used to implement efficiently the finite fields arithmetic in the hardware,
consumming less power than other bases. But normal bases can also be used as a
theoretic tools to understand field extensions, and it is probably why normal bases were
studied since the $19^\textrm{th}$ century. Gauss, for example, used normal
bases to study when regular polygons can be drawn with ruler and compass
alone, a \emph{very} old problem. The notion of normal bases is not linked
with finite fields, if $\mathbb{K}$ is a field and $\mathbb{L}$ is a finite
Galois extension of $\mathbb{K}$ of Galois group $G$, a normal basis of
$\mathbb{L}$ over $\mathbb{K}$ is a basis of the form $\left\{
  \sigma(\alpha)\;|\;\sigma\in G
\right\}$ where $\alpha\in\mathbb{L}$. In other words, it is a basis composed
of an element $\alpha$ and all its conjugates. In the case of finite fields,
the definition that we give is indeed the same as this one, since, in this case,
$G$ is generated by the Frobenius automorphism. Given a finite Galois extension
$\mathbb{L}/\mathbb{K}$, normal bases can also be used to realize the Galois
correspondence of $\mathbb{L}/\mathbb{K}$. Theory tells us that there exists a
correspondence between intermediate extensions of $\mathbb{L}/\mathbb{K}$ and subgroups
of $G$, but it does not give an \emph{effective} way of realising that
correspondence. Normal bases are a way to solve that problem. In this report, we
focus on the problem of finding normal elements in finite fields. That is
why, from now on, we do not look at cases other then finite extensions
of finite fields (\ie extensions of type
$\mathbb{F}_{q^d}/\mathbb{F}_q$).
\subsection{Recalls and notations}
In all the text, we denote by $\mathbb{F}_n$ the field with $n$ elements. We
recall that $n$ must be a \emph{prime power} (\ie $n=p^d$ where $p$ is a prime
number and $d$ is a positive number). We have
$\mathbb{F}_{p^d}\cong\mathbb{F}_p[X]/(P)$, where $P$ is an irreducible
polynomial of degree $d$ in $\mathbb{F}_p[X]$, and this is the representation
that will be used in the C code. From now, we note
$X$ both for the indeterminate $X$ and for its image $\bar X$ in the
quotient $\mathbb{F}_p[X]/(P)$. $\mathbb{F}_{p^d}$ is a vector space over
$\mathbb{F}_p$, of dimension $d$, the basis $\left\{ 1, X, X^2, \dots,
X^{d-1} \right\}$ is called the \emph{polynomial basis} in the following. We also recall
that $\mathbb{F}_{p^d}$ is a \emph{field extension} of $\mathbb{F}_p$ and the
characteristic of $\mathbb{F}_{p^d}$ is $p$. Lastly, we denote by $\sigma$ the
\emph{Frobenius map}, defined by $\sigma(x)=x^p$ for all
$x\in\mathbb{F}_{p^d}$. This map is a $\mathbb{F}_p$-automorphism of
$\mathbb{F}_{p^d}$ (\ie $\sigma$ is a field morphism, a bijection, and $\forall
y\in\mathbb{F}_p,\;\sigma(y)=y$), as a consequence, $\sigma$ is also a linear
map.

Note that we look only
at \emph{prime extensions}, (\ie extensions of type
$\mathbb{F}_{p^d}/\mathbb{F}_p$). This choice has been made because the theory
of normal elements in extensions of type $\mathbb{F}_{q^n}/\mathbb{F}_q$ is not
different (replacing $p$ by $q$ in the following demontrations would be
sufficient), but the implementation of the algorithms is simpler in the case of
prime extensions. Speaking about implementation, we must introduce Flint
(Fast Library for Number Theory), because \emph{every} function wrote in this
project uses Flint.

\subsection{Flint}

Flint~\cite{Ha10} is a C library maintained by William Hart, and developed
by him and many others since 2007. In Flint, we use the type
\texttt{fq\_t}, where the elements of $F_{p^d}\cong\mathbb{F}_p[X]/(P)$ are
represented as polynomials of degree less than $d$. The underlying data
structure is \texttt{fmpz\_poly\_t}, that is a \texttt{struct} containing
\begin{enumerate}
  \item a pointer to arbitrary large integers \texttt{fmpz}, representing the
    coefficients of the polynomial;
  \item the number of memory allocations for the pointer of type
    \texttt{slong}: Flint's own \texttt{unsigned long};
  \item the degree of the polynomial, a \texttt{slong} too.
\end{enumerate}
In Flint, it is possible to work with finite fields of arbitrary large
order and with polynomials or matrices over these fields. The basic functions
are already implemented, such as gcd or derivative for polynomials, or reduced
row echelon form for matrices. Using Flint save a lot of time, but it also has
its limits. It is possible to deal with arbitrary \emph{prime}
extensions. For extension of type $\mathbb{F}_{q^n}/\mathbb{F}_q$, it is still
possible, using polynomial arithmetic over $\mathbb{F}_q$. But using
polynomials
over $\mathbb{F}_{q^n}$ would have required polynomial in two
indeterminates, and everything would be more complicated. That is the reason we
look only at prime extensions.

All these recalls being made, and the true hero
(Flint) of our functions being introduced, we can begin our journey.

\section{Basics on normal bases}

In all this section, we set $p$ a prime number and $d$ a positive number. We
work with the extension $\mathbb{F}_{p^d}/\mathbb{F}_p$ and the Frobenius
morphism $\sigma$ defined above. We are now able to give a formal definition
of a normal element and a normal basis.

\begin{defi}[normal element, normal basis]
  Let $\alpha\in\mathbb{F}_{p^d}$, we say that $\alpha$ is a \emph{normal
  element} if $\left\{ \alpha, \sigma(\alpha),\dots,\sigma^{d-1}(\alpha)
  \right\}=\left\{ \alpha, \alpha^{p}, \dots, \alpha^{p^{d-1}} \right\}$ is a
  basis of $\mathbb{F}_{p^d}$. Such a basis is called a \emph{normal basis}.
\end{defi}

In order to recognize a normal element $\alpha$, we can compute the dimension of the
linear span of $\left\{ \sigma^i(\alpha) \;|\; 0\leq i < k \right\}$. A way of
doing that is to construct the matrix $M$ which columns are the coordinates of the
elements $\sigma^i(\alpha)$ in the polynomial basis, and to check if $M$ is
non-singular. This can be done using Gauss algorithm. This method is not
efficient so we work on other characterizations of normal elements. We need
two more definitions to be able to state our results.

\begin{defi}[trace function]
  The \emph{trace function}
  $\Tr_{p^d|p}:\mathbb{F}_{p^d}\to\mathbb{F}_p$ of the extension
$\mathbb{F}_{p^d}/\mathbb{F}_p$ is the function defined by
\[
  \Tr_{p^d|p}(\alpha) = \sum_{i=0}^{d-1}\alpha^{p^i}.
\]
It takes its values in $\mathbb{F}_p$ since $(\Tr_{p^d|p}(\alpha))^p
= \Tr_{p^d|p}(\alpha)$.
\end{defi}

The trace is a tool that permit to know when a family of elements $\alpha_1,
\dots, \alpha_d$ forms a basis of $\mathbb{F}_{p^d}$ over
$\mathbb{F}_p$, using the discrinant of these elements.

\begin{defi}[discriminant]
  Let $\alpha_1, \dots, \alpha_d$ be elements in $\mathbb{F}_{p^d}$, the
  \emph{discriminant} $\Delta(\alpha_1, \dots, \alpha_d)$ of these elements
  is the determinant
  \[
    \Delta(\alpha_1, \dots, \alpha_d)=\det
    \begin{pmatrix}
      \Tr(\alpha_1\alpha_1) & \Tr(\alpha_1\alpha_2) & \dots &
      \Tr(\alpha_1\alpha_d) \\
      \Tr(\alpha_2\alpha_1) & \Tr(\alpha_2\alpha_2) & \dots &
      \Tr(\alpha_2\alpha_d) \\
      \vdots & \vdots & & \vdots \\
      \Tr(\alpha_d\alpha_1) & \Tr(\alpha_d\alpha_2) & \dots &
      \Tr(\alpha_d\alpha_d)
    \end{pmatrix}
  \]
  where $\Tr$ is the same trace as before. We ommit the indices because
  the extension is always the same.
\end{defi}

With this last tool, we can state our first theorem.

\begin{thm}[Theorem 2.2.1, \cite{Ga93}]
  For any $n$ elements $\alpha_1, \dots, \alpha_d$ in
  $\mathbb{F}_{p^d}$, they form a basis of $\mathbb{F}_{p^d}$ over
  $\mathbb{F}_p$ if and only if $\Delta(\alpha_1, \dots,
  \alpha_d)\neq0$.
\end{thm}

\begin{proof}
  First assume that $\alpha_1, \dots, \alpha_d$ form a basis of
  $\mathbb{F}_{p^d}$ over $\mathbb{F}_p$. We prove that
  $\Delta(\alpha_1, \dots, \alpha_d)\neq0$ by showing that the row vectors
  $L_1, \dots, L_d$ of
  the matrix in the definition of $\Delta(\alpha_1, \dots, \alpha_d)$ are
  linearly independent over $\mathbb{F}_p$, and thus the matrix is
  nonsingular. Suppose that there exists $c_1, \dots,
  c_d\in\mathbb{F}_p$ with $\sum_i c_iL_i = 0$, it means
  \[
    c_1\Tr(\alpha_1\alpha_j) + \dots + c_n\Tr(\alpha_d\alpha_j) =
    0\textrm{ for }1\leq j\leq d.
  \]
  Then with $\beta=c_1\alpha_1 + \dots + c_d\alpha_d$, by linearity of the
  trace function, we get $\Tr(\beta\alpha_j) = 0$ for $1\leq j\leq d$. Since
  $\alpha_1, \dots, \alpha_d$ is a basis of $\mathbb{F}_{p^d}$, it follows
  that $\Tr(\beta\alpha) = 0$ for all $\alpha\in\mathbb{F}_{p^d}$. If
  $\beta\neq0$, it means that $\Tr(\gamma)=0$ for all
  $\gamma\in\mathbb{F}_{p^d}$ : we can see that this is impossible by thinking
  of $\Tr(\gamma)$ as the trace of the multiplication-by-$\gamma$ linear map. So we have $\beta=0$, and then
  $c_1\alpha_1 + \dots + c_d\alpha_d = 0$ implies $c_1=\dots=c_d=0$.

  Conversely, assume that $\Delta(\alpha_1, \dots, \alpha_d)\neq0$ and
  $c_1\alpha_1+\dots+c_d\alpha_d=0$ for some $c_1, \dots,
  c_d\in\mathbb{F}_p$. By multiplying by $\alpha_j$, we have
  \[
    c_1\alpha_1\alpha_j+\dots+c_d\alpha_d\alpha_j\textrm{ for }1\leq j \leq d,
  \]
  and by applying the trace function, we get
  \[
    c_1\Tr(\alpha_1\alpha_j)+\dots+c_n\Tr(\alpha_d\alpha_j)\textrm{ for }1\leq
    j\leq d.
  \]
  But this is the same as $\sum c_iL_i=0$ where the $L_i$ are the rows of the
  matrix in $\Delta(\alpha_1, \dots, \alpha_d)$. Since this matrix is
  nonsingular by hypothesis, its rows are linearly independent and it follow
  that $c_1=\dots=c_d$. Therefore $\alpha_1, \dots, \alpha_d$ are linearly
  independant over $\mathbb{F}_p$, and form a family of $d$ vectors in a
  $d$-dimensionnal vector space, so $\alpha_1, \dots, \alpha_d$ form  basis of
  $\mathbb{F}_{p^d}$ over $\mathbb{F}_p$.
\end{proof}

This characterization is interesting in itself, but the matrix defined in
$\Delta(\alpha_1, \dots, \alpha_d)$ is quite complicated. Fortunately,
we can use a much simpler matrix.

\begin{cor}
  \label{circulantCor}
 The elements $\alpha_1, \dots, \alpha_d$ form a basis of
 $\mathbb{F}_{p^d}$ over $\mathbb{F}_p$ if and only if the matrix $A$ is
 nonsigular, where
 \[
   A =
   \begin{pmatrix}
     \alpha_1 & \alpha_2 & \dots & \alpha_d \\
     \alpha_1^p & \alpha_2^p & \dots & \alpha_d^p \\
     \vdots & \vdots & & \vdots \\
     \alpha_1^{p^{d-1}} & \alpha_2^{p^{d-1}} & \dots &
     \alpha_d^{p^{d-1}}.
   \end{pmatrix}
 \]
\end{cor}
\begin{proof}
  We have $\Delta(\alpha_1, \dots, \alpha_d)=\det(A^tA)=(\det A)^2$.
\end{proof}

We now have a simpler matrix to study, but it is still a matrix, so the nature
of the problem is the same as before. Next lemma allow us to change the nature
of the problem, by working with polynomials.

\begin{lm}
  \label{henselLm}
  For any $n$ elements $a_0, a_1, \dots, a_{d-1}\in\mathbb{F}_{p^d}$, the
  $d\times d$ circulant matrix
  \[
    c[a_0, a_1, \dots, a_{d-1}] =
    \begin{pmatrix}
      a_0 & a_1 & a_2 & \dots & a_{d-1} \\
      a_{d-1} & a_0 & a_1 & \dots & a_{d-2} \\
      a_{d-2} & a_{d-1} & a_0 & \dots & a_{d-3} \\
      \vdots & \vdots & \vdots & & \vdots \\
      a_1 & a_2 & a_3 & \dots & a_0
    \end{pmatrix}
  \]
  is nonsigular if and only if the polynomial $\sum a_iX^i$ is relatively
  prime to $X^d-1$.
\end{lm}
\begin{proof}
  Let $A$ be the following $d\times d$ permutation matrix
  \[
    \begin{pmatrix}
      0 & 1 & 0 & \dots & 0 & 0 \\
      0 & 0 & 1 & \dots & 0 & 0 \\
      \vdots & \vdots & \vdots & & \vdots & \vdots \\
      0 & 0 & 0 & \dots & 0 & 1 \\
      1 & 0 & 0 & \dots & 0 & 0
    \end{pmatrix}.
  \]
  Then we see that $c[a_0, a_1, \dots, a_{d-1}] =
  \sum_{i=0}^{d-1}a_iA^i=f(A)$, where $f(X)=\sum_{i=0}^{d-1}a_iX^i$. On the
  first line of $A^i$, there is only one nonzero value, it is a $1$ in
  position $i+1$. Hence the matrices $A^0, A, \dots, A^{d-1}$, are linearly
  independant, and since $A^d=I_d$, we know that the minimal polynomial
  of $A$ is $X^d-1$. Assume that $f(X)$ is relatively prime to $X^d-1$. Then
  there are polynomials $a(X), b(X)$ such that
  \[
  a(X)f(X) + b(X)(X^d-1)=1,
\]
and so
\[
  a(A)f(A) = I_d,
\]
as $A^d-I_d=0$. This implies that $f(A)$ is invertible and so
nonsigular. Now assume that $f(X)$ and $X^d-1$ are not relatively prime, and
note $d(X)\neq1$ their gcd. Let $f(X)=f_1(X)d(X)$ and $X^d-1=h(X)d(X)$.
Since $\deg h < n$, we have $h(A)\neq0$. But $h(A)d(A)=0$, so we see that
$d(A)$ is singular. Therefore $f(A)=f_1(A)d(A)$ is also singular. We have shown
that $f(A)$ is singular if and only if $f(X)$ is relatively prime to $X^d-1$.
\end{proof}

We are now able to state the last result oh this section.

\begin{thm}[Hensel, 1888]
  \label{hensel}
  Let $\alpha\in\mathbb{F}_{p^d}$, $\alpha$ is a normal element of the
  extension $\mathbb{F}_{p^d}/\mathbb{F}_p$ if and only if the polynomial
  $\alpha^{p^{d-1}}X^{d-1}+\dots+\alpha^pX+\alpha\in\mathbb{F}_{p^d}[X]$
  is relatively prime to $X^d-1$.
\end{thm}
\begin{proof}
  $\alpha$ is a normal element, if, by definition, the elements $\alpha,
  \alpha^p, \dots, \alpha^{p^{d-1}}$ form a basis of
  $\mathbb{F}_{p^d}$ over $\mathbb{F}_p$. By
  Corollary~\ref{circulantCor}, this is
  true if and only if the matrix
  \[
    A =
    \begin{pmatrix}
      \alpha & \alpha^p & \alpha^{p^2} & \dots & \alpha^{p^{d-1}} \\
      \alpha^p & \alpha^{p^2} & \alpha^{p^3} & \dots & \alpha \\
      \vdots & \vdots & \vdots & & \vdots \\
      \alpha^{p^{d-1}} & \alpha & \alpha^p & \dots & \alpha^{p^{d-2}}
    \end{pmatrix}
  \]
  is nonsingular. But, reversing the order of the rows in $A$, from the second
  row to the last, we get the matrix $c[\alpha, \alpha^p, \dots,
  \alpha^{p^{d-1}}]$, that is nonsingular if and only if $A$ if
  nonsingular. By Lemma~\ref{henselLm}, $c[\alpha, \alpha^p, \dots,
  \alpha^{d-1}]$ is nonsingular if and only if $X^d-1$ and
  $\alpha^{p^{d-1}}X^{d-1}+\dots+\alpha^pX+\alpha$ are relatively prime.
  This proves the theorem.
\end{proof}
We now have a new characterization of normal elements. We can decide if an
element is normal or not by computing a gcd. It is more efficient that the
naive matrix method because we have efficient algorithms to compute the gcd.
The function \texttt{is\_normal} is exactly the implementation of this
method. The code of this function, and of all the others, is available at
\url{github.com/erou/normalBases}.

\section{Computation of normal bases}
Before trying to compute normal elements, a natural question is to wander if
normal elements always exist in extensions of type
$\mathbb{F}_{p^{d}}/\mathbb{F}_p$. We do not prove this result here (it is done
in~\cite{Ga93}), but the answer is yes. Now that this natural fear has been
eliminated, we give some ways to compute normal elements. We use the same
notations as in the previous section.

\subsection{Randomized algorithm}
First, we give a randomized algorithm. Such algorithms are often based on the
same strategy.
\begin{enumerate}
  \item Take a random element, following a certain protocol.
  \item Check if this element verify the wanted property.
\end{enumerate}
Our algorithm also uses this strategy, that is why our first function
\texttt{is\_normal} is very important. But the protocol is also essential.
For example, if we just take an element completly at random in
$\mathbb{F}_{p^d}$, our algorithm will not be very efficient. The following
theorem helps us defining a better protocol.
\begin{thm}
  Let $f(X)$ be an irreducible polynomial of degree $d$ over
  $\mathbb{F}_p$ and $\alpha$ a root of $f(X)$. Let
  \[
    g(X)=\cfrac{f(X)}{(X-\alpha)f'(\alpha)}.
  \]
  Then there are at least $p -d(d-1)$ elements $u$ in $\mathbb{F}_p$ such that
  $g(u)$ is a normal element of $\mathbb{F}_{p^d}$ over $\mathbb{F}_p$.
\end{thm}
\begin{proof}
  Let $\sigma_i=\sigma^i$ be the automorphism
  $\theta\to\theta^{p^i},\theta\in\mathbb{F}_{p^d}$, for $0\leq i <d$. Then
  $\alpha_i=\sigma_i(\alpha)$ is also a root of $f(X)$, for $0\leq i <d$. The
  automorphism $\sigma_i:\mathbb{F}_{p^d}\to\mathbb{F}_{p^d}$ can be extended
  into a ring morphism
  $\sigma_i:\mathbb{F}_{p^d}[X]\to\mathbb{F}_{p^d}[X]$ by setting
  $\sigma_i(X)=X$. We get
  \[
    g_i(X):=\sigma_i(g(X)) = \cfrac{f(X)}{(X-\alpha_i)f'(\alpha_i)},
  \]
  and we note that $\sigma_i\sigma_i(g(X)) = \sigma_{i+j}(g(X))$. Then
  $g_i(X)$ is a polynomial in $\mathbb{F}_{p^d}[X]$ having $\alpha_k$ as a root
  for $k\neq i$ and $g_i(\alpha_i)=1$. Hence, for $i\neq k$, we have that every
  root of $f(X)$ is also a root of $g_i(X)g_k(X)$, so
  \begin{equation}
    g_i(X)g_k(X) \equiv 0 \;(\Mod f(X)), \textrm{ for }i\neq k.
    \label{3.1}
  \end{equation}
  Note that
  \begin{equation}
    g_1(X)+g_2(X)+\dots+g_d(X)-1=0,
    \label{3.2}
  \end{equation}
  since the left side is a polynomial of degree at most $d-1$ (all the
  $g_i$
  are of degree $d-1$) and has $\alpha_0,
  \alpha_1, \dots, \alpha_{d-1}$ as roots. Multiplying (\ref{3.2}) by $g_i(X)$
  and using (\ref{3.1}) to simplify, we have
  \begin{equation}
    g_i(X)^2 \equiv g_i(X)\;(\Mod f(x)).
    \label{3.3}
  \end{equation}
  We next set the matrix
  \[
    D=(\sigma_{i+j}(g(X)))_{0\leq i,j <0}
  \]
  and we study its determinant, $\Delta(X)$. From the equations
  (\ref{3.1}), (\ref{3.2}) and (\ref{3.3}), we wee that the entries of $D^tD$
  molulo $f(X)$ are all $0$, except on the main diagonal where they are all
  $1$. It follows that
  \[
    \Delta(X)^2 = \det(D^tD) \equiv 1\;(\Mod f(x)).
  \]
  This proves that $\Delta(X)$ is a nonzero polynomial of degree at most
  $d(d-1)$, since it is a sum of $d$ products of polynomials of degree
  $d-1$. Therefore $\Delta(X)$ has at most $d(d-1)$ roots in
  $\mathbb{F}_p$. The result follows from Corollary~\ref{circulantCor}, since the matrix
  $D(u)$ is exactly the one defined in the corollary applied to the elements
  $g(u), \sigma(g(u)),\dots,\sigma^{d-1}(g(u))$.
\end{proof}
Now we have our protocol. We take $u\in\mathbb{F}_p$ at random, then we check
if $g(u)$ is normal. If $p$ is large enough, for example $p>2d(d-1)$, then
$g(u)$ is normal with probability at least $1/2$. We will not discuss it, but
the entire computation takes $O((n+\log q)(n\log q)^2)$ bit operations.
A reference for this result is given in~\cite{Ga93}. As always, we are not
\emph{completely} satisfied with randomized algorithm, so we give two
deterministic algorithms.

\subsection{Deterministic algorithms}

First, we define the $\sigma$-Order of an element
$\theta\in\mathbb{F}_{p^d}$, both Lenstra's and Lüneburg's algorithms use it.

\subsubsection{The $\sigma$-Order polynomials}

\begin{defi}[$\sigma$-Order]
  Let $\theta\in\mathbb{F}_{p^d}$ be an arbitrary element. Let $k$ be the least
  positive integer such that $\sigma^k(\theta)=\theta^{p^k}$ belongs to
  $\mathbb{F}_p$-linear span of $\left\{ \sigma^i\theta\;|\;0\leq i < k
  \right\}$. If $\sigma^k\theta=\sum_{i=0}^{k-1}c_i\sigma^i\theta$ for that
  $k$, then the \emph{$\sigma$-Order} of $\theta$ is the polynomial
  \[
    \Ord_\theta(X)=X^k-\sum_{i=0}^{k-1}c_iX^i.
  \]
\end{defi}
The $\sigma$-Order polynomials are widely used in our functions. We compute
them using Gauss algorithm and the row reduced echelon form. To know if
$\sigma^j(\theta)$ is in the linear span of $\left\{
\sigma^i(\theta)\;|\; 0\leq i <j \right\}$, we construct the matrix $M$ whose columns are the
vectors $\theta, \sigma(\theta), \dots, \sigma^{j}(\theta)$, we compute
the rank using Gauss algorithm. If the rank is $j+1$ then $\theta,
\sigma(\theta), \dots, \sigma^j(\theta)$ are linearly independant. In order to
compute $k$, we do this operation for $j=1$ and while the vectors are linearly
independant, we increase the value of $j$ by $1$. When we get that the
vectors are dependant, we use the reduced row echelon form to get a
$d\times(k+1)$ matrix of type
\[
\begin{pmatrix}
  1 & 0 & 0 & \dots & 0 & c_0 \\
  0 & 1 & 0 & \dots & 0 & c_1 \\
  0 & 0 & 1 & \dots & 0 & c_2 \\
  \vdots & \vdots & & \ddots & & \vdots \\
  0 & 0 & 0 & \dots & 1 & c_{k-1} \\
  0 & 0 & 0 & \dots & 0 & 0 \\
  \vdots & \vdots & \vdots & & \vdots & \vdots \\
  0 & 0 & 0 & \dots & 0 & 0 \\
\end{pmatrix}
\]
where the coefficients $c_i$ are in the last column. This strategy is
implemented in the function \texttt{sigma\_order}.

We also see that an element
$\alpha\in\mathbb{F}_{p^d}$ is normal is and only if
$\Ord_\alpha(X)=X^d-1$. This characterization is the same as the
definition, thus it is not very interesting. The following property is used in
further demonstrations.

\begin{prop}
  \label{orderDiv}
  Let $P\in\mathbb{F}_p[X]$, if $P(\sigma)\zeta=0$, then $P$ is divisible
  by $\Ord_{\zeta}$.
\end{prop}
\begin{proof}
  Let $I_\zeta$ be the set defined by $I_\zeta=\left\{P\in\mathbb{F}_p[X]\;|\; P(\sigma)\zeta
  = 0
\right\}$. This is an ideal of $\mathbb{F}_p[X]$, so $I_\zeta$ is principal. The
polynomial
$\Ord_\zeta$ belongs to $I_\zeta$ and has minimum degree in $I_\zeta$ by definition.
Hence $I_\zeta=(\Ord_\zeta)$.
\end{proof}
We always have $(X^d-1)(\sigma)\theta=0$, so for any
$\theta\in\mathbb{F}_{p^d}$, we have that $X^d-1$ is divisible by
$\Ord_\theta$. The result below is also used both in Lenstra's and Lüneburg's
algorithm.
\begin{prop}
  \label{orderLcm}
  Let $\alpha$ and $\beta$ be two elements in $\mathbb{F}_{p^d}$. Then we
  have \[
    \Ord_{\alpha+\beta}\;|\;\lcm(\Ord_\alpha,\Ord_\beta).
  \]
\end{prop}
\begin{proof}
 Let $P=\lcm(\Ord_\alpha,\Ord_\beta)$. By linearity, we have
  \[
  P(\sigma)(\alpha+\beta)=P(\sigma)\alpha+P(\sigma)\beta=0+0=0.
\]
It follows from Proposition~\ref{orderDiv} that $\Ord_{\alpha+\beta}$ divides
$P$.
\end{proof}
\begin{cor}
  \label{orderMul}
  Let $\alpha$ and $\beta$ be two elements in $\mathbb{F}_{p^d}$ such that
  $\Ord_\alpha$ and $\Ord_\beta$ are relatively prime. Then we have
  $\Ord_{\alpha+\beta}=\Ord_\alpha\Ord_\beta$.
\end{cor}
\begin{proof}
  Since $\Ord_\alpha$ and $\Ord_\beta$ are relatively prime, we have
  $\lcm(\Ord_\alpha, \Ord_\beta)=\Ord_\alpha\Ord_\beta$. It follows from
  Proposition~\ref{orderLcm} that $\Ord_{\alpha+\beta}$ divides
  $\Ord_\alpha\Ord_\beta$. Now, note that for any
  $\theta\in\mathbb{F}_{p^d}$, $\Ord_{-\theta}=\Ord_\theta$, then we also have
  \[
    \Ord_{\alpha}\;|\;\lcm(\Ord_{\alpha+\beta}, \Ord_\beta)
  \]
  \[
    \Ord_{\beta}\;|\;\lcm(\Ord_{\alpha+\beta}, \Ord_\alpha).
  \]
  Since $\Ord_\alpha$ and $\Ord_\beta$ are relatively prime, it follows
  that
  \[
    \Ord_{\alpha}\;|\;\Ord_{\alpha+\beta}
  \]
  \[
    \Ord_{\beta}\;|\;\Ord_{\alpha+\beta}.
  \]
  Therefore, $\Ord_\alpha\Ord_\beta$ divides $\Ord_{\alpha+\beta}$. Both
  polynomial are monic so we have the wanted equality.
\end{proof}

\subsubsection{Lenstra's algorithm}

Lenstra's algorithm is based on linear algebra, so our implementation is based
on the matrices module \texttt{fq\_mat} of Flint. Before describing the
algorithm, we need two lemmas.
\begin{lm}
  \label{lmExist}
  Let $\theta\in\mathbb{F}_{p^d}$ with $\Ord_\theta(X)\neq X^d-1$. Let
  $g(X)=(X^d-1)/\Ord_\theta(X)$. Then there exists
  $\beta\in\mathbb{F}_{p^d}$ such that
  \begin{equation}
    g(\sigma)\beta=\theta.
    \label{3.4}
  \end{equation}
\end{lm}
\begin{proof}
  Let $\gamma$ be a normal element of $\mathbb{F}_{p^d}$ over
  $\mathbb{F}_p$. Then, by definition of being a normal element, there
  exists a polynomial $f(X)\in\mathbb{F}_{p}[X]$ such that
  $f(\sigma)\gamma=\theta$. Since $\Ord_\theta(\sigma)\theta=0$, we have
  $(\Ord_\theta(\sigma)f(\sigma))\gamma=0$. So, by
  Proposition~\ref{orderDiv}, $\Ord_\theta(X)f(X)$ is
  divisible by $X^d-1$. We have $g(X)=(X^d-1)/\Ord_\theta(X)$ and $X^d-1|\Ord_\theta(X)f(X)$, so
$g(X)|f(X)$. Let $f(X)=g(X)h(X)$. Then
\[
  g(\sigma)(h(\sigma)\gamma)=\theta.
\]
This proves that $\beta=h(\sigma)\gamma$ is a solution of (\ref{3.4}).
\end{proof}
\begin{lm}
  \label{lmDegree}
  Let $\theta\in\mathbb{F}_{d}$ with $\Ord_\theta(X)\neq X^d-1$. Assume
  that there exists a solution $\beta$ of (\ref{3.4}) such that
  $\deg\Ord_{\beta}\leq\deg\Ord_{\theta}$. Then there exists a nonzero
  element $\zeta\in\mathbb{F}_{p^d}$ such that
  \begin{equation}
    g(\sigma)\zeta=0,
    \label{3.5}
  \end{equation}
  where $g(X)=(X^d-1)/\Ord_\theta(X)$. Moreover any such $\zeta$ has the
  property that
  \begin{equation}
    \deg\Ord_{\theta+\zeta}>\deg\Ord_\theta
    \label{3.6}
  \end{equation}
\end{lm}
\begin{proof}
  Let $\gamma$ be a normal element in $\mathbb{F}_{p^d}$ over
  $\mathbb{F}_p$. We can see that
  $\zeta=\Ord_\theta(\sigma)\gamma\neq0$ is a solution of~(\ref{3.5}). In
  fact, we prove that~(\ref{3.6}) is true for any solution $\zeta$
  of~(\ref{3.5}). From~(\ref{3.4}) and Proposition~\ref{orderDiv}, it follows
  that $\Ord_\theta$ divides $\Ord_\beta$, so the hypothesis that
  $\deg\Ord_\beta\leq\deg\Ord_\theta$ implies that
  $\Ord_\beta=\Ord_\theta$. It follows that $g$ and $\Ord_\theta$ are relatively
  prime: suppose $\gcd(g, \Ord_\theta)=d$ and $\deg d>0$, let
  $\Ord_\theta=\Ord_\theta'd=\Ord_\beta$ and $g=g'd$, then
  $\Ord_\theta'(\sigma)\theta=(\Ord_\theta'
  g)(\sigma)\beta=(\Ord_\beta g')(\sigma)\beta=0$ and
  $\deg\Ord_\theta'<\deg\Ord_\beta$,
  that is a contradiction. Since $\Ord_\zeta$ is a divisor of $g$,
  $\Ord_\zeta$ and $\Ord_\theta$ are also relatively prime. Therefore,
  by Corollary~\ref{orderMul}
  \[
    \Ord_{\theta+\zeta}=\Ord_\theta\Ord_\zeta,
  \]
  and then (\ref{3.6}) follows from the fact that $\zeta\neq0$ and then
  $\deg\Ord_\zeta\geq1$.
\end{proof}
We now have a deterministic algorithm to compute a normal element of
$\mathbb{F}_{p^d}$ over $\mathbb{F}_p$.
\begin{enumerate}
  \item Take an element $\theta\in\mathbb{F}_{p^d}$ and compute
    $\Ord_\theta$.
  \item If $\Ord_\theta(X)=X^d-1$ then the algorithm stops ($\theta$ is
    normal).
  \item Calculate $g(X)=(X^d-1)/\Ord_\theta$ and then find
    $\beta\in\mathbb{F}_{p^d}$ such that $g(\sigma)\beta=\theta$.
  \item Determine $\Ord_\beta$. If $\deg\Ord_\beta>\deg\Ord_\theta$ then
    replace $\theta$ by $\beta$ and go to $2$; otherwise if
    $\deg\Ord_\beta\leq\deg\Ord_\theta$ then find a nonzero element $\zeta$
    such that $g(\sigma)\zeta=0$, replace $\theta$ by $\theta+\zeta$ and
    determine the $\sigma$-Order of the new $\theta$, then go to $2$.
\end{enumerate}
This algorithm ends because with each replacement of $\theta$, the degree of
$\Ord_\theta$ increases by at least one, by Lemma~\ref{lmDegree}, and because
this degree is bounded by $d$ since the only possible $\sigma$-Order of degree
$d$ is $X^d-1$. The existence of the elements $\beta$ of step $3$ is guaranteed
by Lemma~\ref{lmExist} and the existence of $\zeta$ of step $4$ by
Lemma~\ref{lmDegree}. Since $g(\sigma)$ is a linear map, we compute $\beta$ and
$\zeta$, using Gauss algorithm and reduced row echelon form again. Computing
elements of type $g(\sigma)\theta$ is not just polynomial evaluation,
and is widely used in our algorithms, so we have a special function
\texttt{sigma\_composition} to do it. The rest of the algorithm is
essentially linear algebra, it uses Flint matrices and is in the function
\texttt{lenstra}.

\subsubsection{Lüneburg's algorithm}
Lüneburg algorithm is more elementary, it is not based on a powerfull theory
like linear algebra. Let $f$ be an irreducible polynomial of degree $d$ over
$\mathbb{F}_p$ and $\alpha$ a root of $f$. Then $\left\{ 1, \alpha, \dots,
  \alpha^{d-1}
\right\}$ is a basis of $\mathbb{F}_{p^d}$ over $\mathbb{F}_p$. Let
$f_i=\Ord_{\alpha^i}$, for $0\leq i < d$. Let
$\gamma=\sum_{i=0}^{d-1}a_i\alpha^i$ be a normal element of
$\mathbb{F}_{p^d}$ over $\mathbb{F}_p$. Note that for any
$a_i\in\mathbb{F}_p$, $\Ord_{a_i\alpha^i}=\Ord_{\alpha^i}=f_i$, and as seen in the
demonstration of Corollary~\ref{orderMul}, $\Ord_\gamma=X^d-1$ divides
$\lcm(f_0, \dots, f_{d-1})$. Since for all $i$, $f_i$ divides $X^d-1$, we have that
$\lcm(f_0, \dots, f_{d-1})$ divides $X^d-1$. Therefore $\lcm(f_0, \dots,
f_{d-1})=X^d-1$. Then we apply the \emph{factor
refinement}~\cite{BaDrSh93}. It is an algorithm that, given a list of
polynomials $f_0, \dots, f_{d-1}$, compute a new list $g_1, \dots, g_r$ of
polynomials, pairwise relatively prime, such that
\[ \prod_{i=0}^{d-1} f_i=\prod_{j=1}^{r} g_j^{e_j}. \]
The factor refinement works that way: we first set $h_i=f_i$ and $e_i=1$ for $0\leq i
<d$ and we consider the list $(h_i, e_i)_{i}$. While there exists $i\neq j$ with $p:=\gcd(h_i, h_j)\neq1$, we delete
$(h_i,e_i)$ and $(h_j,e_j)$ from the list and we add $(p, e_i+e_j),( h_i/p,
e_i), (h_j/p, e_j)$, except for the cases where the first entry is $1$. This
algorithm stops and compute what we expect. In our case, we use it to compute
the pairwise relatively prime polynomials $g_j$ $(1\leq j \leq r)$, and then
we compute the integers $e_{ij}$ such that
\[
  f_i=\prod_{1\leq j \leq r}g_j^{e_{ij}}\textrm{, for all }0\leq i<d.
\]
This algorithm is implemented in the function
\texttt{factor\_refinement}, it uses Flint's type
\texttt{fq\_poly\_factor} to represent the list $(h_i, e_i)_i$ and to let Flint
deal with the memory allocation.

Next, for each $j$, $1\leq j \leq r$, we find an index $i(j)$ such that
$e_{ij}$ is maximized. Let
\[
  h_j=f_{i(j)}/g_j^{e_{i(j)j}}
\]
and take $\beta_j=h_j(\sigma)\alpha^{i(j)}$. Then
\[
  \beta=\sum_{j=1}^r\beta_j
\]
is a normal element of $\mathbb{F}_{p^d}$ over $\mathbb{F}_p$. In fact, the
$\sigma$-Order of $\beta_j$ is $g_j^{e_{i(j)j}}$ ($\beta_j$ has been
constructed in that purpose) for $1\leq j \leq r$. As $g_1, \dots, g_r$ are
pairwise relatively prime, Corollary~\ref{orderMul} states that the
$\sigma$-Order of $\beta$ must
\[
  \prod_{j=1}^rg_j^{e_{i(j)j}}=\lcm(f_0, \dots, f_{d-1})=X^d-1,
\]
meaning that $\beta$ is a normal element.

To implement this algorithm, we use our functions
\texttt{sigma\_composition},
\texttt{sigma\_order}, \texttt{factor\_refinement}, and Flint's basic functions to deal with
polynomials. The implementation is a translation of the steps we followed
to obtain $\beta$.

\section{Experimental results}
In the source code, one can find two directories named \texttt{fq} and
\texttt{fq\_nmod} with very little differences between the files inside these
directories. The name of the file determines the type used in the functions. The
type \texttt{fq} supports arbitrary large integers $p$ for the characteristic of
the fields $\mathbb{F}_{p^d}$, whereas \texttt{fq\_nmod} supports only small
integers for $p$ but is faster.

\subsection{Random algorithms}
We implemented the naive algorithm for computing normal elements, in order to
compare this algorithm to the more elaborate one. Surprinsigly, as we see in
Figure~\ref{NaiveElab}, the speed is
often the same, because when we chose an element randomly in
$\mathbb{F}_{p^d}$, it is often a normal element. If the naive algorithm finds a
normal element with its first pick, there is no way that the the elaborate one is
faster, since it does more calculus. There are still a few cases where the naive
algorithm does not find a normal element easily, and where the elaborate one is
faster.

The time spent in those algorithms is essentially to time spent to perform the
tests.

\begin{figure}
  \begin{center}
\includegraphics[scale=0.6]{../benchmarks/benchNaiveElab.pdf}
\end{center}
\caption{Benchmark of naive and elaborate algorithms with $p=90001$.}
\label{NaiveElab}
\end{figure}

\subsection{Lüneburg's and Lenstra's algorithms}

Lenstra's algorithm is faster than Lüneburg algorithm, though the complexity
obtained seems to match with the theory in both algorithms. Also, Lenstra's
algotihm is mush more stable than Lüneburg's, the latter can have really
different behaviours for entries of the same sizes. The core of
both Lüneburg's and Lenstra's algorithms is the computation of the
$\sigma$-order polynomials.

  \begin{figure}
\begin{center}
    \begin{tabular}{ccc}
      \textbf{d} & \textbf{Lüneburg} & \textbf{Lenstra} \\
     $50$ & $1,87$ s & $0,65$ s \\
     $114$ & $93,0$ s  & $11,7$ s \\
     $205$ & $290$ s & $187$ s \\
     $249$ & $1960$ s & $253$ s
    \end{tabular}
    \caption{Benchmark of Lüneburg's and Lenstra's algorithms with $p=90001$.}
    \label{LuneburgLenstra}
\end{center}
  \end{figure}

\clearpage
\bibliographystyle{unsrt}
\bibliography{biblio}
\end{document}


\documentclass[10pt]{extarticle}

\usepackage[utf8]{inputenc}
\usepackage{extsizes} % Allow for more sizes suchas 14pt or 17pt on document class.
\usepackage{mathtools} % Allows conditional math expressions, etc.

% Don't output references in case they're empty - http://tex.stackexchange.com/questions/74476/how-to-avoid-empty-thebibliography-environment-bibtex-if-there-are-no-refere

\let\myBib\thebibliography
\let\endmyBib\endthebibliography

\renewcommand\thebibliography[1]{\ifx\relax#1\relax\else\myBib{#1}\fi}

\begin{document}

%% Front page.
\title{Advise for Applying Machine Learning}

    \date{}

    \maketitle

\newpage
%% Abstract page.

%% Table of contents page.

%% Body start.
\section{Deciding What to Try Next.}\label{deciding-what-to-try-next.}

Introduction: In this section how to not waste time choosing / debugging
algorithms will be explained.

Example: Say that we have a regularized linear regression
implementation, and it makes unacceptably large errors. What should we
do?

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  More training examples? Might not be the best idea - can waste time.
\item
  Try smaller sets of features. Might prevent overfitting.
\item
  Try getting additional features. We can waste time - need to know if
  it will work.
\item
  Try additional features.
\item
  Decreasing, increasing $\lambda$.
\item
  etc\ldots{}.
\end{itemize}

We can waste a lot of time doing this if we are not sure why our
algorithm fails, so we're gonna try to see what shall be the best
option, or at least rule out some of them so we don't lose time doing
something that is not going to increase the prediction accuracy.

The tests done to solve this problems will be called \textbf{Machine
learning diagnostic tests}.

\section{Evaluating a Hypothesis}\label{evaluating-a-hypothesis}

How do you tell if a hypothesis is overfitting?\\Just split the data in
two portions: The first portion is going to be the training set and the
other one the test set (70\%-30\% split is a good estimation).

We train with the training data and evaluate our model with the test
data. We can evaluate the model using different metrics, such as mse (eq
\ref{eq:mse}) for regression or the 1/0 error for classification (eq
\ref{eq:one_zero_error}).

\begin{equation} \label{eq:mse}
J_{test}(\theta) = \frac{1}{2m_{test}} \sum_{i=1}^{m_{test}}(h_\theta(x_{test}^{(i)}) - y_{test}^{(i)})^2
\end{equation}

\begin{equation}
error(h_\theta(x), y) =
\begin{dcases}
    1, & \text{if\,} h_\theta(x) \geq 0.5, y = 0 \\
       & \text{or\,} h_\theta(x) < 0.5, y = 1 \\
    0, & \text{otherwise}
\end{dcases}
\end{equation}

\begin{equation} \label{eq:one_zero_error}
\text{test error} = \frac{1}{m_{test}} \sum_{i=1}^{m_{test}} error(h_\theta(x_{test}^{(i)}), y^{(i)})
\end{equation}

\section{Model Selection and Train/Validation/Test
Sets}\label{model-selection-and-trainvalidationtest-sets}

Say that we want to fit a problem. We want to decide the degree (d) of
the polynomial to use (amount of features extracted from a single
feature using the feature \^{}2, \^{}3, etc).

Each single polynomial will fit to different $\theta$ (d = 1 to
$\theta^{(1)}$, d = 2 to $\theta^{(2)}$, etc).

To find the best d, we could minimize the test error. If we do that,
though, what happens is that we would overfit the d parameter for the
test set, and we don't want to do that.

\subsection{Welcome, Cross Validation
set.}\label{welcome-cross-validation-set.}

In order to fix that problem, we simply split the data in 3: Training,
CrossValidation and Test (with a typical ratio of 60\% - 20\% - 20\%).
The CV error is calculated with the same formulas, only with the CV
data, obviously.

Once we have that split, we train every different model (each with a
different d) and choose the one that has the lowest CV error. This way,
we can still use the test error in order to know whether our model is
overfitting on ``real life'' data.

\section{Diagnosting Bias
vs.~Variance}\label{diagnosting-bias-vs.variance}

We continue with the example of the previous section.

When we increase the degree of the polynomial, we get less error on the
training set, which means we're overfitting. With the CV/Test sets,
though, the error increases at a certain point, where the model fails to
generalize the data (overfitting).

When we have a high CV error, we can have either high variance or high
bias. High bias - Underfitting is when both training error and CV error
are high. On the other hand, High variance - Overfitting is when
training error is very low and the CV error is high.

We can see all these in figure \ref{fig:error_degree_polynomial}.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{img/error_degree_polynomial.eps}
\caption{Error on degree of the polynomial.}
\label{fig:error_degree_polynomial}
\end{figure}

\section{Regularization and
Bias/Variance}\label{regularization-and-biasvariance}

Suppose we're fitting a high order polynomial with regularization to
evade the overfitting trap.

In figure \ref{fig:lambda_bias_variance} we can see that, given the case
of a high order polynomial, when we have a large $\lambda$ we are
underfitting the data and when we have a small $\lambda$ we're
overfitting it.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{img/lambda_bias_variance.eps}
\caption{Different $\lambda$ values comparisson on a high degree polynomial.}
\label{fig:lambda_bias_variance}
\end{figure}

An important detail here is noticing that, even though we use
regularization when training, we don't use it for the evaluation cost
functions, where we use the simple mse function (equation \ref{eq:mse}).

In order to find the correct $\lambda$ value, we train a model with
every different value and pick up the one that gives the min CV error
function. We can then evaluate the chosen $\theta$ with the test error
function to see how well it's working (figure
\ref{fig:lambda_cv_train_error}).

\begin{figure}
\centering
\includegraphics[width=\textwidth]{img/lambda_cv_train_error.eps}
\caption{CV and Train error functions related to $\lambda$ values.}
\label{fig:lambda_cv_train_error}
\end{figure}

\section{Learning Curves}\label{learning-curves}

Tool to diagnose a algorithm and find out if it suffers from bias or
variance.

Artificially reduce training set size and train with that. On very small
datasets the training error will be very small because it will fit
perfectly. As the size of the dataset increases, though, the training
error increases.

With the CV error, it starts with huge errors when training with a small
dataset and then the error decreases as the training set size increases.

We can see the ``perfect'' graph we should expect if our model isn't
overfitting or underfitting the data in figure
\ref{fig:error_training_size_perfect}.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{img/error_training_size_perfect.eps}
\caption{Error on training set size without both variance and bias.}
\label{fig:error_training_size_perfect}
\end{figure}

If we have high bias (underfitting), the cross validation error will end
up close to the train error (figure \ref{fig:error_training_size_bias}).
This means that if we have a high bias, getting more data doesn't help
much.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{img/error_training_size_bias.eps}
\caption{Error on training set size with bias (underfitting).}
\label{fig:error_training_size_bias}
\end{figure}

If we have high variance (overfitting), the model will have a very low
training error and very high CV error (figure
\ref{fig:error_training_size_variance}). In this case, getting more data
is likely to help.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{img/error_training_size_variance.eps}
\caption{Error on training set size with variance (overfitting).}
\label{fig:error_training_size_variance}
\end{figure}

\section{Deciding What to Do Next
Revisited}\label{deciding-what-to-do-next-revisited}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  Getting more training examples $\rightarrow$ Fixes high variance.
\item
  Smaller sets of features $\rightarrow$ Fixes high variance.
\item
  Additional features $\rightarrow$ Fixes high bias.
\item
  Adding polynomial features $\rightarrow$ Fixes high bias.
\item
  Decreasing $\lambda$ $\rightarrow$ Fixes high bias.
\item
  Increasing $\lambda$ $\rightarrow$ Fixes high variance.
\end{itemize}

\subsection{Neural Networks}\label{neural-networks}

A small neural network is prone to underfitting and a large one is prone
to overfitting.

%% Body end.

%% Bibliography.

    \nocite{*}

\bibliographystyle{plain}
\bibliography{references}

\end{document}

theory/06.01 - Advice for Applying Machine Learning/06.01 - Advice for Applying Machine Learning.tex