\subsection{The rotation periods of cool main-sequence stars}

The \citet{mcquillan2014} catalog was the first large-scale stellar rotation
period catalog, generated from analyzing light curves from the \kepler\
spacecraft and remains one of the most valuable community products in the
field of stellar astronomy.
It contains around 34,000 rotation periods of FGKM dwarfs and subgiants,
measured from autocorrelation functions (ACFs) of light curves.
This catalog revealed a gap in the rotation periods of K and M dwarfs: a strip
in the rotation vs effective temperature plane that is under-populated
compared to the surrounding parts of parameter space.
This gap can be seen in figure \ref{fig:the_gap}.
\begin{figure}
  \caption{
The rotation periods of 18,259 FGKM dwarf stars, measured by \mct, vs. their
effective temperatures.
We applied cuts to the \gaia\ color-magnitude diagram of these stars (figure
\ref{fig:CMD_cuts}) in order to remove equal-mass binaries and subgiants from
the sample.
The rotation period gap can be seen as an almost-horizontal gap toward the
right of this figure.
}
  \centering
    \includegraphics[width=1\textwidth]{the_gap}
\label{fig:the_gap}
\end{figure}
The rotation period gap is most prevalent for low-mass stars, of spectral type
K and M, but was recently shown to extend through G, F and A types as well
\citep{davenport2017}.

Three explanations for the origin of this gap have been proposed.
Firstly, since the gap falls on what is thought to be an isochrone in
period-effective temperature space, it follows that it may be caused by a
discontinuity in the age distribution of the sample.
The gap could be created if there exist two populations of stars with
different age distributions, with the distribution of young ages peaking at
around 800 Myr, and the distribution of old ages peaking at around 2-3 billion
years.
This idea was investigated by \mct, who suggested that the two populations
might be the thin and thick disk of the Milky Way.
However, they found no sharp change in kinematic properties between the two
samples.
In addition, the thin/thick disk transition is thought to be around 6-8 Gyr,
which is older than the majority of stars in the \mct\ sample
\racomment{(citations)}.
A short pause in star formation is difficult to explain by invoking, \eg\ the
passage of a spiral arm through the Solar neighbourhood, since an increase in
gas density would be expect to boost, not quench, star formation rate.
However perhaps the Solar neighbourhood passed between two spiral arms around
1 billion years ago, however pattern speed, orbital period (200 Myrs)... seems
unlikely.
Possible passage of a dwarf galaxy but, again, this would be expected to boost
star formation, not suppress it.

The second explanation for the rotation period gap is that stars either
transition rapidly across the gap, \ie\ experience a rapid phase of angular
momentum loss at an age of around 1 billion years, or become magnetically
inactive for a short period of time such that their rotation periods become
undetectable.
This first scenario could be caused by a sudden recoupling between the core
and envelope, where core and envelope rotate differentially before the gap,
then angular momentum is radially redistributed and stars rotate as solid
bodies after the gap.
The latter scenario is hinted at by \citet{reinhold2017} who noticed that the
amplitudes of light curves decrease smoothly either side of the gap, as though
stars gradually become more inactive until they have no large-scale surface
features and their rotation periods become almost impossible to detect.
\citet{reinhold2017} suggest that a transition between a spot-dominated and
faculae-dominated regime could lead to a cancelling-out of surface features.
% do not lose
% angular momentum uniformly over time: instead they undergo a period of rapid
% spin down at an age of around 1.1 Gyr, or a Rossby number of around...
% \racomment{(Calculate this!)}.
% Since the rotation period gap is located along a line of constant Rossby
% number \racomment{(what is this number?)}, and since the rate of stellar
% angular momentum loss is known to be related to magnetic field strength , it
% follows that the gap could be caused by some phenomenon related to magnetic
% braking.
% (this
% is the principle behind gyrochronology \citep[\eg][]{kawaler1989,
% pinsonneault1989, barnes2003, barnes2007, angus2015, vansaders2016,
% vansaders2018, angus2019})

The final explanation for the rotation period gap is one of measurement error
or confounding by binarity.

As of May 2019, the \citet{mcquillan2014} catalog has been used more than 250
times, for a range of studies spanning fields from exoplanets to white dwarfs,
to galactic archaeology.
Since it underpins so many astronomical studies, a large number of incorrectly
measured rotation periods in this catalog would undermine several of the
studies built upon it.

\subsection{Using kinematics to solve the mystery}

Stars are thought to be born in the thin disk of the Milky Way, orbiting the
center of the galaxy with a low out-of-plane, or vertical, velocity ($W$, or
$v_z$).
Young stars have relatively small vertical velocities, but gain momentum in
the vertical direction over time.
Although the cause of orbital heating is not well understood, interactions
with giant molecular clouds and spiral arms are thought to play an important
role \racomment{(citations)}.
Although the velocity of any individual star will only provide a weak age
constraint, the velocity dispersion of a group of stars can indicate whether,
on average, that group is old or young relative to other groups.
In this work we compare the velocity dispersions of groups of stars to
ascertain which groups are older and which younger and draw conclusions based
on the implied relative ages of populations.
% Stellar velocities have a history of being used as an age proxy, with several
% notable examples within stellar astronomy \citep[\eg][]{faherty2009,
% west2011}.
Since the age-velocity dispersion relations (AVRs) are themselves still
actively being calibrated, it is difficult to directly compare gyrochronal
ages with kinematic ones.
However, regardless of the exact relation between velocity dispersion and
stellar age, it is expected to be a monotonic relationship, therefore velocity
dispersion can be used effectively to {\it rank} groups of stars by age.

% Vertical {\it actions} are better age indicators than velocities, because
% actions are calculated by integrating angular momentum over the Milky Way's
% potential, and are therefore position invariant -- \eg\ a star will have the
% same action at periapsis and apoapsis.
% In contrast, orbital velocities are different at periapsis and apoapsis -- so
% in this sense, {\it actions} are the natural quantities to use as age proxies.
Vertical action is a better age indicator than vertical velocity, however both
vertical action ($J_z$) and vertical velocity (\vz/W) can only be calculated
with full 6-dimensional position and velocity information.
Unfortunately, most stars with measured rotation periods do not have radial
velocity (RV) measurements because they are relatively faint \kepler\ targets
($\sim$11th-18th magnitudes).
For this reason, we used an alternative age proxy: velocity in the direction
of galactic latitude, \vb.
The \kepler\ field is positioned at low galactic latitude (b=5-20\degrees), so
\vb\ is a close approximation to \vz.

% The rotation periods of middle-aged FGKM stars are mostly determined by their
% mass, age and evolutionary stage, and this is due, for the most part, to
% angular momentum loss through magnetic braking.
% Although stars are born with a range of rotation periods, a steep dependence
% of angular momentum loss, $J$ angular frequency, $\omega$, $J \propto
% \omega^3$ causes stellar rotation periods to converge after a characteristic
% timescale.
% This timescale is on the order of a few tens of millions of years for F and G
% stars, but hundreds of millions of years for K and M stars.
% Once stars exceed this age, they steadily lose angular momentum via magnetized
% stellar winds that remove angular momentum from the star at a radius that is
% determined by the magnetic field strength.
% The Rossby number, the ratio of rotation period to convective turnover
% timescale, is linked to the strength of stellar magnetic dynamos.
% Large-scale polodial and toroidal stellar magnetic fields are thought to be
% generated by the $\alpha-\Omega$ mechanism, whereby convective motion of
% plasma and differential stellar rotation cause the winding up of magnetic
% field lines, generating a strong magnetic field \racomment{(citations)}.
% In general, the ratio between a star's rotation period and its characteristic
% timescale of convective motion (the Rossby number) determines the strength of
% the magnetic field \racomment{(citation)}.
% Stars that rotate quickly and have deep convection zones, therefore long
% convective turnover times, have small Rossby numbers and strong magnetic
% fields.
% Young M dwarfs typically have the strongest magnetic fields.
% Slowly rotating stars with shallow convective zones, such as old F and G
% dwarfs, have weak magnetic fields \racomment{(citations)}.
% Since magnetic field strength depends on stellar rotation period and mass via
% the Rossby number, it follows that angular momentum loss driven by magnetized
% stellar winds also depends on Rossby number.


\chapter{Introduction}\label{introduction}
\section{Problem statement}
With the advance of \emph{deep reinforcement learning} in $2013$, \emph{intelligent agents} have increasingly been able to solve more complex problems \cite{deeprl}. In reinforcement learning (RL), an agent is trained to solve a problem through trial and error, using feedback in the form of rewards. This way it tries to find an optimal \emph{policy}: a mapping of a state and its available actions to a valuation for each action. For \emph{deep} RL, \emph{artificial neural networks} are used to train such an agent. The usage of increasingly complex problem settings, comes with more complex environments for an agent to work in; observations representing the current state of the environment are becoming dimensionally higher.

This means that the \emph{state-space} of the environment, the number of possible states the environment can be in, is increasing. In general, the larger the state-space, the more difficult and time intensive the training of the agent will be, and thus the more computing cost is needed. This is due to several reasons. Firstly, with a larger state-space, the agent will need to explore more states before having explored the entire state-space. This leads to having to gather larger datasets of state observations which can be impractical \cite{AE_2019}.

Secondly, for deep RL, the agent uses artificial neural networks. These networks need to be of a size proportional to the state-space to be able to accurately approximate an optimal policy; otherwise it could lead to under-fitting or over-fitting \cite{rlfitting}. Generally, the larger the state-space, the larger such a network needs to be. This means the network will contain more trainable parameters, leading to more computation for its training and its usage \cite{AE_2019}.

Lastly, higher dimensional states may include more irrelevant information and noise. This can lead to longer training times for an agent and even to agents not being able to find good policies \cite{AE_2016}.

\section{Research questions}
A way for dealing with large state-spaces is needed for RL to be scalable. One way to deal with this, is by lowering the state-space of a specific problem, whilst retaining enough state information for the agent to find an optimal policy. This is known as \emph{state-space dimensionality reduction}. Here, a state observation would be projected to a lower dimensional space; this lower dimensional representation is called the \emph{latent representation}. The main problem with state-space dimensionality reduction, is the loss of possibly essential information, which could lead to RL agents not being able to find an optimal policy.

In this thesis, we will examine the effect of using state-space dimensionality reduction on an RL agent, focusing on whether an agent can find optimal policies on a lower dimensional state-space. More specifically, we will look at the effect on a model-free, value based RL agent (as explained in section \ref{pl-rl}). For this, we will use the \emph{double deep-Q-learning} (DDQN) learning algorithm (see section \ref{pl-dqn}) \cite{ddqn}. We will project the state observation to a lower dimensional space using three different methods, whose effects on the RL agent we will compare: \emph{principal component analysis}, \emph{autoencoders} and \emph{DeepMDPs} (see sections \ref{pl-pca}, \ref{pl-ae} and \ref{pl-deepmdp} respectively for information about each method). We will apply these methods on two environments that use high dimensional grid-based observations: Starcraft II MoveToBeacon (see \ref{research-env-pysc2}) and OpenAI Atari Pong (see \ref{research-env-pong}) \cite{pysc2}\cite{pong}.

The main research question we will examine is: \textit{what is the effect of state-space dimensionality reduction on model-free value-based reinforcement learning in an environment using grid-based observations, by using PCA, autoencoders or DeepMDPs}? To answer this, we will answer the research sub-question: \textit{how do the training results of a double deep-Q-learning reinforcement learning agent change when using PCA, autoencoders and DeepMDPs for state-space dimensionality reduction in Starcraft II MoveToBeacon and OpenAI Atari Pong}?

\section{Method, results and possible benefits}
We gathered the training results for several RL agents trained in Starcraft II and Pong. The first agent is a baseline agent: a DDQN RL agent using the observations given by the environment. The second agent uses PCA to reduce the dimensionality of the observations. Thus, the observations given by the environment first go through PCA, thereby projected to a lower dimensional space, before being used by the RL agent. The PCA is trained on previously stored observation traces, before being used by the RL agent. There are also two agents using an autoencoder, that work similar to the PCA agent. The first of the two uses a pre-trained autoencoder, again trained on stored observations. The second autoencoder agent, the online trained autoencoder agent, is trained simultaneously with the RL agent, using the observations received by the agent. Lastly, there is a DeepMDP agent, which is only used in the Starcraft II environment.

In both environments, the PCA agent lost all spatial information, leading to RL agents learning sub-optimal policies. The DeepMDP was also unable to train to an optimal policy in Starcraft II. However, the autoencoder agents were able to learn good policies. In the Starcraft II environment, the pre-trained autoencoder agent learned a better, more stable policy in less episodes than the baseline agent. The online trained autoencoder agent had training results similar to the baseline agent, yet slightly more inconsistent, and needing more episodes to train well. Similar results were found for the second environment, Pong. However, the pre-trained autoencoder agent learned a slightly worse and less consistent policy than the baseline agent. This resulted from the agent needing more precise spatial information than in the Starcraft II environment. However, the online trained autoencoder agent reached a policy similar to the baseline agent, though needing more episodes to converge. Its better converging policy can be explained by the autoencoder being trained on more frames than the pre-trained autoencoder, therewith giving a more precise latent representation. However, the precision needed in the spatial information for the RL to find an optimal policy, show a limit with regards to the usage of an autoencoder agent.

Several benefits may arise from training agents on lower state-spaces. Though we will mention these benefits now here, the extend of these benefits are not examined in this thesis and are left for future research.

Firstly, RL agents training in a lowered state-space might need less training time and less observation data, and therefore less computational cost for training and using its neural network.

Secondly, in very complex, large scaled environments, a normal RL agent possibly cannot find an optimal policy in a feasible amount of time; here, state-space dimensionality reduction could to lead to a better policy.

Thirdly, depending on the dimensionality reduction method, we could reuse the dimensionality reduction component in different RL agents solving different problems in the same environment. Thus, we would only have to train a dimensionality reduction component once, then being able to train different agents on the same latent representation. This possibility though depends on two things. It would need to be a dimensionality reduction method that is trainable separately from the RL agent. This is the case for only two of the three methods we consider in our research: PCA and autoencoders. Secondly, the latent representation would have to be problem-independent: it would have to be a good representation of the entire state-space, without losing information essential to any of the RL problems being used in this environment.

\section{Paper structure}
We will start by giving preliminary information in section \ref{preliminaries}. Here we will first give an overview of reinforcement learning principles in section \ref{pl-rl}. This will include an explanation of artificial neural networks and the RL learning algorithm DDQN, which we use for our agents. After this, we will discuss state-space dimensionality reduction in section \ref{pl-dimensionality}, which will include an examination of the three methods used in our research: PCA, autoencoders and DeepMDPs.

After the preliminaries, we will show our methodology in section \ref{research-method}. Here we will give a general overview of the different RL agents that we will use for our experiments. Then we will give information about the environments used and the specific agent setups per environment.

After having explained the experiment setups, we will give the results of our experiments in section \ref{research-results}. We will show and discuss the results per environment, before discussing the results of both environments as a whole.

Then, we will discuss related work in section \ref{relatedwork}, before giving our conclusions and suggestions for future research in section \ref{conclusions}. Lastly, the appendices in section \ref{appendix} will contain more details on the specific agent architectures per environment, including hyperparameter settings.

Our code repository can be found at: \url{https://github.com/Niels-vv/Safe-RL-With-DR.git}.


\documentclass[main.tex]{subfiles}
\begin{document}

\subsubsection{Rømer delay (observer)}

\marginpar{Monday\\ 2020-4-6, \\ compiled \\ \today}

% We are discussing the Hulse-Taylor pulsar, which is a very precise clock.
% There are many effects by which the time-of-arrival is shifted, if we take them all out we can get the effect from the source.

This effect is denoted as \(\Delta_{R, {\odot}}\), it is due to the fact that if Earth is on the far side of the Sun the radio signal takes longer to get to it than it would if it were on the near side to the pulsar. The simplest way to model it is
%
\begin{align}
\Delta_{R, {\odot}} = t_0 \cos(\Omega t - \lambda ) \cos(\beta )
\,,
\end{align}
%
where \(t_0 = R / c\), \(R\) being the radius of the Earth's orbit (here taken to be circular), while \(\lambda \) and \(\beta \) define the position of the source in the orbital plane.

Proper modelling would require us to also take into account the ellipticity of the orbit and the Earth's rotation.
In general, the effect can be written as
%
\begin{align}
\Delta_{R, {\odot}} = - \frac{\vec{r} _{\text{ob}} \cdot \hat{n}}{c}
\,,
\end{align}
%
where \(\hat{n}\) denotes the direction of the source in barycentric coordinates for the solar system, while \(\vec{r} _{\text{ob}}\) is the vector connecting the barycenter of the Solar System to the observer, which can be decomposed as a sum of vectors to the Sun's center, Earth's center and finally to the observer:
%
\begin{align}
\vec{r} _{\text{ob}} = \vec{r}_{\text{ob}, \oplus} + \vec{r}_{\oplus, \odot} + \vec{r}_{\odot, \text{barycenter}}
\,.
\end{align}

Modeling this allows us get information about \(\lambda \) and \(\beta \), which define \(\hat{n}\).
The \textbf{effect size} is dominated by the Earth's motion around the Sun, and it has a magnitude of about \(\pm \SI{e3}{s}\) in the span of six months.

\subsubsection{Shapiro delay (observer)}

% We will have the Shapiro delay from the Sun, the Einstein delay at the receiver from the curvature of the spacetime around the Earth, which is given by

This effect is due to the curvature introduced by the gravitational potential of the Sun.
The Schwarzschild metric in Cartesian coordinates reads:
%
\begin{align}
\dd{s^2} &= - \qty(1 + 2 \phi (x)) c^2 \dd{t^2} + \frac{1}{\qty(1 + 2 \phi (x))} \dd{\vec{x}}^2  \\
&\approx - \qty(1 + 2 \phi (x)) c^2 \dd{t^2} + \qty(1 - 2 \phi (x)) \dd{\vec{x}}^2
\,,
\end{align}
%
where \(\phi (x) = -GM / \abs{x}c^2 \ll 1\) is the gravitational potential in the weak-field approximation.
Photons travel along null geodesics, so they have \(\dd{s}^2 = 0\): this means that
%
\begin{align}
c\dd{t} = \pm\sqrt{\frac{1 - 2 \phi }{1 + 2 \phi }} \abs{\dd{x}}
\approx \pm \qty(1 - 2 \phi ) \abs{ \dd{x}}
\,,
\end{align}
%
so we can compute the Shapiro delay as the correction to the total travel time:
%
\begin{align}
c \qty(t _{\text{obs}} - t _{\text{emit}})
&= \int_{r _{\text{obs}}}^{t _{\text{emit}}} \abs{ \dd{x}} \abs{1 - 2 \phi (r)}
\,,
\end{align}
%
which automatically incorporates the Rømer delay, since we find
%
\begin{align}
t _{\text{obs}} - t _{\text{emit}} =  \underbrace{\frac{\abs{\vec{r} _{\text{emit}} - \vec{r} _{\text{bary}}}}{c}}_{\text{simple propagation}} + \underbrace{\frac{\vec{r} _{\text{obs}} \cdot \hat{n}}{c}}_{\text{Rømer delay}}
\underbrace{- \frac{2}{c} \int_{r _{\text{obs}}}^{r _{\text{emit}}} \abs{ \dd{x}} \phi (x)}_{\text{Shapiro delay: } \Delta_{S, \odot}}
\,.
\end{align}
%
\todo[inline]{The sign of the Rømer effect is different here! Intuitively we should expect the delay \(\Delta_R \) to be \(>0\) when \(\hat{n} \cdot \vec{r} < 0\), since then the Earth is \emph{behind the barycenter}.}

Let us explicitly compute \(\Delta_{S, \odot}\): we consider a reference frame in which the Sun is at the center, the Earth is on the \(x\) axis at a distance \(r_{\oplus \odot}\), while the pulsar is at a distance \(\rho \) from the Earth and at an angle \(\theta \) with respect to the Sun-Earth axis: so, the Sun-pulsar distance \(r\) is given by
%
\begin{align}
r^2 = \qty(r_{\oplus \odot} + \rho \cos \theta )^2 + \qty(\rho \sin \theta )^2
= r_{\oplus \odot }^2 \qty(u^2 + 2 u \cos \theta + 1)
\marginnote{Used \(\cos^2\theta+ \sin^2\theta = 1\).}
\,,
\end{align}
%
where \(u = \rho / r_{\oplus \odot}\).
The delay is given by
%
\begin{align}
\Delta_{S, \odot} &= - \frac{2}{c} \int_{r _{\text{obs}}}^{r _{\text{emit}}} \abs{ \dd{x}} \phi (x)
= \frac{2}{c} \int_{0}^{d} \dd{\rho } \frac{G M_{\odot}}{ c^2 r}  \\
&= \frac{2 G M_{\odot}}{c^3} \int_{0}^{d / r_{\oplus \odot} }
\dd{u} \frac{r_{\oplus \odot}}{r}   \\
&= \frac{R_{S, \odot}}{c} \int_{0}^{d / r_{\oplus \odot} }
\dd{u} \frac{1}{\sqrt{u^2 + 2 u \cos \theta  +1}}
\,,
\end{align}
%
where \(R_{S, \odot}\) is the Sun's Schwarzschild radius, while \(d\) is the distance from the Earth to the pulsar.
We used the fact that \(\abs{ \dd{x}} = \dd{ \rho }\): the differential \(\abs{\dd{x}}\) represents the \emph{coordinate distance} along the path from the Earth to the pulsar, which is linearly parametrized by \(\rho \); \(r\) on the other hand can be used to parametrize the curve but that would be a \emph{nonlinear} parametrization, not useful for our purposes.

We need to estimate this integral; it is a divergent one but the divergence is only logarithmic. It is reasonable to estimate it assuming \(d/r_{\oplus \odot} \gg 1\) --- realistically this is of the order of \num{e9}, since the distance to the pulsar is of the order \SI{5}{kpc} \cite[pag.\ L53]{hulseDiscoveryPulsarBinary1975}.

So, we add and subtract \(1/ \sqrt{u^2 + 1}\) in the integrand: then we find
%
\begin{align}
\Delta_{S, \odot} = \frac{R_{S, \odot}}{c} \qty[
    \int_{0}^{d/r_{\oplus \odot}} \dd{u} \frac{1}{\sqrt{u^2+ 1}}
    +
    \int_{0}^{d/r_{\oplus \odot}} \dd{u}
    \frac{1}{\sqrt{u^2 + 2 u \cos \theta +1}}
    -\frac{1}{\sqrt{u^2+ 1}}
]
\,,
\end{align}
%
where the first integral is a hyperbolic arcsine, which can be estimated by \(\log (2 d/ r_{\oplus \odot})\); the second term can be estimated with the following
\begin{claim}
\begin{align}
\int_{0}^{ \infty } \dd{u} \qty[
\frac{1}{\sqrt{u^2 + 2 u \cos \theta +1}}
-\frac{1}{\sqrt{u^2+ 1}}]
= - \log \qty(1 + \cos \theta )
\,.
\end{align}
\end{claim}

\begin{proof}
Mathematica says so.
\end{proof}

Then finally our estimate is
%
\boxalign{
\begin{align}
\Delta_{S, \odot} = \frac{R_{S, \odot}}{c} \qty(\log ( \frac{2d}{r_{\oplus \odot}}) - \log (1 + \cos \theta ) )
\,,
\end{align}}
%
where \(\theta \) varies seasonally, as the Earth moves around the Sun, while \(d\) is basically constant.

It might seem that this can diverge as \(\theta \to \pi \), but this is not actually the case: that would correspond to the radiation coming through the center of the Sun, which it cannot. Actually, there is a range of values of \(\theta \) around \(\pi \) for which the radiation cannot reach us since the Sun is in the way. The maximum value of \(\theta \) that can be reached, as the radiation is tangent to the surface of the Sun, is called the grazing angle \(\theta_{g}\).
This angle can be estimated by \(\theta_{g} \approx \pi - R_{\odot} / r_{\oplus \odot}\).

The delay is maximal when the radiation grazes the Sun, and minimal when it is coming from the opposite direction as the Sun: \(\theta = 0\).
So, the \textbf{maximum size of the effect} is given by the difference
%
\begin{align}
\Delta_{S, \odot} (\theta = \theta_{g}) -
\Delta_{S, \odot} (\theta = 0)
&= \frac{R_{S, \odot}}{c} \qty[
    \log( \frac{2d}{r_{\oplus \odot}} )
    - \log (1 + \theta_{g})
    -\log( \frac{2d - 2 r_{\oplus \odot}}{r_{\oplus \odot}})
] \marginnote{\(\log 1 = 0\).}  \\
&\approx - \frac{R_{S, \odot}}{c}
\log (1 + \cos(\pi - \frac{R_{\odot}}{r_{\oplus \odot}}))  \\
&\approx - \frac{R_{S, \odot}}{c}
\log (1 - 1 + \frac{1}{2}\qty(\frac{R_{\odot}}{r_{\oplus \odot}})^2)
\marginnote{Used \(\cos(\pi - x) = - \cos(x)\) and expanded.}
\\
&\approx - \frac{R_{S, \odot}}{c}
\log (\frac{1}{2}\qty(\frac{R_{\odot}}{r_{\oplus \odot}})^2)
\approx + \SI{56}{\micro s}
\,,
\end{align}
%
where we discarded the difference of the two logarithms with \(d\) since the relative difference between their arguments is on the order \num{e-9}.
\todo[inline]{In the slides this is reported as being on the scale of \SI{100}{\micro s} with the logarithm's argument being \(2 r_{\oplus \odot} / d\): I do not see how that could come about, and a dependence on \(d\) seems not to make sense physically.}

% \todo[inline]{Also, the slide is repeated twice.}

\subsubsection{Einstein delay (observer)}

This effect is about the gravitational shift and \emph{transverse} Doppler shift due to gravitational field of the Earth. We start from the proper time at the observer:
%
\begin{align}
c^2 \dd{\tau^2} &\approx \qty(1 + 2 \phi (x _{\text{obs}})) \dd{t^2}
 - \qty(1 - 2 \phi (x _{\text{obs}})) \dd{\vec{x}}^2 _{\text{obs}}  \\
\frac{ \dd{\tau^2}}{ \dd{t^2}} &\approx 1 + 2 \phi (x) - \frac{1 - 2 \phi (x)}{c^2} \frac{ \dd{x^2}}{ \dd{t^2}}  \\
&= 1 + 2 \phi (x) - \frac{v^2}{c^2} + \mathcal{O}(\phi v^2 / c^2)
\,,
\end{align}
%
and if we discard the higher order terms (both \(\phi \) and \(v/c\) are small for an Earth-bound observer, so a term containing their product is negligible) we can expand the square root as
%
\begin{align}
\dv{\tau }{t} \approx 1 + \phi \qty(x _{\text{obs}}) - \frac{v^2 _{\text{obs}}}{2c^2}
\,,
\end{align}
%
so
%
\begin{align}
\tau \approx t + \int^{t} \dd{\widetilde{t}} \qty(\phi \qty(x _{\text{obs}} (\widetilde{t})) - \frac{v^2 _{\text{obs}} (\widetilde{t})}{2c^2})
= t - \Delta_{\oplus, \odot}
\,,
\end{align}
%
where the lower bound of the integration is arbitrary, amounting only to a shift in the origin from which we measure times.

If most of the velocity is due to the motion of the Earth in its elliptic orbit, from Keplerian dynamics we know that the energy is given by the \emph{vis viva} equation:
%
\begin{align}
E = - \frac{G M \mu }{2 a} = \frac{1}{2} \mu v _{\text{obs}}^2- \frac{GM \mu }{r}
\,,
\end{align}
%
which means that the square velocity can be written as
%
\begin{align}
\frac{v^2 _{\text{obs}}}{2} = \frac{GM}{r} - \frac{GM}{2a}
\,,
\end{align}
%
so that we find
%
\begin{align}
\dv{ \Delta_{E, \odot}}{t} \approx \frac{v^2}{2c^2} - \phi = \frac{GM_{\odot}}{c^2} \qty( \frac{1}{r} - \frac{1}{2a} + \frac{1}{r})
= R_{S, \odot} \qty( \frac{1}{r} - \frac{1}{4a})
\,,
\end{align}
%
where \(R_{S, \odot} = 2 GM_{\odot} / c^2\) is the Schwarzschild radius of the Sun.
\todo[inline]{Wrong sign in the slides!}

The term depending on \(r\) is relevant to the modulation of our signal; the other one is a correction to apply to our estimates of the parameters of the binary and pulsar but it is not relevant to the modulation as it is a constant.

In order to estimate the magnitude of this modulation we need to ask by how much the radius of the Earth's orbit changes: the average orbital distance is around \SI{150e9}{m}, with annual variations of the order of \SI{2e9}{m}. So, the magnitude of the first term is around \num{e-8} seconds per second, with annual variations on the order of \num{e-10} seconds per second.

\todo[inline]{Not sure about the \(\pm \SI{4}{ns} / \SI{}{s}\): that would mean \num{4e-9} seconds per second\dots but the number we get is not the \emph{amplitude} of the oscillation, it is the value of the correction: the oscillation is much smaller, corresponding to the variation of the orbital radius of the Earth!}

\subsubsection{Interstellar medium propagation}

% We must also consider the group velocity of a signal of frequency which travels through the ISM, which is ionized gas.
The ISM is made of ionized gas mostly, and it causes the group velocity of the radio signal to not be precisely \(c\): there is a correction depending on the frequency \(\nu \) of the signal, given by
%
\begin{align}
v_g \approx c \qty(1 - \frac{n_e e^2}{2 \pi m_e} \frac{1}{\nu^2})
\,,
\end{align}
%
which causes a total time lag in the arrival of a pulse of fixed frequency:
%
\begin{align}
t_d &= \int_{0}^{d} \frac{ \dd{l}}{v_g}
\approx \frac{d}{c} + \frac{e^2}{2 \pi m_e} \frac{1}{\nu^2} \int_{0}^{d} n_e \dd{l}
\marginnote{First order approximation of \(1/(1-x) \sim 1+x\).} \\
&= \frac{d}{c} + \frac{e^2}{2 \pi m_e c} \frac{1}{\nu^2} DM
\,,
\end{align}
%
where \(DM\) is the Dispersion Measurement: it corresponds to the column number density of electrons between us and the pulsar, and it has the units of an inverse area.

For the HT pulsar, the bandwidth of the interesting signal is around \SI{4}{MHz}, and from the top to the bottom of the band the time delay is of the order of \SI{70}{ms}!

Fortunately we can measure precisely in the spectral domain: the arrival times for spectral bands which are near to each other will be very close, and they will form a continuous line:
so, we can ``connect the dots'' to find what corresponds to a single pulse.
The process of doing this and correcting for the DM is called \textbf{de-dispersion}; after doing it we can sum all the components in order to get the original pulse.

\subsubsection{Combination of local effects}

Taking all the effects pertaining to the source into account, we get
%
\begin{align}
t _{\text{ssb}} = \tau - \frac{D}{\nu^2} + \Delta_{E, {\odot}} - \Delta_{S, {\odot}} + \Delta_{R, {\odot}}
\,,
\end{align}
%
where \(t _{\text{ssb}}\) means ``time in solar-system barycenter'', and
%
\begin{align}
D = \frac{e^2}{2 \pi m_e c} DM
\,.
\end{align}

\subsubsection{Einstein delay at the source}

We need to look at the gravitational time delay at the source: the gravitational field will be able to be written as
%
\begin{align}
\phi (x) =
- \frac{G m_p}{c^2 \abs{\vec{x} - \vec{x}_p}}
- \frac{G m_c}{c^2 \abs{\vec{x} - \vec{x}_c}}
\,,
\end{align}
%
% there is a contribution from the gravitational field of the NS itself, which is hard to calculate but constant, so we do not worry about it.
where \(p\) and \(c\) denote ``pulsar'' and ``companion'' respectively.
When we compute the delay due to the source we cannot apply the weak-field approximation, which makes the calculation hard; however, this effect gives a constant frequency shift, which we cannot observe.

The potential from the companion, on the other hand, modulates the signal and is therefore observable. So, like we did for the Earth-Sun system we can write the proper-to-coordinate time ratio as
%
\begin{subequations}
\begin{align}
\dv{T}{t} &= 1 - \frac{G m_c}{c^2 \abs{x_p - x_c}} - \frac{v_p^2}{2 c^2}  \\
\dv{T}{u} &\approx \frac{P_b}{2 \pi } \qty(1 - \frac{G}{c^2} \frac{2 m_c m_p + 3 m_c^2}{2 a (m_p + m_c)})\qty(1 - e \cos u \qty(1 + \frac{G}{c^2} \frac{m_c (m_p + 2 m_c)}{a (m_p + m_c)}))  \\
&= \frac{P_b}{2\pi } \qty(1 - e \cos u) - \gamma \cos u
\,,
\end{align}
\end{subequations}
%
where we plugged in the equation for relativistic eccentric motion; \(u\) is the adimensional angular parameter describing the orbit, which is related to the coordinate time by \(u - e \sin u = (2 \pi / P_b) (t - t_0 )\), while \(e\) is the eccentricity.
This expression is post-Newtonian: it is valid up to first order in \(G\).
Its simpler formulation is given in terms of the \textbf{Einstein parameter}
%
\begin{align}
\gamma = e \frac{P_b}{2 \pi } \frac{G}{c^2} \frac{m_c (m_p + 2 m_c)}{a (m_p + m_c)}
= e \qty(\frac{P_b}{2 \pi })^{1/3} \frac{G^{2/3}}{c^2}  \frac{m_c (m_p + 2 m_c)}{(m_p + m_c)^{4/3}}
\,.
\end{align}

What is the magnitude of this effect? Well, we define the Einstein time difference \(\Delta_{E}\) by \(T = t - \Delta_{E}\); we can differentiate this equality with respect to \(u\) to find
%
\begin{align}
\dv{ \Delta_{E}}{u} = \dv{t}{u} - \dv{T}{u}
= \frac{P_b}{2 \pi } \qty(1 - e \cos u) -
\frac{P_b}{2\pi } \qty(1 - e \cos u) + \gamma \cos u = \gamma \cos u
\,,
\end{align}
%
where we computed \(\dv*{t}{u}\) starting from \(t = (P_b / 2\pi ) (u - e \sin u) + \const\), which comes from the definition.
So, the amplitude of the modulation comes from the parameter \(\gamma \), which has the units of a time, roughly on the order of magnitude of the period of the binary multiplied by the ratio between the Schwarzschild radius of the companion and the semimajor axis of the binary: for the Hulse-Taylor pulsar it comes out to be of the order \SI{4.292}{ms}.

\subsubsection{Rømer delay (source)}

Like in the Solar System, we have the Rømer delay, which is given in general by
%
\begin{align}
\Delta_{R} = \frac{\hat{z} \cdot x_{pb}}{c}
\,,
\end{align}
%
where \(\hat{z}\) is the direction of the Earth, while \(x_{pb}\) is the vector connecting the binary barycenter to the pulsar.

The polar coordinates \(r_1 \) and \(\psi \) for a Keplerian orbit are given by
%
\begin{align}
r_{pb} = r_1 = a_1 \qty(1 - e \cos u )
\qquad \text{and} \qquad
\cos \psi = \frac{\cos u - e}{1- e \cos u}
\,,
\end{align}
%
where \(u\) is the angular parameter of the orbit, while \(a_1 \) is its semimajor axis.

The Rømer delay can then be written in terms of the \emph{observation angle} \(\iota \) (the angle between the observation direction and the normal to the orbital plane) and the \emph{argument of periapsis} \(\omega\) (the angle of the orbit at the point in which the body is closest to us --- this is simply \(\pi /2\) for circular orbits, it can differ for elliptic ones):
%
\begin{align}
\Delta_{R} = r_1 \sin \iota \sin(\omega+\psi )
&= r_1 \sin \iota \qty(\cos \psi \sin \omega + \cos \omega \sin \psi )  \\
& =r_1 \sin \iota \qty( (\cos u - e) \sin \omega + \sqrt{1 - e^2} \sin u \cos \omega )
\,.
\end{align}
%
% where \(\iota \) is the observation angle, while \(\psi \) is the angle from the line of notes (see drawing).

Note that if \(\psi \) is measured from the \emph{line of nodes}, the line where the orbital plane crosses the plane drawn from the barycenter and normal to the observation direction, \(\omega + \psi \) gives the angle of the pulsar from the line of nodes.

This is all classical --- the relativistic corrections, however, are large.
We will not do the calculation, instead we show the result: it is written similarly,
%
\begin{align}
\Delta_{R} = a_1 \sin \iota
\qty((\cos u - e_r ) \sin \omega + \sqrt{1 - e^2_{\theta }} \sin u \cos \omega )
\,,
\end{align}
%
where
%
\begin{subequations}
\begin{align}
e_{r, \theta } &= e (1 + \delta_{r, \theta })  \\
\delta_{r} &= \frac{G}{c^2} \frac{3 m_p^2 + 6 m_p m_c + 2 m_c^2}{a (m_p + m_c)}  \\
\delta_{\theta } &= \frac{G}{c^2}
\frac{(7/2) m_p^2 + 6 m_p m_c + 2 m_c^2}{a (m_p + m_c)}
\,.
\end{align}
\end{subequations}

To get a feeling for the numbers: if the masses of the objects are comparable and similar to \(m\), the order of magnitude of the corrections is given by the ratio \(G m / a c^2\), the ratio of the Schwarzschild radius corresponding to the mass to the binary's radius.

Also, note that here the advance of the periastron is much more significant than it is for Mercury, it is given by
%
\begin{align}
\expval{\dot{\omega}}
= \frac{3G }{c^2} \qty(m_p +m_c)^{2/3} \qty( \frac{2 \pi }{P_b})^{5/3}
\frac{1}{1 - e^2} \sim \SI{4.2}{\degree / yr}
\,.
\end{align}

\subsubsection{Shapiro delay (source)}

The computation to get the Shapiro delay at the source is similar to that of the Solar System.
We only need to account for the companion's gravitational field, the pulsar's is included in the Einstein delay.

The expression comes out to be
%
\begin{align}
\Delta_{S} = - 2 \frac{G m_c}{c^3} \log
((1 - e \cos u) - \sin \iota \qty(\sin \omega \qty(\cos u - e) + \sqrt{1 - e^2} \cos \omega \sin u))
\,.
\end{align}

To get all the corrections we need to account for the \emph{relativistic aberration} due to the motion of the source, as well as the \emph{Doppler shift} due to the relative motion of the barycenters of the Solar System and the binary system.

% If we get all the Keplerian parameters and two of the post-Newtonian ones then we should know everything.

\subsection{Putting all the parameters together}

The parameters of the system can be categorized into:
\begin{enumerate}
    \item the pulsar's parameters: position, frequency, phase, spin-down and others;
    \item the Keplerian parameters of the orbit: \(P_b, T_0, x= (a/c) \sin \iota , e, \omega \);
    \item the relativistic corrections to the orbit: \(\dot{\omega}
    , \gamma, P_b, r, s, \delta_{\theta }, \dot{e}, \dot{x}\).
\end{enumerate}

The system is fully characterized by all the classical parameters, plus two of the relativistic ones.
If we can measure three relativistic parameters, we can use the third as a GR test.

We extract from the data the classical parameters, the precession of the periastron \(\expval{\dot{\omega}}\) and the Einstein parameter \(\gamma \). Then,  we use the derivative of the orbital period \(\dot{P}\) to test GR.
% We measure \(P_b, T_0, x= a \sin \iota /c, e, \omega \) and the post-Newtonian \(\dot{\omega}, \gamma \) and finally we make a prediction for \(\dot{P}\). This matches the data very well.

The prediction from the calculations is
%
\begin{align}
\dot{P}_{b, GR} &= - \frac{192 \pi G^{5/3}}{5 c^{5}}
\frac{m_p m_c}{(m_p + m_c)^{1/3}} \qty( \frac{P_b}{2 \pi })^{- 5/3}
\frac{1}{(1 - e^2)^{7/2}} \qty(1 + \frac{73}{24} e^2 + \frac{37}{96}e^{4})  \\
&\approx -\num{2.40242(2)e-12}
\,,
\end{align}
%
and the measured value is \(- \num{2.4056(51)e-12}\).
The GR prediction (not a fit to the data, the theoretical prediction starting from the measured parameters!) fits the data extremely well.

\section{GW from a rotating rigid body}

\subsubsection{Classical mechanics recap}

The moment of inertia tensor can be defined as
%
\begin{align}
I^{ij} = \int \dd[3]{x} \rho (x) \qty(r^2 \delta^{ij} - x^{i} x^{j})
\,.
\end{align}

There exists an \emph{orthogonal} frame in which this tensor is diagonal, its eigenvalues are the moments of inertia, its eigenvectors are the axes of inertia: \(I^{\prime, ij} = \diag{I_1 , I_2 , I_3 }\). This is called the \textbf{body frame}.
The moments in this frame can be written as:
%
\begin{align}
I_1' = \int \dd[3]{x}' \rho \qty(\qty(x_1^{\prime, 2} + x_2^{\prime, 2} + x_3^{\prime, 2}) \delta^{11} - x^{\prime, 1} x^{\prime, 1})
= \int \dd[3]{x} \rho \qty(x_2^{\prime, 2} + x_3^{\prime, 2})
\,.
\end{align}

If the body is shaped like an ellipsoid with axes \(a, b, c\) and mass \(m\) then we have
%
\begin{align}
I_1 = \frac{m}{5} \qty(b^2 +c^2)
\,.
\end{align}

In general the angular momentum is given in terms of the angular velocity by the tensor equation \(J_{i} = I_{ij} \omega_{j} \); in the body frame this simplifies to three scalar equations: \(J_1' = I_1' \omega_1' \) and so on.

The rotational kinetic energy is given by
%
\begin{subequations}
\begin{align}
E _{\text{rot}} &= \frac{1}{2} I_{ij} \omega_{i} \omega_{j} \\
&= \frac{1}{2} I_i' \omega_{i}^{\prime, 2}
\,,
\end{align}
\end{subequations}
%
where the last equality holds in the body frame.

\subsection{GW emission from a rigid body}

Suppose we have a body spinning its principal axis \(x_3'\) with angular velocity \(\omega_{r}\); then the position of any point can be recovered from the body frame using a time-dependent rotation matrix \(R_{ij}\):
%
\begin{align}
x^{\prime }_{i} = R_{ij} x_{j} =
\left[\begin{array}{ccc}
\cos(\omega_{r}t) & \sin(\omega_{r}t) & 0 \\
- \sin(\omega_{r} t) & \cos(\omega_{r}t) & 0 \\
0 & 0 & 1
\end{array}\right]_{ij} x_{j}
\,.
\end{align}

The inertia tensor transforms as \(I^{\prime }_{ij} = R_{ik} R_{jl} I_{kl} = (R I R^{\top})_{ij}\), and we can reverse this relation to express the time-dependent inertia tensor as a time-dependent rotation of the fixed body frame tensor:
%
\begin{align}
I = R^{\top} I' R
\,.
\end{align}

Let us write this transformation law explicitly, using \(c = \cos(\omega_{r}t)\) and \(s = \sin(\omega_{r}t)\) for compactness and writing only the \(xy\) components, since the \(I_{3j}\) components for any \(j\) are unchanged:
%
\begin{align}
R^{\top} I' R &= \left[\begin{array}{cc}
c & -s \\
s & c
\end{array}\right]
\left[\begin{array}{cc}
I_1  & 0 \\
0 & I_2
\end{array}\right]
\left[\begin{array}{cc}
c & s \\
-s & c
\end{array}\right]  \\
&= \left[\begin{array}{cc}
c I_1  & -s I_2  \\
cI_1  & c I_2
\end{array}\right]
\left[\begin{array}{cc}
c & s \\
-s & c
\end{array}\right]  \\
&= \left[\begin{array}{cc}
c^2 I_1 - s^2 I_2  & cs (I_1 - I_2 )  \\
cs (I_1 - I_2 ) & s^2 I_1 - c^2 I_2
\end{array}\right]  \\
&= \left[\begin{array}{cc}
\frac{I_1 + I_2 }{2} + \frac{I_1 - I_2 }{2} \cos(2 \omega_{r}t)
& \frac{I_1 - I_2 }{2} \sin(2 \omega_{r} t) \\
\frac{I_1 - I_2 }{2} \sin(2 \omega_{r} t)
& \frac{I_1 + I_2 }{2} - \frac{I_1 - I_2 }{2} \cos(2 \omega_{r}t)
\end{array}\right]
\,,
\end{align}
%
where we used the identities \(2\cos^2x = \cos(0) + \cos x\), \(2 \sin^2(x) = \cos(0) - \cos x\) and \(2 \sin x \cos x = \sin (2x) \).

The second mass moment \(M^{ij}\) can be written in the nonrelativistic approximation as
%
\begin{align}
M^{ij} = \frac{1}{c^2} \int \dd[3]{x} T^{00} (x) x^{i} x^{j}
\approx \int \dd[3]{x} \rho (x) x^{i} x^{j} =
 - I^{ij} + \int \dd[3]{x} \rho (x) r^2 \delta^{ij}
\,.
\end{align}

\begin{claim}
When we compute the time derivative of \(M^{ij}\) the integral of \(\rho r^2 \delta^{ij}\) does not contribute.
\end{claim}

\begin{proof}
We can write the term (without the delta, which is a constant anyway) as
%
\begin{align}
\int \dd[3]{x} \rho (x) r^2  &=
\Tr[ \int \dd[3]{x} \rho (x) x^{i}x^{j}] = \Tr[K^{ij}]
\,,
\end{align}
%
but this \(K^{ij}\) is a tensor, so it can be written as \(K = R^{\top} K' R\) where \(K'\) is in the body frame.
Then, we can use the cyclic property of the trace to show that \(\Tr[R^{\top} K' R] = \Tr[K' R R^{\top}] = \Tr[K']\), which is a constant.
\end{proof}

So, we can write \(\ddot{M}^{ij} = - \ddot{I}^{ij}\): explicitly,
%
\begin{align}
\ddot{M}^{11}= - \ddot{M}^{22} = - \ddot{I}^{11} = + \ddot{I}^{22}
&= 2 \omega_{r}^2 (I_1 - I_2 ) \cos(2 \omega_{r} t)   \\
\ddot{M}^{12} = - \ddot{I}^{12}
&= 2 \omega_{r}^2 (I_1 - I_2 ) \sin(2 \omega_{r} t)
\,,
\end{align}
%
while the other components of \(\ddot{M}^{ij}\) vanish.

% Suppose we had a body whose angular momentum is not aligned with the moment of inertia: we can use Euler angles to express the rotation matrix.
We have general expressions for the GW amplitudes in the two polarizations if we observe from a generic direction \((\theta, \phi )\). Let us select a direction \((\theta, \phi ) = (\iota , 0)\) --- we do not lose generality by fixing \(\phi \), since it only amounts to a phase shift.
Then, the expressions give us:
%
\begin{align}
h_{+} &= \frac{G}{rc^{4}} \qty(\ddot{M}_{11} - \ddot{M}_{22} \cos^2 \iota ) = \frac{4}{r}  \frac{G \omega_{r}^2}{c^{4}} (I_1 - I_2 ) \frac{1 + \cos^2\iota }{2}  \cos(2 \omega_{r}t )   \\
h_{ \times } &= \frac{G}{rc^{4}} 2 \ddot{M}_{12} \cos \iota
= \frac{4}{r} \frac{G \omega_{r}^2}{c^{4}} (I_1 - I_2 ) \cos \iota \sin(2 \omega_{r} t)
\,.
\end{align}

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% UMB-CS110-2015S: Introduction to Computing
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/UMB-CS110-2015S
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\def \topDirectory {.}
\def \texDirectory {\topDirectory/src/main/tex}

\documentclass[12pt,letterpaper,twoside]{article}
\usepackage{\texDirectory/template/style/directives}
\usepackage{\texDirectory/template/style/assignment}
\input{\texDirectory/template/config}

\begin{document}

\doc{title}{Quiz 2(b)}
\doc{date-pub}{Mar 05, 2015 at 01:00 PM}
\doc{date-due}{Mar 05, 2015 at 11:00 PM}
\doc{points}{4}

\prepare{header}

\section*{Question 1}

Write a program \texttt{PrimeCounter.java} that takes a command-line argument \texttt{N} and finds the number of primes less than or equal to \texttt{N}. Use it to \textbf{efficiently} print out the number of primes less than or equal to 10 million.

\section*{Question 2}

Write a program \texttt{MatrixMultiply.java} that asks for elements of two three-by-three matrix and prints their multiplication.

\prepare{footer}

\end{document}


\section{Assignment}

The reader should try to take a look at the factorial math thesis, as that is what this document has gone through, but not explained. It should give a hint on how math can be applied to programming and maybe why it could make some problems hard to solve through programming.

\href{https://en.wikipedia.org/wiki/Factorial}{Factorial}

As this document is only theoretical, there is no practical assignment.


\section{LEDs}\label{s:leds}

\project{Controlling LEDs with BASIC}{In this project, you will learn how to connect and control LEDs (Light Emitting Diode) with the Raspberry Pi.}

\subsection*{Equipment Required}

You will need:
\begin{itemize}
\item Your Raspberry Pi
\item 400 Point Breadboard
\item 1 x Red LED
\item 1 x Yellow LED
\item 1 x Green LED
\item 3 x 330$\Omega$ Resistors
\item 4 x M/F jumper wires
\end{itemize}

\subsection*{The Parts}

In this first circuit, you will be connecting three LEDs to the GPIO header of your Raspberry Pi and using BASIC to turn the LEDs on and off.

\textbf{It is important that you read this section, as you need to understand the Raspberry Pi GPIO pins, how the holes in the breadboard are connected together, and which leg of the LED is which.}

Before you build the circuit, let us look at the parts you are going to use.

\subsubsection*{Raspberry Pi GPIO Pins}

\limage{0.2}{gpio-pins}

First, lets look at the Raspberry Pi's `\textbf{GPIO}' pins.  GPIO stands for \textbf{General Purpose Input Output}.  It is a way the Raspberry Pi can control and monitor the outside world when connected to electronic circuits.  The Pi is able to control LEDs, turning them on and off, or motors, or many other things.  It is also able to detect whether a switch has been pressed, or what the temperature is, or whether there is light.  With this CamJam EduKit you will learn to control LEDs and a buzzer, and detect when a button has been pressed.

The disgram on the left shows the pin layout for all the Raspberry Pi models built for the last few years; they have 40 GPIO pins.  The original Raspberry Pi models A and B only had 26 pins and will not work for these worksheets.

Some pins have different functions.  There are pins that provide power at 5 volts and 3.3 volts, ground pins (0 volts), input/output pins and some pins that interface to external circuits in more complex ways (which are not currently supported by BMC64).  You are going to use 8 of the GPIO pins and the ground pins in these worksheets.  Note that all of the pins marked as ground (GND) are equivalent.

In BMC64, we access the GPIO pins through the User Port.  For these worksheets, we will assume that you are operating in C64 mode.  In this case, the user port is represented by memory location 56577.  Each memory location can hold a number from 0 to 255.  This is because the computer uses 8 \textbf{bits} to hold each value and each bit can be 0 or 1.  Table~\ref{tbl:port-pins} shows the numerical value of each bit and which GPIO pin it accesses.

\begin{table}[htb]
\caption{GPIO pins used by the user port.}\label{tbl:port-pins}
\begin{center}
\begin{tabular}{|l|r|r|r|r|r|r|r|r|}
\hline
\textbf{Bit}      &   7 &  6 &  5 &  4 &  3 &  2 &  1 &  0 \\
\hline
\textbf{Value}    & 128 & 64 & 32 & 16 &  8 &  4 &  2 &  1 \\
\hline
\textbf{GPIO Pin} &  21 & 20 & 26 & 16 &  8 & 25 & 24 &  4 \\
\hline
\end{tabular}
\end{center}
\end{table}

The bits are numbered from the right (lowest valued) position, starting at bit number 0.  To specify which bits we want turned on, we add the value for those bits.  For example, if we want to access bits 1 and 0, we see that they have values 2 and 1.  Now, $2+1 = 3$ which means we would need a value of 3 to represent those two bits.

Each pin in the user port can be either an input or an output.  Location 56579 holds this direction information and is known as the \textbf{data direction register} or \textbf{DDR}.  In the DDR, each bit that is 0 means the corresponding bit in the user port is an input and each bit that is 1 means the user port bit is an output.  The default value in the DDR is 0, so all of the pins default to inputs.  If the DDR had a value of 255, then all of the pins would be outputs and a value of 15 would mean the first four pins would be inputs and the last four pins would be outputs.

The BASIC commands you will use to read and write these values are \textbf{PEEK} and \textbf{POKE}.  Since you will be using the user port and DDR locations a lot, it will be best to put them into variables.  Let's use \textbf{UP} and \textbf{DDR} for these purposes.  In this worksheet, you will be performing output, so you will need to be POKEing values into UP and DDR.

To set all of the user port pins to inputs, you will use \textbf{POKE DDR,0} and to set them all as outputs, you use \textbf{POKE DDR,255}.  When pins are set as inputs, you can read the state of them using \textbf{PEEK(UP)} and if you want to set pins to be on, then you will use \textbf{POKE UP,X} (where X will represent which pins are on and which are off).

In this way, using the user port, we will be able to access and control up to eight devices at the same time.

\subsubsection*{The Breadboard}

The breadboard is a way of connecting electronic components to each other without having to solder them together.  They are often used to test a circuit design before creating a Printed Circuit Board (PCB).

\limage{0.7}{breadboard}

The holes on the breadboard are connected in a pattern.

With the breadboard in the CamJam EduKit, the top row of holes are all connected together---marked with blue dots.  And so are the second row of holes---marked with red dots.  The same goes for the two rows of holes at the bottom of the breadboard.

In the middle, the columns of holes are connected together with a break in the middle.  So, for example, all the green holes marked are connected together, but they are not connected to the yellow holes, nor the purple ones.  Therefore, any wire you poke into the green holes will be connected to other wires poked into the other green holes.

\subsubsection*{The LEDs}

\limage{0.15}{leds}

Three LEDs are supplied in the EduKit---one red, one yellow, and one green.  LED stands for Light Emitting Diode.  An LED glows when electricity is passed through it.

When you pick up an LED, you will notice that one leg is longer that the other.  The longer leg (known as the `anode') is always connected to the positive supply of the power supply.  The shorter leg (known as the `cathode') is connected to the negative side of the power supply (known as `ground').

LEDs will only work if power is supplied the correct way round (i.e. if the `polarity' is correct).  You will not break the LEDs if you connect them the wrong way round---they will just not light.  If you find that they do not light in your circuit, if may be because they have been connected the wrong way round.

\subsubsection*{The Resistors}

\limage{0.1}{resistors}

Resistors are a way of limiting the amount of electricity going through a circuit; specifically, they limit the amount of `current' that is allowed to flow.  The measure of resistance is called the Ohm ($\Omega$), and the larger the resistance, the more it limits the current.  The value of a resistor is marked with coloured bands along the length of the resistor body.

The EduKit is supplied with two sets of resistors.  There are three 330$\Omega$ resistors and one 4.7k$\Omega$ (or 4700$\Omega$) resistor.  In the LED circuit, you will be using the three 330$\Omega$ resistors.  You can identify the 330$\Omega$ resistors by the colour bands along the body.  The colour coding will depend on how many bands there are on the resistors supplied:
\begin{itemize}
\item If there are four colour bands, they will be Orange, Orange, Brown, and then Gold.
\item If there are five bands, then the colours will be Orange, Orange, Black, Black, Brown.
\end{itemize}

You have to use resistors to connect LEDs up to the GPIO pins of the Raspberry Pi.  The Raspberry Pi can only supply a small current on the GPIO pins.  The LEDs will want to draw more and, if allowed to, they will burn out the Raspberry Pi.  Therefore, putting resistors in the circuit will ensure that only this small current will flow and the Pi will not be damaged.

It does not matter which way round you connect the resistors.  Current can flow in both directions through them.

\subsubsection*{The Jumper Wires}

\limage{0.2}{jumpers}

Jumper wires are used on breadboards to `jump' from one connection to another.  The ones you will be using in this circuit have different connectors on each end.  The end with the `pin' will go into the breadboard, and is known as the `male' end.  The end with the piece of plastic with a hole in it will go onto the Raspberry Pi's GPIO pins.  This is the `female' end.

The jumper wires supplied in the EduKit will vary in colour and are unlikely to match the colours used in the diagrams.

\subsection*{Building the Circuit}

While you can build the circuit with the Pi turned on, it is best to turn it off at this stage.

You will be using one of the `ground' (GND) pins to act like the `negative' or 0 volt end of a battery.

\limage{0.5}{led-circuit}

The `positive' ends of the battery will be provided by three of the other GPIO pins, one for each of the three LEDs.  You will be using the pins marked 4, 24, and 25 for the Red, Yellow, and Green LEDs respectively.

When they are `taken high', which means they output 3.3 volts, the LEDs will light.

Now take a look at the circuit diagram on the left.

The power for each LED will be provided by the Pi, from GPIO pins 4, 24, and 25.  You can control them from BASIC, meaning you can make the GPIO pins supply either 0 volts (off) or 3.3 volts (on).

There are, in fact, three separate circuits in the diagram.  Each one consists of the power supply (the Pi), an LED that lights when power is applied, and a resistor to limit the current that can flow through the circuit.

Each circuit is going to share a `common ground rail'.  In other words, you will be connecting all of the circuits to the same `ground' (0 volts) pin of the Raspberry Pi.  You are going to use the second row up from the bottom of the breadboard.  Remember that the holes on the two top and two bottom rows are connected together?  So, connect one of the jumper wires from the third pin from the left on the top row of the Pi to the second row up of the breadboard, as shown in the diagram (the black wire).

Next, push three LEDs legs into the breadboard, with the long leg on the right as shown in the circuit diagram.

Then connect the three 330$\Omega$ resistors between the `common ground rail' and the left leg of the LEDs.  You will need to bend the legs of each of the resistors to fit, but please make sure that the wires of each leg do not touch one another.

Lastly, using three jumper wires, complete the circuit by connecting pins 4, 24, and 25 to the right-hand leg of each LED.  These are shown here with orange, yellow, and blue wires.

You are now ready to write some code to switch the LEDs on.

\subsection*{Code}

Follow the instructions in Section~\ref{s:intro} to turn on your Pi.  Type in the following code:
\begin{basic}
10 REM LED
20 REM SETUP LOCATIONS
30 UP=56577:DDR=56579
40 REM PIN VALUES FOR LEDS
50 R=1:Y=2:G=4:A=R+Y+G
60 REM SET PINS 0,1,2 FOR OUTPUT
70 POKE DDR,A
80 REM CLEAR USER PORT (ALL PINS OFF)
90 POKE UP,0
100 PRINT "LEDS ON"
110 POKE UP,A
120 PRINT "WAIT FOR ONE SECOND"
130 FOR I=1 TO 720:NEXT I
140 PRINT "LEDS OFF"
150 POKE UP,0
\end{basic}

Once you have typed all the code and checked it, save it to disk as ``2 LED''.

So, what is happening in the code?  Let's go through it a section at a time:

\begin{basic}
10 REM LED
20 REM SETUP LOCATIONS
30 UP=56577:DDR=56579
\end{basic}

\explain{This introduces the variables UP, to refer to the user port, and DDR, to refer to the data direction register.  This is so that we don't have to type the locations multiple times.  It will also mean that we are less likely to make mistakes.  If you decide to try these worksheets using a different emulator model (eg. VIC20), then it will likely have different locations for these two.  Putting them into variables means that you will have to make fewer changes to be able to run the code.}

\begin{basic}
40 REM PIN VALUES FOR LEDS
50 R=1:Y=2:G=4:A=R+Y+G
\end{basic}

\explain{This is telling BASIC that you are going to use pins 0, 1, and 2 (with values 1, 2, and 4) of the user port.  To reference all of them together, remember that we just add them.}

\begin{basic}
60 REM SET PINS 0,1,2 FOR OUTPUT
70 POKE DDR,A
80 REM CLEAR USER PORT (ALL PINS OFF)
90 POKE UP,0
\end{basic}

\explain{We have connected the three LEDs to pins 0, 1, and 2 of the user port so we need to tell BASIC that we are going to use them for outputting information, which means you are going to be able to turn the pins `on' and `off'.  We have also set all pins on the user port to 0 (off) so that the LEDs will start in the off state.}

\begin{basic}
100 PRINT "LEDS ON"
\end{basic}

\explain{This line prints some information to the screen so you can see what the program is trying to do.}

\begin{basic}
110 POKE UP,A
\end{basic}

\explain{This turns the three LEDs `on'.  What this actually means is that these three pins are made to provide power of 3.3 volts to the three GPIO pins.}

\begin{basic}
120 PRINT "WAIT FOR ONE SECOND"
130 FOR I=1 TO 720:NEXT I
\end{basic}

\explain{These two lines print a message to the screen, followed by an empty loop which will pause BASIC for about 1 second.}

\begin{basic}
140 PRINT "LEDS OFF"
150 POKE UP,0
\end{basic}

\explain{This prints out a message again, then turns all three LEDs off.}

\subsection*{Running the Code}

You are now ready to run the code.  Run it by typing in \textbf{RUN} followed by the \textbf{ENTER} or \textbf{RETURN} key.  You should see your LEDs light for one second, then turn off again.

If you find that the code does not run correctly there may be an error in the code you have typed.  You should check and re-edit the code, save again and re-run it.  Also check that you have configured the GPIO option to enable the user port in the settings.

\note{Do not disassemble this circuit as it will be used in the following worksheets.}


\section{Empirical Application}
\subsection{Out-of-sample Forecast under Conditional Homoskedasticity}
Here we apply our theoretical results to forecasting excess US stock returns based on Goyal and Welch's \cite{goyal_welch_RFS2008} monthly dataset \footnote{Data can be obtained at \url{http://www.hec.unil.ch/agoyal/}.}. Yin \cite{yin2012} shows that there is detected structural breaks in the simple univariate forecasting model with stock market variance but the evidence supporting breaks is not strong: the BIC is almost the same between the break model and the stable model, so there is no clear cut on which one we should choose to forecast. Model selection may not be a good option in this case, so we use the model averaging method studied in this paper to forecast out-of-sample and compare its results with other popular forecast combination methods.

Our monthly data runs from September 1974 to December 2012. We reserve the last 36 observations as the evaluation sample while the rest as training sample. The estimation is based on fixed window, though including more window schemes, namely, recursive window and rolling window, is desirable. To calculate optimal CV weights, for the penalty term, we choose the value corresponding to the one with two regressors and with trimming parameter of 0.15 in Hansen \cite{hansen2009averaging} \footnote{MATLAB code to compute the penalty coefficient at different combinations of number of regressors and trimming parameters can be obtained from the author upon request.}.We compute the out--of--sample MSFE for CV averaging, post-break window, equal weights, Bayesian averaging and GR combination and the results are shown in table 1.

From table 1, we can see that the CV model averaging performs significantly better than equal weights, Bayesian model averaging and Granger--Ramanathan (\textbf{GR}) combination in this empirical study. Granger--Ramanathan combination performs the worst among all five methods confirming the prevailing forecast combination puzzle which says that equal weights scheme dominates the GR combination. One fact worth pointing out is that the post-break window forecast performs equally well as CV averaging in this case.


\documentclass[a4paper]{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{textcomp}
\usepackage[english]{babel}
\usepackage{amsmath, amssymb}
\usepackage{physics}

% figure support
\usepackage{import}
\usepackage{xifthen}
\pdfminorversion=7
\usepackage{pdfpages}
\usepackage{transparent}
\newcommand{\incfig}[1]{%
	\def\svgwidth{\columnwidth}
	\import{./figures/}{#1.pdf_tex}
}

\pdfsuppresswarningpagegroup=1
\title{ME 766 - High Performance Scientific Computing}
\begin{document}
\section*{Introduction}
Course content
\begin{itemize}
	\item Introduction to HPSC and scientific computing
	\item Processor performance, memory hierarchy, multi-core computing and vector computing
	\item Introduction  to parallel programming concepts and parallel algorithms
	\item Effective use of Bash scripting
	\item Effective use of tools like git, SVN, Mercurial etc
	\item OpenMP, MPI, GPGPU, Vector programming
	\item debuggers
	\item Performance analysis
	\item Numerical methods, applications of
\end{itemize}

This course is all about making you a better researcher. It
is very hands on.
The processors keep changing every two years.
How to exploit hardware-level optimization techniques will also
change every two years. This course is targeted at those who are
pursuing a project - help them improve their code over there. So,
include this in your SoPs and course description.

The biggest component of this course the project. All the best.
This project is a distillation of the professor's experience over
B.Tech. This man wants to \emph{teach}. So you better \emph{imbibe}.

The traditional scientific method paradigms have been about observation
$\to $ hypothesis $\to $ testing by experimentation and going back
to refine the hypothesis. Think Archimedes' eureka moment and
the fable of an apple falling on Newton's head.

A more concrete example would be the Higgs Boson discovery; the
existence of the Higgs Boson was hypothesized much earlier.
The experiments at LHC set out to test this hypothesis and the
veracity of the standard model in general.

The computational modelling paradigm is much recent compared to the
traditional paradigms. It picked up during the second world war,
the Los Alamos project, for example

Alexei Krylov - general in the imperial soviet army (remember Krylov
subspaces and all that), his work led to important iterative methods
in linear algebra (typically systems with millions to billions of students).
Professor gave an example of computational fluid dynamics (CFD). I imagine
that a lot of examples will be from mechanical engineering.

Consider trying to solve for $\vec{x}$ in a linear system of size $n$

\begin{equation}
	\mathrm A \vec{x} = \mathrm B.
\end{equation}
If you do Gaussian elimination, you'll have an algo that goes as
$\mathcal{O}(n^3)$. Imagine if $n = 1\text{M}$. Unscalable.  Practical
applications using this method would be intractable even on the largest supercomputers.
Using iterative solvers would reduce the complexity to something
like  $\mathcal{O}(n^2)$. If you're doing any serious computational
work, you will be using these methods instead of na\"ive Gaussian
elimination kind of methods.

\subsection*{Floating Point Operations (FLOPs)}
A FLOP is a binary operation between two floating point operands.
ENIAC was able to achieve 300 FLOPs per second (FLOPS, note the capitalisation). The ENIAC was used for writing artillery firing tables.
Some factors that would come in
\begin{itemize}
	\item Elevation of target
	\item Horizontal distance
	\item Aerodynamics of the shell
		\begin{itemize}
			\item wind direction
			\item geometry and weight of the shell
			\item density of air
			\item altitude
		\end{itemize}
	\item \ldots
\end{itemize}

ENIAC was insanely slow by modern computing standards. Head over to
\texttt{https://top500.org}. (AMD would have us believe that exascale
begins with their \texttt{Instinct MI100} GPUs). We are on the cusp of
the first exascale
machine. Thanks to Alexei Krylov's work.

The fastest supercomputer (SC) is Fugaku at RIKEN (Kobe, Japan). It
clocks a peak 537 Peta FLOPS, and consumes about 30 MegaWatts (as much
as a small town/city). IITB's SC, SpaceTime2, costs a few crores in
electricity alone. Fugaku is about 4 times more power efficient than
SpaceTime2. SpaceTime2 is also some four years old, and newer machines
are becoming more and more power efficient.

\subsection*{Application Areas}
\begin{itemize}
	\item CFD - sneeze and cough droplet simulations in these trying times
	\item Structural Mechanics - stresses, strains, deformations, fractures and all that
	\item Computational Electromagnetics - scattering of EM waves off e.g. an aircraft body (RADAR signature detection for military aircraft). Stealth
	\item Magnetohydrodynamics - e.g. molten ferromagnetic material flowing. Nuclear fusion reactors use a blanket of Lithium-Lead liquid
	\item Computational astrophysics - mostly exclusively computational, i.e. no experimental counterpart. You cannot really sit around to observe the evolution of a galaxy over a gajillion years
	\item Computational Finance - models made using partial differential equations (PDEs), usually involve all time scales (microtrading to long term investments et cetera)
	\item Molecular Dynamics (MD)
\end{itemize}

Computational science cannot completely replace experiments, of course.
You can explore the parameter space freely in computational models, and
then cherry pick promising results to be tested by experimental rigs.
``Computational science is the partial truth fully revealed. Experimental
science is the full truth partially revealed". When you make a computational
model, you have to start off with some simplifications.

\subsection*{What makes a good computational scientist}
\begin{itemize}
	\item Domain specific knowledge - obviously
	\item Programming proficiency - you should be able to write your models and math in code. You need to be able to use these large
		machines \emph{efficiently}.
	\item Excellent background in applied mathematics and numerical analysis
\end{itemize}

Your model on a machine should be able to procure a three-day prediction
in a few hours. It should also not run at some 30 megawatts just for
a three-day prediction for one city. We will use parallel computing
and serial computing optimization techniques.

Related to course logistics - the domain specific knowledge should
come from your end. This course is targeted at senior undergrads and
postgrads. All the best.

Standard benchmarking programs do exist for supercomputers. E.g. LINPACK.

\section*{Why Parellel?}
So, by now we have learned that the high performance comes from
parallelization. Why can we not just go on making more and more
powerful sequential processing hardware?

\subsection*{Moore's Law}
VLSI circuits are a lot of transistors on a chip. We can only pack
so many transistors on a processor. What is the best that we can do?
One transistor per lattice site of the solid state material, if you
could get that level of control.

Point is that there is a physical limit to the number of transistors
that you can pack into a single chip. We are at the cusp of where
Moore's law will break down. Any year now.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.8\textwidth]{figures/moore.png}
	\caption{Screenshot of a curve showing Moore's law}
	\label{fig:figures-moore-png}
\end{figure}

The MOSFET is a building block of these chips; each MOSFET functions
as a transistor.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.8\textwidth]{figures/mosfet.png}
	\caption{MOSFET}
	\label{fig:figures-mosfet-png}
\end{figure}

Today in 2021, gate sizes in production are at 5 nanometers. The
Apple M1 chips are built on a 5 nm tech. In research labs people
already have 1 nm gates and sizes.

While Moore's law is holding up just fine (for now) Dennard scaling
has already broken down.

\begin{table}[h]
	\centering
	\caption{Dennard Scaling of MOSFETs (1974)}
	\label{tab:dennard}
	\begin{tabular}{c c}
	 Parameter & Factor\\
	 \hline
	 Dimension & l/k\\
	 Voltage & l/k\\
	 Current & l/k\\
	 Capacitance & l/k\\
	 Delay time & l/k \\
	 Power dissipation & l/k$^2$
	\end{tabular}
\end{table}

When you specify something like 1 volt, you know that it is not
exactly one volt; thermal motion of electrons et cetera will cause
fluctuations. At smaller scales, the transients and fluctuations
become comparable.

Thermal losses also increase a lot as you go smaller and smaller.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.8\textwidth]{figures/proctrends.png}
	\caption{35 Years of Microprocessor Trend Data}
	\label{fig:figures-proctrends-png}
\end{figure}

Essentially, single-thread performance, switching times and clock frequencies
et cetera are asymptoting, which means that single core performance
is becoming peak. You absolutely cannot run away from parallel
programming in the future.

Also, parallel programmers are needed because automatic parallelization
is very non-trtivial and is an area of active research.
An anecdote - around 2004-2005 some of the first dual core machines
were showing up, and people started thinking seriously about
parallel programming. ``Why don't we invest in a compiler that does
automatic parallelization". They did, the code became slower by 50\%.

It is very easy to slow down the task by trying to parallelize it.

\subsection*{Amdahl's Law}
Your program will not be completely parallelizable; it will have both
serial and parallel components. If $P$ is the proportion of the program
that can be parallelized. Then the maximum speedup possible by parallelization
of the problem is
\begin{equation} \label{amdahl}
	S(N) = \frac{1}{1 - P + \frac{P}{N}}
\end{equation}
where $N$ is the number of processors available at hand.

Consider a trivial problem like addition of an array of integers.
Sequentially, this is an $O(n)$. If we have 10 cores, then we
can partition the array into 10 parts, relegate each part to one
core. We end up with 10 partial sums, and then a single core
could sequentially add the partial sums to return the final sum.

So, we have $N = 10$. The original sum has $n$ floating point sum
operations. The parallel algorithm has $n/10 + 10$ floating point
summations (loosely speaking, I am assuming multiple FP adds occuring
in parallel as a single FP add). So, consider table \ref{tab:tabam}.
\begin{table}[h]
	\centering
	\caption{Amdahl's Law in action ($N = 10$) for an array sum}
	\label{tab:tabam}
	\begin{tabular}{c c c}
		$n$ &   $n/10 + 10$ & Speedup\\
		100 & 20 & 5 \\
		1000 & 110 & 9 \\
		10000 & 1010 & 9.9 \\
		\vdots & \vdots & \vdots
	\end{tabular}
\end{table}

This is a very rosy picture. You can lose the speedup because of
communication overheads, latencies, load imbalance et cetera.
Communication between different nodes in the system takes a significant
amount of time. Think about RAM v/s cache, RAM is much slower than
the speed at which your processor works. If you have to communicate
with the RAM frequently during a single process, R.I.P. Doesn't
matter how many cores you have, you have a major bottleneck.

Also, there is no golden test that tells you how much parallism your
sequential algo has. You have to take your sequential algo to the
whiteboard.

%%%%% TODO: complete the last example on Laplace equation
%%%%% Note that there is an intentional typo in this part of the slides
\end{document}


\documentclass{article}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amsfonts}
\usepackage{tikz}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}
\usepackage{fancyvrb}
\usepackage{multirow}
\usepackage{booktabs}
\usetikzlibrary{automata,positioning}

%
% Basic Document Settings
%

\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\pagestyle{fancy}
\lhead{\hmwkAuthorName}
\chead{\hmwkClass\ : \hmwkTitle}
\rhead{\firstxmark}
\lfoot{\lastxmark}
\cfoot{\thepage}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

\setlength\parindent{0pt}

%
% Create Problem Sections
%

\newcommand{\enterProblemHeader}[1]{
    \nobreak\extramarks{}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
}

\newcommand{\exitProblemHeader}[1]{
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \stepcounter{#1}
    \nobreak\extramarks{Problem \arabic{#1}}{}\nobreak{}
}

\setcounter{secnumdepth}{0}
\newcounter{partCounter}
\newcounter{homeworkProblemCounter}
\setcounter{homeworkProblemCounter}{1}
\nobreak\extramarks{Problem \arabic{homeworkProblemCounter}}{}\nobreak{}

%
% Homework Problem Environment
%
% This environment takes an optional argument. When given, it will adjust the
% problem counter. This is useful for when the problems given for your
% assignment aren't sequential. See the last 3 problems of this template for an
% example.
%
\newenvironment{homeworkProblem}[1][-1]{
    \ifnum#1>0
        \setcounter{homeworkProblemCounter}{#1}
    \fi
    \section{Problem \arabic{homeworkProblemCounter}}
    \setcounter{partCounter}{1}
    \enterProblemHeader{homeworkProblemCounter}
}{
    \exitProblemHeader{homeworkProblemCounter}
}

%
% Homework Details
%   - Title
%   - Due date
%   - Class
%   - Section/Time
%   - Instructor
%   - Author
%

\newcommand{\hmwkTitle}{Homework\ \#2}
\newcommand{\hmwkDueDate}{\today}
\newcommand{\hmwkClass}{MATH6644 Iterative Methods}
\newcommand{\hmwkClassTime}{}
\newcommand{\hmwkClassInstructor}{}
\newcommand{\hmwkAuthorName}{Komahan Boopathy}

%
% Title Page
%

\title{
    \vspace{2in}
    \textmd{\textbf{\hmwkClass:\ \hmwkTitle}}\\
    \normalsize\vspace{0.1in}\small{\hmwkDueDate}\\
    %\vspace{0.1in}\large{\textit{\hmwkClassInstructor\ \hmwkClassTime}}
    \vspace{3in}
}

\author{\textbf{\Large\hmwkAuthorName}}
\date{}

\renewcommand{\part}[1]{\textbf{\large Part \Alph{partCounter}}\stepcounter{partCounter}\\}

%
% Various Helper Commands
%

% Useful for algorithms
\newcommand{\alg}[1]{\textsc{\bfseries \footnotesize #1}}

% For derivatives
\newcommand{\deriv}[1]{\frac{\mathrm{d}}{\mathrm{d}x} (#1)}

% For partial derivatives
\newcommand{\pderiv}[2]{\frac{\partial #1}{\partial #2}}

% Integral dx
\newcommand{\dx}{\mathrm{d}x}

% Alias for the Solution section header
\newcommand{\solution}{\textbf{\large Solution}}

% Probability commands: Expectation, Variance, Covariance, Bias
\newcommand{\E}{\mathrm{E}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\Cov}{\mathrm{Cov}}
\newcommand{\Bias}{\mathrm{Bias}}

\begin{document}
\maketitle
\thispagestyle{empty}

\pagebreak

\begin{homeworkProblem}
Consider a matrix
$$A =
\begin{bmatrix}
  6 & -1 & 0 \\ -1 & 5 & 0 \\ 0 & 0 & 2 \\
\end{bmatrix}$$

\paragraph{a.} \emph{Show that A is positive definite.}
\medskip

The eigenvalues of the matrix $A$ is found using $|A-\lambda
I|=0$. The characterisitic polynomial is $(\lambda^2 -11\lambda +
29)(\lambda-2)=0$. The eigenvalues are $\lambda=2,
\frac{11-\sqrt{5}}{2}, \frac{11+\sqrt{5}}{2}$, all of which are real
(due to symmetry of $A$) and greater than zero (due to positive
definiteness of $A$).

\paragraph{b.} \emph{Calculate the A-norm of the vector.}
$$ \mathbf{b} =
\begin{bmatrix}
  1 \\ 2 \\ -1
\end{bmatrix} $$
\medskip
The A-norm of vector $||b||_{A} = \sqrt{(b, Ab)}$. We find
$$ \mathbf{c} = \mathbf{Ab} =
\begin{bmatrix}
  6 & -1 & 0 \\
 -1 &  5 & 0 \\
  0 &  0 & 2 \\
\end{bmatrix}
\begin{bmatrix}
  1 \\ 2 \\ -1
\end{bmatrix} =
\begin{bmatrix}
  4 \\ 9 \\ -2
\end{bmatrix}
$$
The A-norm of vector $b$ is therefore $\sqrt{(b,c)}=4.899$

\paragraph{c.} \emph{Find a vector that is $A-$orthogonal to $\mathbf{b}$.}

This means that we need to find a vector $\mathbf{d}$ that such that
$(\mathbf{d}, \mathbf{Ab}) = (\mathbf{d}, \mathbf{c}) = 0$. We are
essentially finding a conjugate vector to $\mathbf{b}$. We can use
Gram-Schmidt orthogonalization to get that vector $\mathbf{d}$ that is
orthogonal to $\mathbf{c}$ as follows:
\begin{equation*}
  \begin{split}
    \mathbf{d} & = \mathbf{b} - \frac{(\mathbf{b}, \mathbf{c})}{(\mathbf{c},\mathbf{c})} \mathbf{c} \\
    & = \begin{bmatrix}
      1 \\
      2 \\
      -1
    \end{bmatrix}
    - \dfrac{24}{101}
    \begin{bmatrix}
      4 \\
      9 \\
      -2
    \end{bmatrix}
    =
    \begin{bmatrix}
    0.0495\\
   -0.1386\\
   -0.5248
    \end{bmatrix}
  \end{split}
\end{equation*}
This can be normalized if necessary to the following vector
$$
\mathbf{\hat{d}} =
\begin{bmatrix}
  0.0908\\
  -0.2543\\
  -0.9628
\end{bmatrix}
$$

\end{homeworkProblem}

\clearpage
\begin{homeworkProblem}
 Assume that $A$ is a $n\times n$ symmetric positive definite
  matrix. If $\kappa(A) = {\cal{O}}(n)$, give a rough estimate of the
  number of steepest descent iterations required to reduce the error
  in $A-norm$ to ${\cal{O}}(1/n)$.

  \medskip

  Let $m$ be the number of steepest descent iterations required to
  reduce the error to ${\cal{O}}(1/n)$ of the initial error. We have
  $$ ||e||_{A}^{m+1} \le \left( \dfrac{\kappa(A)-1}{\kappa(A)+1} \right)^{m+1} ||e||_{A}^{0} $$

  Taking logarithm on both sides:
  $$
  (m + 1) \ln \left( \dfrac{\kappa(A)-1}{\kappa(A)+1} \right) \ge \ln \left( {\dfrac{||e||_{A}^{m+1}}{||e||_{A}^{0}}} \right)
  $$
  Using the given estimates for $A$ , this expression becomes
  $$
  (m + 1) \ln \dfrac{n-1}{n+1} \ge \ln {\dfrac{1}{n}}
  $$
  On rearranging, we get
$$
  m \ge  \dfrac{\ln {\dfrac{1}{n}}}{\ln \dfrac{n-1}{n+1}} - 1
$$ This is the theoretical lower bound for the number of steepest
  descent iterations required to achive the ${\cal{O}}(1/n)$ error
  reduction.
\end{homeworkProblem}

\clearpage
\begin{homeworkProblem}
\emph{Assume that $A$ is a $n\times n$ symmetric positive definite
  matrix with eigenvalues in the intervals $(1, 1.1) \cup (10,10.2)$.
  Assume that the cost of a matrix vector multiplication is about $4n$ floating point
  multiplications. Estimate the number of floating point opearations reduce the
  $A-$norm of the error  by a factor of $10^{-3}$ using CG iterations. }

  \medskip

  Let $m$ be the number of conjugate gradient iterations required to
  reduce the error to ${\cal{O}}(1/n)$ of the initial error.
  We have
  $$ ||e||_{A}^{m+1} \le \left( \dfrac{\sqrt{\kappa(A)}-1}{\sqrt{\kappa(A)}+1} \right)^{m+1} ||e||_{A}^{0} $$

  Taking logarithm on both sides:
  $$
  (m + 1) \ln \left( \dfrac{\sqrt{\kappa(A)}-1}{\sqrt{\kappa(A)}+1} \right) \ge
  \ln \left( {\dfrac{||e||_{A}^{m+1}}{||e||_{A}^{0}}} \right)
  $$
  The condition number $\kappa(A) =
  \dfrac{|\lambda_{max}|}{|\lambda_{min}|} = \dfrac{10.2}{1} = 10.2$.
  The required relative reduction in error is $10^{-3}$. Using this
  information
  $$
  m \ge \dfrac{\ln {10^{-3}}}{\ln \dfrac{\sqrt{10.2}-1}{\sqrt{10.2}+1}} - 1 = 9.66 \approx 10
  $$

  At each CG iteration there are 4 matrix-vector
  multiplications. However, they are not unique operations (rather
  redundant). Therefore, an efficient implementation will require only
  one matrix-vector multiplication per iteration.  It is given that
  each matrix-vector product costs $4n$ floating point
  operations. Thus, the number of floating point operations would be
  $10 \times 4n = 40n$, where $n$ is the size of the matrix.

\end{homeworkProblem}

\clearpage
\begin{homeworkProblem}

\emph{Discretize the following differential equation}
$$ -u^{\prime\prime} = f(x) = 2x -\frac{1}{2}$$
$$u(0) = 1, u(1) = 0,~(case~a)$$
$$ u(0) = 1, u^\prime(1) = 0~(case~b)$$
$$x \in [0,1]$$

\emph{by the central difference scheme with $n$ interior mesh
  points. Solve the resulting linear system by Conjugate Gradient
  (CG), Preconditioned CG with Sine transform $[S]_{j,k} =
  \sqrt{\frac{2}{n+1}}\sin\left(\frac{\pi j k }{n+1}\right)  \quad \forall \quad 1 \le j,k \le
  n$ to construct the preconditioner. Run your code with $n = 64,
  128, 256, 512, 1024, \ldots$.  Compare the iteration numbers used
  for each case in a table. Discuss your own results.}

\cite{sine-transform}

  \paragraph{(a) Linear System}

  \medskip

  The discretized ODE is $$- u_{i+1} +2 u_i - u_{i-1} = h^2f(x_i),$$
  where $h = \frac{b-a}{n+1}$, $x_i=ih$, and $i = 0, \ldots,
  n+1$. This leads to a tridiagonal banded linear system.  For the
  case $(a)$ (Dirichlet) we get the following linear system.

  $$ \begin{bmatrix}
    2 & -1  &  0 & 0 & 0\\
    -1 & 2 & -1  &  0 & 0 \\
    0 & \ddots & \ddots & \ddots & 0 \\
    0 & 0 & -1 & 2 & -1 \\
    0 & 0 &  0 & -1 & 2 \\
  \end{bmatrix}
  \begin{bmatrix}
    u(x_1) \\
    u(x_2) \\
    \vdots\\
    u(x_{n-1}) \\
    u(x_{n}) \\
  \end{bmatrix}
  =
  \begin{bmatrix}
    1 +  h^2f(x_1) \\
    h^2f(x_2) \\
    \vdots\\
    h^2f(x_{n-1}) \\
    0 + h^2f(x_{n}) \\
  \end{bmatrix}$$

  For case $(b)$, we impose a Neumann boundary condition that the unknown itself is
  not given at the right boundary so we have to solve for it. This
  means that our unknowns are not just at the interior points but also
  at any point where a Neumann condition is specified. Therefore, the
  size of the linear system is $m=n+1$, where $m$ is the total number
  of points and $n$ is the number of interior points.  The resulting
  linear system is
  $$ \begin{bmatrix}
      2 & -1  &  0 & 0 & 0\\
    -1 & 2 & -1  &  0 & 0 \\
     0 & \ddots & \ddots & \ddots & 0 \\
     0 & 0 & -1 & 2 & -1 \\
     0 & 0 &  0 & -1 & 1 \\
  \end{bmatrix}
  \begin{bmatrix}
    %  u(x_1) \\
    u(x_1) \\
    u(x_2) \\
    \vdots\\
    u(x_{n}) \\
    u(x_{n+1}) \\
  \end{bmatrix}
  =
  \begin{bmatrix}
    1 +  h^2f(x_1) \\
    h^2f(x_2) \\
    \vdots\\
    h^2f(x_{n}) \\
    \frac{h^2}{2} f(x_{n+1}) \\
  \end{bmatrix}$$

  In formulating the Neumann BC, a second order central difference
  operator is used by introducing a ghost node beyond the boundary \emph{i.e.,}
  $$ \dfrac{u_{n+2}-u_{n}}{2h} = 0 \rightarrow u_{n+2} = u_{n}$$
  and the discretized boundary equation
  $$ - u_{n} +2 u_{n+1} - u_{n+2} = h^2f(x_{n+1}). $$

  \paragraph{(b) Preconditioning based on sine transform}

  The preconditioner is $M = S D^{-1} S$, where $D=SAS$ (a similarity
  transformation) is a diagonal matrix~\cite{sine-transform}.  The
  diagonal matrix is inverted in $n$ operations, simply by finding the
  reciprocal of the diagonal elements. We exploit the properties of
  the sine-transform matrix $S$ that is is orthonormal (i.e. $S^T =
  S^{-1}$ and $SS^T = I$).

  \paragraph{(c) Comparison of CG and PCG}

  The CG and PCG algorithms were implemented in the computer code. The
  performances of each method for both test cases is summarized in
  Table~\ref{tab:summary}. It can be seen that the preconditioned algorithms
  perform much better than the CG algorithm. This is due to the nature
  of the preconditioning matrix $M$ is approximating the inverse using
  similarity transformation involing sine-transforms. The results are
  similar to the ones reported in the
  literature~\cite{sine-transform}.  An observation that can be made
  about the case $a$ is the vectors $S$ matrix are the actual
  eigenfunctions and therefore the preconditioner $M$ is the actual
  inverse. This leads to the theoretical convergence expected in one
  iteration. However, for case $b$, $M$ is not the actual inverse but
  indeed a close enough approximation which drastically reduced the
  number of iterations required to converge to the actual
  solution. This exercise emphasizes the importance of the choice of
  preconditioners.

\begin{table}[H]
  \caption{Comparison of number of iterations taken to converge with
    CG and PCG methods.}  \centering
  \begin{tabular}{c|c|c}
    \toprule
      n   &  Case $(a)$  & Case $(b)$ \\
      &  CG/PCG    & CG/PCG \\
      \midrule
      64    &  64/1   & 65/3  \\
      128   &  128/1   & 129/3  \\
      256   &  256/1   & 257/4 \\
      512   &  512/1 &  513/4\\
      1024  &  1024/1 &  1025/4\\
      \bottomrule
  \end{tabular}
  \label{tab:summary}
\end{table}

We will now take a look at the solution profile obtained with each
method. Figure~\ref{fig:profile} compares the solutions produced by
CG, PCG and exact (direct) solution. All iterative methods were
converged to the same tolerance $10^{-8}$. The Dirichlet BC condition
can be seen to be satisfied for case (a) and the zero-slope can be
seen in the other case (b).

  \begin{figure}[H]
    \centering
    \includegraphics[width=0.5\textwidth]{profile.pdf}
    \caption{Plot of solution profile in the domain for different
      solution methods.}
    \label{fig:profile}
  \end{figure}

\begin{thebibliography}{9}
\bibitem{sine-transform} Chan, R. H., Ng, M. K., Wong,
  C. K. \emph{Sine transform based preconditioners for
    symmetric Toeplitz systems}. Linear Algebra and Its Applications,
  232, pp 237--259, 1996. doi:
  https://doi.org/10.1016/0024-3795(94)00049-2
\end{thebibliography}

\end{homeworkProblem}

\end{document}