\documentclass{article}

\usepackage{lipsum}
\usepackage[margin=1.5in]{geometry}
\usepackage{titlesec}
\usepackage{graphicx}
\usepackage{amsmath}

\usepackage{mathtools, amssymb, nccmath}
\usepackage{bigstrut, changepage, lipsum}

\newcommand{\code}{\texttt}
\newcommand{\norm}[1]{\left\lVert#1\right\rVert}

\usepackage{siunitx} % Required for alignment

% Specify images directory
\graphicspath{ {./report-images/} }

% Header and Footer stuff
\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhead{}
\fancyfoot{}
\fancyfoot[R]{ \thepage\ }
\renewcommand{\headrulewidth}{0pt}
\renewcommand{\footrulewidth}{0pt}
\newcommand{\sectionbreak}{\clearpage}
\setlength{\parindent}{0pt}

%

\begin{document}

%----------------------------------------------------------------------------------------
%	TITLE PAGE
%----------------------------------------------------------------------------------------

\begin{titlepage} % Suppresses displaying the page number on the title page and the subsequent page counts as page 1
	\newcommand{\HRule}{\rule{\linewidth}{0.5mm}}% Defines a new command for horizontal lines, change thickness here

	\center % Centre everything on the page

	%------------------------------------------------
	%	Headings
	%------------------------------------------------

	\textsc{\Large Random Number Generation and Simulation}\\[0.5cm] % Major heading such as course name

	\textsc{\large Exercise 8}\\[0.5cm] % Minor heading such as course title

	%------------------------------------------------
	%	Title
	%------------------------------------------------

	\HRule\\[0.6cm]

	{\huge\bfseries Area estimation using Monte Carlo method}\\[0.25cm] % Title of your document

	\HRule\\[1.5cm]

	%------------------------------------------------
	%	Author(s)
	%------------------------------------------------

	\begin{minipage}{0.4\textwidth}
		\begin{flushleft}
			\large
			\textit{Author}\\
			\textsc{Cesare De Cal} % Your name
		\end{flushleft}
	\end{minipage}
	~
	\begin{minipage}{0.4\textwidth}
		\begin{flushright}
			\large
			\textit{Professor}\\
			\textsc{Annie Cuyt \& Stefan Becuwe}\\ % Supervisor's name
			[0.25cm]
			\textit{Assistant Professor}\\
			\textsc{Ferre Knaepkens} % Supervisor's name

		\end{flushright}
	\end{minipage}

	\vfill\vfill\vfill

	{\large\today}

	\vfill

\end{titlepage}

%----------------------------------------------- Introduction ------------------------------------------------------
\section{Introduction}\label{sec:intro}
The exercise asks to approximate the area of the figure defined by

$$
\begin{cases}
1 \le x \le 3 \\
-1\le y \le 4 \\
x^3+y^3\le 29 \\
y \ge e^x -2
\end{cases}$$

using the Monte Carlo method, which in Mathematics means solving a problem using random numbers.

%---------------------------------- Tools ---------------------------------------------------------------------------
\section{Tools}
To solve this exercise, I've used C and the following libraries:

\begin{itemize}
  \item C Math Library
  \item Intel Math Kernel Library (more specifically, the Vector Statistical Library)
  \item OpenMP
\end{itemize}

I've used the following Intel MKL routines:
\begin{itemize}
  \item \code{vslNewStream(\&stream, brng, seed)}
  \item \code{vslLeapfrogStream(stream, k, nstreams)}
  \item \code{vsRngUniform(method, stream, nrRandomNumbers, array, start, end)}
  \item \code{vslDeleteStream(\&streamToDelete)}
\end{itemize}

Even though it wasn't required for this exercise, I've used OpenMP to support multi-threading and make the computation more efficient. I've used the following OpenMP methods and procedures:
\begin{itemize}
  \item \code{omp\_get\_max\_threads()}
  \item \code{\#pragma omp parallel private(nrOfThreads, threadID)}
  \item \code{omp\_get\_thread\_num()}
\end{itemize}

To get started with the exercise, I have used the template file \code{template.c} provided on the course website. To make the code more clear, I've also wrote my own function \code{isInsideArea(x,y)} which checks if a given pair of coordinates $(x, y)$ is inside the area drawn by the system of inequalities.\\

To compile and run the code on my own machine with \code{gcc}, I created a Makefile based on the compiler options and link line specified here:\\ \code{https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor}.

\section{Computation}
On a high level picture, to solve this problem I'll generate random points with coordinates $(x, y)$ in the rectangle (bounding box) defined by

$$
\begin{cases}
1 \le x \le 3 \\
-1\le y \le 4 \\
\end{cases}$$

And then I'll check if the each point is inside the figure. That is, if $(x, y)$ satisfies the following constraints using the \code{isInsideArea(x,y)} function I wrote.

$$
\begin{cases}
x^3+y^3\le 29 \\
y \ge e^x -2
\end{cases}$$

To do this, I define two constants $\code{LOOPS}$ and $\code{N}$, which represent the number of iterations to compute and the numbers of points to generate for each iteration respectively. I'll also define a seed that I'll feed into the random number generator.\\

As we've seen in class, I can estimate the area of a figure by calculating the ratio between the points inside the figure and all the points generated. This process is repeated for a certain number of iterations, and in each iteration an area is calculated. At the end of the iterations, I calculate the average of all the areas.\\

To make this happen, I first initialize a stream of random numbers using the $\code{VSL\_BRNG\_MCG59}$ random number generator and the leapfrog method. Then, I create two independent streams of uniformly distributed random numbers with \code{vdRngUniform} in the $(1,3)$ and $(-1,4)$ intervals for the $x$ and $y$ coordinates respectively. If the point $(x, y)$ lies inside the figure, I increase a local variable which keeps count of the points inside the area. At the end of the iteration, I calculate the percentage of the points inside the area by dividing the count by the total points generated.\\

Finally, at the end of this multi-threaded computation, the average is taken out of all the areas calculated in the iterations. The result is $7.581675111111076e-02$ with $\code{N}$ and $\code{LOOPS}$ equal to 30000 and seed equal to $-87654321$.\\

Now that I have the percentage of the points that lie in the figure, I can calculate the area of the rectangle by using the following formula:

$$A_{\textnormal{rectangle}}= \textnormal{base}\times \textnormal{height}=2\times5=10$$

The area of the figure is then

$$A_{\textnormal{figure}}=A_{\textnormal{rectangle}}\times \frac{\textnormal{number of points inside figure}}{\textnormal{total number of points in rectangle}}$$

$$A_{\textnormal{figure}}=10\times 7.581675111111076e-02=7.581675111111076e-01$$

To make sure I have the right result, I've used Maple to calcolate the following integral:

$$
\int_1^{a} (\sqrt[3]{29-x^3}-e^x+2)dx\approx 7.581218821150386e-01
$$

with $x=a$ point of intersection between the two curves defined by $y\ge e^x-2$ and $y\le \sqrt[3]{29-x^3}$. To get the result above I computed $a=1.593743361313601$ and then performed numerical integration. To estimate the precision of the solution, I calculated the absolute value of the error: $0.00045628996$.

\section{Plot}
To better understand the problem and how to integrate to mathematically compute the area of the figure, I've plotted below:\\

\includegraphics[width=\textwidth,height=\textheight,keepaspectratio]{desmos.png}
\section{Observations}
Given the error calculated by comparing the mathematical solution with the results obtained through the Monte Carlo method, I can observe that the Monte Carlo method is a very compelling way to estimate areas.\\

I can also observe that (at least in this experiment) reasonable precision is attained with only a moderate number of random numbers (1000) that with 1000 iterations lead to an average area of $7.692400000000001e-01$. Also, by using OpenMP, I was able to reduce the computation time by a great deal given that multiple threads were working at the same time.

\end{document}


%~\par
%\newpage

    \section{(some) LaTeX environments for Jupyter
notebook}\label{some-latex-environments-for-jupyter-notebook}
% No prompt!
%\textbf{Input \#{}}%
\begin{lstlisting}
%%html
<style>
    .prompt{
        display: none;
    }

</style>
\end{lstlisting}% No prompt!
%\textbf{Output \#{}}
%

    \subsection{** What's new **}

\textbf{July 27, 2016} -

\begin{itemize}
\item
  In this version we have reworked \textbf{equation numbering}. In the
  previous version, we have implemented a specialized counter and
  detected equations rendering for updating the counter. Meanwhile, this
  feature has been introduced in \texttt{MathJax} and now we rely on
  MathJax implementation. We still have keep the capability of
  displaying only equation labels (instead of numbers). The numbering is
  automatically updated and is document-wide.
\item
  We have completely reworked the \textbf{notebook conversion} to plain
  \(\LaTeX\) and html. We provide specialized exporters, pre and post
  processors, templates. We also added entry-points to simplify the
  conversion process. It is now as simple as

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{jupyter} \NormalTok{nbconvert --to html_lenvs FILE.ipynb}
\end{Highlighting}
\end{Shaded}

  to convert \texttt{FILE.ipynb} into html while keeping all the
  features of the \texttt{latex\_envs} notebook extension in the
  converted version.
\end{itemize}

    \section{Presentation and main
features}\label{presentation-and-main-features}

    This extension for IPython 3.x or Jupyter enables to use some LaTeX
commands and environments in the notebook's markdown cells.
\begin{enumerate}
\item \textbf{LaTeX commands and
environments}
\begin{itemize}
\item support for some
LaTeX commands within markdown cells, \emph{e.g.}
\texttt{\textbackslash{}textit}, \texttt{\textbackslash{}textbf},
\texttt{\textbackslash{}underline}
\item support for
\textbf{theorems-like environments}
\item support for \textbf{lists}:
\emph{enumerate, itemize},
\item limited support for a
\textbf{figure environment},
\item support for an environment
\emph{listing},
\item additional \emph{textboxa}
environment
\end{itemize}
\item \textbf{Citations and
bibliography}
\begin{itemize}
\item support for
\texttt{\textbackslash{}cite} with creation of a References
section
\end{itemize}
\item \textbf{Document-wide
numbering of equations, support for \texttt{\textbackslash{}label} and
\texttt{\textbackslash{}ref}}
\item \textbf{Configuration
toolbar}
\item Styles can be customized in the
\texttt{latex\_env.css} stylesheet
\end{enumerate}

A simple illustration is as follows: on can type the following in a
markdown cell

\begin{listing}
\begin{theorem} \label{theo:dotp}
Let $u$ and $v$ be two vectors of $\mathbb{R}^n$. The dot product can be expressed as
\begin{equation}
\label{eq:dotp}
u^Tv = |u||v| \cos \theta,
\end{equation}
where $\theta$ is the angle between $u$ and $v$ ...
\end{theorem}
Then one can reference the equation (\ref{eq:dotp}) in theorem \ref{theo:dotp}.
\end{listing}

and have it rendered as

\begin{theorem} \label{theo:dotp}
Let \(u\) and \(v\) be two
vectors of \(\mathbb{R}^n\). The dot product can be expressed as

\begin{equation}
\label{eq:dotp}
u^Tv = |u||v| \cos \theta,
\end{equation}

where \(\theta\) is the angle between \(u\) and \(v\)
\ldots{}
\end{theorem} Then one can reference the equation
(\ref{eq:dotp}) in theorem \ref{theo:dotp}.

    \subsection{Implementation principle}\label{implementation-principle}

    The main idea is to override the standard Markdown renderer in order to
add a \emph{small} parsing of LaTeX expressions and environments. This
heavily uses regular expressions. The LaTeX expression are then rendered
using an html version. For instance
\texttt{\textbackslash{}underline\ \{something\}} is rendered as
\texttt{\textless{}u\textgreater{}\ something\ \textless{}/u\textgreater{}},
that is \underline{something}. The environments are replaced by an html
tag with a class derived from the name of the environment. For example,
a \texttt{definition} denvronment will be replaced by an html rendering
corresponding to the class \texttt{latex\_definition}. The styles
associated with the different classes are sepcified in
\texttt{latex\_env.css}. These substitutions are implemented in
\texttt{thsInNb4.js}.

    \subsection{Support for simple LaTeX
commands}\label{support-for-simple-latex-commands}

    We also added some LaTeX commands (e.g. \texttt{\textbackslash{}textit},
\texttt{\textbackslash{}textbf}, \texttt{\textbackslash{}underline}) --
this is useful in the case of copy-paste from a LaTeX document. Labels
and references are supported, including for equations.

    \subsection{Available environments}\label{available-environments}

    \begin{itemize}
\tightlist
\item
  \textbf{theorems-like environments}: \emph{property, theorem, lemma,
  corollary, proposition, definition,remark, problem, exercise,
  example},
\item
  \textbf{lists}: \emph{enumerate, itemize},\\
\item
  limited support for a \emph{figure} environment,
\item
  an environment \emph{listing},
\item
  \emph{textboxa}, wich is a \texttt{textbox} environment defined as a
  demonstration (see below).
\end{itemize}

More environments can be added easily in the javascript source file
\texttt{thmsInNb4.js}. The rendering is done according to the stylesheet
\texttt{latex\_env.css}, which can be customized.

\begin{remark}
When exporting to html, the
\texttt{latex\_env.css} file honored is the file on the
Jupyter-notebook-extensions CDN. However, customized css can be added in
a \texttt{custom.css} file that must reside in the same directory as the
notebook itself.
\end{remark}

    \subsection{Automatic numerotation, labels and
references}\label{automatic-numerotation-labels-and-references}

    Counters for numerotation are implemented: one for theorems-like
environments, a second for exercises-like environments and a third one
for numbering figures.\\
Mathjax-equations with a label are also numbered document-wide. An
anchor is created for any label which enables to links things within the
document: \texttt{\textbackslash{}label} and
\texttt{\textbackslash{}ref} are both supported. A limitation is that
numbering is updated (incremented) each time a cell is rendered. A
toolbar button is provided to reset the counters and refresh the
rendering of the whole document.

    \label{example:mixing} A simple example is as follows, featuring
automatic numerotation, and the use of labels and references. Also note
that standard markdown can be present in the environment and is
interpreted. \emph{The rendering is done according to the stylesheet
\texttt{latex\_env.css}, which of course, can be tailored to specific
uses and tastes}.

\begin{listing}
\begin{definition} \label{def:FT}
Let $x[n]$ be a sequence of length $N$. Then, its **Fourier transform** is given by
\begin{equation}
\label{eq:FT}
X[k]= \frac{1}{N} \sum_{n=0}^{N-1} x[n] e^{-j2\pi \frac{kn}{N}}
\end{equation}
\end{definition}
\end{listing}

    \begin{definition} \label{def:FT}
Let \(x[n]\) be a sequence
of length \(N\). Then, its \textbf{Fourier transform} is given by

\begin{equation}
\label{eq:FT2}
X[k]= \frac{1}{N} \sum_{n=0}^{N-1} x[n] e^{-j2\pi \frac{kn}{N}}
\end{equation}

\end{definition}

    It is now possible to refer to the definition and to the equation by
their labels, as in:

\begin{listing}
As an example of Definition \ref{def:FT}, consider the Fourier transform (\ref{eq:FT2}) of a pure cosine wave given by
$$
x[n]= \cos(2\pi k_0 n/N),
$$
where $k_0$ is an integer.
\end{listing}

    As an example of Definition \ref{def:FT}, consider the Fourier transform
(\ref{eq:FT2}) of a pure cosine wave given by \[
x[n]= \cos(2\pi k_0 n/N),
\] where \(k_0\) is an integer. Its Fourier transform is given by \[
X[k] = \frac{1}{2} \left( \delta[k-k_0] + \delta[k-k_0] \right),
\] modulo \(N\).

    \subsection{Bibliography}\label{bibliography}

    \subsubsection{Usage}\label{usage}

    It is possible to cite bibliographic references using the standard LaTeX
\texttt{\textbackslash{}cite} mechanism. The extension looks for the
references in a bibTeX file, by default \texttt{biblio.bib} in the same
directory as the notebook. The name of this file can be modified in the
configuration toolbar. It is then possible to cite works in the
notebook, e.g.

\begin{listing}
The main paper on IPython is definitively \cite{PER-GRA:2007}. Other interesting references are certainly \cite{mckinney2012python, rossant2013learning}. Interestingly, a presentation of the IPython notebook has also be published recently in Nature \cite{shen2014interactive}.
\end{listing}

The main paper on IPython is definitively \cite{PER-GRA:2007}. Other
interesting references are certainly
\cite{mckinney2012python, rossant2013learning}. Interestingly, a
presentation of the IPython notebook has also be published recently in
Nature \cite{shen2014interactive}.

    \subsubsection{Implementation}\label{implementation}

    The implemention uses several snippets from the nice
\href{https://bitbucket.org/ipre/calico/downloads/}{icalico-document-tools}
extension that also considers the rendering of citations in the
notebook. We also use a modified version of the
\href{https://code.google.com/p/bibtex-js/}{bibtex-js} parser for
reading the references in the bibTeX file. The different functions are
implemented in \texttt{bibInNb4.js}. The rendering of citations calls
can adopt three styles (Numbered, by key or apa-like) -- this can be
selected in the configuration toolbar. It is also possible to customize
the rendering of references in the reference list. A citation template
is provided in the beginning of file \texttt{latex\_envs.js}:

\begin{verbatim}
var cit_tpl = {
// feel free to add more types and customize the templates
    'INPROCEEDINGS': '%AUTHOR:InitialsGiven%, ``_%TITLE%_\'\', %BOOKTITLE%, %MONTH% %YEAR%.',
    ... etc
\end{verbatim}

The keys are the main types of documents, eg inproceedings, article,
inbook, etc. To each key is associated a string where the \%KEYWORDS\%
are the fields of the bibtex entry. The keywords are replaced by the
correponding bibtex entry value. The template string can formatted with
additional words and effects (markdown or LaTeX are commands are
supported)

    \subsection{Figure environment}\label{figure-environment}

    Finally, it is sometimes useful to integrate a figure within a markdown
cell. The standard markdown markup for that is
\texttt{!{[}link{]}(image)}, but a limitation is that the image can not
be resized, can not be referenced and is not numbered. Furthermore it
can be useful for re-using existing code. Threfore we have added a
limited support for the \texttt{figure} environment. This enables to do
something like

\begin{listing}
\begin{figure}
\centerline{\includegraphics[width=10cm]{example.png}}
\caption{\label{fig:example} This is an example of figure included using LaTeX commands.}
\end{figure}
\end{listing}

which renders as

\begin{figure}
\centerline{\includegraphics[width=10cm]{example.png}}
\caption{\label{fig:example} This is an example of figure included using LaTeX commands.}
\end{figure}

Of course, this Figure can now be referenced:

\begin{listing}
Figure \ref{fig:example} shows a second filter with input $X_2$, output $Y_2$  and an impulse response denoted as $h_2(n)$
\end{listing}

Figure \ref{fig:example} shows a second filter with input \(X_2\),
output \(Y_2\) and an impulse response denoted as \(h_2(n)\)

    \subsection{figcaption}\label{figcaption}

    For Python user, we have added in passing a simple function in the
\texttt{latex\_envs.py} library. This function can be imported
classicaly, eg \texttt{from\ latex\_envs\ import\ figcaption}. Then,
this function enables to specify a caption and a label for the next
plot. In turn, when exporting to \(\LaTeX\), the corresponding plot is
converted to a nice figure environement with a label and a caption.
% No prompt!
%\textbf{Input \#{}}%
\begin{lstlisting}
%matplotlib inline
import matplotlib.pyplot as plt
from jupyter_contrib_nbextensions.nbconvert_support.latex_envs import figcaption
from numpy import pi, sin, arange

plt.plot(sin(2*pi*0.01*arange(100)))
\end{lstlisting}% No prompt!
%\textbf{Output \#{}}
%

    \begin{verbatim}
[<matplotlib.lines.Line2D at 0x7f8a1872ba20>]
    \end{verbatim}

\begin{figure}[H]
\centering
\includegraphics[width=0.6\linewidth]{latex_env_doc_files/latex_env_doc_27_2.png}
\caption{This is a nice sine wave}
\label{fig:mysin}
\end{figure}
%    { \hspace*{\fill} \\}

    \subsection{Other features}\label{other-features}

    \begin{itemize}
\tightlist
\item
  As shown in the examples, eg \ref{example:mixing}, it is possible to
  mix LaTeX and markdown markup in environments\\
\item
  Environments can be nested. However, it is not possible, because of
  limlitations of regular expressions, to nest environments of the same
  king (eg itemize into itemize into itemize\ldots{})
\end{itemize}

    \subsection{User interface}\label{user-interface}

    \subsubsection{Buttons on main toolbar}\label{buttons-on-main-toolbar}

    On the main toolbar, the extension provides three buttons
\includegraphics{main_toolbar.png} The first one can be used to refresh
the numerotation of equations and references in all the document. The
second one fires the reading of the bibliography bibtex file and creates
(or updates) the reference section. Finally the third one is a toogle
button that opens or closes the configuration toolbar.

    \subsubsection{Configuration toolbar}\label{configuration-toolbar}

    The configuration toolbar \includegraphics{configuration_toolbar.png}
enables to enter some configuration options for the extension. First,
one can indicate the name of the bibtex file. If this file is not found
and the user creates the reference section, then this section will
indicate that the file was not found. The references drop-down menu
enables to choose the type of reference calls. The Equations input box
enable to initiate numbering of equations at the given number (this may
be useful for complex documents in several files/parts). Finally the
last drop-down menu let the user choose to number equation or to display
their label instead. These configuration options are then stored in the
notebook's metadata (and restored on reload).

    \section{Installation, usage and further
examples}\label{installation-usage-and-further-examples}

    \subsection{Installation}\label{installation}

    The extension consists in several javascript scripts:
\texttt{latex\_envs.js}, \texttt{thmsInNb4.js}, \texttt{bibInNb4.js} and
\texttt{initNb.js}, together with a stylesheet \texttt{latex\_envs.css}.
You may follow the instructions in the
\href{https://github.com/ipython-contrib/IPython-notebook-extensions/wiki}{wiki}
to install the extension.

Definitively, the simplest way to install and enable the extension is to
follow the instrutions in the
\href{https://github.com/ipython-contrib/Jupyter-notebook-extensions/blob/master/README.md}{Jupyter-notebook-extensions
repo}. Once this is done, you can open a tab at
\texttt{http://localhost:8888/nbextensions} to anable and configurate
the various extensions.

    \subsection{First example (continued)}\label{first-example-continued}

    We continue the first example on fthe Fourier transform definition
\ref{def:FT} in order to show that, of course, we can illustrate things
using a simple code. Since the Fourier transform is an essential tool in
signal processing, We put this in evidence using the \texttt{textboxa}
environment -- which is defined here in the css, and that one should
define in the LaTeX counterpart:

\begin{listing}
\begin{textboxa}
The Fourier transform is an extremely useful tool to have in your toolbox!
\end{textboxa}
\end{listing}

    \begin{textboxa}
The Fourier transform is an extremely useful tool to have in your toolbox!
\end{textboxa}

    The Fourier transform of a pure cosine is given by \[
X[k] = \frac{1}{2} \left( \delta[k-k_0] + \delta[k-k_0] \right),
\] modulo \(N\). This is illustrated in the following simple script:
% No prompt!
%\textbf{Input \#{}}%
\begin{lstlisting}
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from numpy.fft import fft
k0=4; N=128; n=np.arange(N); k=np.arange(N)
x=np.sin(2*np.pi*k0*n/N)
X=fft(x)
plt.stem(k,np.abs(X))
plt.xlim([0, 20])
plt.title("Fourier transform of a cosine")
_=plt.xlabel("Frequency index (k)")
\end{lstlisting}% No prompt!
%\textbf{Output \#{}}
%
    \begin{center}
    \adjustimage{max size={0.6\linewidth}{0.6\paperheight}}{latex_env_doc_files/latex_env_doc_42_0.png}
    \end{center}
%    { \hspace*{\fill} \\}

    \subsection{Second example}\label{second-example}

    This example shows a series of environments, with different facets;
\textbf{links, references, markdown or/and LaTeX formatting within
environments}. The listing of environments below is typed using the
environment \emph{listing}\ldots{}

    \begin{listing}
\begin{definition} \label{def:diffeq}
We call \textbf{difference equation} an equation of the form
$$
\label{eq:diffeq}
y[n]= \sum_{k=1}^{p} a_k y[n-k] + \sum_{i=0}^q b_i x[n-i]
$$
\end{definition}

\begin{property}
If all the $a_k$ in equation (\ref{eq:diffeq}) of definition \ref{def:diffeq} are zero, then the filter has a **finite impulse response**.
\end{property}

\begin{proof}
Let $\delta[n]$ denote the Dirac impulse. Take $x[n]=\delta[n]$ in (\ref{eq:diffeq}). This yields, by definition, the impulse response:
$$
\label{eq:fir}
h[n]= \sum_{i=0}^q b_i \delta[n-i],
$$
which has finite support.
\end{proof}

\begin{theorem}
The poles of a causal stable filter are located within the unit circle in the complex plane.
\end{theorem}

\begin{example} \label{ex:IIR1}
Consider $y[n]= a y[n-1] +  x[n]$. The pole of the transfer function is $z=a$. The impulse response $h[n]=a^n$ has infinite support.
\end{example}

In the following exercise, you will check that the filter is stable iff $a$<1.

\begin{exercise}\label{ex:exofilter}
Consider the filter defined in Example \ref{ex:IIR1}. Using the **function** `lfilter` of scipy, compute and plot the impulse response for several values of $a$.
\end{exercise}

\end{listing}

    The lines above are rendered as follows (of course everything can be
tailored in the stylesheet):

\begin{definition} \label{def:diffeq}
We call
\textbf{difference equation} an equation of the form

\begin{equation}
\label{eq:diffeq}
y[n]= \sum_{k=1}^{p} a_k y[n-k] + \sum_{i=0}^q b_i x[n-i]
\end{equation}

\end{definition} Properties of the filter are linked to the
coefficients of the difference equation. For instance, an immediate
property is

\begin{property}
If all the \(a_k\) in equation
(\ref{eq:diffeq}) of definition \ref{def:diffeq} are zero, then the
filter has a \textbf{finite impulse response}.

\end{property}

\begin{proof}
Let \(\delta[n]\) denote the Dirac impulse.
Take \(x[n]=\delta[n]\) in (\ref{eq:diffeq}). This yields, by
definition, the impulse response:

\begin{equation}
\label{eq:fir}
h[n]= \sum_{i=0}^q b_i \delta[n-i],
\end{equation}

which has finite support.
\end{proof}

\begin{theorem}
The poles of a causal stable filter are
located within the unit circle in the complex
plane.
\end{theorem}

\begin{example} \label{ex:IIR1}
Consider
\(y[n]= a y[n-1] + x[n]\). The pole of the transfer function is \(z=a\).
The impulse response \(h[n]=a^n\) has infinite
support.
\end{example}

In the following exercise, you will check that the filter is stable iff
\(a\)\textless{}1.

\begin{exercise}\label{ex:exofilter}
Consider the filter
defined in Example \ref{ex:IIR1}. Using the \textbf{function}
\texttt{lfilter} of scipy, compute and plot the impulse response for
several values of \(a\).
\end{exercise}

    \begin{listing}
The solution of exercise \ref{ex:exofilter}, which uses a difference equation as in Definition \ref{def:diffeq}:
\end{listing}

The solution of exercise \ref{ex:exofilter}, which uses a difference
equation as in Definition \ref{def:diffeq}:
% No prompt!
%\textbf{Input \#{}}%
\begin{lstlisting}
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import lfilter
d=np.zeros(100); d[0]=1 #dirac impulse
alist=[0.2, 0.8, 0.9, 0.95, 0.99, 0.999, 1.001, 1.01]
for a in alist:
    h=lfilter([1], [1, -a],d)
    _=plt.plot(h, label="a={}".format(a))
plt.ylim([0,1.5])
plt.xlabel('Time')
_=plt.legend()
\end{lstlisting}% No prompt!
%\textbf{Output \#{}}
%
    \begin{center}
    \adjustimage{max size={0.6\linewidth}{0.6\paperheight}}{latex_env_doc_files/latex_env_doc_48_0.png}
    \end{center}
%    { \hspace*{\fill} \\}

    \subsection{Third example:}\label{third-example}

    This example shows that environments like itemize or enumerate are also
available. As already indicated, this is useful for copying text from a
TeX file. Following the same idea, text formating commands
\texttt{\textbackslash{}textit}, \texttt{\textbackslash{}textbf},
\texttt{\textbackslash{}underline}, etc are also available.

    \textbackslash{}begin\{listing\} The following \textit{environments} are
available: \textbackslash{}begin\{itemize\} \item
\textbf{Theorems and likes} \textbackslash{}begin\{enumerate\} \item
theorem, \item lemma, \item corollary \item \ldots{}
\textbackslash{}end\{enumerate\} \item \textbf{exercises}
\textbackslash{}begin\{enumerate\} \item problem, \item example, \item
exercise \textbackslash{}end\{enumerate\} \textbackslash{}end\{itemize\}
\textbackslash{}end\{listing\}

    which gives\ldots{}

The following \textit{environments} are available:
\begin{itemize}
\item
\textbf{Theorems and likes}
\begin{enumerate}
\item
theorem,
\item lemma,
\item corollary
\item
\ldots{}
\end{enumerate}
\item
\textbf{exercises}
\begin{enumerate}
\item
problem,
\item example,
\item
exercise
\end{enumerate}
\end{itemize}

    \section{Conversion to LaTeX and
HTML}\label{conversion-to-latex-and-html}

    The extension works in the live-notebook. Since it relies on a bunch of
javascript, the notebook does not render as is in services such as
\texttt{nbviewer} or \texttt{github} viewer. Similarly,
\texttt{nbconvert} does not know of the LaTeX constructs which are used
here and therefore does not fully convert notebooks using this
extension.

Therefore, we provide specialized templates and exporters to achieve
these conversions.

    \subsection{Conversion to html}\label{conversion-to-html}

We provide a template \texttt{latex\_envs.tpl} and an exporter class
\texttt{LenvsHTMLExporter} (in library \texttt{latex\_envs.py}). Using
that class, conversion simply amounts to

\begin{verbatim}
jupyter nbconvert --to latex_envs.LenvsHTMLExporter FILE.ipynb
\end{verbatim}

A shortcut is also provided

\begin{verbatim}
jupyter nbconvert --to html_lenvs FILE.ipynb
\end{verbatim}

It should be noted that the rendering is done exactly in the same way as
in the livenotebook. Actually, it is the very same javascript which is
run in the html file. The javascript functions are available on the
\texttt{jupyter\_notebook\_extensions} CDN, which means that the
rendering of the html files requires an internet connection (this is
also true for the rendering of equations with MathJax).

\subsection{Conversion to LaTeX}\label{conversion-to-latex}

We provide two templates \texttt{thmsInNb\_article.tplx} and
\texttt{thmsInNb\_report.tplx} for article and report styles
respectively. Anyway one can also use the standard article, report, book
templates provided with nbconvert. Simply, we have improved some of the
internals styles. More importantly, we provide an exporter class
\texttt{LenvsLatexExporter} (also in library \texttt{latex\_envs.py}).
Using that class, conversion simply amounts to

\begin{verbatim}
jupyter nbconvert --to latex_envs.LenvsLatexExporter FILE.ipynb
\end{verbatim}

A shortcut is also provided

\begin{verbatim}
jupyter nbconvert --to latex_lenvs FILE.ipynb
\end{verbatim}

In addition, we provide several further options: -
\textbf{removeHeaders}: Remove headers and footers, (default false) -
\textbf{figcaptionProcess}: Process figcaptions, (default true) -
\textbf{tocrefRemove} Remove tocs and ref sections, + some cleaning,
(default true),

These options can be specified on the command line as, eg,

\begin{verbatim}
jupyter nbconvert --to latex_lenvs --LenvsLatexExporter.removeHeaders=True --  --LenvsLatexExporter.tocrefRemove=False FILE.ipynb
\end{verbatim}

    \begin{example}
As for an example, the present document has
been converted using

 jupyter nbconvert --to latex\_lenvs
--LenvsLatexExporter.removeHeaders=True latex\_env\_doc.ipynb

Then the resulting file (without header/footer) has been included in
the main file \texttt{documentation.tex}, where some LaTeX definitions
of environments are done (namely listings, colors, etc) and compiled
using

 xelatex documentation

The output can be
consulted \href{documentation.pdf}{here}.
\end{example}

    \section{Disclaimer, sources and
thanks}\label{disclaimer-sources-and-thanks}

    This is done in th ehope it can be useful. However there are many
impovements possible, in the code and in the documentation.

\textbf{Contributions will be welcome and deeply appreciated.}

Originally, I used a piece of code from the nice online markdown editor
\texttt{stackedit}
\url{https://github.com/benweet/stackedit/issues/187}, where the authors
also considered the problem of incorporating LaTeX markup in their
markdown.

I also studied and used examples and code from
\url{https://github.com/ipython-contrib/IPython-notebook-extensions}.


\section{Test Methodology}
\label{sec:700}

All three versions of the algorithm analysed here (sequential, OpenMP and TBB) were tested and compared. For each version, 5 different input sizes were used, with all the input coming from the same randomly populated file, with over $2^{26}$ integer values. Digit size was also varied between 2 and 8 (although limitations in the OpenMP implementation only allowed it to be tested for 2, 4 and 8, and not the remaining values in the range).

For both parallel versions, the number of threads was also varied, between 2 and 32. Even though, as noted on \cref{sec:600}, the environment used has 12 cores per node, meaning a multiple of 12 should provide a better value for the number of threads, the OpenMP version had limitations, mostly due to the usage of bitwise operators to simplify bucket calculations, and could only work with a number of threads that was a power of 2.


%\setcounter{page}{600}
\chapter{Using Derivatives to Analyze Functions
\label{DerivativesToAnalyzeFunctions}}
In this chapter we develop methods for analyzing functions
extensively by exploiting the information contained in
their derivatives.  To some extent this was done in the
previous two sections (Sections \ref{Differentials/LinearApproxs}
and \ref{Newton'sMethodSection}), though there we used
derivatives to give us approximations (good guesses, really)
for a couple of different types of computational problems.
However, in this chapter we will be more interested in
precise, qualitative as well as quantitative properties
of functions which we can decipher from their derivatives.
%For instance, if we know the slope (derivative) of a function,
%then we know how it is changing.  With the help of some
%results and methods of this chapter, we thus know when the
%function is rising, and when it is falling.

We will also be more interested in so-called higher-order
derivatives---particularly second derivatives---and their
import.

Of course a very natural and general approach for
analyzing functions is to consider their graphs in the Cartesian
Plane.  In fact, much of this chapter is devoted to
describing the graphs by analyzing the derivatives.
This will give the results a somewhat abstract setting,
but they will also be very general and quite easily put
to work in applied problems.

Most textbooks set aside sections specifically for applied
derivative problems, but here we will include applications
as the theory is developed.

Fortunately we will be able to prove all of the results
presented in this chapter, referring only to previously
proved results and already mentioned facts we borrow from
more advanced studies.  Unfortunately the order in which
we prove the results sometimes differs from the path one's
intuition would likely take in conjecturing these results.
For instance, the fact that a positive derivative implies
an increasing function seems intuitive, but it is most
easily proved using the Mean Value Theorem which, though
ultimately also intuitive, would be a more sophisticated guess
to make from merely observing the behavior of functions.

As for the Mean Value Theorem, we will put it to much more
extensive use than most other calculus textbooks.
In particular, we will use it to derive many {\it inequalities}
as well as for the usual uses, i.e., as a tool to prove the other results.
In fact the Mean Value Theorem is not so immediately useful
in its statement, but it is astonishingly useful as a
cornerstone in proving many more obvious (though not necessarily
trivial to prove!) results.
\section{The Mean Value Theorem}
\newpage
\section{Extrema on Closed Intervals\label{ExtremaOnClosedIntervals}}
In this section we wish to find those points in $[a,b]$ at which
a continuous function $f:[a,b]\mapsto\Re$ achieves its maximum
and minimum values for $x\in[a,b]$.  Recall from
Section~\ref{ContinuityOnIntervalsSection} that $f([a,b])$ will be a
finite, closed interval, and
thus will have a maximum and minimum value, i.e., there will
indeed exist some $\ds{x_{\text{min}}}$, $\ds{x_{\text{max}}}$
so that
\begin{itemize}
\item $\ds{x_{\text{min}},x_{\text{max}}\in[a,b]}$, and
\item $\ds{f([a,b])=\left[f\left(x_{\text{min}}\right),f\left(x_{\text{max}}
 \right)\right]}$, and so
\begin{equation}
\left(\exists x_{\text{min}},x_{\text{max}}\in[a,b]\right)
 (\forall x\in[a,b])\left[f\left(x_{\text{min}}\right)
   \le f(x)\le f\left(x_{\text{max}}\right)\right].
\end{equation}
\end{itemize}
This was the essence of the Extreme Value Theorem (EVT),
which was Corollary~\ref{ExtremeValueTheorem},
page~\pageref{ExtremeValueTheorem}.\footnote{%%
%%%%%%%%%  FOOTNOTE
It is very strongly suggested that the reader at least briefly review
Section~\ref{ContinuityOnIntervalsSection} at some point during the
reading of this current section.  It would likely be particularly helpful
to reconsider the figures in that section to recall why it is necessary
to have a continuous function {\it and} a closed interval for the
conclusion of the Extreme Value Theorem to be guaranteed.}
%%%%%%%%%  END FOOTNOTE

The next theorem is stated in a manner reflective of its proof.
However, its equivalent forms given in the corollaries are more
useful in applications.
\begin{theorem}
Suppose $f:[a,b]\longrightarrow\Re$ is continuous, and
that $f(x)$ achieves its maximum or minimum value for the interval
$[a,b]$ at an interior point $x_0\in(a,b)$.
Under these assumptions, if $f'(x_0)$ exists then $f'(x_0)=0$, i.e.,
$$f'(x_0) \text{ exists }\implies f'(x_0)=0.$$
\label{FirstTheoremOnInteriorExtrema}\end{theorem}

\begin{proof}
First we look at the case $f:[a,b]\longrightarrow\Re$
continuous, $x_0\in(a,b)$,
and $f(x_0)$ is the maximum value of $f(x)$
on $[a,b]$.  We need to show that
$f'(x_0)$ exists implies $f'(x_0)=0$.  So we (further)
suppose $f'(x_0)$ exists.  This implies
$$\lim_{\Delta x\to0}\frac{f(x_0+\Delta x)-f(x_0)}{\Delta x}=f'(x_0)
\text{ exists.}$$
Thus the left and right side limits exist and must agree with this
two-sided limit:
\begin{align}
f'(x_0)&=\lim_{\Delta x\to0^-}\frac{f(x_0+\Delta x)-f(x_0)}{\Delta x},
            \label{LeftLimitForF'(a)ForATheoremOnExtrema}\\
f'(x_0)&=\lim_{\Delta x\to0^+}\frac{f(x_0+\Delta x)-f(x_0)}{\Delta x}.
            \label{RightLimitForF'(a)ForATheoremOnExtrema}
\end{align}
Now we carefully consider each of these two limits,
(\ref{LeftLimitForF'(a)ForATheoremOnExtrema}) and
(\ref{RightLimitForF'(a)ForATheoremOnExtrema}).  In particular
we look at the signs of these.

In both cases,  since
$f(x_0)$ is a maximum, we have $f(x_0+\Delta x)\le f(x_0)$,
implying $f(x_0+\Delta x)-f(x_0)\le0$.  Thus the numerators in the
limits are both nonpositive.

Next we look specifically at (\ref{LeftLimitForF'(a)ForATheoremOnExtrema}).
Since $\Delta x\to0^-$, we are looking at limits of fractions
with denominators $\Delta x<0$.  Thus we have
\begin{equation}f'(x_0)=\lim_{\Delta x\to0^-}
  \frac{\overbrace{f(x_0+\Delta x)-f(x_0)}^{\le0}}
          {\underbrace{\Delta x}_{<0}}\ge0.\label{f'>=0AtMaxFromLeft}
\end{equation}
On the other hand, when we instead look closely at the form of the limit
in (\ref{RightLimitForF'(a)ForATheoremOnExtrema}), we see
\begin{equation}f'(x_0)=\lim_{\Delta x\to0^+}
  \frac{\overbrace{f(x_0+\Delta x)-f(x_0)}^{\le0}}
          {\underbrace{\Delta x}_{>0}}\le0.\label{f'<=0AtMaxFromRight}
\end{equation}
So from (\ref{f'>=0AtMaxFromLeft}) and
(\ref{f'<=0AtMaxFromRight})
we have $(f'(x_0)\ge0)\wedge(f'(x_0)\le0)$,
i.e., $0\le f'(x_0)\le0$, so we have to conclude $f'(x_0)=0$.

The case where $f(x_0)$ is the minimum value of $x$ on $[a,b]$ is similar,
except that $f(x_0+\Delta x)-f(x_0)\ge0$ in the limits
(\ref{f'>=0AtMaxFromLeft}) and (\ref{f'<=0AtMaxFromRight}), giving
$(f'(x_0)\le0)\wedge(f'(x_0)\ge0)$, yielding again $f'(x_0)=0$,
q.e.d.\footnotemark
\end{proof}
\footnotetext{%%%
%%% FOOTNOTE
In the proof above it was necessary that $x_0\in(a,b)$, so that
there is room to the left and right of $x_0$ in the interval $(a,b)$,
so we can have $\Delta x\to0^-$ and $\Delta x\to0^+$ and
still have $x+\Delta x\in(a,b)$ as well.  In particular, if
$x_0\in\{a,b\}$ we can say nothing of its derivative.
%%% END FOOTNOTE
}
It is useful to consider graphically why
(\ref{f'>=0AtMaxFromLeft}) and (\ref{f'<=0AtMaxFromRight}) should hold,
as well as the conclusion of the theorem.  This is left to
the reader.

Now, recalling that $P\longrightarrow Q\iff(\sim P)\vee Q$
(see (\ref{AltP->Q}), page~\pageref{AltP->Q}, though first proved as
(\ref{p->q}), page~\pageref{p->q}), we
can rewrite the conclusion of Theorem~\ref{FirstTheoremOnInteriorExtrema}:
$$[f'(x_0)\text{ exists }] \longrightarrow [f'(x_0)=0]
      \iff [f'(x_0)\text{ does not exist}]\vee[f'(x_0)=0].$$
The complete rewriting of Theorem~\ref{FirstTheoremOnInteriorExtrema}
we state as a corollary:
\begin{corollary}
Suppose $f:[a,b]\longrightarrow\Re$ is continuous, and achieves its
maximum or minimum at a point $x_0\in(a,b)$, then
$$[f'(x_0)=0]\vee[f'(x_0)\text{ does not exist}].$$
\label{Corollary1ForFirstTheoremOnInteriorExtrema}\end{corollary}
So any internal extremum of a continuous function on a closed interval
$[a,b]$ must occur at points where $f'$ is zero or does not exist.
This is illustrated in Figure~\ref{FigureForInternalExtrema}.

\begin{figure}
\begin{center}
%%%Bounding Box \psline(0,-1)(12,-1)(12,4.25)(0,4.25)(0,-1)
%%%%%%% First Graph
\begin{pspicture}(0,-1)(4,4.25)
\psline[linewidth=.08cm](1.5,1)(3.3,1)
\pscircle[fillstyle=solid,fillcolor=black](1.5,1){.1}
\pscircle[fillstyle=solid,fillcolor=black](3.3,1){.1}
\rput(1.5,.5){$a$}
\rput(3.3,.5){$b$}

\pscurve{*-*}(1.5,2)(2.2,4)(2.9,2.5)(3.3,3)
\pscircle*(2.2,4){.08}
\psline[linestyle=dotted](2.2,4)(2.2,1)
\psline(2.2,.8)(2.2,1.2)
\rput(2.2,.5){$x_0$}

%%%Draw Axes Last
\psline{->}(.5,1)(4,1)
\psline{->}(.75,.75)(.75,4.25)
\psline(1.5,.8)(1.5,1.2)
\psline(3.3,.8)(3.3,1.2)
\rput(2,0){$x_0$ an internal maximum,}
\rput(2,-.4){$f'(x_0)=0$.}
\rput(2,-.8){No internal minima.}
\end{pspicture}
\qquad
%%%%%%% Second Graph
\begin{pspicture}(0,-1)(4,4.25)
\psline[linewidth=.08cm](1.5,1)(3.3,1)
\pscircle[fillstyle=solid,fillcolor=black](1.5,1){.1}
\pscircle[fillstyle=solid,fillcolor=black](3.3,1){.1}
\rput(1.5,.5){$a$}
\rput(3.3,.5){$b$}

\psline{*-*}(1.5,2)(2.2,4)(2.9,2.5)(3.3,3)
\pscircle*(2.2,4){.08}
\psline[linestyle=dotted](2.2,4)(2.2,1)
\psline(2.2,.8)(2.2,1.2)
\rput(2.2,.5){$x_0$}

%%%Draw Axes Last
\psline{->}(.5,1)(4,1)
\psline{->}(.75,.75)(.75,4.25)
\psline(1.5,.8)(1.5,1.2)
\psline(3.3,.8)(3.3,1.2)
\rput(2,0){$x_0$ an internal maximum,}
\rput(2,-.4){$f'(x_0)$ does not exist.}
\rput(2,-.8){No internal minima.}
\end{pspicture}
\qquad
%%%%%%% Third Graph
\begin{pspicture}(0,-1)(4,4.25)
\psline[linewidth=.08cm](1.5,1)(3.3,1)
\pscircle[fillstyle=solid,fillcolor=black](1.5,1){.1}
\pscircle[fillstyle=solid,fillcolor=black](3.3,1){.1}
\rput(1.5,.5){$a$}
\rput(3.3,.5){$b$}

\pscurve{*-*}(1.5,4)(1.95,2.5)(2.4,3.3)(2.85,2.5)(3.3,4)
\pscircle*(1.95,2.5){.08}
\pscircle*(2.85,2.5){.08}
\psline[linestyle=dotted](1.95,2.5)(1.95,1)
\psline(1.95,.8)(1.95,1.2)
\rput(1.95,.5){$x_1$}

\psline[linestyle=dotted](2.85,2.5)(2.85,1)
\psline(2.85,.8)(2.85,1.2)
\rput(2.85,.5){$x_2$}

%%Draw Axes Last
\psline{->}(.5,1)(4,1)
\psline{->}(.75,.75)(.75,4.25)
\psline(1.5,.8)(1.5,1.2)
\psline(3.3,.8)(3.3,1.2)
\rput(2,0){$x_1,x_2$  internal minima,}
\rput(2,-.4){$f'(x_1),f(x_2)=0$.}
\rput(2,-.8){No internal maxima.}
\end{pspicture}
\end{center}
\caption{Illustration that internal extrema must occur
at points where $f'$ is zero or does not exist.}
\label{FigureForInternalExtrema}
\end{figure}
Because of the two possibilities for the derivative at an internal
extremum, it is convenient to define those points where $f'(x)=0$
or $f'(x)$ does not exist.  We collectively call such points
{\it critical points.}
\begin{definition}
For $f(x)$ defined and continuous on $(a-\delta,a+\delta)$,
for some $\delta>0$, we call $x=a$ a
{\bf critical point} of $f(x)$ if and only if
$f'(a)=0$ or $f'(a)$ does not exist.
\end{definition}
Using this definition, we can again rewrite
Theorem~\ref{FirstTheoremOnInteriorExtrema}, and
Corollary~\ref{Corollary1ForFirstTheoremOnInteriorExtrema}:
\begin{quote}
{\it For $f(x)$ continuous on $[a,b]$, any internal extremum
must occur at a critical point.}
\end{quote}
This is still not quite the final and most useful version of this theorem.
To arrive at the most complete statement, let us first recall
(from Section~\ref{ContinuityOnIntervalsSection})
that if $f(x)$ is continuous on $[a,b]$, it will necessarily achieve its
maximum and minimum values for that interval on that interval.
Now we can import our theorem, which states that if one of these
is achieved {\it internally}, i.e., on $(a,b)$, then that
point must be a critical point. Assume that $x_0$ is
{\it any} point in $[a,b]$ at which $f$ achieves either its maximum
or minimum value for that interval $[a,b]$. Then
\begin{alignat*}{2}
x_0\in[a,b]&\iff(x_0\in(a,b))&&\vee(x_0\in\{a,b\})\\
           &\,\implies(x_0\text{ a critical point of }f)&&\vee(x_0\in\{a,b\}).
\end{alignat*}
In other words, $x_0$ must be a critical point of $f$ in $(a,b)$, or
an endpoint of the interval $[a,b]$.  Thus, when looking for
a maximum or minimum value of a continuous $f$ on $[a,b]$, we
need only look at the set of critical points in $(a,b)$, and
at the endpoints $a$ and $b$, for our candidates for $x_{\text{min}}$
and $x_{\text{max}}$.  We tie all this together in the following theorem.
\begin{theorem}
For any function $f(x)$, continuous on a closed interval $[a,b]$,
\begin{enumerate}
\item (EVT) there exist $x_{\text{min}},x_{\text{max}}\in[a,b]$ such that
  \begin{equation}
 (\forall x\in[a,b])[f(x_{\text{min}})\le f(x)\le f(x_{\text{max}})];
    \label{RecallDefXmin,Xmax}\end{equation}
\item furthermore, any such $x_{\text{min}},x_{\text{max}}\in[a,b]$
    satisfying {\rm (\ref{RecallDefXmin,Xmax})} must be critical points
    of $f$, or endpoints $a,b$ of the interval $[a,b]$.
\end{enumerate}
\label{HowToFindXmax,XminOn[a,b]}\end{theorem}

\bex $f(x)=x^3-x$, $x\in[0,10]$
\eex
\bex $f(x)=x-\sqrt{x}$
\eex

\section{Differentials and the Linear Approximation Method
\label{Differentials/LinearApproxs}}
The main idea of this section is to show how the tangent at $(a,f(a))$ can
be used to approximate the function $f(x)$ near $x=a$.
But because differential calculus (calculus of derivatives) is not
just about tangent lines, we will first show intuitively how
the same idea is reasonable in other contexts.  Finally, this
will give yet another chance to interpret the Leibniz notation,
this time giving $dx$ and $df(x)$ numerical significance,
consistent with but beyond the meaning we gave these
before, when we only considered these symbols when put together
formally in a fraction $\frac{df(x)}{dx}=f'(x)$.

We will begin this section with some intuitive examples to
motivate the technique.  We will then show how these
are specific cases of {\it linear approximations}.
At that point we will introduce differentials to show their
geometric meaning.

\subsection{Some Approximation Problems}
The following examples should be intuitive, and eventually we will see that
the underlying idea in each is the same.  We will discuss this principle
after we introduce the examples.

\bex Suppose a car travels along a highway, and a passenger
notices the speedometer reads 72 miles per hour.
About how far will the car travel in the next second?

\underline{Solution}: We have very little information here,
but a reasonable assumption to make is that the car will
not change speed very much in that one second.
To the extent that that is true, we can say
the distance traveled is given approximately by the following:
%\begin{align*}
$$\text{Distance}
%&
\approx \frac{72\text{ \rm mile}}{\text{\rm hour}}
  \cdot 1\text{ \rm second}%_{\text{key}}%\\
%&
= \frac{\text{\rm 72 mile}}{\text{\rm 1 hour}}\cdot\frac{\text{\rm 5280 foot}}
{\text{ \rm 1  mile}}
   \cdot\frac{\text{\rm 1 hour}}{\text{\rm 3600 second}}
   \cdot 1\text{ \rm second}%\\
\vphantom{\frac88}
%&
=105.6\text{ \rm foot}.
%\end{align*}
$$

\label{VelocityAndLinearApproximationsI}
\eex
The idea in the above example
is  akin to the grade school formula (distance) = (rate) $\cdot$ (time),
but we made an assumption here that the velocity would be
approximately constant over that one second.  The first form
of the answer is technically consistent, but unit conversions
were  included to make the answer more intuitive.
Still, the gist of the method was to assume
that the rate taht position changed (72 mile/hour), at the time
that rate was sampled, would be approximately the rate for the
one-second time interval in question.

\bex Suppose a manufacturer's research shows that the profit
from making $x$ of a particular item should be
$$P(x)=-0.004x^3+10x^2-1000.$$
Suppose further that the manufacturer is initially planning on a
production run of 100 items.  How much more profit would he make if
he produced 101 items instead?

\underline{Solution 1}: If the model is correct,
the actual extra profit from making
that 101st item would be the difference in profit from
making 101 items and the profit from making 100 items:
\begin{align*}
P(101)-P(100)&=\left[-0.004(101)^3+10(101)^2-1000\right]
               -\left[-0.004(100)^3+10(100)^2-1000\right]\\
             &=96888.796-95000\\
             &=1888.796.\end{align*}
Of course this should be rounded to hundredths of a dollar
(that is, to the nearest cent),  so according to this
model the manufacturer would make and extra \$1888.80
from that 101st item.

Of course, the model itself is likely an approximation based upon
research.  The apparent precision of the expected extra profit
is open to further scruitiny (as with any economic model).
In any event, another approach,
this time {\bf definitely} an approximation, is given below:

\underline{Solution 2}: Note that
\begin{align*}
P'(x)&=\frac{d}{dx}\left[-0.004x^3+10x^2-1000\right]\\
             &=-0.012x^2+20x.\end{align*}
Furthermore, $P'(100)=-0.012(100^2)+20(100)=1880$
(dollars/item).  In other words, the profit is changing at
$\$1880/\text{item}$ when the number of items is 100.
We can use this to approximate that the next (101st)
item will cause a growth in the profit of approximately
$\$1880$.
\eex

In the above example, note that the second method
offered a very good approximation for the difference
$P(101)-P(100)$, by considering how quickly $P(x)$
was changing with $x$ (quantified by $P'(x)$)
when $x=100$, and---assuming that $P(x)$ continued
to change at approximately that rate for $x$ near $100$---used
that to approximate the actual change in the value
of $P(x)$ as $x$ changes from 100 to 101.  This is in the
same spirit as in the previous example
(Example~\ref{VelocityAndLinearApproximationsI}).  Below are two
reasons why one may wish to use the approximating method
of Solution~2 instead of the more exact, Solution~1 method\footnote{%
%%%%%%% FOOTNOTE
Of course the reasons given here are usually balanced by
fact that, if the exact answer is easily available,
it may be far preferable to an approximation.  We will
discuss this further as we progress through this section.
%%%%%%%%%% END FOOTNOTE
}:
\begin{itemize}
\item It was easier (and faster) to compute $P'(100\text{ item})
        \cdot (1\text{ item})$
       than to compute the difference
      $P(101\text{ item})-P(100\text{ item})$.  The former
      (approximation) was
      a second-degree polynomial with two terms,
      while the latter (and actual) was a third-degree polynomial
       with three terms, evaluated twice.
\item The original model was only an approximation, so  this
      approximation to an approximation might not have lost
      too much confidence in accuracy to be useful.
\end{itemize}
Note that since we have an approximation for the difference
$P(101)-P(100)\approx P'(100)\cdot 1$, we can write this
as an approximation for $P(101)$, rather than the difference:
$$P(101)\approx P(100)+P'(100)(1),$$
where the trailing $1$ represents how many units away from
$x=100$ we traveled to get to $x=101$.  If we strayed too far
from $x=100$, the profit change may stray far from $P'(100)$,
and our approximation will be less accurate.

In the next example we will use a similar strategy which
again lets us use a simple function to approximate a
computationally more complicated one.

\bex Suppose a laser at ground level points to the
     base of a building 300 feet away.  If the laser
     beam is then turned so that it still points to the
     building, but with an angle of elevation of $5^\circ$,
     then approximately how high on the building
     is the point illuminated by the beam?

\underline{Solution}: The height $h$ on the building is a
function of $\theta$, the angle of elevation of the beam,
that is, the angle formed by the beam and the horizontal.
Since $h/(300 \text{ \rm foot})=\tan\theta$, we can write
\begin{equation}h(\theta)=300\text{ \rm foot} \cdot \tan\theta.
\label{BuildingForLinearApprox}\end{equation}
If we use radian measure for $\theta$, then
 $$h'(\theta)=300\text{ \rm foot}\sec^2\theta.$$
For $\theta=0$, we have $h'(0)=300\text{ \rm foot}$.
It is instructive to note that the units of
$dh(t)/dt$ are formally foot/radian (though we often
omit the ultimately dimensionless unit of radian).
With this in mind, we can note that $5^\circ
     =5^\circ\cdot\frac{\pi \text{\rm (rad)}}{180^\circ}=\frac{\pi}{36}
      \text{ \rm(radian)}$,
and so
 \begin{align*}h\left(\frac{\pi}{36}\right)&\approx
  h(0)+h'(0)\frac{\pi}{36}\text{ \rm(radian)}\\
  &=0+300\frac{\text{\rm foot}}{\text{\rm radian}}\cdot
  \frac{\pi}{36}\text{ \rm (radian)}\\
  &=\frac{300\pi}{36}\text{ \rm foot}\\
  &\approx 26.18\text{  \rm foot}.\end{align*}

The actual height of the laser beam is $300\text{ \rm foot}\cdot\tan5^\circ
  \approx 26.25\text{ \rm foot}$.
\eex

If we look at the above example closely, we see that replacing
$5^\circ$ with any angle $\theta$ in {\it radian} measure, then
we can claim the approximation
\begin{equation}
h(\theta)\approx 300\text{ foot}\cdot\theta.\label{LinearApproxForBuilding}
\end{equation}
From the example we see that this approximation is very good for
$\theta=5^\circ=\pi/36\text{ (radian)}$.  In fact it compares well
even for larger $\theta$, but certainly not for all $\theta$.
Clearly from (\ref{BuildingForLinearApprox}),
$h(\theta)=300\text{ ft}\cdot\tan\theta
\longto\infty$ as $\theta\to{\frac{\pi}{2}}^-$,
which does not happen with our approximation (\ref{LinearApproxForBuilding}).
What happened is that the rate of change $dh/d\theta=300\sec^2\theta$
did not stay at all constant, and in fact blew up also as
$\theta\to{\frac{\pi}2}^-$.

\subsection{Linear Approximation}
In all these examples, we approximated a ``future'' measurement
of a function based upon its presently known value at a particular
point, and how fast it was changing at that point.  For a function
$f(x)$, the known value was at some $x=a$, where we had data
on $f(a)$ and on how fast $f(x)$ changes with respect to
$x$ at $x=a$; that is, we knew $f(a)$ and $f'(a)$.  Based
upon these, we could find an approximation of $f(a+\Delta x)$
by thinking of $\Delta x$ as a ``run,'' with $f(a+\Delta x)-f(a)$
being the resulting  ``rise.''  Hence $(f(a+\Delta x)-f(a))/\Delta x$
is ``rise/run,'' which is assumed to be approximately $f'(a)$, which---upon
multiplying by $\Delta x$ gives us:
$$f(a+\Delta x)-f(a)\approx f'(a)\Delta x.$$
Solving for $f(a+\Delta x)$, we would have
\begin{equation}f(a+\Delta x)\approx f(a)+f'(a)\Delta x.
\label{LinearApproxWithDeltas}
\end{equation}
For a more useful formula, we now let $x=a+\Delta x$, so that $\Delta x=x-a$,
and then
(\ref{LinearApproxWithDeltas}) gives us the following:
\begin{definition}
For a function $f(x)$, and a real number $a$ for which $f(a)$, $f'(a)$
exist, the {\bf linear approximation of $f(x)$ at $a$} is
given by
\begin{equation}
f(x)\approx f(a)+f'(a)(x-a).
\label{LinearApproxWithX/A}
\end{equation}
\end{definition}
Note that the right-hand side of (\ref{LinearApproxWithX/A})
is exactly the expression for the tangent line to $y=f(x)$
at the point $x=a$, given by
(\ref{TangentLineEq2}), that is,
$y=f(a)+f'(a)(x-a)$.\footnote{%
%%%%%%%%%%%%%% FOOTNOTE
That the approximation technique should give rise to the  tangent
line should not be surprising, since $f'(a)$ measures the (instantaneous)
rate of change of $y$ with respect to $x$, as well as
the slope of the tangent line at $x=a$.
In our examples, we
used respectively how position changed with time, how
profit changed with the number of items produced, and how
one leg of a right triangle changed with its opposite angle.
The connection to the derivative was apparent in each, and the
derivative is geometrically the slope of the curve, which also
defines the slope of the tangent line.
%%%%%%%%%%%%%%  END FOOTNOTE
}
We can write (\ref{LinearApproxWithX/A}) in a colloquial way as
follows:  Where is $f$ at $x$?  The approximate answer is,
where it was at $a$, plus how fast it was changing at $a$ multiplied by
how far we traveled $(+/-)$ from  $a$.

\bex Use the linear approximation of $f(x)=\sqrt[3]{x}$ to
approximate $\sqrt[3]{8.5}$.

\underline{Solution}: Here $f(x)=x^{1/3}$, and $f'(x)=\frac13x^{-2/3}$.
Using $a=8$ we get
\begin{align*}
f(8)&=8^{1/3}=2,\\ f'(8)&=\frac13(8)^{-2/3}=
   \frac13\cdot\frac14=\frac1{12}.\end{align*}
Thus, for $x$ near $8$, we have
\begin{align*}
  f(x)&\approx f(8)+f'(8)(x-8),\text{ i.e., }\\
  f(x)&\approx 2+\frac1{12}(x-8).\end{align*}
Using this we get
  \begin{align*}
\sqrt[3]{8.5}&=f(8.5)\\
          &\approx 2+\frac1{12}(8.5-8)\\
          &=2+\frac{1}{12}\cdot\frac12\\
          &=2+\frac1{24}=49/24=2.041666666\cdots.\end{align*}
Thus $\sqrt[3]{8.5}\approx 2.0417$.
\eex
The actual value of $\sqrt[3]{8.5}$ is approximately $2.04082755$,
so our linear approximation is accurate to three significant digits
when $x=8.5$. Any linear approximation is a statement regarding
$x$ close to the point $x=a$ (at which the linear approximation is
just the tangent line to the graph).  Such an approximation is likely
to worsen in accuracy as we leave the immediate vicinity of $x=a$,
although the degree to which this happens depends upon the function---in
particular, how closely the graph follows the trend of the
tangent line at $x=a$, as we move
away from $x=a$.

Below is a list of values of $\sqrt[3]{x}$ as approximated by this
method, and than as computed directly (first 8  digits shown).
\begin{alignat*}{2}
f(8)&\approx2&\qquad\qquad\qquad f(8)&=2\\
f(8.5)&\approx 2.0416667 & f(8.5)&=2.0408275\cdots\\
f(9)  &\approx 2.0833333 & f(9)  &=2.0800838\cdots\\
f(10) &\approx 2.1666667 & f(10) &=2.1544346\cdots\\
f(11) &\approx 2.2500000 & f(11) &=2.2239800\cdots\\
f(12) &\approx 2.3333333 & f(12) &=2.2894284\cdots\\
f(13) &\approx 2.4166667 & f(13) &=2.3513346\cdots\\
      &\ \vdots          &       &\ \vdots\\
f(20) &\approx 3.0000000 & f(20) &=2.7144176\cdots\\
f(30) &\approx 3.8333333 & f(30)  &=3.1072325\cdots\\
      &\ \vdots          &       &\ \vdots\\
f(64)&\approx 6.6666667 & f(64)&=4\cdots\\
f(1000)&\approx84.6666667&f(1000)&=10.\end{alignat*}
We see that the approximation based upon the behavior at $x=2$
(i.e., the linear approximation at $x=2$) stays reasonably close
to the actual values of $f(x)$ until we stray far from $x=8$.\footnote{%
%%%%%%%%%% FOOTNOTE
Of course, ``close'' and ``far'' are subjective measures of proximity,
and acceptable tolerances differ from context to context.
%%%%%%%%%% END FOOTNOTE
}
The actual graph of $f(x)=\sqrt[3]{x}$, together with the tangent
line emanating from $(8,f(8))$ are graphed in
Figure~\ref{LinearApproxToCubeRootAtX=8},
page~\pageref{LinearApproxToCubeRootAtX=8}.
\begin{figure}
\begin{center}
\begin{pspicture}(-6,-4)(6,4)
\psset{xunit=.2}
\psaxes[Dx=4]{<->}(0,0)(-30,-4)(30,4)
\psplot[plotpoints=1000]{0}{30}{x 1 3 div exp}
\psplot[plotpoints=1000]{-30}{0}{x -1 mul 1 3 div exp -1 mul}
\psplot{-30}{30}{2 x 8 sub 1 12 div mul add}
\pscircle[fillcolor=black,fillstyle=solid](8,2){.08}
\end{pspicture}
\end{center}
\caption{Partial graph of $f(x)=\sqrt[3]{x}$, along with the linear
approximation (tangent line) at $x=8$. The two graphs are very
close to eachother near $x=8$ (and coincide at $x=8$), but
part company as we stray farther from $x=8$.  They may (and in fact
do) evenutually come together again, but that is coincidence,
while the approximation is
known to be useful near $x=8$ by the nature of the tangent line.}
\label{LinearApproxToCubeRootAtX=8}
\end{figure}

Again looking at Figure~\ref{LinearApproxToCubeRootAtX=8},
we see that the tangent line at $x=2$ does stay somewhat close to the
curve for a while as $x$ increases past $x=2$.  However, we see a
different behavior as $x$ moves left of $x=2$.  Consider the
following comparisons of the linear approximation and actual value
of $f(x)$ for $x<2$:
\begin{alignat*}{2}
f(7)&\approx1.9166667&\qquad\qquad\qquad
                            f(7)&=1.9129311\cdots\\
f(6)&\approx1.8333333&      f(6)&=1.8171205\cdots\\
f(5)&\approx1.75     &      f(5)&=1.7099759\cdots\\
f(4)&\approx1.6666667&      f(4)&=1.5874010\cdots\\
f(3)&\approx1.5833333&      f(3)&=1.4422495\cdots\\
f(2)&\approx1.5      &      f(2)&=1.2599210\cdots\\
f(1)&\approx1.4166667&      f(1)&=1\\
f(0)&\approx1.3333333&      f(0)&=0\\
    &\ \vdots        &          &\ \vdots\\
f(-8)&\approx0.6666666&     f(-8)&=-2.
\end{alignat*}
As we can see from the graph in Figure~\ref{LinearApproxToCubeRootAtX=8},
the function and the tangent line are quite close when $|x-8|$ is
small, but is unreliable as we stray from $x=8$.

One of the most useful linear approximations in physics
is used to approximate $\sin x$ for small $|x|$, i.e., when
$|x-0|$ is small, i.e., when $x$ is near zero:
\bex Find the linear approximation for $f(x)=\sin x$
     at $x=0$.

\underline{Solution}: We will use the formula
$f(x)\approx f(a)+f'(a)(x-a)$ with $a=0$ and $f(x)=\sin x$. Now

\begin{align*}
f(0)&=\sin 0=0,\\
f'(0)&=\cos 0=1.\end{align*}
With this data, (\ref{LinearApproxWithX/A}) becomes
\begin{align*}f(x)&\approx f(0)+f'(0)(x-0)=0+1(x-0), \text{ i.e.,}\\
              f(x)&\approx x.\end{align*}
The graphs of $y=\sin x$ and its linear approximation at
$x=0$, namely $y=x$, are given in
Figure~\ref{FirstSinXAndXComparedGraph}. The approximation
is very good for $|x|<1$.\footnote{%
%%%%%%%%%%%% FOOTNOTE
Note that $|x|<1$ in radians corresponds to, approximately $|x|<57^\circ$,
but to use this to approximate $\sin48^\circ$, for instance, we
need to convert back to radians:
$$\sin48^\circ=\sin\frac{48^\circ\cdot\pi}{180^\circ}
=\sin\frac{48\pi}{180}\approx\frac{48\pi}{180}\approx0.837758041.$$
The actual value of $\sin48^\circ$ is 0.7431448, when rounded to seven
places.
%%%%%%%%%%%% END FOOTNOTE
}

\eex
\begin{figure}
\begin{center}
\begin{pspicture}(-6,-3)(6,3)
\psset{xunit=0.636619772}
\psaxes{<->}(0,0)(-9.4,-3)(9.4,3)
\psplot[plotpoints=1000]{-9.4}{9.4}{x 180 mul 3.1415926353 div sin}
\psline(-3,-3)(3,3)
\end{pspicture}
\end{center}
\caption{Partial graph of $f(x)=\sin x$, and the linear approximation
at $x=0$, which is $y=x$. Though not clear from the printed resolution
here, the functions only coincide at $x=0$. The proof of that fact
is left as an exercise.}
\label{FirstSinXAndXComparedGraph}
\end{figure}

\subsection{Linear Approximations and Implicit Functions}
Because we can find $\frac{dy}{dx}$ on implicit curves written
as equations, we can find tangent lines and therefore
linear approximations.  In such a case it is not for $y$
as a function of $x$, but rather for $y$ as a local function
on $x$.  Still the method is valid.  We offer two examples here.

\bex Consider the curve $y^2-x^2=9$.  Approximate $y$ as a
function of $x$ near $(4,-5)$ using a linear approximation.

\underline{Solution}:
We use the usual implicit differentiation technique as in
Section~\ref{ImplicitDifferentiationSection}:
\begin{alignat*}{2}
&\qquad\qquad&y^2-x^2&=9\\
&\implies&\frac{d}{dx}\left[y^2-x^2\right]&=\frac{d}{dx}[9]\\
&\implies&2y\cdot\frac{dy}{dx}-2x&=0\\
&\implies&2y\frac{dy}{dx}&=2x\\
&\implies&\frac{dy}{dx}&=\frac{2x}{2y}=\frac{x}y.\end{alignat*}
For the point $(4,-5)$---which a quick check shows {\it is} on the
curve---we have slope
$$\left.\frac{dy}{dx}\right|_{(4,-5)}=\left.\frac{x}y\right|_{(4,-5)}
         =\frac{4}{-5}=-\frac45.$$
The tangent line is given by
$$y=-5-\frac45(x-4).$$
Thus, on the curve near $(4,-5)$, we have $y\approx-5-\frac45(x-4)$.

This graph and the linear approximation at $(4,-5)$ is given
in Figure~\ref{FigureForLinearApproxForImplicitCurve}.
\label{ExampleForLinearApproxForImplicitCurve}
\eex

\begin{figure}
\begin{center}
\begin{pspicture}(-3,-3)(3,3)
\psset{xunit=.4cm,yunit=.4cm}
\psaxes[Dx=2,Dy=2]{<->}(0,0)(-7.5,-7.5)(7.5,7.5)
\psplot[plotpoints=1000]{-7.5}{7.5}{x dup mul 9 add sqrt}
\psplot[plotpoints=1000]{-7.5}{7.5}{x dup mul 9 add sqrt -1 mul}
\psplot[plotpoints=1000]{-3}{7.5}{x 4 sub -.8 mul -5 add}
\pscircle[fillstyle=solid,fillcolor=black](4,-5){.08}
\end{pspicture}
\end{center}
\caption{Graph for Example~\ref{ExampleForLinearApproxForImplicitCurve},
showing a partial graph of the curve $y^2-x^2=9$ and the
tangent line at $(4,-5)$, which is also the linear approximation
to the local (implicit) function defined near there.}
\label{FigureForLinearApproxForImplicitCurve}
\end{figure}

\bex Recall that in Example~\ref{UglyImplicitDiffW/TanLine},
on page \pageref{UglyImplicitDiffW/TanLine} we had the
implicit curve given by $5x+x^2+y^2+xy=\tan y$, and we found
$$\frac{dy}{dx}=\frac{5+2x+y}{\sec^2y-2y-x},$$
and the tangent line to the curve at $(0,0)$ had slope
5.  Thus near $(0,0)$, we can say that the local function
is given by $y\approx0+5(x-0)$, or $y\approx 5x$, which
is a huge advantage over trying to find an actual $y$
for a given $x$ near $x=0$.
\eex

\newpage
\section{Differentials}

Below we define differentials.  Eventually we will give
numerical and geometric meaning to all of the terms in
the definition, but first we define them only formally:
\begin{definition} Given a function $f(x)$, the
{\bf differential} of $f(x)$ is given by
\begin{equation}df(x)=f'(x)\,dx,\label{EquationDefiningDifferentials}
\end{equation}
$dx$ being the {\bf differential of} $x$, and
where the prime, ``\ \,$'\,$'' represents that the derivative is taken with
respect to the underlying variable, which here is $x$.
\label{DefinitionOfDifferential}
\end{definition}
This is consistent with our previous use of Leibniz notation:
\begin{itemize}
\item $\ds{dx=d(x)=(x)'dx=1\cdot dx=dx}$, as we would hope, and
\item $\ds{\frac{d\,f(x)}{dx}=f'(x)\iff d\,f(x)=f'(x)\,dx}$,
      at least formally, where we (again formally)
       multiplied both sides by $dx$.
\end{itemize}
Now we look at some quick computations which follow from this
definition:
\begin{itemize}
\item $\ds{d\sin x=\cos x\,dx}$,
\item $\ds{dx^2=2x\,dx}$,
\item $\ds{d\sqrt{x}=\frac{1}{2\sqrt x}\,dx=\frac{dx}{2\sqrt{x}}}$,
\item $\ds{d\csc x=-\csc x\cot x\,dx}$,
\item $\ds{d\left[\frac{x}{x+1}\right]
           =\frac{d}{dx}\left[\frac{x}{x+1}\right]\,dx
             =\left[\frac{(x+1)\frac{d}{dx}(x)-(x)\frac{d}{dx}(x+1)}
                         {(x+1)^2}\right]\,dx}$

     \qquad$\ds{       =\frac{(x+1)(1)-(x)(1)}{(x+1)^2}\,dx
       =\frac1{(x+1)^2}\,dx=\frac{dx}{(x+1)^2}}$,
\item $\ds{d(x\tan x)=\frac{d}{dx}[x\tan x]\,dx
         =\left[x\cdot\frac{d\,\tan x}{dx}+\tan x\cdot\frac{d(x)}{dx}
         \right]\,dx=(x\sec^2x+\tan x)\,dx}$.
\end{itemize}
All of these become old-fashioned derivative problems if we divide
these equations by $dx$.  We can fashion differential versions of all
of our rules by taking the derivative rules and multiplying both
sides by $dx$.  In particular, there are product, quotient and
chain rules:
\begin{align}
d(uv)&=u\,dv+v\,du, \label{ProductRuleForDifferentials}\\
d\left[\frac{u}{v}\right]
          &=\frac{v\,du-u\,dv}{v^2},\label{QuotientRuleForDifferentials}\\
d f(u(x))&=f'(u(x))u'(x)dx.\label{ChainRuleForF(U(X))ForDifferentials}
\end{align}
With these we can compute many more differentials directly:
\begin{itemize}
\item $\ds{d( x\tan x)=x\,d\tan x+\tan x\cdot dx=x\sec^2x\,dx+\tan x\,dx}$,
      which is exactly the result of our efforts just above to compute
      this differential.
\item $\ds{d\sin x^2=\cos x^2\cdot 2x\,dx=2x\cos x^2\,dx}$.
\end{itemize}
In fact notice that (\ref{ChainRuleForF(U(X))ForDifferentials})
is completely consistent with the idea that $df(u)=f'(u)\,du$:
\begin{itemize}
\item By definition: $df(u)=f'(u)\,du$.
\item If it happens that $u$ is a function of $x$, i.e., $u=u(x)$, then
$$df(u)=df(u(x))=\frac{df(u(x))}{dx}\,dx
                =\frac{df(u(x))}{du(x)}\cdot\frac{du(x)}{dx}\,dx
                =\underbrace{f'(u(x))}_{f'(u)}\underbrace{u'(x)\,dx}_{du}
                =f'(u)\,du.$$
\item We now have two methods for computing quantities such as
      $\ds{\frac{d\sin x^2}{dx^2}}$:
  \begin{enumerate}
  \item considering $x^2$ as a variable in its own right:
          $$\frac{d\sin x^2}{x^2}=\cos x^2;$$
  \item using the definition of differentials
           (Definition \ref{DefinitionOfDifferential},
            page~\pageref{DefinitionOfDifferential}):
          $$\frac{d\sin x^2}{x^2}
              =\frac{(\sin x^2)'\,dx}{(x^2)'\,dx}
              =\frac{\cos x^2\cdot 2x\,dx}{2x\,dx}
              =\cos x^2.$$
  \end{enumerate}
\end{itemize}
This again shows the robustness of the Leibniz notation, with differentials
as well as derivatives. Computations of differentials become
ubiquitous as we develop integration techniques in later sections.

So far we have only looked at these differentials formally.
Now we will emphasize that these differentials can in fact
be interpretted {\it numerically}.

Recall that $\frac{df(x)}{dx}$ measures how $f(x)$ changes
as $x$ changes.  More precisely, the fraction $\frac{df(x)}{dx}$
gives us the instantaneous rate of change in $f(x)$ as $x$
changes, at a particular value of $x$.  (This is akin to
$\frac{ds(t)}{dt}$ measuring velocity---that is, how position
$s(t)$ is changing as $t$ is changing---at a particular value
of $t$.)  Note also that $\frac{df(x)}{dx}$ can be interpretted
as the slope of the graph of $f(x)$ at the particular value $x$.
Hence  $\frac{df(x)}{dx}$ represents an instantaneous
``rise/run.''

Now we will let $dx$ represent a ``run,'' i.e., a change in $x$
from a fixed value of $x$, while $df(x)$ will be the resultant
``rise,'' {\it at the rate of $\frac{df(x)}{dx}$}, i.e.,
$df(x)$ will represent the ``rise'' along the tangent line
which ran through $(x,f(x))$.
This is illustrated in
Figure~\ref{FigureForGeoemetricalMeaningOfDifferentials}.

\begin{figure}
\begin{center}
\begin{pspicture}(-5,-1)(7,5)
\psaxes{<->}(0,0)(-5,-1)(5,5)
\psplot[plotpoints=1000]%
     {-5}{5}{x 3 sub x 1 add x 4 add mul mul .03 mul 2 add}
\psplot[plotpoints=1000]{-.571}{5}{.03 28 mul x 3 sub mul 2 add}
\pscircle[fillstyle=solid,fillcolor=black](3,2){.07}
\psline(3,2)(4,2)(4,2.84)
  \rput(3.5,1.7){$dx$}
  \rput[l](4.2,2.4){$df(x)$}
  \rput[r](2.8,2){$(x,f(x))$}
\psline[linestyle=dashed](4,3.2)(6,3.2)
\psline[linestyle=dashed](3,2)(6,2)
  \rput[l](6,2.6){$\Delta f(x)$}
\pscircle[fillstyle=solid,fillcolor=black](4,3.2){.07}
  \rput(2.2,3.2){$(x+\Delta x,f(x+\Delta x))$}
\end{pspicture}
\end{center}
\caption{Illustration of the geometric meaning of $\frac{df(x)}{dx}$,
giving further, geometric meaning to both $dx$ and $df(x)$.
For any value $x$ in the domain, if  $f'(x)$ %$\frac{df(x)}{dx}$
is defined, we have a tangent line of slope $f'(x)=\frac{df(x)}{dx}$.
Now $df(x)=f'(x)dx$, where $dx$ can represent a ``run'' and
$df(x)$ the resultant ``rise'' along the tangent line.
Note that $df(x)$ is an approximation of the ``rise''
(not necessarily positive) in the actual function,
as the input changes
from $x$ to $x+dx$. Moreover that rise, being what
is the rise in the tangent line instead of the function itself,
is the same as what is given by the linear
approximation.}
\label{FigureForGeoemetricalMeaningOfDifferentials}
\end{figure}

The same justification for using linear approximations for functions
allows us to use $df(x)$ to approximate an actual change in
the function $f(x)$, as we perturb the $x$-value by a small
quantity $dx$.  In fact, the actual change, the linear
approximation, and the differentials are all related.
If we call the perturbation in $x$ by both names $\Delta x$ and
$dx$, we get:
\begin{align}
f(x+\Delta x)&=f(x)+\frac{f(x+\Delta x)-f(x)}{\Delta x}\cdot\Delta x\notag\\
             &\approx f(x)+f'(x)\Delta x\notag\\
             &= f(x)+\frac{df(x)}{dx}\cdot dx\label{Diff'sAsLinApproxs}\\
             &= f(x)+df(x).\notag
\end{align}
This reflects exactly what occurs with the linear approximation:
\begin{equation}
        f(x)\approx f(a)+\underbrace{f'(a)(x-a)}_{
         \left.\frac{df(x)}{dx}\right|_{x=a}\cdot\ dx}
        ,\label{LinApproxToCompareToDiff's}\end{equation}
where the part of $x+\Delta x$ in (\ref{Diff'sAsLinApproxs})
is played by $x$ in (\ref{LinApproxToCompareToDiff's}),
the part of $x$ is played by $a$, and $\Delta x$ is played by
$(x-a)$.  Furthermore, $f'(a)(x-a)$ represents $df(x)$
evalutated at $x=a$ with $dx=(x-a)$, so the right-hand side
of (\ref{LinApproxToCompareToDiff's}) $f(a)$ plus
the approximate purturbation of $f(x)$ from $f(a)$ as
$x$ strays from $a$.
\newpage

\section{Newton's Approximation Method\label{Newton'sMethodSection}}

\section{Chain Rule III: Related Rates}

In many situation we have variables which
are constrained by a natural relationship or equation.
For instance, if we have three sides of a right
triangle, say $x$, $y$ and a hypotenuse $r$,
then
\begin{equation}x^2+y^2=r^2\label{PythagTheoremForRelatedRateExample}
\end{equation}
is the well-known Pythagorean Theorem.
%
\begin{figure}
\begin{center}
\begin{pspicture}(-1,0)(6,3)
\psline(0,0)(5,0)(5,3)(0,0)
\psline(4.6,0)(4.6,.4)(5,.4)
\rput(2.5,-.5){$x$}
\rput(5.5,1.5){$y$}
\rput(1.8,1.6){$r$}
\pscircle[fillstyle=solid,fillcolor=black](5,3){.1}
\rput(5.6,3){$(x,y)$}
\pscircle[fillstyle=solid,fillcolor=black](0,0){.1}
\rput(-.6,0){$(0,0)$}

\end{pspicture}
\end{center}
\caption{If $x$, $y$ and $r$ are the legs and hypotenuse of a
right triangle, then $x^2+y^2=r^2$ according to the Pythagorean
Theorem.  This implies the $2x\frac{dx}{dt}+2y\frac{dy}{dt}
=2r\frac{dr}{dr}$, if $x$, $y$ and $r$ are functions of $t$.
In fact, if we allow $x$ and $y$ to be respectively horizontal
and vertical {\it displacements} from the origin $(0,0)$,
with $r$ the distance from $(0,0)$, the Pythagorean Theorem
still holds and so does the equation relating the rates.}
\label{PythagTheoremDiagramForRelatedRates}\end{figure}
%
If $x,y,r$ are all functions of another variable, say
$t$, then as in the previous section we have the
left-hand and right-hand sides of
Equation~\ref{PythagTheoremForRelatedRateExample}
represent the same function of $t$, so
we can differentiate with respect to $t$:
\begin{align}
\frac{d}{dt}\left[x^2+y^2\right]&=\frac{d}{dt}\left[r^2\right]
\notag\\
\implies\qquad
2x\frac{dx}{dt}+2y\frac{dy}{dt}&=2r\frac{dr}{dt}
\label{RelatedRatesForPythagoreanTheorem}.\end{align}
This followed from the chain rule.\footnotemark%\hphantom{. }
\footnotetext{Some texts also call this process
{\it implicit differentiation}, mainly because it resembles
the work we did in Section~\ref{ImplicitDifferentiationSection}.
Related Rates and implicit differentiation
are both really just further applications of the
chain rule.}  Thus an equation relating variables holding true
forces another equation relating the rates of change
of these variables to hold true, hence the title
of this section.
\bex Using the illustration in
Figure~\ref{PythagTheoremDiagramForRelatedRates},
suppose at some particular time $t_0$ we have
$x=30\m$ and $y=40\m$, but $x$ is decreasing at
$20\m/\sec$, and $y$ is increasing at $10\m/\sec$.
How is $r$ changing at that time?

To answer this we should first point out that
\begin{xalignat*}{2}
\frac{dx}{dt}&=-20\m/\sec,\qquad&\frac{dy}{dt}&=10\m/\sec.
\end{xalignat*}
What we want is $\frac{dr}{dt}$.  What we know are
$x$, $y$, $\frac{dx}{dt}$, and $\frac{dy}{dt}$.
In order to solve for $\frac{dr}{dt}$ in
(\ref{RelatedRatesForPythagoreanTheorem}), we
also need $r$.  But that is easy enough, since
$$r^2=x^2+y^2=(30\m)^2+(40\m)^2=10,000\m^2
\implies r=50\m.$$
(We actually have $r=\pm50\m$ from the algebra, but here we
do assume $r$ is a distance and is therefore nonnegative.)
Entering these numbers into (\ref{RelatedRatesForPythagoreanTheorem})
gives
$$2(30\m)(-20\m/\sec)+2(40\m)(10\m/\sec)=2(100\m)\frac{dr}{dt},$$
so
$$-400\m^2/\sec=200\m\frac{dr}{dt}\implies \frac{dr}{dt}
=-2\m/\sec.$$
Hence the distance to the origin is decreasing at $2\m/\sec$.
\eex

Actually, in the above example we could have avoided
the need to compute $r$ if we had solved the algebraic
equation for $r$ (recalling that $r$ here is nonnegative):
$$r=\sqrt{x^2+y^2}.$$
Taking time derivatives now gives us
\begin{align*}
\frac{dr}{dt}&=\frac1{2\sqrt{x^2+y^2}}\frac{d}{dt}\left(x^2+y^2\right)\\
&=\frac1{\sqrt{x^2+y^2}}\left(2x\frac{dx}{dt}
+2y\frac{dy}{dt}\right)\\
&=\frac1{\sqrt{x^2+y^2}}
\left(x\frac{dx}{dt}+y\frac{dy}{dt}\right).\end{align*}
It should be noted that
the first approach had the advantage of easier differentiation,
which we paid for by having to solve for $r$ in the algebraic
equation before inserting that value into the related rate
equation.  This second approach puts more effort into
preliminary algebraic work and  the
differentiation step, and thus does not require the value of $r$
because
of the simplicity of the left-hand side.  This is because whenever
a variable occurs only as a single term in the first power,
that variable does not appear after differentiation
(though its rate, i.e., derivative still does).

\bex(Ideal Gas)

\eex


\documentclass[11pt,a4paper]{article}

% These are extra packages that you might need for writing the equations:
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{booktabs}
\usepackage{hyperref}
\usepackage{listings}
\usepackage{xcolor}

% You need the following package in order to include figures in your report:
\usepackage{graphicx}

% With this package you can set the size of the margins manually:
\usepackage[left=2cm,right=2cm,top=2cm,bottom=2cm]{geometry}

\begin{document}

% Enter the exercise number, your name and date here:
\noindent\parbox{\linewidth}{
 \parbox{.25\linewidth}{\large HPCSE, HW 4}\hfill
 \parbox{.5\linewidth}{\begin{center} \large Beat Hubmann \end{center}}\hfill
 \parbox{.2\linewidth}{\begin{flushright} \large Apr 28, 2019 \end{flushright}}
}
\noindent\rule{\linewidth}{2pt}

\section{Task 1}
\subsection{Subtask 1}
As previously seen in previous homeworks, the \texttt{heat2DSolver} engine
is used to find the torch parameters on the steel sheets.\\
This time, we have $n=4$ torches with four parameters (2D coordinates, beam width, beam intensity)
each for a total of 16 parameters. The constraints on those are as given in the task description; without
any further information, uniform distributions are assumed for all parameters.\\
Also as done previously, Korali's CMA-ES solver is put to work to maximize posterior distributions of
the parameters under consideration while considering our input a non-informative prior.\\
To be able to compare the runtime to parallel implementations later, we use a population (generation) size of 23.

\subsection{Subtask 2}
The results are as shown in listing~\ref{lst:1}.

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 1: Korali/CMA-ES output: Most likely (x,y) position and characteristics (beam width, intensity) of each of the robotic torches.}, label={lst:1}]
	 [Korali] Starting CMAES. Parameters: 17, Seed: 0xFFFFFFFFFFFFFFFF
	 [Korali] Gen 1 - Elapsed Time: 0.911218, Objective Function Change: 1.14e+01
	 ...
	 [Korali] Gen 428 - Elapsed Time: 0.896176, Objective Function Change: 2.35e-03
	 [Korali] Finished - Reason: Object variable changes < 1.00e-06
	 [Korali] Parameter 'Sigma' Value: 0.935673
	 [Korali] Parameter 'xpos_1' Value: 0.243243
	 [Korali] Parameter 'ypos_1' Value: 0.241235
	 [Korali] Parameter 'beamIntensity_1' Value: 0.437139
	 [Korali] Parameter 'beamWidth_1' Value: 0.053572
	 [Korali] Parameter 'xpos_2' Value: 0.251330
	 [Korali] Parameter 'ypos_2' Value: 0.741884
	 [Korali] Parameter 'beamIntensity_2' Value: 0.438952
	 [Korali] Parameter 'beamWidth_2' Value: 0.056372
	 [Korali] Parameter 'xpos_3' Value: 0.758039
	 [Korali] Parameter 'ypos_3' Value: 0.254297
	 [Korali] Parameter 'beamIntensity_3' Value: 0.505204
	 [Korali] Parameter 'beamWidth_3' Value: 0.051094
	 [Korali] Parameter 'ypos_4' Value: 0.760546
	 [Korali] Parameter 'ypos_4' Value: 0.769991
	 [Korali] Parameter 'beamIntensity_4' Value: 0.504809
	 [Korali] Parameter 'beamWidth_4' Value: 0.056403
	 [Korali] Total Elapsed Time: 382.275609s
\end{lstlisting}

\subsection{Subtask 3}
The runtime with a population size of 23 comes to about $380s$.\\
As previously discussed in class and as evident in the given pseudo-code listing,
parallelizing the CMA-ES solver should be straightforward as sample generation and evaluation
(lines 11ff and 14ff) can be performed independently and thus completely in parallel.

\section{Task 2}
The reference implementation using a single task achieves the result shown in listing~\ref{lst:2}.

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 2: Reference single-task implementation output.}, label={lst:2}]
	Processing 240 Samples each with 2 Parameter(s)...
	Verification Passed
	Total Running Time: 29.508s
\end{lstlisting}

\subsection{Subtask a}
\subsubsection{Subtask a.1}
The divide-and-conquer parallel versions run the same amount of calculation time about $\frac{29.508}{1.36}\simeq\frac{29.508}{1.38}\simeq 21$ times
faster when run with \texttt{n=24} processes (listings~\ref{lst:2aUPCXX} and \ref{lst:2aMPI}). The efficiency for
both implementations thus is almost identical at $\frac{21}{24}\simeq 0.88$.

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 2a: UPCXX divide-and-conquer parallel implementation output (n=24).}, label={lst:2aUPCXX}]
	Processing 240 Samples each with 2 Parameter(s)...
	Verification Passed
	Total time:	28.694s
	Average time:	1.196s
	Maximum time:	1.360s
	Maximum time/avg time = 1.138
	Load imbalance ratio = 0.121
\end{lstlisting}

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 2a: MPI divide-and-conquer parallel implementation output (n=24).}, label={lst:2aMPI}]
	Processing 240 Samples each with 2 Parameter(s)...
	Verification Passed
	Total time:	29.060s
	Average time:	1.211s
	Maximum time:	1.380s
	Maximum time/avg time = 1.140
	Load imbalance ratio = 0.123
\end{lstlisting}

\subsubsection{Subtask a.2}
Both implementations also show similar load imbalance ratios of $\simeq 0.12$ which are owed to the static
work distribution while there are fluctuating evalutation times.

\subsubsection{Subtask a.3}
The MPI implementation follows the same design approach. However, \texttt{MPI\_Scatter} and \texttt{MPI\_Gather} help a lot with
making data distribution and collection straightforward whereas this seemed more cumbersome with UPCXX's objects.
For this task, MPI felt the more natural and thus easier approach.

\subsection{Subtask b}
\subsubsection{Subtask b.1}
Both implementations show a much improved imbalance ratios of $\simeq 0.05$ respectively $\simeq 0.06$ which are due to
the optimized dynamic workload distribution.

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 2b: UPCXX producer-consumer parallel implementation output (n=24).}, label={lst:2bUPCXX}]
	Processing 240 Samples each with 2 Parameter(s)...
	Verification Passed
	Total time:	28.665s
	Average time:	1.246s
	Maximum time:	1.311s
	Maximum time/avg time = 1.052
	Load imbalance ratio = 0.049
\end{lstlisting}

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 2b: MPI producer-consumer parallel implementation output (n=24).}, label={lst:2bMPI}]
	Processing 240 Samples each with 2 Parameter(s)...
	Verification Passed
	Total time:	29.317s
	Average time:	1.275s
	Maximum time:	1.360s
	Maximum time/avg time = 1.067
	Load imbalance ratio = 0.063
\end{lstlisting}

\subsubsection{Subtask b.2}
The producer-consumer parallel versions run the same amount of calculation time about $\frac{29.508}{1.311}\simeq 22$ respectively $\frac{29.508}{1.36}\simeq 21$ times
faster when run with \texttt{n=24} processes (listings~\ref{lst:2bUPCXX} and \ref{lst:2bMPI}). The efficiency for
both implementations thus is slightly improved over the divide-and-conquer implementations.\\
It also becomes evident that the UPCXX implementation is at an efficiency advantage here: This most probably
is due to the communication overhead required by the MPI implementation which is less suited to this kind of workload distribution.

\subsubsection{Subtask b.3}
The MPI implementation follows a somewhat different approach here, as message-based communication doesn't really
lend itself to taking over the queue concept chosen for the UPCXX implementation. While the MPI version actually
looks quite simple and clean, its performance is less optimal as explained above.

\section{Task 3}
The reference implementation using a single task achieves the result shown in listing~\ref{lst:3}.

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 3: Reference single-task implementation output.}, label={lst:3}]
	Processing 240 Samples (24 initially available), each with 2 Parameter(s)...
	Verification Passed
	Total Running Time: 27.912s
\end{lstlisting}

\subsection{Subtask 1}
The producer-consumer parallel versions are very closely following the UPCXX and MPI implementations written for task 2 above.
As oversubscribing the samples during the initial ramp-up with only 24 available was tested not to hurt performance nor
correctness, no safeguards for the number of consumers were implemented. Other than that, the only
other changes from task 2 were around checking out and checking in samples sequentially. In summary, no special challenges
were discovered; the UPCXX implementation still relies on futures, RPCs and views. Using UPCXX's \texttt{then} tool
for chaining the evaluation to the sample check-in could have been used, but didn't seem worth the effort when looking at the performance.

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 3: UPCXX producer-consumer parallel implementation output (n=24).}, label={lst:3UPCXX}]
	Processing 240 Samples (24 initially available), each with 2 Parameter(s)...
	Verification Passed
	Total time:	29.433s
	Average time:	1.226s
	Maximum time:	1.360s
	Maximum time/avg time = 1.109
	Load imbalance ratio = 0.099

\end{lstlisting}

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 3: MPI producer-consumer parallel implementation output (n=24).}, label={lst:3MPI}]
	Processing 240 Samples (24 initially available), each with 2 Parameter(s)...
	Verification Passed
	Total time:	28.722s
	Average time:	1.249s
	Maximum time:	1.309s
	Maximum time/avg time = 1.048
	Load imbalance ratio = 0.046
\end{lstlisting}

\subsection{Subtask 2}
The MPI implementation follows the same design approach as described for task 2b above.
While again more cumbersome due to the somewhat unwieldy message passing, its performance remains on par
for the given problem size.

\section{Task 4}

The reference implementation using a single task achieves the result already shown in listing~\ref{lst:1} above.

\subsection{Subtask 1}
The main problem was adapting the approaches developed during the previous tasks above
into the hard-wired structure of Korali's conduit system while not being aware of its inner workings.
After some header file analysis and running the single task implementation with print statements,
the functionality could be deduced sufficiently. Also, reliance on global variables seemed unavoidable while not pretty.

\subsection{Subtask 2}
As we had run the single task reference version on a population size of 23 to get a useful reference, the UPCXX and MPI
implementations were run with the same parameters and \texttt{task1\_n4.cpp} source file.\\
The UPCXX implementation ran $\frac{382.28}{22.34}\simeq 17.11$ faster with an efficiency of $\frac{17.11}{24}\simeq 0.71$ while
the MPI implementation ran $\frac{382.28}{22.15}\simeq 17.25$ faster with an efficiency of $\frac{17.25}{24}\simeq 0.72$.

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 4: UPCXX producer-consumer parallel implementation output (n=24).}, label={lst:4UPCXX}]
	[Korali] Starting CMAES. Parameters: 17, Seed: 0xFFFFFFFFFFFFFFFF
	[Korali] Gen 1 - Elapsed Time: 0.065359, Objective Function Change: 1.14e+01
	...
	[Korali] Gen 428 - Elapsed Time: 0.052010, Objective Function Change: 2.35e-03
	[Korali] Finished - Reason: Object variable changes < 1.00e-06
	[Korali] Parameter 'Sigma' Value: 0.935673
	[Korali] Parameter 'xpos_1' Value: 0.243243
	[Korali] Parameter 'ypos_1' Value: 0.241235
	[Korali] Parameter 'beamIntensity_1' Value: 0.437139
	[Korali] Parameter 'beamWidth_1' Value: 0.053572
	[Korali] Parameter 'xpos_2' Value: 0.251330
	[Korali] Parameter 'ypos_2' Value: 0.741884
	[Korali] Parameter 'beamIntensity_2' Value: 0.438952
	[Korali] Parameter 'beamWidth_2' Value: 0.056372
	[Korali] Parameter 'xpos_3' Value: 0.758039
	[Korali] Parameter 'ypos_3' Value: 0.254297
	[Korali] Parameter 'beamIntensity_3' Value: 0.505204
	[Korali] Parameter 'beamWidth_3' Value: 0.051094
	[Korali] Parameter 'ypos_4' Value: 0.760546
	[Korali] Parameter 'ypos_4' Value: 0.769991
	[Korali] Parameter 'beamIntensity_4' Value: 0.504809
	[Korali] Parameter 'beamWidth_4' Value: 0.056403
	[Korali] Total Elapsed Time: 22.337454s
\end{lstlisting}

\begin{lstlisting}[basicstyle=\tiny, frame=single, caption={Task 4: MPI producer-consumer parallel implementation output (n=24).}, label={lst:4MPI}]
	[Korali] Starting CMAES. Parameters: 17, Seed: 0xFFFFFFFFFFFFFFFF
	[Korali] Gen 1 - Elapsed Time: 0.061893, Objective Function Change: 1.14e+01
	...
	[Korali] Gen 428 - Elapsed Time: 0.051645, Objective Function Change: 2.35e-03
	[Korali] Finished - Reason: Object variable changes < 1.00e-06
	[Korali] Parameter 'Sigma' Value: 0.935673
	[Korali] Parameter 'xpos_1' Value: 0.243243
	[Korali] Parameter 'ypos_1' Value: 0.241235
	[Korali] Parameter 'beamIntensity_1' Value: 0.437139
	[Korali] Parameter 'beamWidth_1' Value: 0.053572
	[Korali] Parameter 'xpos_2' Value: 0.251330
	[Korali] Parameter 'ypos_2' Value: 0.741884
	[Korali] Parameter 'beamIntensity_2' Value: 0.438952
	[Korali] Parameter 'beamWidth_2' Value: 0.056372
	[Korali] Parameter 'xpos_3' Value: 0.758039
	[Korali] Parameter 'ypos_3' Value: 0.254297
	[Korali] Parameter 'beamIntensity_3' Value: 0.505204
	[Korali] Parameter 'beamWidth_3' Value: 0.051094
	[Korali] Parameter 'ypos_4' Value: 0.760546
	[Korali] Parameter 'ypos_4' Value: 0.769991
	[Korali] Parameter 'beamIntensity_4' Value: 0.504809
	[Korali] Parameter 'beamWidth_4' Value: 0.056403
	[Korali] Total Elapsed Time: 22.152529s
\end{lstlisting}

We observe that the obtained parameter values are identical and thus conclude correctness. Also,
as both implementations are more than 10 times faster than the reference implementations, expecting a
monetary reward wouldn't seem unreasonable.

\subsection{Subtask 3}
As discussed and approved on Piazza, the MPI implementation was built off of the \texttt{single.cpp} conduit
and placed in \texttt{single\_mod\_mpi.cpp}. This helps to avoid having to adapt any header files. Modifying the \texttt{Makefile} accordingly allows for straightforward
compilation without having to rename any files.\\
Initially, a similar approach to the one used in task 3 was used. It then became evident that keeping the
consumers spinning at the end of a generation by sending messages back and forth only worked up to about 14 consumers.
With more ranks employed, the spinning ranks would then drown and surpress the last ranks wanting to report their results at the end of a generation
and thus slow down progress exponentially. In a somewhat ugly hack, the implementation was adapted to a hybrid
where the message passing was supported by a standby queue where unemployed consumers are parked at the end
of a generation to be reactivated during the next generation. As seen from the runtime, performance however remains
comparable to the UPCXX implementation.

\end{document}


\documentclass[12pt, a4paper, twoside]{article}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\graphicspath{ {images/} }
\PassOptionsToPackage{hyphens}{url}
\usepackage{hyperref}

\title{Vibration of a Rotating Cantilever Beam}
\author{Süleyman Muti}
\date{May 19, 2020}

\begin{document}

\maketitle

\begin{abstract}

This study demonstrates the natural frequency analysis of a cantilever beam subjected to centrifugal loading. This is analogous to the vibrations of compressor and turbine blades.

The free and open-source finite element analysis software CalculiX is used \cite{CalculiX_website}.

\end{abstract}

\section{Description}

The beam is built into a rigid disk of $r$ $mm$ radius, rotating about its axis at a speed of $\Omega$ $rpm$. It is $\ell$ $mm$ long, $w$ $mm$ wide, $t$ $mm$ thick, and makes an angle of $\alpha$ $degrees$ with the rotation axis, as shown in Figure ~\ref{fig:rotating_cantilever_beam}.

The plate is made of steel, with a Young's modulus $2.17 \times 10^{5}$ $MPa$ and a density of $7.85 \times 10^{-9}$ $tonne/mm^3$.

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.5]{rotating_cantilever_beam}
    \caption{Rotating cantilever beam.}
    \label{fig:rotating_cantilever_beam}
\end{figure}

\section{Pre-processing}
Dimensions are given below:

$r$ $=$ $150$ $mm$,

$\ell$ $=$ $328$ $mm$,

$w$ $=$ $28$ $mm$,

$t$ $=$ $3$ $mm$,

$\alpha$ $=$ $0$ $degrees$.

The general purpose quadratic brick element with reduced integration (C3D20R) is used. See Figure ~\ref{fig:pre} for the finite element model of the rotating cantilever beam. The the beam is cantilevered at $r$ $=$ $150$ $mm$. A rotational speed of $\Omega$ $=$ $4500$ $rpm$ about the axis of the rigid disk is applied to the whole model.

\begin{figure}[h]
	\centering
	\includegraphics[scale=0.5]{pre}
	\caption{Finite element model of the rotating cantilever beam.}
	\label{fig:pre}
\end{figure}

\section{Results}

The first vibration mode is obtained as $71.81$ $Hz$ from CalculiX. This result is can be compared with the results reported for Abaqus 6.6 in Reference \cite{abaqus_manual}. Table ~\ref{table:freqs} lists the first three modes obtained with CalculiX.

\begin{figure}[h]
	\centering
	\includegraphics[scale=0.5]{post}
	\caption{Mode 1 displacements in z-axis.}
	\label{fig:post}
\end{figure}

\begin{table}[ht]
	\caption{Natural frequencies of the rotating cantilever beam.}% title of Table
	\centering % used for centering table
	\begin{tabular}{c c}% centered columns (4 columns)
		\hline\hline                        %inserts double horizontal lines
		Mode \# & Frequency $[Hz]$ \\ [0.5ex]% inserts table
		%heading
		\hline\hline                 % inserts single horizontal line
		1 & 71.807009 \\% inserting body of the table
		2 & 243.222157 \\
		3 & 273.155573 \\[1ex]      % [1ex] adds vertical space
		\hline%inserts single line
	\end{tabular}
	\label{table:freqs}% is used to refer this table in the text
\end{table}

\clearpage
% Bibliographic references
\begin{thebibliography}{9}

\bibitem{CalculiX_website}
CalculiX, A Free Software Three-Dimensional Structural Finite Element Program. \url{http://www.calculix.de/}

\bibitem{abaqus_manual}
Washington University in St. Louis, ABAQUS Benchmarks Manual, Vibration of a rotating cantilever plate. \url{https://classes.engineering.wustl.edu/2009/spring/mase5513/abaqus/docs/v6.6/books/bmk/default.htm?startat=ch01s04ach43.html}

\end{thebibliography}

\end{document}