\lab{Intro to pandas I}{Intro to pandas I}
\objective{Become acquainted with the data structures and tools that pandas offers for data analysis.}
\label{lab:pandas1}

In volumes 1 and 2, we solved data problems primarily using NumPy and SciPy.
While extremely flexible and useful tools, these libraries lack some of the high-level
data-analytic abstractions present in other popular data packages like R and Stata.
We now turn our attention to \emph{pandas}, a Python library that is more specifically built
for data analysis.

\section*{Data Structures in pandas}
Just as NumPy is built on the \li{ndarray} data structure suited for efficient scientific
and numerical computation, pandas is centered around a handful of core data structures
custom built for data analysis. These data structures include the \li{Series}, \li{DataFrame},
and \li{Panel}, which correspond roughly to one, two, and three-dimensional arrays.
We will explore the first two data structures in some detail.
The interested reader can learn more about the \li{Panel} data structure (the least-used one in pandas) in the online documentation.

\subsection*{Series}
The \li{Series} is a one-dimensional array with labeled entries. The values of the array may be
any data type, including integers, strings, or general Python objects. Further, the array
need not be homogeneous. That is, it can hold values of different data types. Together,
the array values are referred to as the \emph{data} of the \li{Series}.
The labels must consist of hashable types, and are most commonly integers or strings.
Together, the labels are referred to as the \emph{index} of the \li{Series}.

Thus, a \li{Series} consists of data and an index. The most basic way to initialize such an object
is as follows:
\begin{lstlisting}
>>> import pandas as pd
>>> s = pd.Series(data, index=index)
\end{lstlisting}
We don't need to explicitly define the index. The default index is simply \li{np.arange(len(data))}.

For example, we can create a Series containing the integers from 9 down to 0:
\begin{lstlisting}
>>> s1 = pd.Series(range(9, -1, -1))
>>> s1.values    #the data
array([9, 8, 7, 6, 5, 4, 3, 2, 1, 0])
>>> s1.index     #the labels
Int64Index([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype='int64')
>>> s1           #left column is index, right column is data
0    9
1    8
2    7
3    6
4    5
5    4
6    3
7    2
8    1
9    0
dtype: int64
\end{lstlisting}

Here is an example where we create customized labels:
\begin{lstlisting}
>>> import numpy as np
>>> data = np.random.randn(3)
>>> index = ['first', 'second', 'third']
>>> s2 = pd.Series(data, index=index)
>>> s2
first     1.661255
second   -0.033570
third    -2.185991
dtype: float64
\end{lstlisting}

We can create a \li{Series} having constant values in the following manner:
\begin{lstlisting}
>>> val = 4     #desired constant value of Series
>>> n = 6       #desired length of Series
>>> s3 = pd.Series(val, index=range(n))
>>> s3
0    4
1    4
2    4
3    4
4    4
5    4
dtype: int64
\end{lstlisting}

It is also possible to use a Python dictionary when creating a \li{Series}:
\begin{lstlisting}
>>> d = {'e1': 93, 'e2': 95, 'e3': 87, 'e4': 82, 'e5': 94}
>>> s4 = pd.Series(d)
>>> s4
e1    93
e2    95
e3    87
e4    82
e5    94
dtype: int64
\end{lstlisting}
Note that we didn't need to specify the index; the keys of the dictionary are used as the index for the \li{Series}.
There are many more ways to create Series objects.
For a more complete discussion of how to create Series objects, see the online documentation.

\begin{problem}
Create the following pandas \li{Series}.

\begin{itemize}
\item Constant array with value -3, length 5. Labels should be the first five positive even integers.

\item Data is given by the dictionary \li{\{'Bill': 31, 'Sarah': 28, 'Jane': 34, 'Joe': 26\}}.
\end{itemize}
\end{problem}

\subsection*{DataFrame}
The \li{DataFrame} data structure is a two-dimensional generalization of the \li{Series}. It can be viewed
as a tabular structure with labeled rows and columns. The row labels are collectively called the
index, and the column labels are collectively called the columns. An individual column in a
\li{DataFrame} object is a \li{Series}.

There are many ways to initialize a \li{DataFrame}. In the following, we build a \li{DataFrame} out of a
dictionary of \li{Series}.
\begin{lstlisting}
>>> x = pd.Series(np.random.randn(4), ['a', 'b', 'c', 'd'])
>>> y = pd.Series(np.random.randn(5), ['a', 'b', 'd', 'e', 'f'])
>>> d = {1: x, 2: y}
>>> df1 = pd.DataFrame(d)
>>> df1
	        1	        2
a	-0.924259	-0.708301
b	 0.767422	-2.214516
c	 0.399212	      NaN
d	 0.130365	-2.352364
e	      NaN	 0.789419
f	      NaN	-0.859482
\end{lstlisting}
Note that the index of this \li{DataFrame} is the union of the index of \li{Series} \li{x} and that of \li{Series} \li{y}.
The columns are given by the keys of the dictionary \li{d}. Since \li{x} doesn't have a label \li{e}, the
value in row \li{e}, column \li{1} is \li{NaN}. This same reasoning explains the other missing values as well.
Note that if we take the first column of the \li{DataFrame} and drop the missing values, we recover the \li{Series} \li{x}:
\begin{lstlisting}
>>> x == df1[1].dropna()
a    True
b    True
c    True
d    True
dtype: bool
\end{lstlisting}

\begin{warn}
A Pandas \li{DataFrame} cannot be sliced the same way a NumPy array could.
Notice how we just used \li{df1[1]} to access the first \emph{column} of the the \li{DataFrame} \li{df1}.
We will discuss this in more detail later on.
\end{warn}

We can also initialize a \li{DataFrame} using a NumPy array, creating custom row and column labels:
\begin{lstlisting}
>>> data = np.random.random((3, 4))
>>> pd.DataFrame(data, index=['A', 'B', 'C'], columns=range(1, 5))

            1	        2	        3	        4
A	 0.065646	 0.968593	 0.593394	 0.750110
B	 0.803829	 0.662237	 0.200592	 0.137713
C	 0.288801	 0.956662	 0.817915	 0.951016
3 rows     4 columns
\end{lstlisting}
As with Series, if we don't specify the index or columns, the default is \li{range(n)}, where \li{n}
is either the number of rows or columns.

It is also possible to create multi-indexed arrays, for example:
\begin{lstlisting}
>>> grade=['eighth', 'ninth', 'tenth']
>>> subject=['math', 'science', 'english']
>>> myindex = pd.MultiIndex.from_product([grade, subject], names=['grade', 'subject'])
>>> myseries = pd.Series(np.random.randn(9), index=myindex)
>>> myseries
grade   subject
eighth  math       1.706644
        science   -0.899587
        english   -1.009832
ninth   math       2.096838
        science    1.884932
        english    0.413266
tenth   math      -0.924962
        science   -0.851689
        english    1.053329
dtype: float64
\end{lstlisting}

Multi-indexing is visually convenient, but not strictly necessary for most applications.
The interested reader is invited to explore the documentation to learn more.

\section*{Data I/O}
Being able to import and export data is a fundamental skill in data science. Unfortunately, with the
multitude of data formats and conventions out there, importing data can often be a tricky task. The pandas library
seeks to reduce some of the difficulty by providing file readers for various types of formats, including
CSV, Excel, HDF5, SQL, JSON, HTML, or pickle files.

Because CSV files are one of the most popular file formats for exchanging data, we will explore the
\li{read_csv} function in more detail. To learn to read other types of file formats, see the online pandas documentation.
To read a CSV data file into a \li{DataFrame}, call the \li{read_csv} function with the path to the CSV file,
along with the appropriate keyword arguments. Below we list some of the most important keyword arguments:
\begin{itemize}
\item \li{delimiter}:
This argument specifies the character that separates data fields, often a comma or a whitespace character.

\item \li{header}:
The row number (starting at 0) in the CSV file that contains the column names.

\item \li{index_col}:
If you want to use one of the columns in the CSV file as the index for the \li{DataFrame},
set this argument to the desired column number.

\item \li{skiprows}:
If an integer $n$, skip the first $n$ rows of the file, and then start reading in the data. If a list
of integers, skip the specified rows.

\item \li{names}:
If the CSV file does not contain the column names, or you wish to use other column names, specify them
in a list assigned to this argument.

\end{itemize}
There are several other keyword arguments, but this should be enough to get you started.

When you need to save your data, pandas allows you to write to several different file formats.
A typical example is the \li{to_csv} function method attached to \li{Series} and \li{DataFrame} objects,
which writes the data to a CSV file.
Keyword arguments allow you to specify the separator character, omit writing the columns names or index,
and other options. The code below demonstrates its typical usage:
\begin{lstlisting}
>>> df.to_csv("my_df.csv")
\end{lstlisting}

\section*{Viewing and Accessing Data}

Once we have our data ready to go in pandas, how can we interact with it?
In this section we will explore some elementary access, plotting, and querying techniques that enable us to maneuver through
and gain insight into our data.

\subsection*{Basic Data Access}
Some of the basic slicing paradigms in NumPy carry over to pandas.
For example, we can slice a \li{Series} using the usual syntax:
\begin{lstlisting}
>>> s = pd.Series(np.random.randn(5))
>>> s[1:3]

1    3.188112
2    0.080191
dtype: float64
\end{lstlisting}
Notice that both the data and the index are sliced in this manner.

Likewise, we can slice the rows of a \li{DataFrame} much as with a two-dimensional NumPy array:
\begin{lstlisting}
>>> df = pd.DataFrame(np.random.randn(4, 2), index=['a', 'b', 'c', 'd'], columns = ['I', 'II'])
>>> df[:2]

          I        II
a  0.758867  1.231330
b  0.402484 -0.955039

[2 rows x 2 columns]
\end{lstlisting}

More generally, we can select subsets of the data using the \li{.iloc} and \li{.loc} methods.
The \li{.loc} method selects rows and columns based on their labels, while the \li{.iloc} method
selects them based on their integer position.
It is interesting to know that accessing Series and DataFrames using these indexing operations is more efficient than using bracket indexing.
Since Series and DataFrames can be accessed by a numerical index or by label, the bracket indexing has to check many cases before it can determine how to slice the data structure.
By using \li{loc}/\li{iloc} explicitly, you bypass all of the extra checks.
\begin{lstlisting}
>>> # select rows a and c, column II
>>> df.loc[['a','c'], 'II']

a    1.231330
c    0.556121
Name: II, dtype: float64

>>> # select last two rows, first column
>>> df.iloc[-2:, 0]

c   -0.171938
d   -0.814336
Name: I, dtype: float64
\end{lstlisting}
Finally, a column of a \li{DataFrame} may be accessed using simple square brackets and the name of the column:
\begin{lstlisting}
>>> # get second column of df
>>> df['II']

a    1.231330
b   -0.955039
c    0.556121
d    0.173165
Name: II, dtype: float64
\end{lstlisting}

All of these techniques for getting subsets of the data may also be used to set subsets of the data:
\begin{lstlisting}
>>> # set second columns to zeros
>>> df['II'] = 0
>>> df['II']

a    0
b    0
c    0
d    0
Name: II, dtype: int64
\end{lstlisting}

\subsection*{Plotting}
Plotting is often a much more effective way to view and gain understanding of a dataset than simply
viewing the raw numbers. Fortunately, pandas interfaces well with matplotlib, allowing relatively
painless data visualization.

We start by plotting a \li{Series}. Doing so is easy, as the \li{Series} object is equipped with its own plot
function.
Let's start with visualizing a simple random walk. By way of background, a \emph{random walk} is a
stochastic process used to model a non-deterministic path through some space. It is a useful construct in
many fields and can be used to explain things like the motion of a molecule as it travels through a liquid to modeling or the fluctuations
of stock prices. Here we will simulate a one-dimensional symmetric random walk on the integers, which can be
described as follows.
\begin{enumerate}
  \item Start at 0.
  \item Flip a fair coin.
  \item If heads, move one unit to the right. Otherwise, move one unit to the left.
  \item Go to Step 2.
\end{enumerate}
How can we simulate this random walk efficiently? Note that the walk is really characterized by the outcomes
of the coin flip. If we represent heads by the number $1$ and tails by $-1$, then our position at a given moment
is just the cumulative sum of all previous outcomes. Below, we simulate a sequence of coin flips, build the
resulting random walk, and plot the outcome.
\begin{lstlisting}
>>> import matplotlib.pyplot as plt
>>> N = 1000        # length of random walk
>>> s = np.zeros(N)
>>> s[1:] = np.random.binomial(1, .5, size=(N-1,))*2-1 #coin flips
>>> s = pd.Series(s)
>>> s = s.cumsum()  # random walk
>>> s.plot()
>>> plt.ylim([-50, 50])
>>> plt.show()
\end{lstlisting}

The random walk is shown in Figure \ref{fig:PandasRandomWalk}.

\begin{figure}
\centering
\includegraphics[width=.7 \textwidth]{randomWalk.pdf}
\caption{Random walk of length 1000.}
\label{fig:PandasRandomWalk}
\end{figure}

\begin{problem}
Create five random walks of length 100, and plot them together.

Next, create a ``biased'' random walk by changing the coin flip probability of head from 0.5 to 0.51.
Plot this biased walk with lengths 100, 10000, and then 100000. Notice the definite trend that emerges.
Your results should be comparable to those in Figure \ref{pandas:biasedRandomWalk}.
\end{problem}

\begin{figure}
\centering
\includegraphics[width=.7 \textwidth]{biasedRandomWalk.pdf}
\caption{Biased random walk of length 100 (above) and 10000 (below).}
\label{pandas:biasedRandomWalk}
\end{figure}

Using \li{DataFrames}, one can also plot one column against another.

\begin{lstlisting}
>>> xvals = pd.Series(np.sqrt(np.arange(1000)))
>>> yvals = pd.Series(np.random.randn(1000).cumsum())
>>> df = pd.DataFrame({'xvals': xvals, 'yvals': yvals})
>>> df.plot(x='xvals', y='yvals') # specify x and y values
>>> plt.show()
\end{lstlisting}

The result is displayed in Figure \ref{pandas:dfPlot}.

\begin{figure}
\centering
\includegraphics[width=.7 \textwidth]{dfPlot.pdf}
\caption{ Graph generated when one coordinate is taken from the xvals column and the other from the yvals column.}
\label{pandas:dfPlot}
\end{figure}

A variety of other types of plots are possible. One of the more useful plots when trying to estimate or
visualize the distribution of data is a histogram. The code listed below demonstrates how to generate
a histogram for each column in a \li{DataFrame}, with the result shown in Figure \ref{fig:PandasHistogram}.

\begin{lstlisting}
>>> col1 = pd.Series(np.random.randn(1000))         #normal distribution
>>> col2 = pd.Series(np.random.gamma(5, size=1000)) #gamma distribution
>>> df = pd.DataFrame({'normal': col1, 'gamma': col2})
>>> df.hist()
>>> plt.plot()
\end{lstlisting}

\begin{figure}
\centering
\includegraphics[width=.7 \textwidth]{histogram.pdf}
\caption{Histogram of two columns of a DataFrame.}
\label{fig:PandasHistogram}
\end{figure}

\subsection*{SQL Operations in pandas}
The \li{DataFrame}, being a tabular data structure, bears an obvious resemblance to a typical relational
database table. SQL is the standard for working with relational databases, and in this section we will
explore how pandas accomplishes some of the same tasks as SQL. The SQL-like functionality of pandas is
one of its biggest advantages, since it can eliminate the need to switch between programming languages
for different tasks. Within pandas we can handle both the querying \emph{and} data analysis.

For the following examples, we will use this data:
\begin{lstlisting}
>>> #build toy data for SQL operations
>>> name = ['Bill', 'Alice', 'Joe', 'Jenny', 'Ted', 'Taylor', 'Tracy', 'Morgan', 'Liz']
>>> sex = ['M', 'F', 'M', 'F', 'M', 'F', 'M', 'M', 'F']
>>> age = [20, 21, 18, 22, 19, 20, 20, 19, 20]
>>> rank = ['Sp', 'Se', 'Fr', 'Se', 'Sp', 'J', 'J', 'J', 'Se']
>>> ID = range(9)
>>> aid = ['y', 'n', 'n', 'y', 'n', 'n', 'n', 'y', 'n']
>>> GPA = [3.8, 3.5, 3.0, 3.9, 2.8, 2.9, 3.8, 3.4, 3.7]
>>> mathID = [0, 1, 5, 6, 3]
>>> mathGd = [4.0, 3.0, 3.5, 3.0, 4.0]
>>> major = ['y', 'n', 'y', 'n', 'n']
>>> studentInfo = pd.DataFrame({'ID': ID, 'Name': name, 'Sex': sex, 'Age': age, 'Class': rank})
>>> otherInfo = pd.DataFrame({'ID': ID, 'GPA': GPA, 'Financial_Aid': aid})
>>> mathInfo = pd.DataFrame({'ID': mathID, 'Grade': mathGd, 'Math_Major': major})
\end{lstlisting}

Before querying our data, it is important to know some of its basic properties, such as
number of columns, number of rows, and the datatypes of the columns. This can be done
by simply calling the \li{info} method on the desired \li{DataFrame}:
\begin{lstlisting}
>>> mathInfo.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 5 entries, 0 to 4
Data columns (total 3 columns):
Grade         5 non-null float64
ID            5 non-null int64
Math_Major    5 non-null object
dtypes: float64(1), int64(1), object(1)
\end{lstlisting}

Now let's look at the pandas equivalent of some SQL SELECT statements.
\begin{lstlisting}
>>> # SELECT ID, Age FROM studentInfo
>>> studentInfo[['ID', 'Age']]

>>> # SELECT ID, GPA FROM otherInfo WHERE Financial_Aid = 'y'
>>> otherInfo[otherInfo['Financial_Aid']=='y'][['ID', 'GPA']]

>>> # SELECT Math_Major, COUNT(*) FROM mathInfo GROUP BY Math_Major
>>> print mathInfo.groupby('Math_Major').size()
\end{lstlisting}

\begin{problem}
The example above shows how to implement a simple WHERE condition, and it is easy
to have a more complex expression. Simply enclose each simple condition by parentheses,
and use the standard boolean operators \li{\&} (AND), \li{\|} (OR), and \li{\~} (NOT) to
connect the conditions appropriately. Use pandas to execute the following query:
\begin{lstlisting}[language=SQL]
SELECT ID, Name from studentInfo WHERE Age > 19 AND Sex = 'M'
\end{lstlisting}
\end{problem}

Next, let's look at JOIN statements. In pandas, this is done with the \li{merge} function,
which takes as arguments the two \li{DataFrame} objects to join, as well as keyword arguments specifying
the column on which to join, along with the type (left, right, inner, outer).

\begin{lstlisting}
>>> # SELECT * FROM studentInfo INNER JOIN mathInfo ON studentInfo.ID = mathInfo.ID
>>> pd.merge(studentInfo, mathInfo, on='ID') # INNER JOIN is the default
   Age Class  ID    Name Sex  Grade Math_Major
0   20    Sp   0    Bill   M    4.0          y
1   21    Se   1   Alice   F    3.0          n
2   22    Se   3   Jenny   F    4.0          n
3   20     J   5  Taylor   F    3.5          y
4   20     J   6   Tracy   M    3.0          n
[5 rows x 7 columns]

>>> # SELECT GPA, Grade FROM otherInfo FULL OUTER JOIN mathInfo on otherInfo.ID = mathInfo.ID
>>> pd.merge(otherInfo, mathInfo, on='ID', how='outer')[['GPA', 'Grade']]
   GPA  Grade
0  3.8    4.0
1  3.5    3.0
2  3.0    NaN
3  3.9    4.0
4  2.8    NaN
5  2.9    3.5
6  3.8    3.0
7  3.4    NaN
8  3.7    NaN
[9 rows x 2 columns]
\end{lstlisting}

\begin{problem}
Using a join operation, create a \li{DataFrame} containing the ID, age, and GPA of all male individuals.
You ought to be able to accomplish this in one line of code.
\end{problem}

\begin{comment}
This may be going into too much detail for an introductory lab...
It is sometimes desirable to join to DataFrames on their indexes rather than on a particular column.
The \li{join} method makes this operation convenient.
\begin{lstlisting}
>>> #create new DataFrame
>>> sibs = [0, 1, 0, 5, 2, 9]
>>> sibInfo = pd.DataFrame({'Siblings':sibs})
>>> sibInfo
   Siblings
0         0
1         1
2         0
3         5
4         2
5         9
[6 rows x 1 columns]

>>> #now join studentInfo with sibInfo
>>> studentInfo.join(sibInfo)
   Age Class  ID    Name Sex  Siblings
0   20    Sp   0    Bill   M         0
1   21    Se   1   Alice   F         1
2   18    Fr   2     Joe   M         0
3   22    Se   3   Jenny   F         5
4   19    Sp   4     Ted   M         2
5   20     J   5  Taylor   F         9
6   20     J   6   Tracy   M       NaN
7   19     J   7  Morgan   M       NaN
8   20    Se   8     Liz   F       NaN
[9 rows x 6 columns]
\end{lstlisting}
The default is a left join, but this can be altered.
When attempting to join to DataFrames that share common columns on their indexes,
we must rename the columns to avoid contradictions. We do this by simply appending
characters to the existing column names, specified by the \li{lsuffix} and \li{rsuffix}
parameters (for the left and right DataFrames, respectively).
\begin{lstlisting}
>>> #join studentInfo and mathInfo on indexes
>>> studentInfo.join(mathInfo, lsuffix='_left', rsuffix='_right', how='inner')
   Age Class  ID_left   Name Sex  Grade  ID_right Math_Major
0   20    Sp        0   Bill   M    4.0         0          y
1   21    Se        1  Alice   F    3.0         1          n
2   18    Fr        2    Joe   M    3.5         5          y
3   22    Se        3  Jenny   F    3.0         6          n
4   19    Sp        4    Ted   M    4.0         3          n
[5 rows x 8 columns]
\end{lstlisting}

The final SQL-like operation we will discuss is the UNION, or concatenation of DataFrames.
In the simplest setting, this operation is useful when we have two DataFrames that have the
same columns but possibly different rows.
\begin{lstlisting}
>>> #create another df holding more math info
>>> mathID2 = [0, 5, 2, 4]
>>> mathGd2 = [4.0, 3.5, 2.0, 3.0]
>>> major2 = ['y', 'y', 'n', 'y']
>>> mathInfo2 = pd.DataFrame({'Grade':mathGd2, 'ID':mathID2, 'Math_Major':major2})
>>> pd.concat([mathInfo, mathInfo2], ignore_index=True).drop_duplicates()
   Grade  ID Math_Major
0    4.0   0          y
1    3.0   1          n
2    3.5   5          y
3    3.0   6          n
4    4.0   3          n
7    2.0   2          n
8    3.0   4          y
[7 rows x 3 columns]
\end{lstlisting}
The \li{ignore_index=True} argument means we discard the indexes of the two input DataFrames and create
a new one for the concatenated DataFrame. The \li{drop_duplicates} method simply drops duplicate rows.
\end{comment}

Be aware that other types of SQL-like operations are also possible, such as UNION.
When you find yourself unsure of how to carry out a more involved SQL-like operation, the online
pandas documentation will be of great service.

\section*{Analyzing Data}
Although pandas does not provide built-in support for heavy-duty statistical analysis of data, there
are nevertheless many features and functions that facilitate basic data manipulation and computation,
even when the data is in a somewhat messy state. We will now explore some of these features.

\subsection*{Basic Data Manipulation}
Because the primary pandas data structures are subclasses of the \li{ndarray}, they are valid input
to most NumPy functions, and can often be treated simply as NumPy arrays. For example, basic
vectorized operations work just fine:
\begin{lstlisting}
>>> x = pd.Series(np.random.randn(4), index=['a', 'b', 'c', 'd'])
>>> y = pd.Series(np.random.randn(5), index=['a', 'b', 'd', 'e', 'f'])
>>> x**2
a    1.710289
b    0.157482
c    0.540136
d    0.202580
dtype: float64
>>> z = x + y
>>> z
a    0.123877
b    0.278435
c         NaN
d   -1.318713
e         NaN
f         NaN
dtype: float64
>>> np.log(z)
a   -2.088469
b   -1.278570
c         NaN
d         NaN
e         NaN
f         NaN
dtype: float64
\end{lstlisting}
Notice that pandas automatically aligns the indexes when adding two \li{Series} (or \li{DataFrames}),
so that the the index of the output is simply the union of the indexes of the two inputs. The default
missing value \li{NaN} is given for labels that are not shared by both inputs.

It may also be useful to transpose \li{DataFrames}, re-order the columns or rows, or sort according to a
given column. Here we demonstrate these capabilities:
\begin{lstlisting}
>>> df = pd.DataFrame(np.random.randn(4,2), index=['a', 'b', 'c', 'd'], columns=['I', 'II'])
>>> df
          I        II
a -0.154878 -1.097156
b -0.948226  0.585780
c  0.433197 -0.493048
d -0.168612  0.999194

[4 rows x 2 columns]

>>> df.transpose()
           a         b         c         d
I  -0.154878 -0.948226  0.433197 -0.168612
II -1.097156  0.585780 -0.493048  0.999194

[2 rows x 4 columns]

>>> # switch order of columns, keep only rows 'a' and 'c'
>>> df.reindex(index=['a', 'c'], columns=['II', 'I'])
         II         I
a -1.097156 -0.154878
c -0.493048  0.433197

[2 rows x 2 columns]

>>> # sort descending according to column 'II'
>>> df.sort(columns='II', ascending=False)
          I        II
d -0.168612  0.999194
b -0.948226  0.585780
c  0.433197 -0.493048
a -0.154878 -1.097156

[4 rows x 2 columns]
\end{lstlisting}

\subsection*{Basic Statistical Functions}
The pandas library allows us to easily calculate basic summary statistics of our data,
useful when we want a quick description of the data. The \li{describe} function
outputs several such summary statistics for each column in a \li{DataFrame}:
\begin{lstlisting}
>>> df.describe()
              I        II
count  4.000000  4.000000
mean  -0.209630 -0.001308
std    0.566696  0.964083
min   -0.948226 -1.097156
25%   -0.363516 -0.644075
50%   -0.161745  0.046366
75%   -0.007859  0.689133
max    0.433197  0.999194

[8 rows x 2 columns]
\end{lstlisting}

Functions for calculating means and variances, the covariance and correlation matrices, and other
basic statistics are also available. Below, we calculate the means of each row, as well as the
covariance matrix:
\begin{lstlisting}
>>> df.mean(axis=1)
a   -0.626017
b   -0.181223
c   -0.029925
d    0.415291
dtype: float64

>>> df.cov()
           I        II
I   0.321144 -0.256229
II -0.256229  0.929456

[2 rows x 2 columns]
\end{lstlisting}

\subsection*{Dealing with Missing Data}
Missing data is a ubiquitous problem in data science. Fortunately, pandas is particularly well-suited to
handling missing and anomalous data. As we have already seen, the pandas default for a missing value is \li{NaN}.
In basic arithmetic operations, if one of the operands is \li{NaN}, then the output is also \li{NaN}.
The following example illustrates this concept:
\begin{lstlisting}
>>> x = pd.Series(np.arange(5))
>>> y = pd.Series(np.random.randn(5))
>>> x.iloc[3] = np.nan
>>> x + y
0    0.731521
1    0.623651
2    2.396344
3         NaN
4    3.351182
dtype: float64
\end{lstlisting}
If we are not interested in the missing values, we can simply drop them from the data altogether:
\begin{lstlisting}
>>> (x + y).dropna()
0    0.731521
1    0.623651
2    2.396344
4    3.351182
dtype: float64
\end{lstlisting}

This is not always the desired behavior, however. It may well be the case that missing data actually corresponds to
some default value, such as zero. In this case, we can replace all instances of \li{NaN} with a specified value:
\begin{lstlisting}
>>> # fill missing data with 0, add
>>> x.fillna(0) + y
0    0.731521
1    0.623651
2    2.396344
3    1.829400
4    3.351182
dtype: float64
\end{lstlisting}

Other functions, such as \li{sum()} and \li{mean()} treat NaN as zero by default.
When dealing with missing data, make sure you are aware of the behavior of the pandas
functions you are using.

\begin{problem}
Using the dataset contained in the file \li{crime_data.txt} and the techniques learned in this lab,
use pandas to complete the following.
\begin{itemize}
\item Load the data into a pandas \li{DataFrame}, using the column names in the file and the column titled
``Year'' as the index. Make sure to skip lines that don't contain data.

\item Insert a new column into the data frame that contains the crime rate by year (the ratio of ``Total'' column
to the ``Population'' column).

\item Plot the crime rate as a function of the year.

\item List the 5 years with the highest crime rate in descending order.

\item Calculate the average number of total crimes as well as burglary crimes between 1960 and 2012.

\item Find the years for which the total number of crimes was below average, but the number of burglaries
was above average.

\item Plot the number of murders as a function of the population.

\item Select the Population, Violent, and Robbery columns for all years in the 1980s, and save
this smaller data frame to a CSV file \li{crime_subset.txt}.
\end{itemize}
\end{problem}


\documentclass{article}
\title{ALP3 U14}
\author{Übungsgruppe: Qianli Wang und Nazar Sopiha}
\date{\today}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{listings}
\usepackage{color}
\usepackage{graphicx}
\usepackage{booktabs}
\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}

\lstset{ %
	language=Octave,                % the language of the code
	basicstyle=\footnotesize,           % the size of the fonts that are used for the code
	numbers=left,                   % where to put the line-numbers
	numberstyle=\tiny\color{gray},  % the style that is used for the line-numbers
	stepnumber=2,                   % the step between two line-numbers. If it's 1, each line
	% will be numbered
	numbersep=5pt,                  % how far the line-numbers are from the code
	backgroundcolor=\color{white},      % choose the background color. You must add \usepackage{color}
	showspaces=false,               % show spaces adding particular underscores
	showstringspaces=false,         % underline spaces within strings
	showtabs=false,                 % show tabs within strings adding particular underscores
	frame=single,                   % adds a frame around the code
	rulecolor=\color{black},        % if not set, the frame-color may be changed on line-breaks within not-black text (e.g. commens (green here))
	tabsize=2,                      % sets default tabsize to 2 spaces
	captionpos=b,                   % sets the caption-position to bottom
	breaklines=true,                % sets automatic line breaking
	breakatwhitespace=false,        % sets if automatic breaks should only happen at whitespace
	title=\lstname,                   % show the filename of files included with \lstinputlisting;
	% also try caption instead of title
	keywordstyle=\color{blue},          % keyword style
	commentstyle=\color{dkgreen},       % comment style
	stringstyle=\color{mauve},         % string literal style
	escapeinside={\%*}{*)},            % if you want to add LaTeX within your code
	morekeywords={*,...}               % if you want to add more keywords to the set
}

\begin{document}
\maketitle

\section{Aufgabe 92: Adjazenzlisten}
\textbf{Inspirieren durch Radixsort:}\\
Dieser Algorithmus ist schon im Vergleich zum Radixsort vereinfacht, weil in allen Adjazenzlisten jede Kante gleiches i hat.\\
Wir können dann folgendes machen:\\
1. Verteile jedes j in jeder Adjazenzliste in 10 Fächern nach der letzten Ziffer  \\
2. Füge Fach 0, 1, ..., 9 zu einer Liste zusammen.\\
3. Iterieren m mal, wobei m die Anzahl von Bits von Maximum in der Adjazenzliste ist.

\section{Aufgabe 93: Optimaler binärer Suchbaum}
\textbf{Vom Algorthmus erzeugte Tabelle:}\\
\includegraphics[scale=0.4]{test.png}\\
Die Zwischenwerte im Algorithmus, also ${S_i}_j$ in der Tabelle bedeuten das Kosten des optimalen Suchbaums für Schlüssel im Intervall $x_i < x < x_j$. \\
Die mittlere Suchzeit: $\frac{37}{20}$\\
\\
\\
\\
\\
\\
\\
\\
\\
\textbf{Optimaler binärer Suchbaum:}\\
\includegraphics[scale=0.6]{test1.png}\\

\section{Aufgabe 97: Zahlen mit Münzen}
\textbf{Wir nehmen an, dass man möglich weinige Münze bekommen sollte.} \\
\textbf{Teilprobleme:} $A_k$ bedeutet die Liste von Münzen mit dem zu zahlenden Betrag k. \\
\textbf{Randbedingung:} $A_{M_i} = [M_i]$\\
\begin{lstlisting}
coins= [1,2,5,10]

changeMoney(nums):
	dp = [inf] * nums
	dp[0] = 0

	for i = 1 to (nums - 1):
		for j = 0 to length(coins):
			if(coins[j] <= nums):
				dp[i] = min(dp[i], dp[i - coins[j]] + 1)
			endif
		endfor
	endfor
	return dp[nums - 1]

\end{lstlisting}
Python Code sehen Sie unter: $https://github.com/qiaw99/WS2019-20/blob/$\\$master/DataStructure/U14/U14\_97.py$
\end{document}


%
% Chapter 3.6
%

\section*{3.6 Graphing with Calculators}

Read chapter 3.6 on page 251 in James Stewart Calculus Eighth Edition.


\section*{The \sixhundredsection Section}

You are the students taking this course as part of a masters degree in mathematics. Thus, there will be extra homework problems for you and you will be graded on a separate curve.


\section{Conclusions and future work}

We have presented a general technique for processing elements of a
list from the last element to the first.  The implementation-specific
version of our technique is comparable in speed to traversing the list
from the first to the last element for all reasonably-sized lists.
For very long lists, the performance of our technique degrades
modestly and gracefully.

Even the implementation-independent version of our technique performs
well enough that it is preferable to the existing technique of
reversing the list that is used in some implementations.

We have presented our technique in the context of the function
\texttt{count} because, together with \texttt{reduce}, processing the
elements from the end is required by the \hs{}, whereas, for other
func\-tions accep\-ting the key\-word ar\-gument \\
\texttt{from-end},
it is ex\-plicitly al\-lowed to process the elements from the begin\-ning to the
end.

However, our technique is potentially even more interesting to use
with functions such as \texttt{find} and \texttt{position} for which
it is not required to process the elements from the end to the
beginning.  The reason is that when elements are processed form the
beginning to the end in these functions, \emph{all} elements must be
tested.  When the combination of the \texttt{test} and the
\texttt{key} functions has non-trivial computational cost, a
significant amount of work may be wasted.  However, when elements are
processed in reverse order, a result can be returned when the test is
satisfied the first time, thereby avoiding such wasted work.

Since the performance of our technique is not significant\-ly wor\-se than
processing from the beginning to the end, it is very likely that our
technique will be faster in almost all cases.  Only when the last
element of the list to satisfy the test is close to the beginning of
the list will our technique apply the test as many times as when
processing is done from the beginning to the end.  We conjecture that,
on the average, our technique will be faster whenever the cost of
applying the test is at least that of a function call.  If so, our
technique should be used in all cases except for those using a very
inexpensive test functions such as \texttt{eq}, and when the
implementation then uses a special version of the sequence function
where this test is inlined, so as to avoid a function call.

Further research is required in order to verify our conjecture.  In
order to determine the result with some accuracy, additional
parameters have to be taken into account.  In particular, the position
of the last element in the list to satisfy the test must be taken into
account, as well as the cost of calling the test function.  As usual,
benchmarks will have to be performed on a variety of implementations
and processors, further complicating the verification of our
conjecture.


\FloatBarrier
\chapter{Alternative magnetisation re-normalisation methods}
\label{sec:magn-renorm-meth}
\chaptermark{Alternative re-normalisation}

In this \thisref{sec:magn-renorm-meth} we experimentally compare three methods for re-normalising the magnetisation:
\begin{enumerate}
\item Re-normalisation after every step, the simplest magnetisation re-normalisation method.
\item The self-correcting LLG method used by \nmag.
\item Tolerance-based re-normalisation used by \magpar.
\end{enumerate}
These three methods are described in \cref{sec:ensuring-constant-mv}.
As our example problem we use the relaxing nano-sphere described in \cref{sec:imr-ode-llg-numer-exper}.

\section{Implementation details}

As in \cref{sec:impl-deta-ode-llg} we use the Landau-Lifshitz (LL) form of the LLG.
The Newton-Raphson method is used for linearisation with the Jacobian calculated analytically and solved using a direct solver.
The Newton-Raphson tolerance set to $\ntol = 10^{-8}$.
We use BDF2 for these experiments because we need a time integrator with no geometric integration properties in order to properly test the re-normalisation methods (TR is equivalent to IMR for the undamped ODE LLG problem so it has the same geometric integration properties in this case, see \cref{sec:aimr-llgode-numerical-results}).
The time step size is fixed at $\dtn = 0.1$.

Tolerance-based re-normalisation is implemented as follows:
if the error in the magnetisation length, $\errml = \abs{1 - \abs{\mv}}$, is less than the tolerance, $\mltol$, then nothing is done and the integration continues.
On the other hand if $\errml > \mltol$ then we set $\mv_{i}$ to $\mv_{i}/\abs{\mv_{i}}$ for the values of the magnetisation at all times stored by the time integrator (for BDF2 this corresponds to $i=n+1, n, n-1$).
Note that re-normalising the history values of TR is more complex because one of them is a derivative (at least in our implementation, see \cref{eq:impl-tr}), which would need to be recalculated using the newly re-normalised magnetisation values for consistency.

We use tolerance values of $\mltol = 0, 10^{-6}, 10^{-2}, 10^{200}$.
Note that $\mltol \gg 1$ (\eg $\mltol = 10^{200}$) corresponds to no re-normalisation, while $\mltol = 0$ corresponds to re-normalisation after every time step.
The default value used in \magpar\footnote{Based on the source code of version 0.9, the relevant function is \texttt{CheckIterationLL\_Init}.} is $\mltol = 10^{-2}$.

The self correcting LLG is implemented by replacing the standard residual, $\rv_{\mathrm{ll}}$ given in \cref{eq:r-ode-llg}, with the modified residual
\begin{equation}
  \rv = \rv_{\mathrm{ll}}  - \scc \mv \bigb{1 - \abs{\mv}^2},
\end{equation}
where $\scc$ is a parameter that can be tuned.
The Jacobian of the modified residual is given by
\begin{equation}
  \label{eq:J-ode-sc-llg}
  \Jm = \Jm_{\mathrm{ll}} + 2\scc \bigb{\mv \tensorprod \mv} - \scc \bigb{1 - \abs{\mv}^2} \Idm ,
\end{equation}
where $\Jm_{\mathrm{ll}}$ is the Jacobian of $\rv_{\mathrm{ll}}$ and is given in \cref{eq:J-ode-llg}.

We experiment with a range of parameter values: $\scc = 0, 0.1, 1, 10, 100, 1000$.
Larger values were not used because the Newton-Raphson method often failed to converge within ten steps for $\scc \gtrsim 1000$.

Note that, due to implementation details, values are output \emph{after} any re-normalisation for that step has been performed.

\section{Results: tolerance-based renormalisation}
\label{sec:renorm-after-toler}

\begin{figure}
  \centering
  \includegraphics[width=0.8\textwidth]{{{plots//tolrenorm-geom-properties/0-mlengtherrormaxesvstimes}}}
  \caption{
    Error in magnetisation length, $\errml$, over time
    for the relaxing nano-sphere problem with
    $\dampc = 0$.
    Magnetisation length is enforced by tolerance-based renormalisation,
    the legend indicates the value of the tolerance $\mltol$.
  }
  \label{fig:renorm-tol-ml-err-undamped}
\end{figure}

\begin{figure}
  \centering
  \includegraphics[width=0.8\textwidth]{{{plots/tolrenorm-geom-properties/0.01-mlengtherrormaxesvstimes}}}
  \caption{
    Error in magnetisation length, $\errml$, over time
    for the relaxing nano-sphere problem
    with $\dampc = 0.01$.
    Magnetisation length is enforced by tolerance-based renormalisation,
    the legend indicates the value of the tolerance $\mltol$.
  }
  \label{fig:renorm-tol-ml-err}
\end{figure}

In \cref{fig:renorm-tol-ml-err-undamped,fig:renorm-tol-ml-err} we show the error in the magnetisation length over time
\begin{equation}
  \errml = \abs{\abs{\mv} - 1}
\end{equation}
for various values of $\mltol$ and $\dampc=0.01, 0$ respectively.
For the case with $\dampc = 0.01$ and $\mltol = 10^{-6}$ re-normalisation is performed with increasing frequency until $t \gtrsim 280$, after which it is performed after almost every step.
In both the damped and undamped cases, the use of the intermediate tolerance value, $\mltol = 10^{-6}$, results in oscillations of the magnetisation length.
For $\mltol = 10^{-2}$ in the undamped case the error in the magnetisation length never reaches the tolerance and the behaviour is identical to the case of no re-normalisation.
In the damped case the tolerance $\mltol = 10^{-2}$ is reached and the behaviour is similar to that of $\mltol = 10^{-6}$, except that the oscillations in the magnetisation length begin at a later time.

\begin{figure}
  \centering
  \includegraphics[width=0.8\textwidth]{plots/tolrenorm-geom-properties/0-absofenergychangevstimes.pdf}
  \caption{
    Error in energy over time
    for the relaxing nano-sphere problem
    with $\dampc = 0$.
    Magnetisation length is enforced by tolerance-based renormalisation,
    the legend indicates the value of the tolerance $\mltol$.
  }
  \label{fig:renorm-tol-energy-err}
\end{figure}

In \cref{fig:renorm-tol-energy-err} the errors in the energy (see \cref{sec:aimr-llg-problem-definition}) for the undamped case are shown.
As noted previously, $\mltol= 10^{-2}$ behaves as the non-renormalised case for this example.
When $\mltol = 10^{-6}$ we see an error similar to that for the always re-normalised case, except with additional oscillations in the energy corresponding to times where a re-normalisation is carried out.

\begin{figure}
  \centering
  \includegraphics[width=0.8\textwidth]{plots/tolrenorm_llg_ode_convergence/maxofswitchingtimeerrorvsmeanofdts}
  \caption{
    Convergence of the maximum (over all steps) of the error in the switching time
    against step size
    for the relaxing nano-sphere problem with
    $\dampc = 0.01$.
    Magnetisation length is enforced by tolerance-based renormalisation,
    the legend indicates the value of the tolerance $\mltol$.
  }
  \label{fig:tol-renorm-convergence}
\end{figure}

Finally in \cref{fig:tol-renorm-convergence} we show the convergence of the method (in terms of the error in the switching time, \cref{eq:sw-time-error}) as the step size is reduced for the damped case.
The use of $\mltol=10^{-2}$ or no re-normalisation ($\mltol=10^{200}$) gives slightly worse errors than the tighter tolerances.

\FloatBarrier
\section{Results: the self-correcting LLG}
\label{sec:self-correcting-llg-results}

\begin{figure}
  \centering
  \includegraphics[width=0.8\textwidth]{plots/sc-geom-properties/0-mlengtherrormaxesvstimes.pdf}
  \caption{
    Error in magnetisation length, $\errml$, over time
    for the relaxing nano-sphere problem
    with $\dampc = 0$.
    Magnetisation length is enforced by use of the self-correcting LLG.
   The legend indicates the values of $\scc$.
  }
  \label{fig:sc-ml-err-undamped}
\end{figure}

\begin{figure}
  \centering
  \includegraphics[width=0.8\textwidth]{{{plots/sc-geom-properties/0.01-mlengtherrormaxesvstimes}}}
  \caption{
    Error in magnetisation length, $\errml$, over time
    for the relaxing nano-sphere problem
    with $\dampc = 0.01$.
    Magnetisation length is enforced by use of the self-correcting LLG,
    the legend indicates the values of $\scc$.
  }
  \label{fig:sc-ml-err}
\end{figure}

In \cref{fig:sc-ml-err-undamped,fig:sc-ml-err} we show the errors in magnetisation length over time with various values of the parameter $\scc$ for the damped and undamped problems respectively.
As would be expected larger values of $\scc$ reduce the error in both cases.
Note that in the damped case (\cref{fig:sc-ml-err}) with $\scc = 1000$ the curve stops at $t \sim 480$, this is due to non-convergence of the Newton-Raphson method within the ten iteration limit.

\begin{figure}
  \centering
  \includegraphics[width=0.8\textwidth]
  {{{plots/sc-geom-properties/0-absofenergychangevstimes}}}
  \caption{
    Error in energy over time
    for the relaxing nano-sphere problem
    with $\dampc = 0$.
    Magnetisation length is enforced by use of the self-correcting LLG,
    the legend indicates the values of $\scc$.
  }
  \label{fig:sc-energy-err}
\end{figure}

In \cref{fig:sc-energy-err} we show the error in the energy for the undamped problem for various values of the parameter $\scc$.
We see that any $\scc > 0$ causes an error in the energy similar to that caused by the various re-normalisation approaches shown in \cref{fig:renorm-tol-energy-err}.

\begin{figure}
  \centering
  \includegraphics[width=0.8\textwidth]
  {{{plots/self_correcting_llg_ode_convergence/maxofswitchingtimeerrorvsmeanofdts}}}
  \caption{
    Convergence of the maximum (over all steps) of the error in the switching time
    against step size
    for the relaxing nano-sphere problem with
    $\dampc = 0.01$.
    Magnetisation length is enforced by use of the self-correcting LLG or re-normalisation after every step (\ie tolerance based renormalisation with $\mltol = 0$).
    The legend indicates, respectively, the values of $\scc$ and the tolerance $\mltol$.
  }
  \label{fig:sc-convergence}
\end{figure}

In \cref{fig:sc-convergence} we show the convergence of the maximum error in the switching time against the time step size $\dtn$ with two values of $\scc$.
For comparison we also show the results when re-normalisation after every step is used instead of the self correcting term (\ie $\mltol = 0$ and $\scc = 0$).
We see that the resulting errors are essentially identical for all three cases.

\begin{table}
  \begin{subtable}{.5\textwidth}
    \centering
    \begin{tabular}{ll}
      $\scc$ & Mean Newton iterations \\
      \hline
      0 & 2.0 \\
      0.1 & 2.0 \\
      1 & 2.46 \\
      10 & 2.68 \\
      100 & 3.13 \\
      1000 & 3.12* \\
    \end{tabular}%
    \caption{Solved with BDF2 and $\ntol = 10^{-8}$. Magnetisation length is enforced by use of the self-correcting LLG.}
  \end{subtable}%
  \begin{subtable}{.5\textwidth}
    \centering
    \begin{tabular}{ll}
      $\scc$ & Mean Newton iterations \\
      \hline
      0 & 2.0 \\
      0.1 & - \\
      1 & - \\
      10 & - \\
      100 & - \\
      1000 & - \\
    \end{tabular}%
    \vfill
    \caption{Solved with IMR and $\ntol = 10^{-12}$}
  \end{subtable}%
  \caption{
    Mean number of Newton iterations (over all time steps) for convergence
    for the relaxing nano-sphere problem with
    $\dampc = 0.01$.
    The time integration methods used are indicated in the sub-captions.
    Values of $\scc \neq 0$ are not considered when using IMR because no self-correcting term is needed to conserve $\abs{\mv}$.
    *With $\scc = 1000$ the Newton-Raphson method fails to converge at $t \sim 480$.
  }
  \label{tab:sc-newton-iters}
\end{table}

In \cref{tab:sc-newton-iters} we show the number of Newton-Raphson iterations required for convergence for each value of $\scc$.
For comparison we also show the result when using IMR with Newton tolerance $\ntol=10^{-12}$ (recall that a sharp linearisation tolerance is required for IMR to attain good geometric integration properties).
We see that as $\scc$ is increased the mean number of Newton iterations to converge also increases.
In addition we see that the tighter tolerance required for geometric integration with IMR has no effect on the number of iterations required.

Since $\scc \geq 100$ is required to control $\errml$ to even the fairly loose tolerance of $10^{-6}$, the use of the self correcting LLG requires, on average, at least one additional Newton-Raphson iteration per time step.
As the majority of the computation time for a time step is spent within the Newton-Raphson method, this increases the cost of each step by approximately $50\%$.
Since the use of the self correcting LLG does not provide any improvement in the overall accuracy or energy error over re-normalising after every step (so the same $\dtn$ is required to obtain the same accuracy), this increases the overall cost of the method by the same factor.

\begin{figure}
  \centering
  \includegraphics[width=\textwidth]{{{plots/sc-adaptivity/maxofswitchingtimeerrorvsmeanofdts}}}
  \caption{
    Convergence of the maximum (over all steps) of the error in the switching time
    against step size
    for the relaxing nano-sphere problem with
    $\dampc = 0.01$
    solved using adaptive BDF2.
    Magnetisation length is enforced by use of the self-correcting LLG or re-normalisation after every step (\ie tolerance based renormalisation with $\mltol = 0$).
    The legend indicates, respectively, the values of $\scc$ and the tolerance $\mltol$.
  }
  \label{fig:sc-adaptivity-convergence}
\end{figure}

Finally we study the effect of the use of the self-correcting LLG on adaptive time integration.
Since the self correcting term is part of the differential equation (as opposed to standard renormalisation, which is carried out separately) we might expect adaptive time integration methods to be able to better predict the error, and choose more appropriate step sizes.
In \cref{fig:sc-adaptivity-convergence} we show the convergence behaviour when using adaptive BDF2 (as described in \cref{sec:aimr-implementation}) with tolerances of $\toltt = 10^{-4}, 3\E{-5}, 10^{-5}, 3\E{-6}, 10^{-6}, 3\E{-7}, 10^{-7}, 10^{-8}$ with various values of $\scc$.
For comparison we also show the convergence behaviour when the magnetisation is re-normalised after every step.
If the use of the self-correcting LLG is beneficial we would expect to see lower errors for the same average step size, but the plots are indistinguishable.

\section{Conclusions and future work}

By design, the use of tolerance-based renormalisation allows larger errors to appear in $\abs{\mv}$ than re-normalisation after every step.
It also introduces spurious oscillations in $\abs{\mv}$ and in the error in the energy for the undamped case.
It has a slight negative effect on convergence when compared to re-normalisation after every step, and no effect on the computational cost of the re-normalisation.

The self correcting LLG is worse at controlling errors in $\abs{\mv}$ than re-normalisation after each step, and gives very similar errors in the energy.
It gives essentially identical convergence properties as re-normalisation after every step.
Finally, it requires additional step(s) of the Newton-Raphson method convergence (typically one more step is needed, increasing the computation time for the simulation by a factor of $\sim 50\%$).

Hence there appears to be no obvious reason to prefer either tolerance-based renormalisation or the self correcting LLG over re-normalisation after every step for ODE problems.
Since the methods used here match those used for PDE problems, except for the obvious lack of any spatial discretisation, this conclusion is likely to extend to PDE problems.

Ideally we would also show wall-clock timing results for CVODE or VODE (two widely used ODE integration packages) when using these re-normalisation methods, and compare these results with adaptive IMR.
However the VODE implementation used previously in this thesis uses a derivative function, $\fv$, written in python.
This has a huge performance penalty: for the relaxing nano-sphere problem VODE is 500x slower than oomph-lib, so any timing results would be meaningless.
Hopefully in the future adaptive IMR will be implemented in the CVODE/VODE framework and a fair comparison of the wall-clock timings can be made.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


\documentclass[twocolumn,11pt]{article}
\setlength{\textheight}{9truein}
\setlength{\topmargin}{-0.9truein}
\setlength{\parindent}{0pt}
\setlength{\parskip}{10pt}
\setlength{\columnsep}{.4in}

\usepackage{amsmath,amsfonts,amssymb,amsthm,bm,caption,calc,ifthen,graphicx,url,hyperref}

\begin{document}
\pagestyle{plain}
\onecolumn
PHYS689
\newline Homework 2
\newline Will Wainwright
\newline Repository: \href{https://github.com/wjwainwright/PHYS689}{https://github.com/wjwainwright/PHYS689}

\section*{Gaussian Noise}
For generating Gaussian noise, I used numpy's random.normal() function to generate the noise about a mean with a given standard deviation. I plotted the Gaussian overlay using scipy's normal PDF function given an initial array of evenly spaced x-values. The time series is a plot of the generated values as a function of their index, i.e. the order they were generated. I created a generic function for this process which takes the mean and standard deviation as variables, so changing these values between part 1 and part 2 of the assignment was trivial. I adjusted the number of bins for the normalized histogram manually until I found a number such that the histogram was not too blocky nor too `hairy'. I defined the bin count as a factor of the number of data points n, though n remains constant for this assignment.

\begin{figure}[!h]
	\centering
	\noindent
	\makebox[\textwidth]{
      \includegraphics[width=5in]{Gaussian_Timeseries_0_1.pdf}}
      \caption{Gaussian normal distributed time series with $\mu=0$ and $\sigma=1$.}
\end{figure}

\begin{figure}[!h]
	\centering
	\noindent
	\makebox[\textwidth]{
      \includegraphics[width=5in]{Gaussian_0_1.pdf}}
      \caption{Gaussian normal distribution histogram with $\mu=0$ and $\sigma=1$.}
\end{figure}

\begin{figure}[!h]
	\centering
	\noindent
	\makebox[\textwidth]{
      \includegraphics[width=5in]{{Gaussian_Timeseries_10.345_2.338}.pdf}}
      \caption{Gaussian normal distributed time series with $\mu=10.345$ and $\sigma=2.338$.}
\end{figure}

\begin{figure}[!h]
	\centering
	\noindent
	\makebox[\textwidth]{
      \includegraphics[width=5in]{{Gaussian_10.345_2.338}.pdf}}
      \caption{Gaussian normal distribution histogram with $\mu=10.345$ and $\sigma=2.338$.}
\end{figure}

\clearpage
\section*{Poisson Noise}
For generating the Poisson noise, I used numpy's random.poisson() function with a given mean. I again plotted a time series of the generated values as a function of index. I plotted a normalized histogram in the same way as before, but for Poisson distribution overlay, I took two different approaches. Depending on the resolution of x values that I defined the Poisson function with, I got either sharp spikes on integer values or a more gradual curve. I found that the sharp spikes were better for identifying the difference between the histogram columns and the expected values. The curve, while less accurate in regards to the value of the function, gives a better view of the expected shape of a continuous or integrated Poisson distribution. For that reason, I decided to keep both types of plots and labeled them `Sharp' and `Fuzzy' accordingly.

\begin{figure}[!h]
	\centering
	\noindent
	\makebox[\textwidth]{
      \includegraphics[width=5in]{Poisson_Timeseries_2.pdf}}
      \caption{Poisson distributed time series with $\mu=2$.}
\end{figure}

\begin{figure}[!h]
	\centering
	\noindent
	\makebox[\textwidth]{
      \includegraphics[width=5in]{Poisson_Sharp_2.pdf}}
      \caption{Poisson distribution histogram with $\mu=2$. The PDF peaks sharply as it is defined with a high resolution.}
\end{figure}

\begin{figure}[!h]
	\centering
	\noindent
	\makebox[\textwidth]{
      \includegraphics[width=5in]{Poisson_Fuzzy_2.pdf}}
      \caption{Poisson distribution histogram with $\mu=2$. The PDF peaks gradually as it is defined with a low resolution.}
\end{figure}

\begin{figure}[!h]
	\centering
	\noindent
	\makebox[\textwidth]{
      \includegraphics[width=5in]{{Poisson_Timeseries_3.45}.pdf}}
      \caption{Poisson distributed time series with $\mu=3.45$.}
\end{figure}

\begin{figure}[!h]
	\centering
	\noindent
	\makebox[\textwidth]{
      \includegraphics[width=5in]{{Poisson_Sharp_3.45}.pdf}}
      \caption{Poisson distribution histogram with $\mu=3.45$. The PDF peaks sharply as it is defined with a high resolution.}
\end{figure}

\begin{figure}[!h]
	\centering
	\noindent
	\makebox[\textwidth]{
      \includegraphics[width=5in]{{Poisson_Fuzzy_3.45}.pdf}}
      \caption{Poisson distribution histogram with $\mu=3.45$. The PDF peaks gradually as it is defined with a low resolution.}
\end{figure}

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{indentfirst}

\title{CS131 B1: HW 1}
\author{Duy Nguyen}
\date{13 September 2016}

\begin{document}

\maketitle

\section*{Question 1}
In the truth table for $P \land Q$, when P is true and Q is true, the result is true. However, this case should be false if it is a truth table for $P \oplus Q$.

\begin{table}[h]
\centering
\caption{Question 1b}
\begin{tabular}{cc|cc}
P & Q & $P \oplus Q$ & $(P \land \neg Q) \lor (Q \land \neg P)$ \\ \hline
T & T & F          & F                                    \\
T & F & T          & T                                    \\
F & T & T          & T                                    \\
F & F & F          & F
\end{tabular}

\end{table}

Therefore, $P \oplus Q \equiv (P \land \neg Q) \lor (Q \land \neg P)$.

\section*{Question 2}
\begin{table}[h]
\centering
\caption{2a. First De Morgan's law}
\begin{tabular}{cc|cc}
P & Q & $\neg (P \land Q)$ & $\neg P \lor \neg Q$ \\ \hline
T & T & F          & F                                    \\
T & F & T          & T                                    \\
F & T & T          & T                                    \\
F & F & T          & T
\end{tabular}

\end{table}

\begin{table}[h]
\centering
\caption{2b. First Domination Law}
\begin{tabular}{cc|cc}
P & T & $P \lor T$ & $T$ \\ \hline
T & T & T          & T                                    \\
F & T & T          & T
\end{tabular}
\end{table}

\begin{table}[h]
\centering
\caption{2c. First Negation Law}
\begin{tabular}{cc|cc}
P & $\neg P$& $P \lor \neg P$ & $T$ \\ \hline
T & F & T          & T                                    \\
F & T & T          & T
\end{tabular}
\end{table}

\begin{table}[h]
\centering
\caption{2d. First Absorption Law}
\begin{tabular}{cc|cc}
P & Q & $P \lor (P \land Q)$ & $P$ \\ \hline
T & T & T          & T                                    \\
T & F & T          & T                                    \\
F & T & F          & F                                    \\
F & F & F          & F
\end{tabular}
\end{table}
\section*{Question 3}
\begin{tabular}{ll}
$\neg (\neg P \land Q) \lor (P \land \neg R)$ & \\
$(P \lor \neg Q) \lor (P \land \neg R)$ & (De Morgan's law) \\
$P \lor (P \land \neg R) \lor  \neg Q$ & (Commutative) \\
$(P \lor (P \land \neg R)) \lor  \neg Q$ & (Associative) \\
$P \lor \neg Q$ & (Absoption) \\
\end{tabular}

Therefore, $\neg (\neg P \land Q) \lor (P \land \neg R) \equiv P \lor \neg Q$.

\section*{Question 4}
\subsection*{4a. I can program using Python or Java, but not C\#.}

I can program using Python is \textbf{P}

I can program using Java is \textbf{J}

I can program using C\# is \textbf{C}

Then our statement is $(P \lor J) \land \neg C$.

\subsection*{4b. The number of a leap year is divisible by 4 but not divisible by 100, unless it is divisible by 400.}

Leap year divisible by 4 is \textbf{A}

Leap year divisible by 100 is \textbf{B}

Leap year divisible by 400 is \textbf{C}

Then our statement is $A \land \neg B \land C$.

\subsection*{4c. If I study hard, then I get good grades.}

I study hard is \textbf{H}

I get good grade is \textbf{G}

Then our statement is $H \rightarrow G$.

\subsection*{4d. Pluto is called a planet if and only if the sky bodies located in Kuiper Belt are called planets.}

Pluto is called a planet is \textbf{P}

Sky bodies located in Kuiper Belt are called planets is \textbf{K}

Then our statement is $P \leftrightarrow K$.

\end{document}