\documentclass[../PHYS306Notes.tex]{subfiles}

\begin{document}
\section{Lecture 22}
\subsection{Lecture Notes - Spinning Top with Torque, Nutation, Intro to Hamiltonian Mechanics}
\subsubsection{Returning to a Question}
We determined the scalar product $\v{L}^T\v{L}$ in the body frame. How does this compare to the scalar product in the space frame?
\begin{s}
The scalar product should be the same under rotation (invaraince under rotation). Going from the body frame to the space frame is just a rotation, and rotation preserves the angle between and the magnitude of two vectors. Another way to think about it:
\[\v{L}^T\v{L} = \v{L}'^TR^TR\v{L}' = \v{L}'^T\v{L}'\]
\end{s}
How does the rotational kinetic energy in the body frame $T = \frac{1}{2}\bm{\omega}^T\II\bm{\omega}$ compare to the rotational kinetic energy in the space frame.
\begin{s}
As above, the kinetic energy in the body frame and space frames should be equivalent. Mathematically:
\[\frac{1}{2}\bm{\omega}^T\II\bm{\omega} = \frac{1}{2}\bm{\omega}^{T'}RR^T\II RR^T\bm{\omega}' = \frac{1}{2}\bm{\omega}^{T'}\II\bm{\omega}'\]
\end{s}

\subsubsection{Spinning top with gravity/Nutation}
Last time, we derived an equation of motion for $\theta$ of a spinning top, and solved the equations for the case where $\theta$ was held constant. From this, we obtained the precession of the spinning top under gravity. We now consider the more general case where $\theta$ varies with time (is not constant). We will consider explicitly $\theta(t)$, and find "nutation"; that is, nodding repeatedly. Let us consider the energy:
\[E = T + U = \frac{\lambda_1}{2}\dot{\theta}^2 + U_{eff}(\theta)\]
Where:
\[U_{eff}(\theta) = \frac{(p_\phi - p_\psi\cos\theta)^2}{2\lambda_1\sin^2\theta} + \frac{p_\psi^2}{2\lambda_3} + MgR\cos\theta\]
(This was obtained just by replacing terms of $\dot{\phi}$ and $\dot{\psi}$ with the generalized momenta). This looks like the two body problem reduced to a one body problem with effective potential. We first notice that $MgR\cos\theta$ is negative for $\theta > \frac{\pi}{2}$ and $\frac{(p_\phi - p_\psi\cos\theta)^2}{2\lambda_1\sin^2\theta} + \frac{p_\psi^2}{2\lambda_3}$ is positive. At $\theta = 0, \pi$ we have that $U_{eff} \rightarrow \infty$. Graphically:
\begin{center}
    \includegraphics[scale=0.5]{Lecture-22/l22-img1.png}
\end{center}
Now consider:
\begin{center}
    \includegraphics[scale=0.5]{Lecture-22/l22-img2.png}
\end{center}
If a symmetric top has total effective energy $E_1$, then what type of motion does this top undergo?
\begin{s}
The top undergoes constant precession about the vertical axis, at a constant $\theta$; we have not excited the one dimensional system, and $\theta$ must hence be constant (stay in the minimum)
\end{s}
If we now give it some energy:
\begin{center}
    \includegraphics[scale=0.5]{Lecture-22/l22-img3.png}
\end{center}
What is the motion?
\begin{s}
The top will now undergo precession combined with nutation, as $\theta$ oscillates about the minimum.
\end{s}
There are several qualitatively different scenarios that can occur. We have the equation:
\[\dot{\phi} = \frac{p_\phi - p_\psi\cos\theta}{\lambda_1\sin^2\theta}\]
Two possible scenarios are:
\begin{center}
    \includegraphics[scale=1]{Lecture-22/l22-img4.png}
\end{center}
The first scenario occurs where $p_\phi > p_\psi$ and $\dot{\phi} > 0$. The second scenario occurs where $p_\phi \approx p_\psi$, then we have $p_\phi < p_\psi\cos\theta$ and $p_\phi > p_\psi\cos\theta_2$ and hence we have that $\dot{\phi}$ changes direction (which is quite cool!) There is also a third case, where we get cusps:
\begin{center}
    \includegraphics[scale=0.8]{Lecture-22/l22-img5.png}
\end{center}
Where $p_\phi \approx p_\psi$ and we can have $p_\phi = p_\psi\cos\theta_1$ or $p_\phi = p_\psi\cos\theta_2$.

\subsubsection{Intro to Hamiltonian Mechanics - Motivation}
Why do we need another form of mechanics? Newton was nice and intuitive, and Lagrange is much easier if we have constraints in our system and want to solve problems without having to explicitly worry about the constraints. Why can't we just be happy with Lagrange and leave it as it is? Reason 1 is that its cool/beautiful. From a practical perspective, it doesn't give us much of an advantage over Lagrange, but it will allow us to give a richer understanding of dynamical systems and conserved quantities. There is also an advantage when we want to link this to the most important theory of 20th century physics, that is, quantum mechanics.

\subsubsection{What was the Hamiltonian, Again?}
As a reminder, we have the Lagrangian:
\[\LL = \LL(q_1, q_2, \ldots, q_n, \dot{q}_1, \dot{q}_2, \cdots, \dot{q}_n)\]
We also introduced the generalized, or canonical momentum:
\[p_i = \dpd{\LL}{\dot{q}_i}\]
We then defined the Hamiltonian:
\[\HH = \sum_{i=1}^mp_i\dot{q}_i - \LL\]
When the transformation from cartesian to generalized coordinates was natural (did not depend on time), then $\HH$ was just the total energy. For further applications in this course, we will generally assume this is the case. The Hamiltonian is a function:
\[\HH = \HH(q_1, q_2, \ldots, q_n, \dot{q}_1, \dot{q}_2, \cdots, \dot{q}_n, t)\]
The main difference is that we trade between the generalized velocities (Lagrangian) with the generalized momentum (Hamiltonian). How does this occur? We will see in a moment, but let us do a basic example first.

\subsubsection{The Harmonic Oscillator}
The Lagrangian for the Harmonic oscillator is:
\[\LL = \frac{m}{2}\dot{x}^2 - k\frac{x^2}{2}\]
So the generalized momentum is:
\[\dpd{\LL}{\dot{x}} = m\dot{x} = p\]
So calculating $\HH$, we have:
\[\HH = m\dot{x}^2 - \LL = \frac{m}{2}\dot{x}^2 + k\frac{x^2}{2} = \frac{p^2}{2m} + k\frac{x^2}{2} =  T + U\]
Calculating $\HH$ is pretty straightforward.

\subsubsection{The Hamiltonian as the Legendre Transform of L}
Consider a function $f(x,y)$. The total differential is then given by:
\[df = \dpd{f}{x}dx + \dpd{f}{y}dy  = udx + vdy\]
Now, we want to "trade" the differentials, $x \mapsto u$. We can do this via a function $g = f - ux$. Let's check:
\[dg = df - udx - xdu\]
But we've already calculated $df$, so:
\[df = udx + vdy - udx - xdu = vdy - xdu\]
So as advertised, $g$ is the Legendre transform of $f$, and it depends explictly on the variables $u$ and $y$:
\[g = g(u, y)\]
To recap, we look at the total differential, we want to trade one function against the other, we then construct a function such that the independent variables are swapped. This has connections to thermodynamics; we recall that the energy of a thermodynamic system is dependent on the natural variables of number of particles, volume, and entropy, that is $E = E(N, V, S)$. But we had other relevant quantities, such as the Hemholtz free energy; $F = E - TS = F(N, V, T)$. Hence, the varaibles $S$ and $T$ are conjugate to each other. In many cases, its more convenient to use $F$ as it is easier to control the temperature rather than the entropy. In classical mechanics, it might be more convenient to work with the momenta rather than velocities, and easier to work with $\HH$ rather than $\LL$.

\subsubsection{Hamilton Equations of Motion}
We need to see what kind of equations of motion come out of this formulation of classical mechanics. Let's write the hamiltonian as:
\[\HH = \sum_i p_i\dot{q}_i - \LL\]
Now, consider the total differential of $\HH$. We have:
\[d\HH = \sum_i \left(\dot{q}_i dp_i + p_i d\dot{q}_i\right) - \dpd{\LL}{q_i}dq_i - \dpd{\LL}{\dot{q}_i}d\dot{q}_i - \dpd{\LL}{t}dt\]
If we recall the definitions, the $\dpd{\LL}{\dot{q}_i}$ term is just $p_i$, and $\dpd{\LL}{q_i}$ is just $\dot{p}_i$ if we use the Euler-Lagrange equation. Hence the above becomes:
\[d\HH = \sum_i (-\dot{p}_idq_i + \dot{q}_idp_i) - \dpd{\LL}{t}dt\]
Hence we can read off the differential:
\[\boxed{\dpd{\HH}{p_i} = \dot{q}_i, \dpd{\HH}{q_i} = -\dot{p}_i}\]
And these are Hamilton's equations! We will look at these more closely on Friday.

\end{document}


\chapter{Arrays of Objects}

During the next three chapters, we will develop programs that work with playing cards and decks of cards.
Here is an outline of the road ahead:

\begin{itemize}

\item In this chapter, we define a \java{Card} class and write methods that work with cards and arrays of cards.

\item In Chapter~\ref{deck}, we define a \java{Deck} class that encapsulates an array of cards, and we write methods that operate on decks.

\item In Chapter~\ref{eights}, we introduce a way to define new classes that extend existing classes.
Then we use \java{Card} and \java{Deck} to implement the game {\it Crazy Eights}.

\end{itemize}

%Although we will see several versions of the same code, the main advantage of proceeding this way is that the explanations will be smoother.
%It may help you to create a {\tt Card.java} file and paste in the examples as we go.

%The code for this chapter is in the directory {\tt ch12} in the repository for this book.
%Instructions for downloading the repository are on page~\pageref{code}.
%Before proceeding, we recommend that you preview the {\tt Card.java} file.

\index{rank}
\index{suit}

%TODO: Add an image of a deck?

There are 52 cards in a standard deck.
Each card belongs to one of four suits and one of 13 ranks.
The suits are Clubs, Diamonds, Hearts, and Spades.
The ranks are Ace, 2, 3, 4, 5, 6, 7, 8, 9, 10, Jack, Queen, and King.

If you are unfamiliar with traditional playing cards, now would be a good time to get a deck or read through \url{https://en.wikipedia.org/wiki/Standard_52-card_deck}.

\section{Card Objects}

\index{Card}
\index{class!Card}

If we want to define a class to represent a playing card, it is pretty clear what the instance variables should be: \java{rank} and \java{suit}.
It is not as obvious what types they should be.

One possibility is a \java{String} containing things like \java{"Spade"} for suits and \java{"Queen"} for ranks.
A problem with this choice is that it would not be easy to compare cards to see which had a higher rank or suit.

\index{encode}
\index{map to}

An alternative is to use integers to {\bf encode} the ranks and suits.
By encode, we {\em don't} mean to encrypt or translate into a secret code.
We mean to define a mapping between a sequence of numbers and the things we want to represent.

Here is a mapping for suits:

\begin{tabular}{l c l}
Clubs & $\mapsto$ & 0 \\
Diamonds & $\mapsto$ & 1 \\
Hearts & $\mapsto$ & 2 \\
Spades & $\mapsto$ & 3
\end{tabular}

% DW suggests using constants for CLUBS, DIAMONDS, etc.
% ABD thinks good idea, but will require changes in the code and book.
% CSM: an object-oriented solution would use enums instead of constants

We use the mathematical symbol $\mapsto$ to make it clear that these mappings are not part of the program.
They are part of the program design, but they never appear explicitly in the code.

Each of the numerical ranks (2 through 10) maps to the corresponding integer.
For the face cards, we can use:

\begin{tabular}{l c l}
Ace & $\mapsto$ & 1 \\
Jack & $\mapsto$ & 11 \\
Queen & $\mapsto$ & 12 \\
King & $\mapsto$ & 13 \\
\end{tabular}

With this encoding, the class definition for the \java{Card} type looks like this:

\begin{code}
public class Card {
    private int rank;
    private int suit;

    public Card(int rank, int suit) {
        this.rank = rank;
        this.suit = suit;
    }
}
\end{code}

\index{constructor}

The instance variables are \java{private}: we can access them from inside this class, but not from other classes.

The constructor takes a parameter for each instance variable.
To create a \java{Card} object, we use the \java{new} operator:

\begin{code}
Card threeOfClubs = new Card(3, 0);
\end{code}

%The second argument, \java{0} represents the suit Clubs.
The result is a reference to a \java{Card} that represents the ``3 of Clubs''.

\section{Card toString}

\index{print!Card}

When you create a new class, the first step is to declare the instance variables and write constructors.
A good next step is to write \java{toString}, which is useful for debugging and incremental development.

\index{string!array of}
\index{array!of strings}

To display \java{Card} objects in a way that humans can read easily, we need to ``decode'' the integer values as words.
A natural way to do that is with an array of \java{String}s.
For example, we can create the array like this:

\begin{code}
String[] suits = new String[4];
\end{code}

And then assign values to the elements:

\begin{code}
suits[0] = "Clubs";
suits[1] = "Diamonds";
suits[2] = "Hearts";
suits[3] = "Spades";
\end{code}

Or we can create the array and initialize the elements at the same time, as we saw in Section~\ref{printarray}:

\begin{code}
String[] suits = {"Clubs", "Diamonds", "Hearts", "Spades"};
\end{code}

\index{memory diagram}
\index{diagram!memory}
\index{reference}
\index{string!reference to}

The memory diagram in Figure~\ref{fig.stringarray} shows the result.
Each element of the array is a reference to a \java{String}.

\begin{figure}[!ht]
\begin{center}
\includegraphics{figs/stringarray.pdf}
\caption{Memory diagram of an array of strings.}
\label{fig.stringarray}
\end{center}
\end{figure}

We also need an array to decode the ranks:

\begin{code}
String[] ranks = {null, "Ace", "2", "3", "4", "5", "6",
           "7", "8", "9", "10", "Jack", "Queen", "King"};
\end{code}

The zeroth element should never be used, because the only valid ranks are 1--13.
We set it to \java{null} to indicate an unused element.

Using these arrays, we can create a meaningful \java{String} using \java{suit} and \java{rank} as indexes.

\begin{code}
String s = ranks[this.rank] + " of " + suits[this.suit];
\end{code}

The expression \java{ranks[this.rank]} means ``use the instance variable \java{rank} from \java{this} object as an index into the array \java{ranks}.''
We select the string for \java{this.suit} in a similar way.

Now we can wrap all the previous code in a \java{toString} method.

\begin{code}
public String toString() {
    String[] ranks = {null, "Ace", "2", "3", "4", "5", "6",
               "7", "8", "9", "10", "Jack", "Queen", "King"};
    String[] suits = {"Clubs", "Diamonds", "Hearts", "Spades"};
    String s = ranks[this.rank] + " of " + suits[this.suit];
    return s;
}
\end{code}

When we display a card, \java{println} automatically calls \java{toString}.
%slr: 1-11-20:
%The output of the following code is {\tt Jack of Diamonds}.

Here is a trinket implemetation of what we have so far:
% height = 130 + 11 * num_lines
\begin{trinket} [420] {Card.java}
public class Card {
    //class attributes
    private int rank;
    private int suit;

    //default constructor
    public Card(int rank, int suit) {
        this.rank = rank;
        this.suit = suit;
    }

    //toString() method
    public String toString() {
       String[] ranks = {null, "Ace", "2", "3", "4", "5", "6",
               "7", "8", "9", "10", "Jack", "Queen", "King"};
       String[] suits = {"Clubs", "Diamonds", "Hearts", "Spades"};
       String s = ranks[this.rank] + " of " + suits[this.suit];
       return s;
    }

    //main method
    public static void main(String[] args) {
       Card threeOfClubs = new Card(3, 0);
       System.out.println(threeOfClubs);
    }
}
\end{trinket}

Run the trinket and observe the result.

%slr: end 1-11-20

\begin{code}
Card card = new Card(11, 1);
System.out.println(card);
\end{code}

\section{Class Variables}
\label{classvar}

\index{class variable}

So far we have seen local variables, which are declared inside a method, and instance variables, which are declared in a class definition, usually before the method definitions.
Now it's time to learn about {\bf class variables}.
They are shared across all instances of the class.

%Local variables are created when a method is invoked, and their space is reclaimed when the method ends.
%Instance variables are created when you construct an object and reclaimed when the object is garbage-collected.

\index{static}
\index{variable!static}

Like instance variables, class variables are defined in a class definition, before the method definitions.
But they are identified by the keyword \java{static}.
Here is a version of \java{Card} where \java{RANKS} and \java{SUITS} are defined as class variables:

\begin{code}
public class Card {

    public static final String[] RANKS = {
        null, "Ace", "2", "3", "4", "5", "6", "7",
        "8", "9", "10", "Jack", "Queen", "King"};

    public static final String[] SUITS = {
        "Clubs", "Diamonds", "Hearts", "Spades"};

    // instance variables and constructors go here

    public String toString() {
        return RANKS[this.rank] + " of " + SUITS[this.suit];
    }
}
\end{code}

\index{garbage collection}

Class variables are allocated when the program begins and persist until the program ends.
In contrast, instance variables like \java{rank} and \java{suit} are allocated when the program creates \java{new} objects, and they are deleted when the object is garbage-collected.

% ABD: Let's check whether garbage collection ends up before this.

\index{final}

%You can refer to a class variable from anywhere inside the class definition.
Class variables are often used to store constant values that are needed in several places.
In that case, they should also be declared as \java{final}.
Note that whether a variable is \java{static} or \java{final} involves two separate considerations:
\java{static} means the variable is {\em shared}, and \java{final} means the variable is {\em constant}.

Naming \java{static final} variables with capital letters is a common convention that makes it easier to recognize their role in the class.
In the \java{toString} method, we refer to \java{SUITS} and \java{RANKS} as if they were local variables, but we can tell that they are class variables.

One advantage of defining \java{SUITS} and \java{RANKS} as class variables is that they don't need to be created (and garbage-collected) every time \java{toString} is called.
They may also be needed in other methods and classes, so it's helpful to make them available everywhere.
Since the array variables are \java{final}, and the strings they reference are immutable, there is no danger in making them \java{public}.

%slr: 1-11-20
\textbf{Section Exercises:}
\begin{enumerate}
\item Modify the the above trinket \java{Card}.  Add definitions for the class variables \java{RANKS} and \java{SUITS} at the very top of the \java{Card} class as shown above.  Also modify the \java{toString} method as shown.  Test your modified trinket to make sure it still works the same way as before.
\end{enumerate}
%slr: end 1-11-20

\section{The compareTo Method}

\index{equivalent}

As we saw in Section~\ref{equals}, it's helpful to create an \java{equals} method to test whether two objects are equivalent.

\begin{code}
public boolean equals(Card that) {
    return this.rank == that.rank
        && this.suit == that.suit;
}
\end{code}

\index{operator!logical}
\index{logical operator}

It would also be nice to have a method for comparing cards, so we can tell if one is higher or lower than another.
For primitive types, we can use comparison operators like \java{<} and \java{>} to compare values.
But these operators don't work for object types.

For strings, Java provides a \java{compareTo} method, as we saw in Section~\ref{strcmp}.
We can write our own version of \java{compareTo} for the classes that we define, like we did for the \java{equals} method.
%Later we will use this method to sort a deck of cards.

\index{ordering}
\index{complete ordering}
\index{partial ordering}

Some types are ``totally ordered'', which means that you can compare any two values and tell which is bigger.
Integers and strings are totally ordered.
Other types are ``unordered'', which means that there is no meaningful way to say that one element is bigger than another.
In Java, the \java{boolean} type is unordered; if you try to compare \java{true < false}, you get a compiler error.

The set of playing cards is ``partially ordered'', which means that sometimes we can compare cards and sometimes not.
For example, we know that the 3 of Clubs is higher than the 2 of Clubs, and the 3 of Diamonds is higher than the 3 of Clubs.
But which is better, the 3 of Clubs or the 2 of Diamonds?
One has a higher rank, but the other has a higher suit.

\index{compareTo}

To make cards comparable, we have to decide which is more important: rank or suit.
The choice is arbitrary, and it might be different for different games.
But when you buy a new deck of cards, it comes sorted with all the Clubs together, followed by all the Diamonds, and so on.
So for now, let's say that suit is more important.
With that decided, we can write \java{compareTo} as follows:

\begin{code}
public int compareTo(Card that) {
    if (this.suit < that.suit) {
        return -1;
    }
    if (this.suit > that.suit) {
        return 1;
    }
    if (this.rank < that.rank) {
        return -1;
    }
    if (this.rank > that.rank) {
        return 1;
    }
    return 0;
}
\end{code}

\java{compareTo} returns \java{-1} if \java{this} is a lower card, \java{+1} if \java{this} is a higher card, and \java{0} if \java{this} and \java{that} are equivalent.
It compares suits first.
If the suits are the same, it compares ranks.
If the ranks are also the same, it returns 0.

%slr: 1-11-20
\textbf{Section Exercises:}
\begin{enumerate}
\item Now make further modifications to those you made in the previous section to the \java{Card} class.  First add the \java{compareTo} \textbf{instance} method as shown immediately above.  Then modify the \java{main} method so as to declare two \java{Card} objects, compare them with your new method and report the results. (Call this instance method on an instance and pass another as a parameter as the method definition clearly indicates - for example \java{a.compateTo(b);}.
\end{enumerate}
%slr: end 1-11-20

\section{Cards are Immutable}

The instance variables of \java{Card} are \java{private}, so they can't be accessed from other classes.
We can provide getters to allow other classes to read the \java{rank} and \java{suit} values:

\begin{code}
public int getRank() {
    return this.rank;
}

public int getSuit() {
    return this.suit;
}
\end{code}

\index{immutable}

Whether or not to provide setters is a design decision.
If we did, cards would be mutable, so you could transform one card into another.
That is probably not a feature we want, and in general, mutable objects are more error-prone.
So it might be better to make cards immutable.
To do that, all we have to do is {\em not} provide any modifier methods (including setters).

\index{final}

That's easy enough, but it is not foolproof, because a fool might come along later and add a modifier.
We can prevent that possibility by declaring the instance variables \java{final}:

\begin{code}
public class Card {
    private final int rank;
    private final int suit;

    ...
}
\end{code}

You can initialize these these variables inside a constructor,
but if someone writes a method that tries to modify them, they'll get a compiler error.
This kind of safeguard helps prevent future mistakes and hours of debugging.

%slr: 1-11-20
\textbf{Section Exercises:}
\begin{enumerate}
\item Now make yet further modifications to those you made in the previous section exercises to the \java{Card} class.  Modify the declarations for the two instance variables to be \java{final}.  Then add getters.  Make sure the trinket will still compile and run.  Finally, demonstrate that your \java{Card} objects are in fact immutable by adding setters that try to modify the values.  What happens when you try to compile the program?
\end{enumerate}
%slr: end 1-11-20
%slr:  insert trinket implementaton of above.  In Section Exercise ask that they demonstrate immutability by providing, say, a setter

\section{Arrays of Cards}
\label{cardarray}

%\index{composition}

%By now we have seen several examples of composition; that is, the ability to combine language features in a variety of arrangements.
%One of the first examples we saw was using a method invocation as part of an expression.
%Another example is the nested structure of statements: you can put an \java{if} statement within a \java{while} loop, or within another \java{if} statement, etc.

%Having seen this pattern, and having learned about arrays and objects, you should not be surprised to learn that you can make arrays of objects.
%And you can define objects with arrays as instance variables; you can make arrays that contain arrays; you can define objects that contain objects, and so on.

\index{array!of objects}
\index{object!array of}

Just as you can create an array of \java{String} objects, you can create an array of \java{Card} objects.
The following statement creates an array of 52 cards.
Figure~\ref{fig.cardarray} shows the memory diagram for this array.

\begin{code}
Card[] cards = new Card[52];
\end{code}

\begin{figure}[!ht]
\begin{center}
\includegraphics{figs/cardarray.pdf}
\caption{Memory diagram of an unpopulated \java{Card} array.}
\label{fig.cardarray}
\end{center}
\end{figure}

% ABD: I might redraw this to use dashed lines instead of zig-zag to indicate the elided part of the array.

\index{null}

Although we call it an ``array of cards'', the array contains {\em references} to cards; it does not contain the \java{Card} objects themselves.
Initially the references are all \java{null}.

Even so, you can access the elements of the array in the usual way:

\begin{code}
if (cards[0] == null) {
    System.out.println("No card yet!");
}
\end{code}

\index{exception!NullPointer}
\index{run-time error}

But if you try to access the instance variables of non-existent \java{Card} objects, you will get a \java{NullPointerException}.

\begin{code}
System.out.println(cards[0].rank);  // NullPointerException
\end{code}

\index{nesting}
\index{loop!nested}

That code won't work until we put cards in the array.

%slr: trinket which access a null object and tries to prin it.

One way to populate the array is to write nested \java{for} loops:

\begin{code}
int index = 0;
for (int suit = 0; suit <= 3; suit++) {
    for (int rank = 1; rank <= 13; rank++) {
        cards[index] = new Card(rank, suit);
        index++;
    }
}
\end{code}

The outer loop iterates suits from 0 to 3.
For each suit, the inner loop iterates ranks from 1 to 13.
Since the outer loop runs 4 times, and the inner loop runs 13 times for each suit, the body is executed 52 times.

\index{index}

We use a separate variable \java{index} to keep track of where in the array the next card should go.
Figure~\ref{fig.cardarray2} shows what the array looks like after the first two cards have been created.

\begin{figure}[!ht]
\begin{center}
\includegraphics{figs/cardarray2.pdf}
\caption{Memory diagram of a \java{Card} array with two cards.}
\label{fig.cardarray2}
\end{center}
\end{figure}

\index{print!array}

When you work with arrays, it is convenient to have a method that displays the contents.
We have seen the pattern for traversing an array several times, so the following method should be familiar.

% ABD: Maybe add reference to where we introduced enhanced loops?

\begin{code}
public static void printDeck(Card[] cards) {
    for (Card card : cards) {
        System.out.println(card);
    }
}
\end{code}

%\begin{code}
%public static void printDeck(Card[] cards) {
%    for (int i = 0; i < cards.length; i++) {
%        System.out.println(cards[i]);
%    }
%}
%\end{code}

Since \java{cards} has type \java{Card[]}, pronounced ``card array'', an element of \java{cards} has type \java{Card}.
So \java{println} invokes the \java{toString} method in the \java{Card} class.

%slr: 1-11-20:
Here is a trinket which builds an array of \java{Card} objects and prints them out:

% height = 130 + 11 * num_lines
\begin{trinket} [640] {Card.java}
public class Card {
    //class attributes
    private int rank;
    private int suit;

    //default constructor
    public Card(int rank, int suit) {
        this.rank = rank;
        this.suit = suit;
    }

    //toString() method
    public String toString() {
       String[] ranks = {null, "Ace", "2", "3", "4", "5", "6",
               "7", "8", "9", "10", "Jack", "Queen", "King"};
       String[] suits = {"Clubs", "Diamonds", "Hearts", "Spades"};
       String s = ranks[this.rank] + " of " + suits[this.suit];
       return s;
    }

    //printDeck() method - *not* an instance method!
    public static void printDeck(Card[] cards) {
           for (Card card : cards) {
              System.out.println(card);
           }
    }

    //main method
    public static void main(String[] args) {
       Card[] cards = new Card[52];  //array of 52 Card object *references*
       //initialize deck
       int index = 0;
       for (int suit = 0; suit <= 3; suit++) {
           for (int rank = 1; rank <= 13; rank++) {
               cards[index] = new Card(rank, suit);
               index++;
           }
       }
       //print the deck using the *static* method printDeck
       printDeck(cards);
    }
}
\end{trinket}

Run the trinket and observe the result.

%slr: end 1-11-20

Then again, we don't have to write our own \java{printDeck} method.
The \java{Arrays} class provides a \java{toString} method that invokes \java{toString} on the elements of an array and concatenates the results.

\begin{code}
System.out.println(Arrays.toString(cards))
\end{code}

%slr:  Section Exercises:  1. fix trinket above by initializing the array of Card objects and show it worked by printing one out.  2.  Implement and test the printDeck method above.  3.  Try the Arrays.toString method as shown above.

\section{Sequential Search}

\index{sequential search}

The next method we'll write is \java{search}, which takes an array of cards and a \java{Card} object as parameters.
It returns the index where the \java{Card} appears in the array, or \java{-1} if it doesn't.
This version of \java{search} uses the algorithm we saw in Section~\ref{traversal}, which is called {\bf sequential search}:

\index{traverse}
\index{loop!search}

\begin{code}
public static int search(Card[] cards, Card target) {
    for (int i = 0; i < cards.length; i++) {
        if (cards[i].equals(target)) {
            return i;
        }
    }
    return -1;
}
\end{code}

\index{statement!return}
\index{return!inside loop}

The method returns as soon as it discovers the card, which means we don't have to traverse the entire array if we find the target.
If we get to the end of the loop, we know the card is not in the array.

\index{efficiency}

If the cards in the array are not in order, there is no way to search faster than sequential search.
We have to look at every card, because otherwise we can't be certain the card we want is not there.
But if the cards are in order, we can use better algorithms.

Sequential search is relatively inefficient, especially for large arrays.
If you pay the price to keep the array sorted, finding elements becomes much easier.
%We will learn in the next chapter how to sort arrays.

%slr: 1-11-20
\textbf{Section Exercises:}
\begin{enumerate}
\item Modify the version of the \java{Card} class as implemented in Section~\ref{cardarray}.  Add the static \java{search} method and test it.  (You may want to omit the call to \java{printDeck}).
\end{enumerate}
%slr: end 1-11-20

\section{Binary Search}

% ABD:  I wonder if we need a different example.  I don't know how many students have looked up a word in a paper dictionary.
% slr 1-11-20:  I agree, but can't think of one right now.

\index{binary search}

When you look for a word in a dictionary, you don't search page by page from front to back.
Since the words are in alphabetical order, you probably use a {\bf binary search} algorithm:

\begin{enumerate}

\item Start on a page near the middle of the dictionary.

\item Compare a word on the page to the word you are looking for.
If you find it, stop.

\item If the word on the page comes before the word you are looking for, flip to somewhere later in the dictionary and go to step 2.

\item If the word on the page comes after the word you are looking for, flip to somewhere earlier in the dictionary and go to step 2.

\end{enumerate}

This algorithm is much faster than sequential search, because it rules out half of the remaining words each time you make a comparison.
If at any point you find two adjacent words on the page, and your word would be between them, you can conclude that your word is not in the dictionary.

Getting back to the array of cards, we can write a faster version of \java{search} if we know the cards are in order:

\begin{code}
public static int binarySearch(Card[] cards, Card target) {
    int low = 0;
    int high = cards.length - 1;
    while (low <= high) {
        int mid = (low + high) / 2;                 // step 1
        int comp = cards[mid].compareTo(target);

        if (comp == 0) {                            // step 2
            return mid;
        } else if (comp < 0) {                      // step 3
            low = mid + 1;
        } else {                                    // step 4
            high = mid - 1;
        }
    }
    return -1;
}
\end{code}

First, we declare \java{low} and \java{high} variables to represent the range we are searching.
Initially we search the entire array, from \java{0} to \java{cards.length - 1}.

Inside the \java{while} loop, we repeat the four steps of binary search:

\begin{enumerate}

\item Choose an index between \java{low} and \java{high}---call it \java{mid}---and compare the card at \java{mid} to the target.

\item If you found the target, return its index (which is \java{mid}).

\item If the card at \java{mid} is lower than the target, search the range from \java{mid + 1} to \java{high}.

\item If the card at \java{mid} is higher than the target, search the range from \java{low} to \java{mid - 1}.

\end{enumerate}

If \java{low} exceeds \java{high}, there are no cards in the range, so we terminate the loop and return \java{-1}.

This algorithm only depends on the \java{compareTo} method of the object, so we can use this code with any object type that provides \java{compareTo}.

\section{Tracing the Code}

\index{tracing}

To see how binary search works, it's helpful to add the following print statement at the beginning of the loop:

\begin{code}
System.out.println(low + ", " + high);
\end{code}

Using a sorted deck of cards, we can search for the ``Jack of Clubs'' like this:

\begin{code}
Card card = new Card(11, 0);
System.out.println(binarySearch(cards, card));
\end{code}

We expect to find this card at position 10 (since the ``Ace of Clubs'' is at position 0).
Here is the output of \java{binarySearch}:

\begin{stdout}
0, 51
0, 24
0, 11
6, 11
9, 11
10
\end{stdout}

You can see the range of cards shrinking as the \java{while} loop runs, until eventually index 10 is found.
If we search for a card that's not in the array, like \java{new Card(15, 1)} the ``15 of Diamonds'', we get the following:

\begin{stdout}
0, 51
26, 51
26, 37
26, 30
26, 27
-1
\end{stdout}

%\index{testing}
%\index{correctness}
%
%These tests don't prove that this program is correct.
%In fact, no amount of testing can {\em prove} that a program is correct.
%But looking at a few cases and examining the code, you might be able to convince yourself.

Each time through the loop, we cut the distance between \java{low} and \java{high} in half.
After $k$ iterations, the number of remaining cards is $52 / 2^k$.
To find the number of iterations it takes to complete, we set $52 / 2^k = 1$ and solve for $k$.
The result is $\log_2 52$, which is about 5.7.
So we might have to look at 5 or 6 cards, as opposed to all 52 if we did a sequential search.

More generally, if the array contains $n$ elements, binary search requires $\log_2 n$ comparisons, and sequential search requires $n$.
For large values of $n$, binary search is substantially faster.

\section{Vocabulary}

\begin{description}

\term{encode}
To represent one set of values using another set of values by constructing a mapping between them.

\term{class variable}
A variable declared within a class as \java{static}.
There is only one copy of a class variable, no matter how many objects there are.

\term{sequential search}
An algorithm that searches array elements, one by one, until a target value is found.

\term{binary search}
An algorithm that searches a sorted array by starting in the middle, comparing an element to the target, and eliminating half of the remaining elements.

\end{description}

\section{Exercises}

The code for this chapter is in the {\tt ch12} directory of {\tt ThinkJavaCode2}.
See page~\pageref{code} for instructions on how to download the repository.
Before you start the exercises, we recommend that you compile and run the examples.

%TODO make an exercise of validating the input to Card constructor?

\begin{exercise}  %%V6 Ex12.1

Encapsulate the deck-building code from Section~\ref{cardarray} in a method called \java{makeDeck} that takes no parameters and returns a fully-populated array of \java{Card}s.

\end{exercise}

\begin{exercise}  %%V6 Ex12.2

In some card games, Aces are ranked higher than Kings.
Modify the \java{compareTo} method to implement this ordering.

\end{exercise}

\begin{exercise}  %%V6 Ex12.3

In Poker a ``flush'' is a hand that contains five or more cards of the same suit.
A hand can contain any number of cards.

\index{histogram}

\begin{enumerate}

\item Write a method called \java{suitHist} that takes an array of cards as a parameter and that returns a histogram of the suits in the hand.
Your solution should only traverse the array once as in Section~\ref{singlepass}.

\item Write a method called \java{hasFlush} that takes an array of cards as a parameter and returns \java{true} if the hand contains a flush (and \java{false} otherwise).

\end{enumerate}

\end{exercise}

\begin{exercise}  %%V6 Ex12.4

% ABD: I have a mild preference for 2-D rather than 2D.  Worth changing?

Working with cards is more fun if you can display them on the screen.
If you have not already read Appendix~\ref{graphics} about 2D graphics, you should read it before working on this exercise.
In the code directory for this chapter, {\tt ch12}, you will find:

\begin{itemize}

\item {\tt cardset-oxymoron}: A directory containing images of playing cards.

\item {\tt CardTable.java}: A sample program that demonstrates how to read and display images.

\end{itemize}

\index{array!2D}

This code demonstrates the use of a 2D array, specifically an array of images.
The declaration looks like this:

\begin{code}
private Image[][] images;
\end{code}

The variable \java{images} refers to a 2D array of \java{Image} objects, which are defined in the \java{java.awt} package.
Here's the code that creates the array itself:

\begin{code}
images = new Image[14][4];
\end{code}

The array has 14 rows (one for each rank plus an unused row for rank 0) and 4 columns (one for each suit).
Here's the loop that populates the array:

\begin{code}
String cardset = "cardset-oxymoron";
String suits = "cdhs";

for (int suit = 0; suit <= 3; suit++) {
    char c = suits.charAt(suit);

    for (int rank = 1; rank <= 13; rank++) {
        String s = String.format("%s/%02d%c.gif",
                                 cardset, rank, c);
        images[rank][suit] = new ImageIcon(s).getImage();
    }
}
\end{code}

The variable \java{cardset} contains the name of the directory that contains the image files.
\java{suits} is a string that contains the single-letter abbreviations for the suits.
These strings are used to assemble \java{s}, which contains the filename for each image.
For example, when \java{rank=1} and \java{suit=2}, the value of \java{s} is \java{"cardset-oxymoron/01h.gif"}, which is an image of the Ace of Hearts.

The last line of the loop reads the image file, extracts an \java{Image} object, and assigns it to a location in the array, as specified by the indexes \java{rank} and \java{suit}.
For example, the image of the Ace of Hearts is stored in row 1, column 2.

If you compile and run {\tt CardTable.java}, you should see images of a deck of cards laid out on a green table.
You can use this class as a starting place to implement your own card games.

% TODO: ML suggests including a screen shot

%TODO We should make an actual exercise out of this.
% Maybe take an array of cards and display them in a row at a given location?

As a starting place, try placing cards on the table in the starting configuration for the solitaire game Klondike (see \url{https://en.wikipedia.org/wiki/Klondike_(solitaire)}).

You can get the image for the back of the card by reading the file \java{back192.gif}.

\end{exercise}


\documentclass{article}

\usepackage[final]{style}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc}    % use 8-bit T1 fonts
\usepackage{hyperref}       % hyperlinks
\usepackage{url}            % simple URL typesetting
\usepackage{booktabs}       % professional-quality tables
\usepackage{amsfonts}       % blackboard math symbols
\usepackage{nicefrac}       % compact symbols for 1/2, etc.
\usepackage{microtype}      % microtypography
\usepackage{verbatim}
\usepackage{graphicx}       % for figures
\usepackage{amsmath}

\title{Lecture \#6-7: Features and Fitting/Feature Descriptors}

\author{
  Trevor Danielson, Wesley Olmsted, Kelsey Wang, Ben Barnett \\
  Department of Computer Science\\
  Stanford University\\
  Stanford, CA 94305 \\
  \texttt{\{trevord, wolmsted, kyw, ben.barnett\}@cs.stanford.edu} \\
}

\begin{document}

\maketitle

% RANSAC - Ben Barnett (barnett3)
\section{RANSAC}
\subsection{Goal}
RANdom SAmple Consensus (RANSAC) is used for model fitting in images (e.g., line detection); it can be extremely useful for object identification, among other applications. It is often more effective than pure edge detection which is prone to several limitations: edges containing extra points due to the noise/clutter, certain parts of the edges being left out, and the existence of noise in measured edges' orientation.
\subsection{Motivation}
One of the primary advantages of RANSAC is that it is relatively efficient and accurate even when the number of parameters is high. However, it should be noted that RANSAC is likely to fail or produce inaccurate results in images with a relatively large amount of noise.
\subsection{General Approach}
The intuition for RANSAC is that by randomly sampling a group of points in an edge and applying a line of best fit to those points \textit{many times}, we have a high probability of finding a line that fits the points very well. Below is the general process "RANSAC loop":
\begin{enumerate}
\item Randomly select a seed group of points from the overall group of points you are trying to fit with a line.
\item Compute a line of best fit among the seed group. For example, if the seed group is only 2 distinct points, then it is clear to see that there is only one line that passes through both points, which can be determined with relative ease from the points' coordinates.
\item Find the number of \textbf{inliers} to this line by iterating over each point in the data set and calculating its distance from the line; if is less than a (predetermined) threshold value, it counts as an inlier. Otherwise, it is counted as an outlier.
\item If the number of inliers is sufficiently large, conclude that this line is "good", and that at least one line exists that includes the inliers tallied in the previous step. To further improve the line, re-compute the line using a least-squares estimate using all of the inliers that were within the threshold distance. Keep this transformation as the line that best approximates the data.
\end{enumerate}
\subsection{Drawbacks}
The biggest drawback to RANSAC is its inefficient handling of highly noisy data; an increase in the fraction of outliers in a given data set results in an increase in the number of samples required for model fitting (e.g., line of best fit). More importantly, the noisier an image is, the less likely it is for a line to \textit{ever} be considered sufficiently good at fitting the data. This is a significant problem because most real world problems have a relatively large proportion of noise/outliers.

% LOCAL INVARIANT FEATURES - Trevor Danielson
\section{Local Invariant Features}
\subsection{Motivation}
\paragraph{} The local invariant image features and their descriptors are used in a wide range of computer vision applications; they include, but are not limited to, object detection, classification, tracking, motion estimation, panorama stitching, and image registration. The previously discussed methods, such as cross-correlation, are not effective and robust in many of such appl. This method works by finding local, distinctive structures within an image (i.e., features), and it describes each feature using the surrounding region (e.g., a small patch centered on the detected feature). This "local" representation of image features (as opposed to a "global" one, such as cross-correlation) provides a more robust means of addressing the above-mentioned computer vision problems; such strategy is invariant to object rotations, point of view translations, and scale changes.

\subsection{General Approach}
The general approach for employing local invariant features is detailed below:
\begin{enumerate}
\item Find and define a set of distinctive keypoints.
\item Define a local region around the keypoint.
\item Extract and normalize the regional content from the area.
\item Compute a local descriptor from the normalized region (i.e., a function of pixel intensity).
\item Match local descriptors.
\end{enumerate}
\begin{center}
	\includegraphics[scale=0.5]{local_feature_ex.png}\\
  	\textbf{Figure 1:} The local features are identified within similar pictures; descriptors are calculated for normalized patches and compared to each other for quantifying the similarity. Source: Lecture 6 slides. \\
\end{center}
\subsection{Requirements}
Good local features should have the following properties:
\begin{enumerate}
\item \textit{Repeatability}: given the same object or scene under different image conditions such as lighting or change in viewpoint, a high amount of features should be detectable in both images being compared. In other words, the feature detection should be invariant to rotations and viewpoint changes and robustly handle lighting changes, noise, and blur.
\item \textit{Locality}: features should be local to avoid issues caused by occlusion and clutter.
\item \textit{Quantity}: enough features should be chosen to ensure the accuracy of techniques relying on descriptors such as object detection.
\item \textit{Distinctiveness}: results need to contain "salient" image features that show a large amount of variation; this ensures that the detected features can be distinguished from each other and properly matched between different images.
\item \textit{Efficiency}: feature matching in new images should be conducive to real-time applications.
\end{enumerate}

%Keypoint Localization - Kelsey Wang
\section{Keypoint Localization}
\subsection{Motivation}
\paragraph{} The goal of keypoint localization is to detect features consistently and repeatedly, to allow for more precise localization, and to find interesting content within the image.

\subsection{General Approach}
We will look for corners since they are repeatable and distinctive in the majority of images. To find corners, we look for large changes in intensity in all directions. To provide context, a "flat" region would not have change in any direction, and an edge would have no change along the direction of the edge. We will find these corners using the Harris technique.

\subsection{Harris Detector}
The intuition behind Harris detector is that if a window ($w$) slides over an image, the change in the intensity of pixel values caused by the shift is highest at corners. This is because change in pixel intensity is observed in both directions ($x$ and $y$) at corners, while it is limited to only one direction at the edges, and it is negligible at flat image regions.  To calculate the change of intensity due to the shift $[u,v]$:
$$E(u,v) = \sum_{x,y}^{} w(x,y)[I(x+u,y+v)-I(x,y)]^2$$
To find corners, we must maximize this function. Taylor Expansion is then used in the process to get the following equation:
\[
E(u,v)=
  \begin{bmatrix}
    u & v\\
  \end{bmatrix}
  M
  \begin{bmatrix}
      u \\
      v
    \end{bmatrix}
\]
where we have M defined as:
\[M =
\sum_{x,y}^{} w(x,y)
  \begin{bmatrix}
      I_x I_x & I_x I_y\\
      I_x I_y & I_y I_y
    \end{bmatrix}
\]
This matrix reveals:
\[M =
  \begin{bmatrix}
      \sum I_x I_x & \sum I_x I_y\\
      \sum I_x I_y & \sum I_y I_y
    \end{bmatrix}
    =
    \begin{bmatrix}
      \lambda_1 & 0\\
      0 & \lambda_2
    \end{bmatrix}
\]
Corners have both large and similar eigenvalues, whereas edges have one significantly greater eigenvalue, and flat regions have two small eigenvalues. Because the calculation of eigenvalues is computationally intensive, an alternative approach is employed where the Corner Response Function ($\theta$) is used to compute a score for each window:
$$\theta = det(M) - \alpha \textnormal{trace} (M)^2 $$
where $\alpha$ is a constant around $.04-.06$.
\begin{center}
	\includegraphics[scale=0.5]{eigenvalues_harris.png}\\
    \textbf{Figure 2:} The visualization of theta thresholds from Corner Response Function ($\theta$) indicating corner, edge, or flat regions. Source: Lecture 6 slides\\
\end{center}

This is not rotation invariant. To allow for rotation invariance, we will smooth with Gaussian, where the Gaussian already performs the weighted sum:

\[M =
g( \sigma )
  \begin{bmatrix}
      I_x I_x & I_x I_y\\
      I_x I_y & I_y I_y
    \end{bmatrix}
\]

Illustrated below is an of keypoints identified by Harris detector.
\begin{center}
	\includegraphics[scale=0.5]{harris_response.png}\\
    \textbf{Figure 3:} An example of Harris keypoint detection on an image. Source: Lecture 6 slides \\
\end{center}

%Scale invariant keypoint detection - Wesley Olmsted
\section{Scale Invariant Keypoint Detection}
\subsection{Motivation}
Earlier we used the Harris detector to find keypoints or corners. This detector used small windows in order to maintain good locality. Since the Harris detector uses this small window, if an image is scaled up, the window now no longer sees the same gradients it had with a smaller image.\\
\begin{center}
	\includegraphics{sift_scale_invariant.jpg} \\
    \textbf{Figure 4:} Harris detector windows on an increased scale. Source: \cite{opencv}
\end{center}
Figure 4:
Looking at the above image, we can see how the three windows on the right no longer see the sharp gradient in the x and y directions. All three windows are, therefore, classified edges. In order to address this problem, we need to normalize the scale of the detector.
\subsection{Solution}
We design a function to be scale-invariant, meaning that the function values for the corresponding regions are the same regardless of the scale. We can use a circle to represent this scalable function. A point on the circle is a function of the region size of the circle's radius.
\subsection{General Approach}
We can find the local maximum of a function. Relative to the local maximum, the region size should be the same regardless of the scale. This also means that the region size is co-variant with the image scale. A "good" function results in a single and distinct local maximum. In general, we should use a function that responds well to stark contrasts in intensity. \\
\begin{center}
	\includegraphics[scale=0.5]{scaleinvariant.png}\\
    \textbf{Figure 5:} The examples of different functions for finding local maximums. Source: Lecture 7 slides.
\end{center}
The function is defined as: $f = kernel*image$. Two such kernels include the Laplacian and the Difference of Gaussians (DoG). $$L = \sigma^2(G_{xx}(x,y,\sigma)+G_{yy}(x,y,\sigma))$$ $$DoG = G(x,y,k\sigma) - G(x,y,\sigma)$$ $$G(x,y,\sigma) = \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{x^2+y^2}{2\sigma^2}}$$
Both these kernels are scale and rotation invariant.

% References
\small
\bibliographystyle{plain}
\bibliography{bibliography}
\end{document}


\documentclass{beamer}
%
% Choose how your presentation looks.
%
% For more themes, color themes and font themes, see:
% http://deic.uab.es/~iblanes/beamer_gallery/index_by_theme.html
%
\mode<presentation>
{
  \usetheme{default}      % or try Darmstadt, Madrid, Warsaw, ...
  \usecolortheme{default} % or try albatross, beaver, crane, ...
  \usefonttheme{default}  % or try serif, structurebold, ...
  \setbeamertemplate{navigation symbols}{}
  \setbeamertemplate{caption}[numbered]
  \setbeamertemplate{footline}[frame number]
  \setbeamertemplate{itemize items}[circle]
  \setbeamertemplate{theorems}[numbered]
  \setbeamercolor*{structure}{bg=white,fg=blue}
  \setbeamerfont{block title}{size=\normalsize}
  \setbeamertemplate{frametitle continuation}{}
}

\newtheorem{proposition}[theorem]{Proposition}
\theoremstyle{definition}
\newtheorem{algorithm}[theorem]{Algorithm}
\newtheorem{idea}[theorem]{Idea}

\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{aligned-overset}
\usepackage{alltt}
\usepackage{amsmath}
\usepackage{csquotes}
\usepackage{multicol}
\usepackage{stmaryrd}
\usepackage{tabularx}

\renewcommand\tabularxcolumn[1]{m{#1}}
\newcolumntype{R}{>{\raggedleft\arraybackslash}X}

\def\code#1{\texttt{\frenchspacing#1}}
\def\padding{\vspace{0.5cm}}
\def\spadding{\vspace{0.25cm}}
\def\b{\textcolor{blue}}
\def\r{\textcolor{red}}
\def\g#1{{\usebeamercolor[fg]{block title example}{#1}}}

% fix for \pause in align
\makeatletter
\let\save@measuring@true\measuring@true
\def\measuring@true{%
  \save@measuring@true
  \def\beamer@sortzero##1{\beamer@ifnextcharospec{\beamer@sortzeroread{##1}}{}}%
  \def\beamer@sortzeroread##1<##2>{}%
  \def\beamer@finalnospec{}%
}
\makeatother

% \usepackage[sorting=ynt,style=alphabetic]{biblatex}
% \addbibresource{sources.bib}

\title[Theoretical Computer Science]{Theoretical Computer Science \\ Complexity Theory}
\author{Jonas Hübotter}
\date{}

\begin{document}

\begin{frame}
  \titlepage
\end{frame}

% Uncomment these lines for an automatically generated outline.
\begin{frame}[allowframebreaks]{Outline}
 \tableofcontents[subsubsectionstyle=hide]
\end{frame}
% \AtBeginSection[]
%   {
%      \begin{frame}
%      \frametitle{Plan}
%      \tableofcontents[currentsection, sectionstyle=show/hide, hideothersubsections]
%      \end{frame}
%   }

\section{$\mathcal{P}$}

\begin{frame}{$\mathcal{P}$}
    $\mathcal{P}$ is the class of problems that can be solved by a DTM in polynomial time.\pause
    \begin{definition}
        We define \begin{align*}
            \text{time}_M(w)  = (&\text{\#steps until DTM $M[w]$ halts}) \in \mathbb{N} \cup \{\infty\} \pause\\
            \text{TIME}(f(n)) = \{&A \subseteq \Sigma^* \mid \exists \text{DTM}\ M.\ A = L(M) \land \forall w \in \Sigma^*.\ \\&\text{time}_M(w) \leq f(|w|)\} \\&\text{for a total function $f : \mathbb{N} \to \mathbb{N}$}
        \end{align*}\pause
        Then, the complexity class $\mathcal{P}$ is given as \begin{align*}
            \mathcal{P} = \bigcup_{p \in \text{polynomial}} \text{TIME}(p(n)) \pause= \bigcup_{k \geq 0} \text{TIME}(\mathcal{O}(n^k))
        \end{align*} where $\text{TIME}(\mathcal{O}(f)) = \bigcup_{g \in \mathcal{O}(f)} \text{TIME}(g)$.
    \end{definition}
\end{frame}

\section{$\mathcal{NP}$}

\begin{frame}{$\mathcal{NP}$}
    $\mathcal{NP}$ is the class of problems that can be solved by a NTM in polynomial time.\pause
    \begin{definition}
        We define \begin{align*}
            \text{ntime}_M(w) = \begin{cases}
                (\text{minimal \#steps for NTM $M[w]$ to halt}) & w \in L(M) \\
                0 & w \not\in L(M)
            \end{cases}
        \end{align*}\pause \begin{align*}
            \text{NTIME}(f(n)) = \{&A \subseteq \Sigma^* \mid \exists \text{NTM}\ M.\ A = L(M) \land \forall w \in \Sigma^*.\ \\&\text{ntime}_M(w) \leq f(|w|)\} \\&\text{for a total function $f : \mathbb{N} \to \mathbb{N}$}
        \end{align*}
    \end{definition}
\end{frame}

\begin{frame}{$\mathcal{NP}$}
    To simplify the notation, we do not require that the NTM $M$ terminates for inputs $w \not\in L(M)$. This is not a restriction as we can always define the NTM $M'$ which returns $0$ after $p(|w|)$ steps (timeout).\pause
    \begin{definition}
        The complexity class $\mathcal{NP}$ is given as \begin{align*}
            \mathcal{NP} = \bigcup_{p \in \text{polynomial}} \text{NTIME}(p(n)) \pause= \bigcup_{k \geq 0} \text{NTIME}(\mathcal{O}(n^k)).
        \end{align*}
    \end{definition}
\end{frame}

\subsection{Polynomially Bounded Verifier}

\begin{frame}{Polynomially Bounded Verifier}
    A problem $A$ is in $\mathcal{NP}$ if and only if solutions (which are described by \emph{certificates}) to the problem can be verified in polynomial time by a DTM (a \emph{polynomially bounded verifier}).\pause\par\spadding
    \begin{block}{Intuition}
        \begin{itemize}
            \item A decision problem can be thought of an exploration of the search space consisting of all instances with the goal of finding a solution.\pause
            \item Problems in $\mathcal{NP}$ may be harder than problems in $\mathcal{P}$ as a NTM is able to pursue exponentially many paths in the search tree.\pause
            \item However, once the NTM found a path, the length of this path must be polynomial in the size of the input.\pause
            \item Thus, a DTM must be able to verify that a given path is correct in polynomial time.
        \end{itemize}
    \end{block}
\end{frame}

\begin{frame}{Polynomially Bounded Verifier}
    \begin{definition}
        Let $M$ be a DTM with $L(M) = \{w\#c \mid w \in \Sigma^*, c \in \Delta^*\}$.\pause
        \begin{itemize}
            \item If $w\#c \in L(M)$, $c$ is a \b{certificate} for $w$.\pause
            \item $M$ is a \b{polynomially bounded verifier} for the language $\{w \in \Sigma^* \mid \exists c \in \Delta^*.\ w\#c \in L(M)\}$ (i.e. the language of all words that have a certificate) if there exists a polynomial $p$ such that $\text{time}_M(w\#c) \leq p(|w|)$.
        \end{itemize}\pause
        \r{Especially: $c \leq p(|w|)$, i.e. the size of the certificate must be polynomially bounded by the size of the input.}
    \end{definition}
\end{frame}

\subsection{Polynomial Reduction}

\begin{frame}{Polynomial Reduction}
    \begin{definition}
        Given problems $A \subseteq \Sigma^*, B \subseteq \Gamma^*$, A is \b{polynomially reducable} to $B$ (denoted $A \leq_p B$) if there exists a total and by a DTM in polynomial time computable function $f : \Sigma^* \to \Gamma^*$ such that \begin{align*}
            \forall w \in \Sigma^*.\ w \in A \iff f(w) \in B.
        \end{align*}
    \end{definition}\pause
    The complexity classes $\mathcal{P}$ and $\mathcal{NP}$ are closed under polynomial reduction.
\end{frame}

\subsection{$\mathcal{NP}$-completeness}

\begin{frame}{$\mathcal{NP}$-completeness}
    \begin{definition}
        \begin{itemize}
            \item The language $L$ is \b{$\mathcal{NP}$-hard} if $\forall A \in \mathcal{NP}.\ A \leq_p L$.\pause
            \item The language $L$ is \b{$\mathcal{NP}$-complete} if $L \in \mathcal{NP}$ and $L$ is $\mathcal{NP}$-hard.
        \end{itemize}
    \end{definition}\pause
    \begin{example}[Proving $\mathcal{NP}$-completeness]\pause
        \begin{itemize}
            \item If $A \leq_p B$ and $A$ is $\mathcal{NP}$-hard, then $B$ is $\mathcal{NP}$-hard.\pause
            \item If $A \leq_p B$ and $B \in \mathcal{NP}$, then $A \in \mathcal{NP}$.\pause
            \item If there exists a polynomially bounded verifier for $A$, then $A \in \mathcal{NP}$.
        \end{itemize}
    \end{example}
\end{frame}

\section{Approximation Algorithm}

\begin{frame}{Approximation Algorithm}
    \begin{definition}
        A \b{$d$-approximation algorithm} ($d \in \mathbb{R}$) for an optimization problem is an algorithm that computes in polynomial time a solution to the problem that is at most $d$ times worse than the optimal solution.
    \end{definition}
\end{frame}

% \begin{frame}[allowframebreaks]{References}

% \printbibliography

% \end{frame}

\end{document}


\chapter{Theory}
\label{ch:2}

This section is largely based on the work done for the course \textit{TDT4501 Computer Science, Specialization Project}\cite{0-SimilarityMeasures}.
It is assumed that the readers of this thesis will not have read the final report and thus some of its content is restated.
This chapter aims to summarize the related to trajectories and the manners of computing their similarity.
We define terms and notation that will be used throughout this work and close the chapter by providing background for trajectory similarity usages.

\section{Movement Data}

Collecting data on how \textit{something} moves is done in a wide variety of fields.
Some of the applications of movement data analysis are found in zoology, finance, forecasts, navigation, and medicine\cite{15-BasicConcepts,89-FinancialTimeseries, 90-InternationalGNSS,91-EnhancedRepresentation}.
The fields process the movement data differently and there different types of moment data that each process.

The type of movement data we will examine here are trajectories, a data format whose prevalence is unmistakable.

\subsection{Collection Techniques}
A trajectory is a record of movement data commonly represented as a timestamped sequence of positions. N. Andrienko et al. noted that there are four ways to observe and record movement data\cite{15-BasicConcepts}:

\begin{itemize}
\item \textit{Time-based} collection means that the position is recorded at equidistant time intervals.
For instance, if we were to record the location of an entity every other minute it would be time-based collection.
\medskip

\item \textit{Change-based} collection means that a new location is recorded when an entity has moved a set distance from its previous location.
An example of this would be sensors in smartwatches that count steps.
\medskip

\item \textit{Location-based} collection means that records are created when an entity enters a specific area. This is useful for tracking location changes over large distances and when precision is not vital.
\medskip

\item For \textit{event-based} data collection the data is only recorded if a set of predetermined events occur. In particular, the moving entity itself should trigger the data sampling. A triggering event could be a user electing to share their current location.
\medskip

\end{itemize}

Lastly, the authors discussed a \textit{mixed}-solution.
As the name indicates, this collection method acknowledges that the listed techniques are not mutually exclusive.
In fact, depending on the application it may be desirable to combine them.
In the case of tracking taxi trips, data collection may be done via time-based techniques during the ride.
However, the only time the data is of interest is when there is a passenger in the vehicle.
A passenger getting into a taxi would then become the triggering event for movement data collection.

\subsection{Notation}

\begin{table}[b]
\centering
\caption{Notation}
\label{tb:notation}
\renewcommand{\arraystretch}{1.4}
\begin{tabular}{|l|l|}
\hline
\textbf{Notation}   & \textbf{Description}                    \\
% \hline
% $D(T_A, T_B)$       & An arbitrary distance measure for trajectory A and B          \\
\hline
$T_A$ or $A$   &  Trajectory A         \\
\hline
$a$           &  An element of $T_A$         \\
\hline
$a_i$           &  The  $i$th element of $T_A$         \\
\hline
$n_A$         &  The length of $T_A$ \\
\hline
$Dist_{X}(A, B)$ or ${X}(A, B)$       & Distance between trajectory A and B with measure X          \\
\hline
$d(a_i, b_j)$ or $d(a, b)$           &  Distance between given trajectory elements of $A$ and $B$        \\
\hline
$Rest(A)$         &  Trajectory A without its leading element \\
\hline
\end{tabular}
\end{table}

As stated, a trajectory is a time-stamped sequence of positions.
Phrased another way, it is a sequence of measurements with a temporal component which is the definition of a \textit{time-series}, also referred to as simply \textit{series}.
Consequently terms and techniques from time-series analysis will be used to characterize trajectories in this work.

In particular, we will use the terms time-series and series when referring to the trajectories.
We shall use $T_A$ to denote trajectory $A$ and $T_B$ denotes trajectory $B$.
A trajectory contains a several elements that each encompass temporal and spatial information, see  \Cref{eq:trajectory_ts}

\clearpage
\begin{equation}\label{eq:trajectory_ts}
T_A =  [a_1,\cdots , a_n] \quad \text{where }  a_i = ( \text{t}_i , \; a_{i,\text{LOC}})
\end{equation}
Where $n$ is the number of trajectory elements in $T_A$,  t$_i$ the timestamp, and $a_{i,\text{LOC}}$ is the location. The dimensionality of this component is most commonly either two or three\cite{17-TrajectorySimilarity}.

In this thesis, trajectories have been simplified to consist of a series of locations in 2-d space and the temporal aspect is implicit.
See \Cref{eq:trajectory_elem} for the format of the trajectory elements. This simplification is common where time-based collection is used to gather data.

\begin{equation}\label{eq:trajectory_elem}
a_i = [ a_{i,X}, \; a_{i, Y} ]
\end{equation}

where $X$ is the longitudinal aspect and $Y$ is the latitudinal one.  See \Cref{tb:notation} for additional notation used throughout in this thesis.

\section{Trajectory Similarity Distance}
% Part of what makes the computing the similarity of trajectories an expensive operation is the data structure itself .
% The constituent elements are themselves composite, even when just considering the spatial features.
% \subsection{Trajectories and the Matching Problem}
% \textit{The notion of similarity and distance are not completely standardized in the literature. As a reference we shall use....  and then (specify) the more terms (as we go) }

Informally, a measure is something that numerically quantifies something.
The mathematical definition of a measure requires precise terminology; studying that definition is far beyond the scope of this thesis.
Consequently, we present a simplified definition of a measure that complies with the proper one\cite{28-Measure, 29-MeasureMathematics}:

\medskip
\begin{definition} %[measure]
\label{def:measure}
A \textit{measure} on the set $S$ is a function $\mu$ that assigns a non-negative numeric value to each subset of $S$ such that for  $E, F \subset S$:
\begin{align*}
&\mu(E) \geqslant 0 & \text{Non-negativity}& \\
&\mu(\emptyset) = 0 &\text{Null empty set}&  \\
E, F \text{ disjoint } \Rightarrow \; &\mu(E \cup F) = \mu(E) + \mu(F)   &\quad \text{Countable additivity}&  \\
\end{align*}
\end{definition}

Another concept from mathematics we wish to bring forth is the notion of a \textit{metric}\cite{30-Metric}.

\medskip

\begin{definition} %[metric]
\label{def:metric}
A function $f(a, b)$ on $S$ is a metric if for $a, b, c \subset S$ the following requirements are met:
\begin{align*}
 f(a, b) &\geqslant 0 & \text{Non-Negativity} \\
 f(a, b) &= 0 \Leftrightarrow  a=b &\text{Identity}  \\
 f(a, b) &= f (b,a) &\text{Symmetry}  \\
 f(a, c) &\leqslant f(a, b) + d(b, c)  &\text{Triangle Inequality}  \\
\end{align*}
\end{definition}

In mathematics, a metric is an abstraction of what distance means between members of a set.
There are a number of generic methods that are designed to work with metric distance functions.
Two examples of applications where a measure's metricity is required are sub-linear time approximation for clustering and index-based search and pruning\cite{85-VehicleTrajectory,54-IndexdrivenSimilarity,87-SublinearTime}.
About half of the similarity algorithms we discussed in \Cref{ch:3} are not proper metrics.
Nevertheless, a distance function does not have to be a metric to be applicable for some selected data mining tasks. \cite{47-ClassificationNonmetric}.

Now that we have established what a measure is, we turn our focus to what similarity means for trajectory data.
We refer to work done by Golding and Kanellakis who formalized the notion of two series' similarity distance.\cite{16-SimilarityQueries}.
They did so by modifying a version of \textit{The Matching Problem} which we have simplified and restated below:

\begin{quote}
\textit{Given a query series \textbf{Q} of length  \textbf{n} and another series \textbf{S} of length  \textbf{N}, where \textbf{N} is much larger than \textbf{n}, we are searching for a contiguous subsequence within \textbf{S} that is identical to  \textbf{Q}}
\end{quote}

Their work defined term \textit{Similarity Distance} as seen in \Cref{def:sim}.

% \medskip
% \begin{definition} %[Approximate matching]
% \label{def:approx}
% Given a tolerance $\epsilon \geq 0$ and a distance measure $X$, $T_A$ and $T_B$ are said to be \text{approximately matching} within $\epsilon$ when $D_X(T_A, T_B) \leq \epsilon$.
% There is an an \textit{exact similarity} between $T_A$ and $T_B$ if in $\epsilon$ is set to 0.
% \end{definition}

\medskip
\begin{definition} %[Similarity distance]
\label{def:sim}
The \text{similarity distance}, $D$, between $T_A$ and  $T_B$ is a numerical value produced by a given distance measure $X$:
\begin{equation*}
Dist_X(T_A, T_B) = D
\end{equation*}
\end{definition}

In essence, this thesis is a comparative study of how different measures affect similarity distance.
It is generally understood that a short distance signifies closeness; in other words, the measure $X$ has to be a dissimilarity measure.

The distinction between similarity and dissimilarity measures is generally is useful as they are opposite maximizing functions.
The similarity function is useful for detecting duplicates and automatic grouping whereas the dissimilarity function is more appropriate for identifying outliers in a data set. \cite{20-SimilarityDissimilarity}.
Nevertheless, the term similarity will be used to describe both types of alikeness quantification.
This is both due to being a natural shorting of \textit{similarity distance}  and it being being used as such in the literature\cite{9-FastDTWAccurate, 19-ClusterAnalysis,23-DiscoveringSimilar,43-TrajectoryDistance}.

% , as the algorithms that are discussed in \Cref{ch:3}\cite{17-TrajectorySimilarity}.

\subsection{Elasticity of Methods and Time Shifting}\label{sec:elasticity}
 Although the trajectories in this thesis are defined with implicit temporal components, there are characteristics of time-series, and thereby time-series similarity, which are easier understood when the temporal value is explicitly stated.

%  For that reason this section discusses time-series of as shown in  \Cref{eq:ts_time}.
% \begin{equation}
% \label{eq:ts_time}
% A =  [(t_1, a_1),\cdots , (t_n, a_n) ]
% \end{equation}
% where $t_i$ is the timestamp for for when position $a_i$ was recorded.

%  Add more of a subsection intro .. also define warping

\medskip
\begin{definition}\label{def:elasticity} %[Elasticity ]
A similarity function is \textbf{elastic} if trajectory elements can be compared one-to-many or one-to-none.
In contrast, a \textbf{lock-step} function compares $a_i$ to $b_i$ at every step. It requires the input trajectories to be of equal length.
\end{definition}

\medskip
\begin{definition}\label{def:timeshift_g} %[global time shifting]
\textbf{Global time shifting} is a distortion or relative lag between trajectories\cite{33-TimeSeries}. Formally we say that  $B$  is \textbf{time shifted} by $\delta$ from $A$ if the following  holds:
\begin{align*}
A &= [(t_1, a_1), \cdots, (t_n, a_n)] \\
B  &= [(t_1, b_1), \cdots, (t_n, b_n)] \\
B = \text{Shift}(A, \delta)  & = [(t_1+\delta, a_1), \cdots, (t_n+\delta, a_n)]
\end{align*}

\end{definition}
\medskip
\begin{definition} \label{def:timeshift_l}%[local time shifting ]
\textbf{Local time shifting} refers to time-shifts that occur non-uniformly and locally.
\end{definition}
\medskip

In order to account for similarity under \textit{local time shifts}, the measure has to be \textit{elastic}.
This is because these measures can realign trajectories so that corresponding trajectory sections can line up which maximizes shape similarity.
We note that it is possible to eliminate the effects of global time shifting by statistical normalization.
When a measure adapts to local time shifts, we say that it has \textit{warped} time.
The warping could refer to single trajectory elements or whole trajectory segments.

%For our data data set, with an implicit temporal component and time-based collection, we do not need to handle global time shifts. We note that it it possible to eliminate the effects of global time shifting by statistical normalization.

\subsection{Sectioning Trajectories}

With the aim of defining how similar whole trajectories are, we have to determine to decompose the trajectories into \textit{sections}.
Next, we must determine how to quantify the distance between those.
The manner in which this is done varies according to the different trajectory similarity algorithms.

A straightforward idea is to simply use the raw observations in trajectories as the basis; indeed that is a popular choice.
All but one of the algorithms discussed in this thesis uses this approach.
This is not the only approach, we observe that there are measures that remap the trajectories to direction vectors or sets before computing their similarity \cite{31-ShapebasedSimilarity, 88-SetbasedSimilarity}.

If the choice has been made to evaluate individual points as trajectory sections, it is possible to use any number of point-to-point distance functions.
The most common choices are the \textit{$L_p$- metrics} and the \textit{Haversine distance}.
If a trajectory spans a great distance, we need to account for the curvature of the Earth.
The Haversine distance is was developed so that distances could be accurately computed at a scale.
However, in this thesis, we assume that the trajectories span a small area, allowing for the assumption is of a flat geometry.
This lets us safely use the $L_p$- metrics, of which $L_1$ and $L_2$ are the most used ones.
The latter of them is known as the \textit{Euclidean distance}, and almost all of the similarity algorithms examined use it to quantify the distance between trajectory elements.
Its definition is seen in \Cref{eq:l2norm} and for the remainder of this thesis, $d(a, b)$ denotes the Euclidean distance between trajectory elements unless otherwise specified.

\begin{equation}\label{eq:l2norm}
d(a, b) = \sqrt{(a_X - b_X)^2 + (a_Y - b_Y)^2}
\end{equation}

\subsection{Classes of Trajectory Similarity}
With both spatial and temporal components or trajectories explicitly expressed, it becomes evident that weighing them differently gives rise to different types of similarity.
Pelekis et al. categorized four classes of trajectory similarity which represent four different ways to account for the trajectories' components\cite{32-SimilaritySearch}.
Below is a summary of each of the classes, each with an example of application.

\begin{enumerate}
\item When analyzing trajectories, we could value their \textit{Spatio-temporal similarity}.
If so,  we require trajectories to be alike in shape and to have occurred at around the same time for them to be considered similar.
\medskip

An example of where one would need this type of similarity would be when analyzing traffic networks and attempting to detect the roads that have the most congestion during rush hours.

\medskip
\item \textit{Time-relaxed}, or \textit{Spatial-only, Similarity} as the name implies considers the shape similarity of trajectory.
Under this type of similarity the temporal, component does not necessarily affect the final similarity score.
A hybrid approach primarily values the shape and then examines the temporal aspect afterward.
This differs from \textit{Spatio-Temporal} similarity as the temporal component is treated as a secondary aspect rather than a primary one.
\medskip

An example of where one would prefer this kind of similarity is “identifying highways at sea”.
The routes themselves are the subjects of  to interest while the time of which a vessel was in a specific location is not.

%  and which is characterizing difference between this class and the previous one. A

\medskip
\item \textit{Speed-Pattern based spatial similarity} can be conceptualized as an extension of \textit{Spatial-Only similarity}.
As the name indicates, the shape of a trajectory and the velocities of the entities are the components that are important for this class.
\medskip

This similarity class is instrumental when analyzing data sets to classify entries that move at different velocities.
If velocity data have been collected, the temporal component can be disregarded. On the other hand, if the velocity is not recorded the temporal data is implicitly needed so that the rate of change of location, the velocity, can be derived.

\medskip
\item The last class is \textit{directional similarity}.
With this type of similarity, entities that moved in the same directions in the same order are considered to be similar, regardless of the entities' origin. For instance, two entities will have a high similarity score if they both did some movement east and then north-west.
As with spatial-only similarity, the temporal component of the trajectory does not affect the similarity score.

\medskip
This kind of similarity is used when one wants to examine how something is moving. In particular, it is useful for looking into weather phenomena such as cyclones, hurricanes, and typhoons.

\end{enumerate}

\clearpage
The class of similarity this thesis will focus on is the Spatial-Only class.
In other words, we will weigh the geometrical resemblance and disregard the temporal and directional aspects of the trajectories.
This leads us back the the notation we laid out in \Cref{tb:notation} where $T_A = [a_1, \dotsc a_n]$ and $a_i = [ a_{i,X}, \, a_{i, Y} ]$ .

\section{Trajectory Clustering}
\label{sec:traj-mining}
Data mining is a field that describes how one can process large data sets so that underlying patterns become apparent. The field is broad and encompasses clustering, classification, outlier detection as well as other tasks\cite{81-DataMining}.
In this thesis, we use trajectory mining, and specifically clustering, as an application for the similarity measures.

% This is so that we may compare different similarity measures.

To balance out the biases of specific clustering techniques, we have chosen to utilize two of them.
It is a well-known issue that there is no ground truth to how to precisely define a cluster\cite{64-WhyMany}.
Optimizing trajectory clustering is itself an interesting research area, and great efforts are being put into it\cite{82-BigTrajectory,69-MiningSpatial}.
Nevertheless, we deemed that a proper exploration of trajectory clustering methods was out of scope for this thesis.
We have settled using well-known clustering techniques.
% Nearest-neighbor and spectral clustering techniques require the underlying similarity measure to be a metric \cite{50-ReviewPerspective}, and thus they were out of the question for this analysis.
We briefly explain the theory of the two clustering methods we chose and an evaluation index for cluster results.

 \subsection{Affinity Propagation (AP) }
Affinity Propagation (AP) was first introduced in 2007 and it creates clusters by passing messages between the data elements\cite{60-ClusteringPassing}.
The passing of the messages is an iterative process and the information passed relates to how well-suited the elements are for being the cluster representative.
Whether or not a given element is suited to be the representative changes as both the number of clusters and the member count of each cluster change.
The algorithm completes either when it has converged, meaning a new iteration does not update the clusters, or when an iteration limit has met.

An advantage of AP is that it does not require the number of final clusters to be known at the start.

 \subsection{Hierarchical Clustering Analysis (HCA)}
 Hierarchical Clustering Analysis (HCA), or just Hierarchical Clustering, refers to a family of algorithms wherein there are different variation\cite{78-Lesson10}. The version we focus on in this thesis is \textit{agglomerative complete-linkage} clustering.
The agglomerative part of the name means that each data point starts out as a cluster, then the most similar clusters are grouped into a new cluster.
This happens recursively until all data points are in the same cluster.
The second part of the name specifies how one determines which clusters should be merged.
Under complete-linkage, each cluster finds the farthest neighbors of the other clusters. The clusters that get merged are ones that have the closest “farthest neighbors”.
This iterative merging can be displayed in a dendrogram, see \Cref{fig:toy-dendrogram}

\begin{figure}[t]
  \centering
  \includegraphics[width=.6\linewidth]{figs/deno/toy-dendrogram.png}
  \caption{Top: A simple scatter plot of some arbitrary data points
  \newline Bottom: A dendrogram illustrating the agglomerative process.
  \newline Notice that $(b,c)$ are merged after $(a,d)$ indicating that latter are more similar}
  \label{fig:toy-dendrogram}
\end{figure}

Under hierarchical clustering, any number of final clusters can be set.
This means that the number of final clusters is determined up to human discretion, thereby not guaranteeing clearly defined clusters.

\subsection{Evaluating Clusters}
 After the clusters have been generated, it is useful to evaluate the result.
%  well the observations have been grouped.
One way to do this is with the Davis-Bouldin index (DB-index or DB-criterion) is defined as the ratio of the “within-cluster”-distance and “between-cluster”-distance. “Within-cluster”-distance is a description of dispersed elements of a cluster is and “between-cluster”-distance  is a description of how far apart the cluster are from each other. This means that a low DB-index is an indication of a good clustering result.

In the context of this, the information gained from the DB-index is minimal.
This is because it as well as all other numerical evaluations,  depend on a ground truth of how to correctly quantify the similarity distance.
The index may provide some insight, however, we expect the true information gain to be minimal and its ranking to be biased.

As a closing remark, we acknowledge that there is no general standard for what constitutes a “reasonable clustering” on account of it being highly domain-dependent.
Human inspection remains a vital tool for evaluating final clusterings\cite{79-SilhouetteAnalysis}.
In order to discuss the clustering results from a visual perspective, we define the terms \textit{Fuzzy} and \textit{Crisp} as seen in \Cref{def:cluster_defs_fuzz,def:cluster_defs_cri}.

\begin{definition}\label{def:cluster_defs_fuzz}
A cluster is \textbf{fuzzy} if its members appear spread out and with seemly little relation to each other.
For a "good" clustering result, we wish to minimize the number of fuzzy clusters.

\end{definition}

\begin{definition}\label{def:cluster_defs_cri}
A cluster is \textbf{crisp} if its members create a clear silhouette with seemly a clear correspondence.
For a "good" clustering result, we wish to have as many crisp clusters as possible
\end{definition}


%
% 6.006 problem set 1
%
\documentclass[12pt,twoside]{article}

\usepackage{amsmath}

\input{macros}

\setlength{\oddsidemargin}{0pt}
\setlength{\evensidemargin}{0pt}
\setlength{\textwidth}{6.5in}
\setlength{\topmargin}{0in}
\setlength{\textheight}{8.5in}

% Fill these in!
\newcommand{\theproblemsetnum}{1}
\newcommand{\releasedate}{September 8, 2011}
\newcommand{\partaduedate}{Thursday, September 15}
\newcommand{\tabUnit}{3ex}
\newcommand{\tabT}{\hspace*{\tabUnit}}

\begin{document}

\handout{Problem Set \theproblemsetnum}{\releasedate}

\newif\ifsolution
\solutiontrue
\newcommand{\solution}{\textbf{Your Solution:}}

\textbf{Both theory and programming questions} are due {\bf \partaduedate} at {\bf 11:59PM}.
%
Please download the .zip archive for this problem set, and refer to the
\texttt{README.txt} file for instructions on preparing your solutions.
%
Remember, your goal is to communicate. Full credit will be given only
to a correct solution which is described clearly. Convoluted and
obtuse descriptions might receive low marks, even when they are
correct. Also, aim for concise solutions, as it will save you time
spent on write-ups, and also help you conceptualize the key idea of
the problem.

We will provide the solutions to the problem set 10 hours after the problem set
is due, which you will use to find any errors in the proof that you submitted.
You will need to submit a critique of your solutions by \textbf{Tuesday,
September 20th, 11:59PM}. Your grade will be based on both your solutions and
your critique of the solutions.

\setlength{\parindent}{0pt}

\medskip

\hrulefill

\textbf{Collaborators:}
%%% COLLABORATORS START %%%
None.
%%% COLLABORATORS END %%%

\begin{problems}

\problem \points{15} \textbf{Asymptotic Practice}

For each group of functions, sort the functions in increasing order of
asymptotic (big-O) complexity:

\begin{problemparts}

\problempart \points{5} \textbf{Group 1:}

$$
\begin{array}{rcl}
f_1(n) &=& n^{0.999999} \log n \\
f_2(n) &=& 10000000 n \\
f_3(n) &=& 1.000001^n \\
f_4(n) &=& n^2
\end{array}
$$

\ifsolution \solution{}
%%% PROBLEM 1(a) SOLUTION START %%%
1, 2, 3, 4
%%% PROBLEM 1(a) SOLUTION END %%%
\fi

\problempart \points{5} \textbf{Group 2:}

$$
\begin{array}{rcl}
f_1(n) &=& 2^{2^{1000000}} \\
f_2(n) &=& 2^{100000n} \\
f_3(n) &=& \displaystyle \binom{n}{2} \\
f_4(n) &=& n \sqrt{n}
\end{array}
$$

\ifsolution \solution{}
%%% PROBLEM 1(b) SOLUTION START %%%
1, 2, 3, 4
%%% PROBLEM 1(b) SOLUTION END %%%
\fi

\problempart \points{5} \textbf{Group 3:}

$$
\begin{array}{rcl}
f_1(n) &=& n^{\sqrt{n}} \\
f_2(n) &=& 2^n \\
f_3(n) &=& n^{10} \cdot 2^{n / 2} \\
f_4(n) &=& \displaystyle\sum_{i = 1}^{n} (i + 1)
\end{array}
$$

\ifsolution \solution{}
%%% PROBLEM 1(c) SOLUTION START %%%
1, 2, 3, 4
%%% PROBLEM 1(c) SOLUTION END %%%
\fi

\end{problemparts}

\problem \points{15} \textbf{Recurrence Relation Resolution}

For each of the following recurrence relations,
pick the correct asymptotic runtime:

\begin{problemparts}

\problempart \points{5}
Select the correct asymptotic complexity
of an algorithm with runtime $T(n, n)$
where
$$
\begin{array}{rcll}
T(x, c) &=& \Theta(x) & \textrm{ for $c \le 2$}, \\
T(c, y) &=& \Theta(y) & \textrm{ for $c \le 2$, and} \\
T(x, y) &=& \Theta(x + y) + T(x / 2, y / 2).
\end{array}
$$

\begin{enumerate}
\item $\Theta(\log n)$.
\item $\Theta(n)$.
\item $\Theta(n \log n)$.
\item $\Theta(n \log^2 n)$.
\item $\Theta(n^2)$.
\item $\Theta(2^n)$.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 2(a) SOLUTION START %%%
1
%%% PROBLEM 2(a) SOLUTION END %%%
\fi

\problempart \points{5}
Select the correct asymptotic complexity
of an algorithm with runtime $T(n, n)$
where
$$
\begin{array}{rcll}
T(x, c) &=& \Theta(x) & \textrm{ for $c \le 2$}, \\
T(c, y) &=& \Theta(y) & \textrm{ for $c \le 2$, and} \\
T(x, y) &=& \Theta(x) + T(x, y / 2).
\end{array}
$$

\begin{enumerate}
\item $\Theta(\log n)$.
\item $\Theta(n)$.
\item $\Theta(n \log n)$.
\item $\Theta(n \log^2 n)$.
\item $\Theta(n^2)$.
\item $\Theta(2^n)$.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 2(b) SOLUTION START %%%
1
%%% PROBLEM 2(b) SOLUTION END %%%
\fi

\problempart \points{5}
Select the correct asymptotic complexity
of an algorithm with runtime $T(n, n)$
where
$$
\begin{array}{rcll}
T(x, c) &=& \Theta(x) & \textrm{ for $c \le 2$}, \\
T(x, y) &=& \Theta(x) + S(x, y / 2), \\
S(c, y) &=& \Theta(y) & \textrm{ for $c \le 2$, and} \\
S(x, y) &=& \Theta(y) + T(x / 2, y).
\end{array}
$$

\begin{enumerate}
\item $\Theta(\log n)$.
\item $\Theta(n)$.
\item $\Theta(n \log n)$.
\item $\Theta(n \log^2 n)$.
\item $\Theta(n^2)$.
\item $\Theta(2^n)$.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 2(c) SOLUTION START %%%
1
%%% PROBLEM 2(c) SOLUTION END %%%
\fi

\end{problemparts}

\section*{Peak-Finding}

In Lecture 1,
you saw the peak-finding problem.
As a reminder,
a \emph{peak} in a matrix
is a location with the property that its four neighbors
(north, south, east, and west)
have value less than or equal to the value of the peak.
We have posted Python code for solving this problem
to the website in a file called \texttt{ps1.zip}.
In the file \texttt{algorithms.py},
there are four different algorithms
which have been written
to solve the peak-finding problem,
only some of which are correct.
Your goal is to figure out
which of these algorithms are correct
and which are efficient.

\problem \points{16} \textbf{Peak-Finding Correctness}

\begin{problemparts}

\problempart \points{4} Is \texttt{algorithm1} correct?
\begin{enumerate}
\item Yes.
\item No.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 3(a) SOLUTION START %%%
1
%%% PROBLEM 3(a) SOLUTION END %%%
\fi

\problempart \points{4} Is \texttt{algorithm2} correct?
\begin{enumerate}
\item Yes.
\item No.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 3(b) SOLUTION START %%%
1
%%% PROBLEM 3(b) SOLUTION END %%%
\fi

\problempart \points{4} Is \texttt{algorithm3} correct?
\begin{enumerate}
\item Yes.
\item No.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 3(c) SOLUTION START %%%
1
%%% PROBLEM 3(c) SOLUTION END %%%
\fi

\problempart \points{4} Is \texttt{algorithm4} correct?
\begin{enumerate}
\item Yes.
\item No.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 3(d) SOLUTION START %%%
1
%%% PROBLEM 3(d) SOLUTION END %%%
\fi

\end{problemparts}

\problem \points{16} \textbf{Peak-Finding Efficiency}

\begin{problemparts}

\problempart \points{4} What is the worst-case runtime of \texttt{algorithm1} on a problem of size $n \times n$?
\begin{enumerate}
\item $\Theta(\log n)$.
\item $\Theta(n)$.
\item $\Theta(n \log n)$.
\item $\Theta(n \log^2 n)$.
\item $\Theta(n^2)$.
\item $\Theta(2^n)$.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 4(a) SOLUTION START %%%
1
%%% PROBLEM 4(a) SOLUTION END %%%
\fi

\problempart \points{4} What is the worst-case runtime of \texttt{algorithm2} on a problem of size $n \times n$?
\begin{enumerate}
\item $\Theta(\log n)$.
\item $\Theta(n)$.
\item $\Theta(n \log n)$.
\item $\Theta(n \log^2 n)$.
\item $\Theta(n^2)$.
\item $\Theta(2^n)$.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 4(b) SOLUTION START %%%
1
%%% PROBLEM 4(b) SOLUTION END %%%
\fi

\problempart \points{4} What is the worst-case runtime of \texttt{algorithm3} on a problem of size $n \times n$?
\begin{enumerate}
\item $\Theta(\log n)$.
\item $\Theta(n)$.
\item $\Theta(n \log n)$.
\item $\Theta(n \log^2 n)$.
\item $\Theta(n^2)$.
\item $\Theta(2^n)$.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 4(c) SOLUTION START %%%
1
%%% PROBLEM 4(c) SOLUTION END %%%
\fi

\problempart \points{4} What is the worst-case runtime of \texttt{algorithm4} on a problem of size $n \times n$?
\begin{enumerate}
\item $\Theta(\log n)$.
\item $\Theta(n)$.
\item $\Theta(n \log n)$.
\item $\Theta(n \log^2 n)$.
\item $\Theta(n^2)$.
\item $\Theta(2^n)$.
\end{enumerate}

\ifsolution \solution{}
%%% PROBLEM 4(d) SOLUTION START %%%
1
%%% PROBLEM 4(d) SOLUTION END %%%
\fi

\end{problemparts}

\problem \points{19} \textbf{Peak-Finding Proof}

Please modify the proof below to construct a proof of correctness
for the \emph{most efficient correct algorithm}
among \texttt{algorithm2}, \texttt{algorithm3}, and \texttt{algorithm4}.

The following is the proof of correctness
for \texttt{algorithm1},
which was sketched in Lecture 1.

\begin{quote}
We wish to show that \texttt{algorithm1}
will always return a peak,
as long as the problem is not empty.
To that end,
we wish to prove the following two statements:

{\bf 1. If the peak problem is not empty,
then \texttt{algorithm1} will always return a location.}
Say that we start with a problem of size $m \times n$.
The recursive subproblem examined by \texttt{algorithm1}
will have dimensions
$m \times \lfloor n / 2 \rfloor$ or
$m \times \left(n - \lfloor n / 2 \rfloor - 1 \right)$.
Therefore, the number of columns in the problem
strictly decreases with each recursive call
as long as $n > 0$.
So \texttt{algorithm1} either returns a location at some point,
or eventually examines a subproblem with a non-positive
number of columns.
The only way for the number of columns to become strictly negative,
according to the formulas that determine the size of the subproblem,
is to have $n = 0$ at some point.
So if \texttt{algorithm1} doesn't return a location,
it must eventually examine an empty subproblem.

We wish to show that there is no way that this can occur.
Assume, to the contrary,
that \texttt{algorithm1} does examine an empty subproblem.
Just prior to this,
it must examine a subproblem of size
$m \times 1$ or $m \times 2$.
If the problem is of size $m \times 1$,
then calculating the maximum of the central column
is equivalent to calculating the maximum of the entire problem.
Hence, the maximum that the algorithm finds must be a peak,
and it will halt and return the location.
If the problem has dimensions $m \times 2$,
then there are two possibilities:
either the maximum of the central column is a peak
(in which case the algorithm will halt and return the location),
or it has a strictly better neighbor in the other column
(in which case the algorithm will recurse
on the non-empty subproblem with dimensions $m \times 1$,
thus reducing to the previous case).
So \texttt{algorithm1} can never recurse into an empty subproblem,
and therefore \texttt{algorithm1} must eventually return a location.

{\bf 2. If \texttt{algorithm1} returns a location,
it will be a peak in the original problem.}
If \texttt{algorithm1} returns a location $(r_1, c_1)$,
then that location must have the best value in column $c_1$,
and must have been a peak within some recursive subproblem.
Assume, for the sake of contradiction,
that $(r_1, c_1)$ is not also a peak within the original problem.
Then as the location $(r_1, c_1)$ is passed up the chain of recursive calls,
it must eventually reach a level where it stops being a peak.
At that level, the location $(r_1, c_1)$
must be adjacent to the dividing column $c_2$ (where $|c_1 - c_2| = 1$),
and the values must satisfy the inequality
$val(r_1, c_1) < val(r_1, c_2)$.

Let $(r_2, c_2)$ be
the location of the maximum value found by \texttt{algorithm1}
in the dividing column.
As a result, it must be that $val(r_1, c_2) \le val(r_2, c_2)$.
Because the algorithm chose to recurse
on the half containing $(r_1, c_1)$,
we know that $val(r_2, c_2) < val(r_2, c_1)$.
Hence, we have the following chain of inequalities:
$$val(r_1, c_1) < val(r_1, c_2) \le val(r_2, c_2) < val(r_2, c_1)$$
But in order for \texttt{algorithm1} to return $(r_1, c_1)$ as a peak,
the value at $(r_1, c_1)$ must have been the greatest in its column,
making $val(r_1, c_1) \ge val(r_2, c_1)$.
Hence, we have a contradiction.
\end{quote}

\ifsolution \solution{}
%%% PROBLEM 5 SOLUTION START %%%
Write your proof here.
%%% PROBLEM 5 SOLUTION END %%%
\fi

\problem \points{19} \textbf{Peak-Finding Counterexamples}

For each incorrect algorithm,
upload a Python file giving a counterexample
(i.e. a matrix for which the algorithm returns a location
that is not a peak).

\ifsolution \solution{}
%%% PROBLEM 6 SOLUTION START %%%
\begin{verbatim}
problemMatrix = [
    [0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0]
]

problemMatrix = [
    [0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0],
    [0, 0, 0, 0, 0]
]
\end{verbatim}
%%% PROBLEM 6 SOLUTION END %%%
\fi

\end{problems}

\end{document}

6.006 Introduction to Algorithms MIT OCW/Assignments/ps1/ps1_sol.tex