\documentclass{article}

% start meta data
\title{Naive Bayes and Text Classification I \\ \vspace{2 mm} \large Introduction and Theory}

\author{Sebastian Raschka\\ \texttt{se.raschka@gmail.com}}
\date{October 4, 2014}
% end meta data

% general packages
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{cite}
\usepackage{amsmath}
\usepackage[font=small,labelfont=bf]{caption}
\usepackage{float}

%TOC
%\setcounter{secnumdepth}{0}

% start header and footer
\usepackage{fancyhdr}
\pagestyle{fancy}
\lhead{Sebastian Raschka}
\rhead{Naive Bayes and Text Classification I}
\cfoot{\thepage} % centered footer
\renewcommand{\headrulewidth}{0.4pt}
\renewcommand{\footrulewidth}{0.4pt}
% end header and footer

\begin{document} % start main document

\maketitle % makes the title page from meta data
\tableofcontents

\section{Introduction}
\label{sec:introduction}

Starting more than half a century ago, scientists became very serious about addressing the question: "Can we build a model that learns from available data and automatically makes the right decisions and predictions?" Looking back, this sounds almost like a rhetoric question, and the answer can be found in numerous applications that are emerging from the fields of pattern classification, machine learning, and artificial intelligence.

Data from various sensoring devices combined with powerful learning algorithms and domain knowledge led to many great inventions that we now take for granted in our everyday life: Internet queries via search engines like Google, text recognition at the post office, barcode scanners at the supermarket, the diagnosis of diseases, speech recognition by Siri or Google Now on our mobile phone, just to name a few.

One of the sub-fields of \emph{predictive modeling} is \emph{supervised pattern classification}; supervised pattern classification is the task of training a model based on labeled training data which then can be used to assign a pre-defined class label to new objects. One example that we will explore throughout this article is spam filtering via naive Bayes classifiers in order to predict whether a new text message can be categorized as spam or not-spam.
Naive Bayes classifiers, a family of classifiers that are based on the popular Bayes' probability theorem, are known for creating simple yet well performing models, especially in the fields of document classification and disease prediction.

\begin{figure}[h!]
\includegraphics[width=\linewidth]{../images/learning_algorithm_1.png}
\caption{A simplified diagram of the general model building procedure for pattern classification.}
\label{fig:learning_model}
\end{figure}

A more detailed overview of predictive modeling can be found in my previous article \emph{\href{http://sebastianraschka.com/Articles/2014_intro_supervised_learning.html}{Predictive Modeling, Supervised Machine Learning, and Pattern Classification - The Big Picture}}.

\section{Naive Bayes Classification}
\label{sec:naive_bayes_classification}

\subsection{Overview}
\label{sec:overview}

\emph{Naive Bayes classifiers} are linear classifiers that are known for being simple yet very efficient. The probabilistic model of naive Bayes classifiers is based on Bayes' theorem, and the adjective \emph{naive} comes from the assumption that the features in a dataset are mutually independent. In practice, the independence assumption is often violated, but naive Bayes classifiers still tend to perform very well under this unrealistic assumption \cite{rish2001empirical}. Especially for small sample sizes, naive Bayes classifiers can outperform the more powerful alternatives \cite{domingos1997optimality}.

Being relatively robust, easy to implement, fast, and accurate, naive Bayes classifiers are used in many different fields. Some examples include the diagnosis of diseases and making decisions about treatment processes \cite{kazmierska2008application}, the classification of RNA sequences in taxonomic studies \cite{wang2007naive}, and spam filtering in e-mail clients \cite{sahami1998bayesian}.
However, strong violations of the independence assumptions and non-linear classification problems can lead to very poor performances of naive Bayes classifiers.
We have to keep in mind that the type of data and the type problem to be solved dictate which classification model we want to choose. In practice, it is always recommended to compare different classification models on the particular dataset and consider the prediction performances as well as computational efficiency.

In the following sections, we will take a closer look at the probability model of the naive Bayes classifier and apply the concept to a simple toy problem. Later, we will use a publicly available SMS (text message) collection to train a naive Bayes classifier in Python that allows us to classify unseen messages as spam or ham.

\begin{figure}[h!]
\includegraphics[width=\linewidth]{../images/linear_vs_nonlinear_problems.png}
\caption{Linear (A) vs. non-linear problems (B). Random samples for two different classes are shown as colored spheres, and the dotted lines indicate the class boundaries that classifiers try to approximate by computing the decision boundaries.
A non-linear problem (B) would be a case where linear classifiers, such as naive Bayes, would not be suitable since the classes are not linearly separable. In such a scenario, non-linear classifiers (e.g.,instance-based nearest neighbor classifiers) should be preferred.}
\label{fig:nonlinear_probs}
\end{figure}

\subsection{Posterior Probabilities}
\label{sec:posterior_probabilities_1}

In order to understand how naive Bayes classifiers work, we have to briefly recapitulate the concept of Bayes' rule.
The probability model that was formulated by Thomas Bayes (1701-1761) is quite simple yet powerful; it can be written down in simple words as follows:

\begin{equation} \text{posterior probability} = \frac{\text{conditional probability} \cdot \text{prior probability}}{\text{evidence}} \end{equation}

 Bayes' theorem forms the core of the whole concept of naive Bayes classification. The \emph{posterior probability}, in the context of a classification problem, can be interpreted as: "What is the probability that a particular object belongs to class $i$ given its observed feature values?" A more concrete example would be: "What is the probability that a person has diabetes given a certain value for a pre-breakfast blood glucose measurement and a certain value for a post-breakfast blood glucose measurement?"

\begin{equation}  P(\text{diabetes} \mid \textbf x_i) \;, \quad \textbf x_i = [90 \text{mg/dl}, 145 \text{mg/dl}] \end{equation}

 Let

\begin{itemize}
	\item $\textbf x_i$ be the feature vector of sample $i, \; i \in \{1, 2, ..., n\}$,
	\item $\omega_j$ be the notation of class $j, \; j \in \{1, 2, ..., m\}$,
	\item and $P(\textbf x_i \mid \omega_j)$ be the probability of observing sample $\textbf x_i$ given that is belongs to class $\omega_j$.
\end{itemize}

 The general notation of the posterior probability can be written as

\begin{equation} P(\omega_j \mid \textbf x_i) = \frac{P(\textbf x_i \mid \omega_j) \cdot P(\omega_j)}{P(\textbf x_i)} \end{equation}

The objective function in the naive Bayes probability is to maximize the posterior probability given the training data in order to formulate the decision rule.

\begin{equation} \text{predicted class label} \leftarrow \underset{j = 1 ..., m} {\text{arg max}} \; P(\omega_{j} \mid \textbf x_i)   \end{equation}

To continue with our example above, we can formulate the decision rule based on the posterior probabilities as follows:

\begin{equation}
\begin{split}
   & \text{person has diabetes if} \\
   & P(\text{diabetes} \mid \textbf x_i) \ge P(\text{not-diabetes} \mid \textbf x_i), \\
   & \text{else classify person as healthy}.
\end{split}
   \end{equation}

\subsection{Class-conditional Probabilities}
\label{sec:class-conditional_probabilities_1}

One assumption that Bayes classifiers make is that the samples are \emph{i.i.d.}
The abbreviation \emph{i.i.d.} stands for "independent and identically distributed" and describes random variables that are independent from one another and are drawn from a similar probability distribution. Independence means that the probability of one observation does not affect the probability of another observation (e.g., time series and network graphs are not independent).  One popular example of  \emph{i.i.d.} variables is the classic coin tossing: The first coin flip does not affect the outcome of a second coin flip and so forth. Given a fair coin, the probability of the coin landing on "heads" is always 0.5 no matter of how often the coin if flipped.

An additional assumption of naive Bayes classifiers is the \emph{conditional independence} of features. Under this \emph{naive} assumption, the \emph{class-conditional probabilities} or (\emph{likelihoods}) of the samples can be directly estimated from the training data instead of evaluating all possibilities of \textbf{x}. Thus, given a $d$-dimensional feature vector \textbf{x}, the class conditional probability can be calculated as follows:

\begin{equation} P(\textbf x \mid \omega_j) = P(x_1 \mid \omega_j) \cdot P(x_2 \mid \omega_j) \cdot \ldots \cdot P(x_d \mid \omega_j) =  \prod_{k=1}^{d} P( x_k \mid \omega_j) \end{equation}

Here, $P(\textbf x \mid \omega_j)$ simply means: "How likely is it to observe this particular pattern $\textbf x$ given that it belongs to class $ \omega_j$?" The "individual" likelihoods for every feature in the feature vector can be estimated via the maximum-likelihood estimate, which is simply a frequency in the case of categorical data:

\begin{equation} \hat{P}(x_i \mid \omega_j) = \frac{N_{x_i, \omega_j}}{N_{\omega_j}}  \quad (i = (1, ..., d))\end{equation}

\begin{itemize}
	\item $N_{x_i, \omega_j}$: Number of times feature $x_i$ appears in samples from class $\omega_j$.
	\item  $N_{\omega_j}$: Total count of all features in class $\omega_j$.
\end{itemize}

To illustrate this concept with an example, let's assume that we have a collection of 500 documents where 100 documents are \emph{spam} messages. Now, we want to calculate the class-conditional probability for a new message "Hello World" given that it is spam.
Here, the pattern consists of two features: "hello" and "world," and the class-conditional probability is the product of the "probability of encountering 'hello' given the message is spam" --- the probability of encountering "world" given the message is spam."

\begin{equation} P(\textbf x=[\text{hello, world}] \mid \omega=\text{spam}) = P(\text{hello} \mid \text{spam}) \cdot P(\text{world} \mid \text{spam}) \end{equation}

Using the training dataset of 500 documents, we can use the maximum-likelihood estimate to estimate those probabilities: We'd simply calculate how often the words occur in the corpus of all spam messages. E.g.,

\begin{equation} \hat{P}(\textbf x=[\text{hello, world}] \mid \omega=\text{spam}) = \frac{20}{100} \cdot \frac{2}{100} = 0.004 \end{equation}

However, with respect to the \emph{naive} assumption of conditional independence, we notice a problem here: The \emph{naive} assumption is that a particular word does not influence the chance of encountering other words in the same document. For example, given the two words  "peanut" and "butter" in a text document, intuition tells us that this assumption is obviously violated: If a document contains the word "peanut" it will be more likely that it also contains the word "butter" (or "allergy"). In practice, the conditional independence assumption is indeed often violated, but naive Bayes classifiers are known to perform still well in those cases \cite{zhang2004optimality}.

\subsection{Prior Probabilities}
\label{sec:prior_probabilities_1}

In contrast to a frequentist's approach, an additional \emph{prior probability} (or just \emph{prior}) is introduced that can be interpreted as the \emph{prior belief} or \emph{a priori} knowledge.

\begin{equation} \text{posterior probability} = \frac{\text{conditional probability} \cdot \text{prior probability}}{\text{evidence}} \end{equation}

In the context of pattern classification, the prior probabilities are also called \emph{class priors}, which describe "the general probability of encountering a particular class." In the case of spam classification, the priors could be formulated as

\begin{equation}P(\text{spam})=\text{"the probability that any new message is a spam message"} \end{equation} and

\begin{equation} P(\text{ham})= 1-P(\text{spam}). \end{equation}

If the priors are following a uniform distribution, the posterior probabilities will be entirely determined by the class-conditional probabilities and the evidence term. And since the evidence term is a constant, the decision rule will entirely depend on the class-conditional probabilities (similar to a frequentist's approach and maximum-likelihood estimate).

Eventually, the \emph{a priori} knowledge can be obtained, e.g., by consulting a domain expert or by estimation from the training data (assuming that the training data is \emph{i.i.d.} and a representative sample of the entire population. The maximum-likelihood estimate approach can be formulated as

\begin{equation} \hat{P}(\omega_j) = \frac{N_{\omega_j}}{N_c}  \end{equation}

\begin{itemize}
	\item $N_{\omega_j}$: Count of samples from class $\omega_j$.
	\item $N_c$: Count of all samples.
\end{itemize}

And in context of  \emph{spam classification}:

\begin{equation}\hat{P}(\text{spam}) = \frac{\text{\# of spam messages in training data}}{\text{ \# of all messages in training data}} \end{equation}

Figure \ref{fig:effect_priors} illustrates the effect of the prior probabilities on the decision rule. Given an 1-dimensional pattern \textbf{x} (continuous attribute, plotted as "x" symbols) that follows a normal distribution and belongs to one out of two classes (\emph{blue} and \emph{green}). The patterns from the first class ($\omega_1=\text{blue}$) are drawn from a normal distribution with mean  $x=4$ and a standard deviation $\sigma=1$. The probability distribution of the second class ($\omega_2=\text{green}$) is centered at x=10 with a similar standard deviation of $\sigma=1$. The bell-curves denote the probability densities of the samples that were drawn from the two different normal distributions. Considering only the class conditional probabilities, the maximum-likelihood estimate in this case would be

\begin{equation}
\begin{split}
&P(x=4 \mid \omega_1) \approx 0.4 \text{ and } P(x=10 \mid \omega_1) < 0.001\\
&P(x=4 \mid \omega_2) < 0.001 \text{ and } P(x=10 \mid \omega_2) \approx 0.4.
\end{split}
\end{equation}

Now, given uniform priors, that is $P(\omega_1) = P(\omega_2) = 0.5$, the decision rule would be entirely dependent on those class-conditional probabilities, so that the decision rule would fall directly between the two distributions

\begin{equation}  P(x \mid \omega_1) = P(x \mid \omega_2).\end{equation}

However, if the prior probability was $P(\omega_1) > 0.5$, the decision region of class $\omega_1$ would expand as shown in Figure \ref{fig:effect_priors}. In the context of spam classification, this could be interpreted as encountering a new message that only contains words which are equally likely to appear in \emph{spam} or \emph{ham} messages. In this case, the decision would be entirely dependent on \emph{prior knowledge}, e.g., we could assume that a random message is in 9 out of 10 cases not \emph{spam} and therefore classify the new message as \emph{ham}.

\begin{figure}[h!]
\includegraphics[width=\linewidth]{../images/effect_priors_1.png}
\caption{The effect of prior probabilities on the decision regions. The figure shows an 1-dimensional random sample from two different classes (blue and green crosses). The data points of both the blue and the green class are normally distributed with standard deviation 1, and the bell curves denote the class-conditional probabilities. If the class priors are equal, the decision boundary of a naive Bayes classifier is placed at the center between both distributions (gray bar). An increase of the prior probability of the blue class ($\omega_1$) leads to an extension of the decision region R1 by moving the decision boundary (blue-dotted bar) towards the other class and vice versa.}
\label{fig:effect_priors}
\end{figure}

\subsection{Evidence}
\label{sec:evidence}

After defining the \emph{class-conditional probability} and \emph{prior probability}, there is only one term missing in order to compute \emph{posterior probability}, that is the \emph{evidence}.

\begin{equation} \text{posterior probability} = \frac{\text{conditional probability} \cdot \text{prior probability}}{\text{evidence}} \end{equation}

The evidence $P(\textbf x)$ can be understood as the probability of encountering a particular pattern $\textbf x$ independent from the class label. Given the more formal definition of posterior probability
\begin{equation} P(\omega_j \mid \textbf x_i) = \frac{P(\textbf x_i \mid \omega_j) \cdot P(\omega_j)}{P(\textbf x_i)}, \end{equation}

the evidence can be calculated as follows ($\omega_j^C$ stands for "complement" and basically translates to "\textbf{not} class $\omega_j$." ):

\begin{equation}P(\textbf x_i) = P(\textbf x_i \mid \omega_j) \cdot P(\omega_j) + P(\textbf x_i \mid \omega_j^C) \cdot P(\omega_j^C)\end{equation}

 Although the evidence term is required to accurately calculate the posterior probabilities, it can be removed from the \emph{decision rule} "Classify sample $\textbf x_i$ as $\omega_1$ if $P(\omega_1 \mid \textbf x_i) > P(\omega_2 \mid \textbf x_i)$ else classify the sample as $\omega_2$,"   since it is merely a scaling factor:

\begin{equation}  \frac{P(\textbf x_i \mid \omega_1) \cdot P(\omega_1)}{P(\textbf x_i)} > \frac{P(\textbf x_i \mid \omega_2) \cdot P(\omega_2)}{P(\textbf x_i)} \end{equation}

\begin{equation}\propto P(\textbf x_i \mid \omega_1) \cdot P(\omega_1) > P(\textbf x_i \mid \omega_2) \cdot P(\omega_2)\end{equation}

\subsection{Multinomial Naive Bayes - A Toy Example}
\label{sec:multinomial_naive_bayes-a_toy_example}

After covering the basics concepts of a naive Bayes classifier, the \emph{posterior probabilities} and \emph{decision rules}, let us walk through a simple toy example based on the training set shown in Figure \ref{fig:toy_dataset}.

\begin{figure}[h!]
\includegraphics[scale=0.5]{../images/toy_dataset_1.png}
\caption{A simple toy dataset of 12 samples 2 different classes $+, -$  . Each sample consists of 2 features: color and geometrical shape.}
\label{fig:toy_dataset}
\end{figure}

Let

\begin{itemize}
	\item $\omega_j$ be the class labels: $\omega_j \in \{+, -\}$
	\item and $\textbf x_i$  be the 2-dimensional feature vectors: $\textbf x_i = [x_{i1} \; x_{i2}], \quad x_{i1} \in \{ \text{blue}, \text{green}, \text{red}, \text{yellow} \}, \quad x_{i2} \in \{\text{circle}, \text{square} \}.$
\end{itemize}

The 2 class labels are $\omega_j \in \{+, -\}$ and the feature vector for sample $i$ can be written as

\begin{equation}
\begin{split}
& \textbf{x}_i = [x_{i1} \; x_{i2}] \\
& \text{for } i \in \{1, 2, ...,  n\}, \; \text{ with } n=12 \\
& \text{and } x_{i1} \in \{ \text{blue}, \text{green}, \text{red}, \text{yellow} \}, \quad x_{i2} \in \{\text{circle}, \text{square} \}
\end{split}
\end{equation}

 The task now is to classify a new sample --- pretending that we don't know that its true class label is "+":

 \begin{figure}[h!]
\includegraphics[scale=0.5]{../images/toy_dataset_2.png}
\caption{A new sample from class $+$ and the features $\textbf{x} = \text{[blue, square]}$ that is to be classified using the training data in Figure \ref{fig:toy_dataset}.}
\label{fig:new_sample1}
\end{figure}

\subsubsection{Maximum-Likelihood Estimates}

The \emph{decision rule} can be defined as

\begin{equation}
\begin{split}
& \text{Classify sample as } +  \text{ if}   \\
& P(\omega=\text{+} \mid \textbf x = \text{[blue, square]}) \geq P(\omega=\text{-} \mid \textbf x = \text{[blue, square]})\\
& \text{else classify sample as} -.
\end{split}
\end{equation}

Under the assumption that the samples are \emph{i.i.d}, the  \emph{prior probabilities} can be obtained via the maximum-likelihood estimate (i.e., the frequencies of how often each class label is represented in the training dataset):

\begin{equation}
\begin{split}
& P(\text{+}) = \frac{7}{12} = 0.58 \\
& P(\text{-}) = \frac{5}{12} = 0.42
\end{split}
\end{equation}

Under the \emph{naive} assumption that the features "color" and "shape" are mutually independent, the \emph{class-conditional probabilities} can be calculated as a simple product of the individual conditional probabilities.

Via maximum-likelihood estimate, e.g., $P(\text{blue} \mid -)$ is simply the frequency of observing a "blue" sample among all samples in the training dataset that belong to class $-$.

\begin{equation}
\begin{split}
& P(\textbf{x} \mid +) = P(\text{blue} \mid +) \cdot P(\text{square} \mid +) = \frac{3}{7} \cdot \frac{5}{7} = 0.31  \\
& P(\textbf{x} \mid -) = P(\text{blue} \mid -) \cdot P(\text{square} \mid -) = \frac{3}{5} \cdot \frac{3}{5} = 0.36
\end{split}
\end{equation}

Now,  the \emph{posterior probabilities} can be simply calculated as the product of the class-conditional and prior probabilities:

\begin{equation}
\begin{split}
& P(+ \mid \textbf{x}) = P(\textbf{x} \mid +) \cdot P(+) = 0.31  \cdot  0.58 = 0.18  \\
& P(- \mid \textbf{x}) = P(\textbf{x} \mid -) \cdot P(-) = 0.36  \cdot  0.42 = 0.15
\end{split}
\end{equation}

\subsubsection{Classification}

Putting it all together, the new sample can be classified by plugging in the posterior probabilities into the decision rule:

\begin{equation}
\begin{split}
&\text{If } P(+ \mid \textbf x) \geq P(\text{-} \mid \textbf{x})  \\
&\text{classify as }+, \\
& \text{else } \text{classify as } -
\end{split}
\end{equation}

Since $0.18 > 0.15$  the sample can be classified as $+$. Taking a closer look at the calculation of the posterior probabilities, this simple example demonstrates the effect of the prior probabilities affected on the decision rule. If the prior probabilities were equal for both classes, the new pattern would be classified as $-$ instead of $+$. This observation also underlines the importance of \emph{representative} training datasets; in practice, it is usually recommended to additionally consult a domain expert in order to define the prior probabilities.

\subsubsection{Additive Smoothing}
\label{sec:additive_smoothing}

The classification was straight-forward given the sample in Figure \ref{fig:new_sample1}. A trickier case is a sample that has a "new" value for the color attribute that is not present in the training dataset, e.g., \emph{yellow}, as shown in Figure \ref{fig:new_sample1}.

 \begin{figure}[h!]
\includegraphics[scale=0.5]{../images/toy_dataset_3.png}
\caption{A new sample from class $+$ and the features $\textbf{x} = \text{[yellow, square]}$ that is to be classified using the training data in Figure \ref{fig:toy_dataset}.}
\label{fig:new_sample2}
\end{figure}

If the color \emph{yellow} does not appear in our training dataset, the class-conditional probability will be 0, and as a consequence, the posterior probability will also be 0 since the posterior probability is the product of the prior and class-conditional probabilities.

\begin{equation}
\begin{split}
& P(\omega_1 \mid \textbf x) = 0  \cdot  0.42 = 0 \\
& P(\omega_2 \mid \textbf x) = 0  \cdot  0.58 = 0
\end{split}
\end{equation}

In order to avoid the problem of \emph{zero} probabilities, an additional smoothing term can be added to the \emph{multinomial Bayes} model. The most common variants of additive smoothing are the so-called \emph{Lidstone smoothing} ($\alpha<1$) and  \emph{Laplace smoothing} ($\alpha=1$).

\begin{equation} \hat{P}(x_i \mid \omega_j) = \frac{N_{x_i, \omega_j}+\alpha}{N_{\omega_j} + \alpha \, d}  \quad (i = (1, ..., d))\end{equation}

where

\begin{itemize}
\item $N_{x_i, \omega_j}$: Number of times feature $x_i$ appears in samples from class $\omega_j$.
\item  $N_{\omega_j}$: Total count of all features in class $\omega_j$.
\item $\alpha$: Parameter for additive smoothing.
\item $d$: Dimensionality of the feature vector $\textbf x = [x_1, ..., x_d]$.

\end{itemize}

 \section{Naive Bayes and Text Classification}

This section will introduce some of the main concepts and procedures that are needed to apply the naive Bayes model to text classification tasks. Although the examples are mainly concerning a two-class problem --- classifying text messages as \emph{spam} or \emph{ham} --- the same approaches are applicable to multi-class problems such as classification of documents into different topic areas (e.g., "Computer Science", "Biology", "Statistics", "Economics", "Politics", etc.).

\subsection{The Bag of Words Model}
\label{sec:the_bag_of_words_model}

One of the most important sub-tasks in pattern classification are \emph{feature extraction} and \emph{selection}; the three main criteria of good features are listed below:

\begin{itemize}
	\item \emph{Salient}. The features are important and meaningful with respect to the problem domain.
	\item \emph{Invariant}. Invariance is often described in context of image classification: The features are insusceptible to distortion, scaling, orientation, etc. A nice example is given by C. Yao \emph{et al.} in \emph{Rotation-Invariant Features for Multi-Oriented Text Detection in Natural Images} \cite{yao2013rotation}.
	\item \emph{Discriminatory.} The selected features bear enough information to distinguish well between patterns when used to train the classifier.
\end{itemize}

Prior to fitting the model and using machine learning algorithms for training, we need to think about how to best represent a text document as a feature vector. A commonly used model in \emph{Natural Language Processing} is the so-called \emph{bag of words} model. The idea behind this model really is as simple as it sounds. First comes the creation of the \emph{vocabulary} --- the collection of all different words that occur in the training set and each word is associated with a count of how it occurs. This vocabulary can be understood as a set of non-redundant items where the order doesn't matter. Let $D_1$ and $D_2$ be two documents in a training dataset:

\begin{itemize}
	\item $D_1$: "Each state has its own laws."
	\item $D_2$: "Every country has its own culture."
\end{itemize}

Based on these two documents, the vocabulary could be written as \\

\begin{equation}
\begin{split}
&V= \{each: 1, state: 1, has: 2, its: 2, own: 2, \\
& laws: 1, every: 1, country: 1, culture: 1\}
\end{split}
\end{equation}

The vocabulary can then be used to construct the $d$-dimensional feature vectors for the individual documents where the dimensionality is equal to the number of different words in the vocabulary ($d=|V|$). This process is called \emph{vectorization}.

\begin{table}[h]

\caption{Bag of words representation of two sample documents $D_1$ and $D_2$.}
\begin{tabular}{l l l l l l l l l l l} \hline
& each   & state & has & its & own & laws & every & country & culture &   \\ \hline
$\textbf{x}_{D1}$             & 1     & 1   & 1   & 1   & 1    & 1     & 0       & 0       & 0  \\
$\textbf{x}_{D2}$             & 0     & 0   & 1   & 1   & 1    & 0     & 1       & 1       & 1 \\
$\sum$ & 1     & 1   & 2   & 2   & 2    & 1     & 1       & 1       & 1 \\ \hline
\end{tabular}
\label{fig:bag_of_words}
\end{table}

Given the example in Table \ref{fig:bag_of_words} one question is whether the 1s and 0s of the feature vectors are binary counts (1 if the word occurs in a particular document, 0 otherwise) or absolute counts (how often the word occurs in each document). The answer depends  on which probabilistic model is used for the naive Bayes classifier: The \emph{Multinomial} or \emph{Bernoulli} model --- more on the probabilistic models in Section \ref{sec:bernoulli_bayes} and Section \ref{sec:multinomial_bayes}.

\subsubsection{Tokenization}
\label{sec:tokenization}

\emph{Tokenization} describes the general process of breaking down a text corpus into individual elements that serve as input for various natural language processing algorithms. Usually, tokenization is accompanied by other optional processing steps, such as the removal of stop words and punctuation characters, stemming or lemmatizing, and the construction of \emph{n-grams}. Below is an example of a simple but typical tokenization step that splits a sentence into individual words, removes punctuation, and converts all letters to lowercase.

\begin{table}[H]
\caption{Example of tokenization.}
\begin{center}
\begin{tabular}{ | c | }

\hline
A swimmer likes swimming, thus he swims. \\ \hline
\end{tabular}

$\downarrow$

\begin{tabular}{ | c | c | c |  c | c | c | c |}
\hline
a & swimmer & likes & swimming & thus & he & swims \\ \hline
\end{tabular}
\end{center}

\end{table}

\subsubsection{Stop Words}
\label{sec:stopwords}

\emph{Stop words} are words that are particularly common in a text corpus and thus considered as rather un-informative (e.g., words such as \emph{so}, \emph{and}, \emph{or}, \emph{the}, ..."). One approach to stop word removal is to search against a language-specific stop word dictionary. An alternative approach is to create a  \emph{stop list} by sorting all words in the entire text corpus by frequency. The stop list --- after conversion into a \emph{set} of non-redundant words --- is then used to remove all those words from the input documents that are ranked among the top \emph{n} words in this stop list.

\begin{table}[H]
\caption{Example of stop word removal.}
\begin{center}
\begin{tabular}{ | c | }
\hline
A swimmer likes swimming, thus he swims. \\ \hline
\end{tabular}

$\downarrow$

\begin{tabular}{ | c | c | c |  c | c | c | }
\hline
swimmer & likes & swimming & , &  swims &. \\ \hline
\end{tabular}
\end{center}
\end{table}

\subsubsection{Stemming and Lemmatization}
\label{sec:stemming_and_lemmatization}

\emph{Stemming} describes the process of transforming a word into its root form. The original stemming algorithm was developed my Martin F. Porter in 1979 and is hence known as \emph{Porter stemmer} \cite{porter1980algorithm}.

\begin{table}[H]
\caption{Example of Porter stemming.}
\begin{center}
\begin{tabular}{ | c | }
\hline
A swimmer likes swimming, thus he swims. \\ \hline
\end{tabular}

$\downarrow$

\begin{tabular}{ | c | c | c |  c | c | c | c | c | c | }
\hline
a & swimmer &  like & swim & , & thu & he & swim & .\\ \hline
\end{tabular}
\end{center}
\end{table}

Stemming can create non-real words, such as "thu" in the example above. In contrast to stemming, \emph{lemmatization} aims to obtain the canonical (grammatically correct) forms of the words, the so-called \emph{lemmas}. Lemmatization is computationally more difficult and expensive than stemming, and in practice, both stemming and lemmatization have little impact on the performance of text classification \cite{toman2006influence}.

\begin{table}[H]
\caption{Example of lemmatization.}
\begin{center}
\begin{tabular}{ | c | }
\hline
A swimmer likes swimming, thus he swims. \\ \hline
\end{tabular}

$\downarrow$

\begin{tabular}{ | c | c | c |  c | c | c | c | c | c | }
\hline
A & swimmer &  like & swimming & , & thus & he & swim & .\\ \hline
\end{tabular}
\end{center}
\end{table}

The stemming and lemmatization examples were created by using the Python NLTK library (\href{http://www.nltk.org}{http://www.nltk.org}).

\subsubsection{\emph{N}-grams}
\label{sec:n-grams}

In the \emph{n-gram} model, a token can be defined as a sequence of \emph{n} items. The simplest case is the so-called \emph{unigram} (1-gram) where each word consists of exactly one word, letter, or symbol. All previous examples were unigrams so far. Choosing the optimal number \emph{n} depends on the language as well as the particular application. For example, Andelka Zecevic found in his study that \emph{n}-grams with $3 \le n \le 7$ were the best choice to determine authorship of Serbian text documents \cite{zevcevic2011n}. In a different study, the \emph{n}-grams of size $4 \le n \le 8$ yielded the highest accuracy in authorship determination of English text books \cite{kevselj2003n} and Kanaris \emph{e. al.} report that \emph{n}-grams of size 3 and 4 yield good performances in anti-spam filtering of e-mail messages  \cite{kanaris2007words}.

\begin{itemize}
\item unigram (1-gram):

\begin{tabular}{ | c | c | c |  c | c | c | c | c | c | }
\hline
a & swimmer &  likes & swimming & thus & he & swims\\ \hline
\end{tabular}

\item bigram (2-gram):

\begin{tabular}{ | c | c | c |  c | c | }
\hline
a swimmer & swimmer likes &  likes swimming & swimming thus & ... \\ \hline
\end{tabular}

\item trigram (3-gram):

\begin{tabular}{ | c | c | c |  c | c | }
\hline
a swimmer likes & swimmer likes swimming &  likes swimming thus &  ... \\ \hline
\end{tabular}

\end{itemize}

\subsection{The Decision Rule for Spam Classification}
\label{sec:decision_rule_spam}

In context of spam classification the decision rule of a naive Bayes classifier based on the posterior probabilities can be expressed as

\begin{equation}
\begin{split}
    & \text{if } P(\textbf{x} \mid \omega = \text{spam}) \ge P(\textbf{x} \mid \omega = \text{ham})  \text{ classify as spam, }\\
    &  \text{else classify as ham. }
\end{split}
\end{equation}

As described in Section \ref{sec:posterior_probabilities_1} the posterior probability is the product of the class-conditional probability and the prior probability; the evidence term in the denominator can be dropped since it is constant for both classes.

\begin{equation}
\begin{split}
    & P(\textbf{x} \mid \omega = \text{spam}) = P(\omega = \text{spam} \mid \textbf{x}) \cdot P(\text{spam}) \\
    & P(\textbf{x} \mid \omega = \text{ham}) = P(\omega = \text{ham} \mid \textbf{x}) \cdot P(\text{ham})
\end{split}
\end{equation}

The prior probabilities can be obtained via the maximum-likelihood estimate based on the frequencies of spam and ham messages in the training dataset:

\begin{equation}
\begin{split}
  & \hat{P}(\omega = \text{spam}) = \frac {\text{\# of spam msg.}}{\text{\# of all msg.}} \\
   &  \hat{P}(\omega = \text{ham}) = \frac {\text{\# of ham msg.}}{\text{\# of all msg.}}
\end{split}
\end{equation}

Assuming that the words in every document are conditionally independent (according to the \emph{naive} assumption), two different models can be used to compute the class-conditional probabilities: The \emph{Multi-variate Bernoulli} model (Section \ref{sec:bernoulli_bayes}) and the \emph{Multinomial} model (Section \ref{sec:multinomial_bayes}).

\subsection{Multi-variate Bernoulli Naive Bayes}
\label{sec:bernoulli_bayes}

The \emph{Multi-variate Bernoulli} model is based on binary data: Every token in the feature vector of a document is associated with the value 1 or 0. The  feature vector has $m$ dimensions where $m$ is the number of words in the whole vocabulary (in Section \ref{sec:the_bag_of_words_model}); the value 1 means that the word occurs in the particular document, and 0 means that the word does
not occur in this document. The Bernoulli trials can be written as

\begin{equation} P(\textbf x \mid \omega_j) = \prod_{i=1}^{m} P(x_i \mid \omega_j)^b \cdot (1-P(x_i \mid  \omega_j))^{(1-b)}  \quad (b \in {0,1}).
\end{equation}

Let  $\hat{P}(x_i \mid \omega_j)$ be the maximum-likelihood estimate that a particular word (or token) $x_i$ occurs in class $\omega_j$.

\begin{equation} \hat{P}(x_i \mid \omega_j) = \frac{df_{xi, y} + 1}{df_y + 2} \end{equation}

where

\begin{itemize}
\item $df_{xi, y}$ is the number of documents in the training dataset that contain the feature $x_i$ and belong to class $\omega_j$.
\item  $df_y$ is the number of documents in the training dataset that belong to class $\omega_j$.
\item  +1 and +2 are the parameters of \emph{Laplace smoothing} (Section \ref{sec:additive_smoothing}).
\end{itemize}

\subsection{Multinomial Naive Bayes}
\label{sec:multinomial_bayes}

\subsubsection{Term Frequency}
\label{sec:term_frequency}

A alternative approach to characterize text documents --- rather than binary values ---  is the \emph{term frequency (tf(t, d))}. The term frequency is typically defined as the number of times a given term \emph{t} (i.e., word or token) appears in a document \emph{d} (this approach is sometimes also called \emph{raw frequency}). In practice, the term frequency is often normalized by dividing the raw term frequency by the document length.

\begin{equation}\text{normalized term frequency} = \frac{tf(t, d)}{n_d}\end{equation}

where

\begin{itemize}
\item  $tf(t, d)$: Raw term frequency (the count of term $t$ in document $d$).
\item $n_d$: The total number of terms in document $d$.
\end{itemize}

The term frequencies can then be used to compute the maximum-likelihood estimate based on the training data to estimate the class-conditional probabilities in the multinomial model:

\begin{equation} \hat{P}(x_i \mid \omega_j) = \frac{\sum tf(x_i, d \in \omega_j) + \alpha}{\sum N_{d \in \omega j} + \alpha \cdot V} \end{equation}

where

\begin{itemize}
\item  $x_i$: A word from the feature vector $\textbf x$ of a particular sample.
\item  $\sum tf(x_i, d \in \omega_j)$: The sum of raw term frequencies of word $x_i$ from all documents in the training sample that belong to class $\omega_j$.
\item  $\sum N_{d \in \omega j}$: The sum of all term frequencies in the training dataset for class $\omega_j$.
\item $\alpha$: An additive smoothing parameter ($\alpha = 1$ for Laplace smoothing).
\item  $V$: The size of the vocabulary (number of different words in the training set).
\end{itemize}

The class-conditional probability of encountering the text $\textbf x$ can be calculated as the product from the likelihoods of the individual words (under the \emph{naive} assumption of conditional independence).

\begin{equation} P(\textbf x \mid \omega_j) = P(x_1 \mid \omega_j) \cdot P(x_2 \mid \omega_j) \cdot   \dotsc \cdot P(x_n \mid \omega_j) = \prod_{i= 1}^{m}  P(x_i \mid \omega_j)  \end{equation}

\subsubsection{Term Frequency - Inverse Document Frequency (Tf-idf)}
\label{sec:tf-idf}

The \emph{term frequency - inverse document frequency (Tf-idf)} is another alternative for characterizing text documents. It can be understood as a weighted \emph{term frequency}, which is especially useful if stop words have  not been removed from the text corpus. The Tf-idf approach assumes that the importance of a word is inversely proportional to how often it occurs across all documents. Although Tf-idf is most commonly used to rank documents by relevance in different text mining tasks, such as page ranking by search engines, it can also be applied to text classification via naive Bayes.

\begin{equation}\text{Tf-idf} = tf_n(t,d) \cdot idf(t)\end{equation}

Let $tf_n(d,f)$ be the normalized term frequency, and $idf$, the inverse document frequency, which can be calculated as follows

\begin{equation}idf(t) = \log\Bigg(\frac{n_d}{n_d(t)}\Bigg), \end{equation}

where

\begin{itemize}
\item  $n_d$: The total number of documents.
\item  $n_d(t)$: The number of documents that contain the term $t$.
\end{itemize}

\subsubsection{Performances of the Multi-variate Bernoulli and Multinomial Model}

Empirical comparisons provide evidence that the multinomial model tends to outperform the multi-variate Bernoulli model if the vocabulary size is relatively large \cite{mccallum1998comparison}. However, the performance of machine learning algorithms is highly dependent on the appropriate choice of features. In the case of naive Bayes classifiers and text classification, large differences in performance can be attributed to the choices of  stop word removal, stemming, and token-length \cite{rudner2002automated}. In practice, it is recommended that the choice between a multi-variate Bernoulli or multinomial model for text classification should precede comparative studies including different combinations of feature extraction and selection steps.

\section{Variants of the Naive Bayes Model}
\label{sec:naive_bayes_variants}

So far, we have seen two different models for categorical data, namely, the multi-variate Bernoulli (Section \ref{sec:bernoulli_bayes}) and multinomial (Section \ref{sec:multinomial_bayes}) models --- and two different approaches for the estimation of class-conditional probabilities. In Section \ref{sec:continuous_variables}, we will take a brief look at a third model: \emph{Gaussian naive Bayes}.

\subsection{Continuous Variables}
\label{sec:continuous_variables}

Text classification is a typical case of categorical data, however, naive Bayes can also be used on continuous data. The \emph{Iris} flower data set would be a simple example for a supervised classification task with continuous features: The Iris dataset contains widths and lengths of petals and sepals measured in centimeters.  One strategy for dealing with continuous data in naive Bayes classification would be to discretize the features and form distinct categories or to use a Gaussian kernel to calculate the class-conditional probabilities. Under the assumption that the probability distributions of the features  follow a normal  (Gaussian)  distribution, the Gaussian naive Bayes model can be written as follows

\begin{equation} P(x_{ik} \mid \omega) = \frac{1}{\sqrt{2\pi\sigma^2_{\omega}}} \exp\left(-\frac{(x_{ik} - \mu_{\omega})^2}{2\sigma^2_{\omega}}\right), \end{equation}

where $\mu$ (the sample mean) and $\sigma$ (the standard deviation) are the parameters that are to be estimated from the training data. Under the naive Bayes assumption of conditional independence, the class-conditional probability can than be computed as the product of the individual probabilities:

\begin{equation} P(\textbf x_i \mid \omega) = \prod_{k=1}^{d} P(\textbf x_{ik} \mid \omega) \end{equation}

\subsection{Eager and Lazy Learning Algorithms}
\label{sec:eager_and_lazy}

Being an \emph{eager learner}, naive Bayes classifiers are known to be relatively fast in classifying new instances. Eager learners are learning algorithms that learn a model from a training dataset as soon as the data becomes available. Once the model is learned, the training data does not have to be re-evaluated in order to make a new prediction. In case of eager learners, the computationally most expensive step is the model building step whereas the classification of new instances is relatively fast.

\emph{Lazy learners}, however, memorize and re-evaluate the training dataset for predicting the class label of new instances. The advantage of \emph{lazy learning} is that the model building (training) phase is relatively fast. On the other hand, the actual prediction is typically slower compared to eager learners due to the re-evaluation of the training data. Another disadvantage of lazy learners is that the training data has to be retained, which can also be expensive in terms of storage space. A typical example of a lazy learner would be a \emph{k-nearest neighbor} algorithm: Every time a new instance is encountered, the algorithm would evaluate the \emph{k}-nearest neighbors in order to decide upon a class label for the new instance, e.g., via the \emph{majority rule} (i.e., the assignment of the class label that occurs most frequently amongst the \emph{k}-nearest neighbors).

\bibliography{../references/bayes}{}
\bibliographystyle{unsrt}

\end{document} % end main document


\section{Results and Discussion}
\label{sec:Results_and_Discussion}
This section contains lists of all important results and summarizes the most important things.

The following figure \ref{fig:results} shows a graphical comparison between the calculated and the litearture values for the Planck constant $h$.

\begin{figure}[H]
	\centering
	\includegraphics[scale=0.94]{results}
	\caption{Graphcal comparison of the three different methods used to determine the Planck constant $h$. The determined values are shown in blue and the respective uncertainty bars are red. The true value of the Planck constant $h$ is marked in green and annotated with an $h$.}
	\label{fig:results}
\end{figure}

Table \ref{tab:Planck_Constants} shows all calculated Planck constants $h^\prime$ and the literature value of the Planck constant $h$. It is clearly visible that the direct measurement was off the most (>$35\ \%$). The calculated Planck constant $h^{(3)}$ from the threshold voltages of the LEDs was the most precise. The true value of the Planck constant $h$ lies almost inside of the error bar.

\begin{table}[H]
	\centering
	\renewcommand{\arraystretch}{1.2}
	\begin{tabular}{|l|c|}
		\cline{2-2}
		\multicolumn{1}{c|}{} & $\textbf{Planck Constant}\ \boldsymbol{h}$ \\
		\multicolumn{1}{c|}{} & in J$\cdot$s \\
		\hline
		\textbf{Litearture \cite{planck_constant}} & $6.626\cdot10^{-34}$ \\
		\hline\hline
		\textbf{Direct Measurement} & $(4.242\pm 0.140)\cdot10^{-34}$ \\ %424.1812e-036 / 13.9708e-036
		\hline
		\textbf{Counter-Field Method} & $(5.437\pm 0.512)\cdot10^{-34}$ \\ %543.7227e-036 / 51.2277e-036
		\hline
		\textbf{Light-Emitting Diode} & $(7.659\pm 1.032)\cdot10^{-34}$ \\ %765.9386e-036 / 103.1796e-036
		\hline
	\end{tabular}
	\caption{List of all calculated Planck constants $h^\prime$ and the literature value of the Planck constant $h$. The measurement with the LEDs resulted in the most precise value for the Planck constant.}
	\label{tab:Planck_Constants}
\end{table}

It is only possible to determine the Planck constant $h$ through experiments. This not so trivial task had to be fulfilled to successfully redefine the kilogram in the last couple of years.


%!TEX TS-program = lualatex
%!TEX encoding = UTF-8 Unicode

\documentclass[12pt]{exam}

%\printanswers

\usepackage{fontspec}
\setmainfont[Ligatures={TeX}, BoldFont={* Bold}, ItalicFont={* Italic}, BoldItalicFont={* BoldItalic}, Numbers={Monospaced, Lining}]{Linux Libertine O}
\setsansfont[Scale=MatchLowercase,Ligatures=TeX]{Linux Biolinum O}
\setmonofont[Scale=MatchLowercase]{Inconsolatazi4}
\newfontfamily{\tablenumbers}[Numbers={Monospaced,Lining}]{Linux Libertine O}
\usepackage{microtype}

\usepackage{amsmath}

\usepackage{unicode-math}
\setmathfont[Scale=MatchLowercase]{TeX Gyre Termes Math}

\usepackage{geometry}
\geometry{letterpaper, left=1.5in, bottom=1in}
%\geometry{landscape}                % Activate for for rotated page geometry
\usepackage[parfill]{parskip}    % Activate to begin paragraphs with an empty line rather than an indent

\usepackage{longtable}

%\usepackage{siunitx}
\usepackage{booktabs}
\usepackage{array}
\newcolumntype{L}[1]{>{\raggedright\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{C}[1]{>{\centering\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{R}[1]{>{\raggedleft\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}

\usepackage{enumitem}
\setlist{leftmargin=*}
\setlist[1]{labelindent=\parindent}
\setlist[enumerate]{label=\textsc{\alph*}.}
\setlist[itemize]{label=\color{gray}\textbullet}
%\usepackage{hyperref}
%\usepackage{placeins} %PRovides \FloatBarrier to flush all floats before a certain point.
%\usepackage{hanging}

\usepackage[sc]{titlesec}

%% Commands for Exam class
\renewcommand{\solutiontitle}{\noindent}
\unframedsolutions
\SolutionEmphasis{\bfseries}

\pagestyle{headandfoot}
\firstpageheader{\textsc{bi}\,063 Evolution and Ecology}{}{\ifprintanswers\textbf{KEY}\else Name: \enspace \makebox[2.5in]{\hrulefill}\fi}
\runningheader{}{}{\footnotesize{pg. \thepage}}
\footer{}{}{}
\runningheadrule

\begin{document}

\subsection*{Mean and standard deviation practice problems}

Here are five practice problems to calculate mean and standard deviation.  Answers are on the second page.

$\overline{Y}$ is the mean. $s$ is the standard deviation.The final answer is rounded to two decimal places. I did not round any values during the calculations.

\begin{tabular}[l]{@{}R{0.5in}R{0.75in}R{0.75in}R{0.75in}R{0.75in}R{0.75in}@{}}
\toprule
	& Problem 1	& Problem 2	& Problem 3	& Problem 4	& Problem 5\\
\midrule
$Y_1$	& 32	& 43	& 48	& 16	& 14\\
$Y_2$	& 23	& 35	& 34	& 15	& 46\\
$Y_3$	& 50	& 12	& 26	& 30	& 27\\
$Y_4$	& 32	& 11	& 29	& 12	& 30\\
$Y_5$	& 29	& 35	& 13	& 50	& 38\\
& 	& 	& 	& \\
$\overline{Y}$	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}\\
&	&	&	& \\
$s$	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}\\
\bottomrule
\end{tabular}

\newpage

\subsubsection*{Answer Key}

\begin{tabular}[l]{@{}R{0.5in}R{0.75in}R{0.75in}R{0.75in}R{0.75in}R{0.75in}@{}}
	\toprule
	& Problem 1	& Problem 2	& Problem 3	& Problem 4	& Problem 5\\
	\midrule
	$Y_1$	& 32	& 43	& 48	& 16	& 14\\
	$Y_2$	& 23	& 35	& 34	& 15	& 46\\
	$Y_3$	& 50	& 12	& 26	& 30	& 27\\
	$Y_4$	& 32	& 11	& 29	& 12	& 30\\
	$Y_5$	& 29	& 35	& 13	& 50	& 38\\
	& 	& 	& 	& \\
	$\overline{Y}$	& 33.2	& 27.2	& 30.0	& 24.6	& 31.0\\
	&	&	&	& \\
	$s$	& 10.08	& 14.70	& 12.71	& 15.81	& 12.04\\
	\bottomrule
\end{tabular}

	\bigskip

	\bigskip

	Mean:
	\begin{equation*}
	\overline{Y} = \dfrac{\sum\limits^n_{i=1} Y_i}{n}
	\end{equation*}

	\bigskip

	Standard Deviation:
	\begin{equation*}
	 s = \sqrt{\dfrac{\sum\left(Y_i - \overline{Y}\right)^2}{n-1}}
	\end{equation*}

\end{document}


% \documentclass[12pt, oneside]{book}

\chapter{Coriolis force and Ekmann Balance}
\label{ch:coriolis}

\begin{tabular}{ll}
    \includegraphics[width=6in]{figs/Coriolis/MerryGoRound}
\end{tabular}

The large-scale circulation of the ocean occurs on spatial scales spanning the size of ocean basins and temporal scales of hours to decades.  On these scales, the earth's rotation matters at leading order to the dynamics of the motions, via an apparent force we call the \Wikiref{Coriolis force}.

\section{Quick review of forces and apparent forces}

First, its helpful to quickly review forces, and then discuss what we mean by an \Wikiref{apparent force}.

\Wikiref{Newton's second law} says that the rate of change of a body's velocity ($\mathbf{u}$) is proportional to the \emph{net} force exerted on it:
\begin{equation}
    \sum{\mathbf{F}_i} = m \frac{d \mathbf{u}}{dt}
\end{equation}
where we have used bold to remind ourselves that the \emph{direction} of the force matters.  As with concentrations, in fluid mechanics it is more natural to consider force per mass, or
\begin{equation}
    \sum{\mathbf{a}_i} = \frac{d \mathbf{u}}{dt}.
\end{equation}

It is possible for a body to have forces being exerted on it, but those forces be in ``balance'' so that the acceleration of the body is zero (or approximately zero).  A common example from physics classes is a falling body that has reached its \Wikiref{terminal velocity}, which is a balance between the force of gravity pulling the body down, and friction resisting the downward motion:
\begin{equation}
    \frac{dw}{dt} = -g + Fr \approx 0
\end{equation}
where $w$ is the vertical component of velocity, and $Fr$ is the wind friction at the terminal velocity (\fref{fig:TermVel}).

\begin{marginfigure}
    \includegraphics[width=1.7in]{figs/Coriolis/TermVel}
    \caption{Sketch of terminal velocity force balance.  In balance, the gravity force $g$ is equal and opposite to the friction force ($F_r$).}
    \label{fig:TermVel}
\end{marginfigure}

Newtons laws of motion apply to an \Wikiref{inertial reference frame}, i.e. a reference frame that is not accelerating.  An east way to think about this is to consider a cup of coffee on the dashboard of your car.  If you are travelling a constant speed, the coffee travels with your dashboard and there are the only forces on it are vertical (gravity and the normal force, \fref{fig:SketchApparent}, left-top).  Now suppose the car accelerates with acceleration $a$.  From an outside observer's point of view, the cup continues to move at the same speed, but the car starts to accelerate away from the coffee cup (\fref{fig:SketchApparent}, left-middle sketch). However, if you are sitting in the car, it is like a force pushed the cup so that it accelerates towards the back of the car (\fref{fig:SketchApparent}, left-bottom sketch). This would be an \Wikiref{apparent force} in the accelerating reference frame of the car.

\begin{figure}[hbt]
  \begin{center}
  \includegraphics{figs/Coriolis/SketchApparent}
    \caption{Apparent force on a cup in a car.  Left side are frictionless cups (or a ball if you prefer) on an accelerating car.  Right hand side is a cup held laterally by a cup holder.}
    \label{fig:SketchApparent}
  \end{center}
\end{figure}

If the cup moves with the car, then something has to supply a lateral force to cause that acceleration.  So, imagine a wall that the coffee can push against, or a cupholder (\fref{fig:SketchApparent}, top-right).  From an outside observers point of view, there is only one force acting on the cup, $a$ to the left supplied by the cup holder, accelerating the cup at the same rate the car is being accelerated.  However, someone inside the car may very well feel that there is a force pushing the cup to the back of the car, and infer an equal-and-opposite force towards the front, so the cup does \emph{not} accelerate in this frame of reference.

Hopefully this change of reference frames makes sense to you.  If you are sitting in a car and the driver slams on the brakes, you don't say that you were "slowed down more slowly than the car", you say that you were ``thrown forward''.  Of you accelerate very quickly, you say you were ``pushed back in your seat''.

The application to the earth is hopefully obvious - the earth is constantly rotating, and hence any point rotating with the earth is constantly changing its velocity (but not its speed!), and hence the rotating frame is not an inertial frame of reference.

\section{Centrifugal and Coriolis forces}

 A rotating reference frame gives rise to \emph{two} apparent forces, the \Wikiref{centrifugal force} and the \Wikiref{Coriolis force}.  The centrifugal force is felt everywhere on the planet and is a constant that depends on position, so we actually roll it into our definition of gravity and promptly forget about it (except perhaps on an exam).  The Coriolis force only arise if a body is moving relative to the rotating frame.

 In oceanography, when we say a parcel of water moves at 1 m/s, it is doing so in the rotating frame.  From space it is moving at 132 m/s (at 49 N), where most of that is the earth's rotation.  Doing everything from a frame that is out in space is possible, but extremely awkward!

 \subsection{Centrifugal force}

\begin{marginfigure}
    \includegraphics{figs/Coriolis/CentrifugalForceCor}
    \caption{Forces on a rotating reference frame observed from an inertial reference frame (i.e. from space).  There needs to be a centripetal force applied to any object going around the circle or it will be flung off the orbit.  An observer in the rotating frame sees two forces: the centripetal force and an apparent centrifugal force that works against the centripetal force to keep the body in place on the rotating frame.}
    \label{fig:CentrifugalForceCor}
\end{marginfigure}

The earth spins about its axis every 24 h, such that it spins counter clockwise when looking down on the north pole.  Because this is an important constant it gets a capital $\Omega = 2\pi/24h =7.27\times10^{-5}\ \mathrm{rad\,s^{-1}}$.  The radius of the earth is $R=6731 \ \mathrm{km}$, and the radius at any given latitude $\lambda$ is given by $r = R\cos(\lambda)$.  As pointed out in \fref{box:centrifugalforce}, the linear speed of a particle is given by $\omega r$ and the velocity is tangential to the circle.  The acceleration needed to keep going in a circle is readily shown to be $a_c = r \omega^2$.  A mass on the rotating system that suddenly becomes unglued will be flung away from the axis of rotation, and we call this the \Wikiref{centrifugal force} (\fref{fig:CentrifugalForceCor}).

So, as you sit in you chair (or comfy sofa) reading this, what keeps you rotating around the axis of the earth's rotation?  There needs to be a centripetal force to counteract the centrifugal force that wants to fling you into space.  This force, at 49 degrees of latitude is
\begin{equation}
    a_{cent} = R\cos\left(\lambda\right)\ \omega^2 = 0.0107 \ \mathrm{m\,s^{-2}}
\end{equation}
so its small compared to gravity, but is not non-zero.  You may argue that it is friction with your chair that keeps you in place, but that doesn't apply if your chair has wheels, or to a hockey puck on an ice rink.  The answer is that the earth is not a sphere, and bulges out at the equator, precisely because it is spinning; the equatorial radius is 22 km more than the polar radius.

This leads to a centripetal force because there is an imbalance between the normal force exerted by the earth's spheroidal surface and the direction that gravity points.  First, consider the forces on a body sitting on a perfect sphere as observed from an inertial frame (i.e. space; \fref{fig:CentrEarth} left panel).  There are two real forces acting on the body - gravity pulls towards the center of the earth, and there is a normal force due to the body of the earth that pushes back.  Because this is a perfect sphere the normal force is in the same direction as the gravity force, pointing away from the center of the sphere.  Combined, then there are no net forces on the object, and hence the object will \emph{not} orbit with the spinning of the earth.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics{figs/Coriolis/CentrEarth}
    \caption{Left panel: forces on an object on a perfect sphere.  There is a normal force that is exactly equal and opposite to the gravity force because the normal on a perfect sphere points towards the centre of the sphere.  Right panel: the same forces on an oblate spheroid; the normal force and the gravity force are at angles to one another leading to a net force towards the axis of rotation.}
    \label{fig:CentrEarth}
  \end{center}
\end{figure}

Naturally, this does not happen on the real earth - if somehow we started with a spinning perfect sphere earth it would tectonically shear apart until it made a bulge at the equator that balanced the spinning.  Such a  bulge is in rotational equilibrium when the normal force at any point on its surface points at an angle to the center of the earth so that the sum of the two forces result in a net force towards the axis of rotation (\fref{fig:CentrEarth}, right panel). The faster a planet spins, the larger this bulge is.  Of course there are irregularities on a real planet, like mountains, but if you put your chair on the mountain side, you would roll off.

Now consider a plumb bob hanging above the surface of the earth.  It is \emph{not} just pulled down towards the center of the earth, but also is flung out slightly by the centrifugal acceleration. Hence it will fall parallel to the normal line in \fref{fig:CentrEarth}, (right side), and not towards the center of the earth. Hence we call this direction "down" and "up" and we slightly modify the value of gravity to account for the centrifugal acceleration.  Hence you will find gravity calculators that depend on the latitude $\lambda$.

\subsection{Coriolis force}

The \Wikiref{Coriolis force} is the second apparent force in a rotating reference frame.  It only occurs if the object has a velocity relative to the rotating frame. It is quite unintuitive, and has no easy analogue like in the accelerating car.  i.e. the cup feels the same acceleration whether it is already moving in the car or not.

The Coriolis force acts exactly 90 degrees to the right of the velocity vector in the Northern hemisphere, and to the left in the Southern. The strength of the force is linearly proportional to the velocity:
\begin{equation}
    |\mathbf{F_{Coriolis}}| = f |\mathbf{u}| \ \ \ \mathrm{to\ the\ right\ of\ \mathbf{u}}
\end{equation}

Understanding how this force arises is not terribly difficult (and the math is even easier if you know some vector calculus).  First, consider a hockey player shooting a puck along a frictionless ice rink.  That person is rotating on the earth, and hence is being pulled into the center of the earth by the net centripetal acceleration (\fref{fig:ThrowerMotion}).  You can think of this as the summation of two vectors, the tangential motion of the hockey player at travelling a distance $\omega r\ \delta t$, and the motion towards the axis of rotation $0.5\omega^2 r \ \delta t^2$.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics{figs/Coriolis/ThrowerMotion}
    \caption{Forces on a person who is stationary in the rotating frame. }
    \label{fig:ThrowerMotion}
  \end{center}
\end{figure}

Now, suppose they aim the puck away from the axis of motion, which on the northern hemisphere is like aiming at a target to the south (\fref{fig:SouthSetup}).  Note that the target moves further to the left than the hockey player, and then is pulled back into orbit by a stronger centripetal force because it is further from the axis of rotation. The hockey player shoots the puck with enough velocity to hit the target, and the aim directly away from the axis of rotation.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics{figs/Coriolis/SouthSetup}
    \caption{Left panel: forces in the inertial frame of a hockey player aiming at a target further from the axis of rotation. Note that the target has a \emph{faster} tangential velocity than the hockey player.  Right panel: setup in the rotating frame observed by the hockey player.  }
    \label{fig:SouthSetup}
  \end{center}
\end{figure}

What happens when the player shoots the puck is that everything continues to rotate because of the centripetal force, but the puck does not have enough tangential velocity to hit the target.  In \fref{fig:SouthThrow}, the puck ends up at the summation of three vectors - the first two are (ahem, almost) the same as the hockey players, the tangential distance $\omega r\ \delta t$, and the motion towards the axis of rotation $0.5\omega^2 r \ \delta t^2$. The third is the velocity component $U\ \delta t$ imparted by the player when they shot the puck. Adding these three together, it should be clear that the puck can't keep up with the target, and ends up ``behind'' it, or to the right of the velocity (\fref{fig:SouthThrow}, Rotating Frame).

\begin{figure}[hbt]
  \begin{center}
    \includegraphics{figs/Coriolis/SouthThrow}
    \caption{Left panel: The result of the shot after a time $\delta t$ in the rotating frame, where the circle is the puck.  the puck doesn't have enough velocity to the ``left'' to keep up with the target and falls behind.   Right panel: in the rotating frame, the puck veers to the right of the target.}
    \label{fig:SouthThrow}
  \end{center}
\end{figure}

Why does a greater velocity result in a greater Coriolis force?  Quite simply the puck would get further away from the axis of rotation in time $\delta t$, and would fall even further behind the new, more distant target.

A similar effect happens if the throw is in the tangential direction.  In this case this would be to the ``east'' if the shot was taken in the Northern hemisphere.  In this case, the puck is given too much tangential velocity (\fref{fig:Response}).  It still gets pulled in towards the axis of rotation the same amount as the hockey player, but it has more tangential velocity so it ends up further from the axis of rotation.  The hockey player sees it fly to the right.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics{figs/Coriolis/Response}
    \caption{Upper plot: a shot in the same direction as rotation.  The puck has (approximately) the same centripetal acceleration towards the axis of rotation, but has too much momentum to be pulled back to the original orbit, and hence moves in a net sense away from the axis of rotation.  Lower plot: as observed by the hockey player, the puck veers to the right of the target.}
    \label{fig:Response}
  \end{center}
\end{figure}

the astute reader will note that the puck now has a component of velocity that has turned to the right of it original motion.  This should mean the Coriolis force turns to the right at the same time.  Indeed what happens is that the Coriolis force pushes the particle around a circle with a frequency given by the proportionality constant $f$.  This type of motion is regularly seen in the ocean (\fref{fig:inertial_currents}), and is called an \Wikiref{inertial oscillation}.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics{figs/Coriolis/inertial_currents}
    \caption{Response of surface drifters in the ocean to a sudden windstorm.  Note the circular motion superimposed on the mean currents.}
    \label{fig:inertial_currents}
  \end{center}
\end{figure}

It is interesting to note that the Coriolis force is the same \emph{everywhere} in the rotating reference frame, and depends only on the velocity and the angle of the velocity with the axis of rotation.  However, a the earth is a sphere, and we specify our velocities as horizontal velocities along the sphere (eastward: $u$, northward: $v$) and upwards perpendicular to the geoid ($w$).  Usually the Coriolis force due to motions in the vertical is very small ($w<<u$), so we only deal with the horizontal terms, but these make an angle to the axis of rotation everywhere on the sphere, except
right at the poles.  That angle means that the horizontal component of the Coriolis force is $f = 2\Omega sin\lambda = 1.45\times10^{-4}\ \mathrm{rad\, s^{-1}}$, where $\lambda$ is the latitude.  So, $f=2\Omega$ at the north pole, $0$ at the equator, and $-2\Omega$ at the south pole.  At the mid latitudes ($\lambda=45^o \ N$) $f\approx10^{-4}\ \mathrm{rad\,s^{-1}}$.

The minus sign in the southern hemisphere means that inertial motions are counter clockwise.  This is just because "up" points closer to the south pole than towards the north pole.  A puck pushed eastward in the southern hemisphere is still flung away from the axis of rotation, its just that away from the axis of rotation in the southern hemisphere means moving northward instead of southward.  You can also think of this as looking down at the earth spinning from the south pole, in which case it appears to turn clockwise and we can think of the rotation rate as being $-\Omega$.

Mathematically, we write the horizontal equations of motion as
\begin{eqnarray}
    \label{eq:motion}
    \frac{du}{dt} & = & +fv  - \frac{1}{\rho}\frac{dP}{dx} +  \sum F_{x}\\
    \frac{dv}{dt} & = & -fu - \frac{1}{\rho_0}\frac{dP}{dy} + \sum F_{y}
\end{eqnarray}
where $F_x$ and $F_y$ are the friction forces in x and y.

\begin{derivbox}[label={box:nonlinear}]{Non-linearity of equations of motion}
The equations in \fref{eq:motion} are adequate for our needs, but note that these are the equations following a parcel of water, just like in classical mechanics how we would follow the forces on a single ball. In fluid mechanics we call this the \Wikiref{Lagrangian reference frame} and usually specify with a capital $D$ instead of lower case $d$ (i.e. $\frac{Du}{Dt}=...$).

However, we usually cannot follow a parcel of water, so the derivatives at a point have terms that are due to the water moving from one point to another.  This is called the Eulerian reference frame, and is what makes water flow so complicated because there are nonlinear terms:
\begin{equation}
    \frac{Du}{Dt} = \frac{\partial u}{\partial t} + u\frac{\partial u}{\partial x} + v\frac{\partial u}{\partial y} + w\frac{\partial u}{\partial z}
\end{equation}
Including these terms makes mathematically solving the equations of motion very difficult!
\end{derivbox}

\section{Friction force}

Now we only need to consider the final terms in \fref{eq:motion}, the friction forces.  Friction forces are quite complicated in general, but lets consider a simpler flow.  Suppose there is a flow in the x-direction, and that it has \Wikiref{shear} in the vertical such that the faster flow is above the slower (\fref{fig:stress}).  Because the fluid is viscous, the faster flow above will exert a stress to the right on the flow below, and the slower flow will exert a stress to the left on the faster flow.  This will decelerate the upper flow and accelerate the lower flow (\fref{fig:Stress}).  This transfer of momentum occurs at a rate proportional to the shear and the viscosity of the fluid:
\begin{equation}
    \frac{\tau_{xz}}{\rho} = \nu \frac{du}{dz}
\end{equation}
where the subscript $xz$ means transfer of x momentum in the z-direction.  Here $\nu$ is the kinematic viscosity and has a molecular value of $\nu \approx 10^{-6}\ \mathrm{m^2\,s^{-1}}$.  Note that the units are the same as for a diffusion co-efficient, and can be thought of as a diffusion of x-momentum in the vertical.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics{figs/Coriolis/Stress}
    \caption{Left: Velocity in x-direction with faster flow above slower (blue arrows). Right: velocity after there has been a downward transfer of x-momentum ($\tau_{xz}$); the faster velocity has slowed, and the slower velocity accelerated.}
    \label{fig:Stress}
  \end{center}
\end{figure}

The net force at any point depends on the difference of the stresses in the vertical (\fref{fig:NetStress}).   The parcel gains momentum from above, but loses it below.  So in this simplified flow
\begin{equation}
    F_x = \frac{1}{\rho}\frac{d\tau_{xz}}{dz}+ other\ friction\ in\ x = \nu \frac{d^2u}{dz^2} + other\ friction\ in\ x
\end{equation}
So just like for diffusion,  the flow needs to have some curvature ($\frac{du}{dz} \neq 0$) for there to be a net effect on the velocity due to friction.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics{figs/Coriolis/NetStress}
    \caption{Net stress on a parcel (yellow dashes) in a sheared flow (blue velocity arrows) is the difference in the stress from top to bottom (red arrows).  The water above speeds up the parcel, but the water below slows it down.}
    \label{fig:NetStress}
  \end{center}
\end{figure}

X-direction momentum doesn't just diffuse in the vertical direction.  If we re-drew \fref{fig:Stress} with $y$ replacing $z$ the same physics would apply, and similarly in the x-direction, so the full value of of the friction in x is:
\begin{equation}
    F_x = \frac{1}{\rho}\left(\frac{d\tau_{xz}}{dz} + \frac{d\tau_{xy}}{dy} + \frac{d\tau_{xx}}{dx}\right)
\end{equation}
and similar equations apply for $F_y$ and $F_z$, leading to nine total terms.

\subsection{External stresses}

Water can have one of two boundary conditions - constant velocity (usually not moving), or constant stress.  Constant velocity means that at the boundary the fluid moves with the boundary, and is usually applicable to solid boundaries.

Constant stress is often applied at the air-sea boundary as a ``wind stress''.  As the wind blows over the top of the ocean, it imparts some momentum to the ocean (and vice-versa).  We usually use the  symbol $\tau_x^w$ for the x-direction wind stress and $\tau^w_y$ where it is understood that the stress is being transmitted vertically.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics[trim=50 140 50 120,clip]{figs/Coriolis/June_SCOW_Wind_Stress_Magnitude-crop}
    \includegraphics[trim=50 90 50 120,clip]{figs/Coriolis/December_SCOW_Wind_Stress_Magnitude-crop}
%    \includegraphics[trim=1200 400 1500 1500,clip]{figs/Coriolis/December_SCOW_Wind_Stress_Magnitude-crop}
  \caption{Wind stress amplitude, averaged for June and December.}
    \label{fig:Windstresses}
  \end{center}
\end{figure}

\section{Ekman balance}
\label{sec:Ekman}

In physics it is often helpful to determine if a steady state exists.  In physical oceanography this means times when $du/dt=0$.  We saw in the previous chapter that the flow in the large-scale ocean actually \emph{does} change relatively slowly, so on long time scale this is approximately true.  Even in the coastal flow, the flow evolved slowly enough that the dominant balances are obeyed.  There are two such balances. The Ekman balance is  between the Coriolis and friction and friction forces, and the \Wikiref{Geostrophic balance} is between Coriolis and the pressure gradient force:
\begin{equation}
    \frac{du}{dt} = \overunderbraces{\br{2}{geostrophic}}
    {- \frac{1}{\rho_0}\frac{dP}{dx} & +fv & + \frac{1}{\rho}\frac{d\tau_{xz}}{dz}}
    {&\br{2}{Ekman}}
\end{equation}
\begin{equation}
    \frac{dv}{dt} = \overunderbraces{\br{2}{geostrophic}}
    {- \frac{1}{\rho_0}\frac{dP}{dy} & -fu & + \frac{1}{\rho}\frac{d\tau_{yz}}{dz}}
    {&\br{2}{Ekman}}
\end{equation}
Here we discuss the first of these two balances, the Ekman balance.

The Ekman balance typically occurs near the boundaries where the shear stresses in the ocean are the largest.  In this region we assume that the friction is balanced by the Coriolis force, and ignore the pressure gradient force:
\begin{eqnarray}
   \frac{1}{\rho} \frac{d\tau_{xz}}{dz} & = & -fv\\
    \frac{1}{\rho}\frac{d\tau_{yz}}{dz} & = & -fu
\end{eqnarray}
The stress profile can be complicated, but if we integrate from some depth where $\tau_{xz}=0$, say at $z=-D$ to the sea surface, then we can write:
\begin{eqnarray}
    \frac{1}{\rho}\tau_x^w & = & -f\overline{v}D\\
    \frac{1}{\rho}\tau_y^w & = & f\overline{u}D
\end{eqnarray}
where $\overline{u} = \frac{1}{D}\int_{-D}^0 u\ \mathrm{d}z$ is the average velocity in the surface layer (same for $\overline{v}$).  In the coastal upwelling example, $D\approx20\ \mathrm{m}$ is probably appropriate, because that is the region where the turbulence is enhanced.  The terms $\overline{v}D$ and $\overline{u}D$ are called the Ekman transport and represent a 2-D transport with units of $\mathrm{m^2\,s^{-1}}$.

So, for example in the coastal upwelling case, the wind is blowing from north to south, so the net force on the upper ocean is to the south (\fref{fig:EkmanLayerForces}).  If this force is in balance with the Coriolis force, then the Coriolis force must be towards the north.  Since the Coriolis force is to the north, that means there must be a net flow to the west (assuming this is northern hemisphere).  The other way to remember this is that the Ekman transport is to the right of the stress.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics[width=3in]{figs/Coriolis/EkmanLayerForces}
    \caption{Sketch of the depth-integrated force balances in the Ekman layer in the coastal-upwelling case. Upper is looking from above, and lower sketch is looking from the south. }
    \label{fig:EkmanLayerForces}
  \end{center}
\end{figure}

Stress can also be exerted at the sea floor by friction of the mean flow with the bottom.  Suppose, as in the coastal circulation example, the flow is towards the south.  Then there will be a friction to the north tending to slow this flow down, which we represent as a bottom stress $\tau_y^B$.  We saw in the observations that there is indeed heightened turbulence near the sea floor.  This also results in an Ekman layer, but now the stress is to the north (\fref{fig:EkmanBottomForces}), so the Ekman transport $\overline{u}D$ is to the east, or onshore.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics[width=3in]{figs/Coriolis/EkmanBottomForces}
    \caption{Sketch of the depth-integrated force balances in a bottom Ekman layer in the coastal-upwelling case. Upper is looking from above, and lower sketch is looking from the south. The bottom stress is the net stress acting to slow down the southward interior current.}
    \label{fig:EkmanBottomForces}
  \end{center}
\end{figure}

Putting some values on these transports, usually we can estimate the windstress or bottom stress from the speed of the water or wind.  For the Oregon shelf case, the bottom stress is estimated to be something like $\tau_B^W\approx 0.1 \ \mathrm{N\,m^{-2}}$, so
\begin{equation}
    \overline{u}D = \frac{\tau_B^W}{\rho_0 f} = \frac{0.1}{10^3 \times 10^-4} = 2\ \mathrm{m^2\,s^{-1}}
\end{equation}
where this is transport per along-shore distance.  If we average this over 20 m, then the average speed of this layer is $0.1\ \mathrm{m\,s^{-1}}$, or about 8 km/d.   We saw from the observations that dense water was seen moving up the shelf at approximately 7 km/d, so this is in agreement with our observations.  (Note that the surface mixed layer was very hard to observe in these observations so we don't make a direct comparison of its motion.)

\subsection{The Ekman spiral: fact and fiction}

Above we elided the structure of the Ekman flow and skipped to the net transport by integrating from $z=-D$ to the surface (or over the bottom Ekman layer).  The details of the Ekman flow are quite interesting, but in the end frustratingly hard to quantify and observe.

The coastal upwelling example clearly shows the effect of Ekman transport pushing water up the shelf.  However, note that the observations did not directly observe the velocity associated with this transport because the Doppler current profiler has a technical limitation near the seafloor.  So we infer the net motion from the tranport of dense water up the shelf.  I would argue that this transport is the important factor, and details of the currents in the Ekman layer are of secondary importance.

Regardless, even elementary text books discuss the \Wikiref{Ekman spiral} which has a very idealized shape.  In this idealization, the surface currents flow at 45 degrees to the wind, and then  smoothly spirals to clockwise as it decays with depth (\fref{fig:ocean-in-motion_clip_image001}).  This ideal solution depends on a constant viscosity $\nu$ in the upper ocean.  However, that is rarely the case as the viscosity is usually a turbulent viscosity and the turbulence in the upper water column varies due to many factors.  Hence observed Ekman spirals are usually found to have more veering than predicted, and a compressed decay scale (\fref{fig:ObservedEkmanSpiral}).

\begin{figure}[hbt]
  \begin{center}
    \includegraphics[width=3in]{figs/Coriolis/ocean-in-motion_clip_image001}
    \caption{Idealized Ekman spiral for flow with a constant viscosity.  \protect\url{http://oceanmotion.org/html/background/ocean-in-motion.htm}}
    \label{fig:ocean-in-motion_clip_image001}
  \end{center}
\end{figure}

\begin{figure}[hbt]
  \begin{center}
    \includegraphics{figs/Coriolis/ObservedEkmanSpiral}
    \caption{Observed Ekman spiral from direct measurements in the upper ocean divided into flow at night and during the day.  Daytime stratification compresses the Ekman layer to the upper ocean.  \citep{priceetal87}}
    \label{fig:ObservedEkmanSpiral}
  \end{center}
\end{figure}

\section{Ekman convergence and divergence}

Ekman transports tend to only affect a thin region through the upper ocean, or near the seafloor.  However, because they move water around they can make convergences and divergences.  In a convergence water piles up and then is pushed downwards, in a divergence the sea level tends to drop, and then water is pulled up from depth.  This creates pressure gradients that can then be felt from the top of the water column to the bottom.

First, if there is a coast, steady offshore Ekman transport leads to a divergence at the coast.  This will tend to make the water level drop at the coast relative to offshore (\fref{fig:EkmanDivCoast}), until there is  replacement flow from below. It is important to remember this cross-shore pressure gradient is caused \emph{before} any replacement flow happens, and indeed this pressure gradient is what ultimately causes the replacement flow.  That there must be a replacement flow should be clear - suppose $1\ \mathrm{m^2\,s^{-1}}$ of offshore transport (see calculation above for bottom boundary layer), and suppose this is applicable 10 km offshore.  Over this 10 km, tbe water level will drop at a rate $10^{-4}\ \mathrm{m\,s^{-1}}$, or 8 m a day.  While winds clearly blow for more than a day, the sea-surface is never drawn down 8 m at the coast.  As we will see in the next chapter, the replacement flow is due to coastal upwelling in the bottom boundary layer, but there is one more step in the process that we will cover next chapter.  However, if we know the offshore transport, we can infer the vertical upwelling rate as $10^{-4}\ \mathrm{m\,s^{-1}}$.

\begin{figure}[hbt]
  \begin{center}
  \includegraphics{figs/Coriolis/EkmanDivCoast}
    \caption{Sketch of divergence on the coast due to a southward wind ($\tau_y^w$).  Average offshore transport is everywhere except at the coast, leading to the sea surface dropping. }
    \label{fig:EkmanDivCoast}
  \end{center}
\end{figure}

Ekman convergences and divergences are also crucially important in the interior of the ocean, away from boundaries.  In this case the convergences must be caused by changes in the wind stress. When discussing the wind-driven flow in the North Pacific, we noted that the westerlies are concentrated at 45 N, and the trades at 15 N (\fref{fig:SketchPacificConv}).  The Ekman transport in the westerlies is to the south, and in the trades it is to the north (\fref{fig:SketchPacificConv}, blue arrows), so everywhere between these two latitudes, we have an Ekman convergence or \href{https://en.wikipedia.org/wiki/Ekman_transport#Ekman_pumping}{Ekman pumping}.  The convergence causes the sea surface to be high in the middle of these winds (as observed) and drive a flow downwards.

\begin{figure}[hbt]
  \begin{center}
    \includegraphics{figs/Coriolis/SketchPacificConv}
    \caption{Sketch of winds in the Pacific, looking from above (top) and from the east (bottom).  There is a convergence between the Westerlies and Trades (red arrows) due to the converging Ekman transports. This leads to high sea level between these latitudes, and downwelling or Ekman pumping into the interior.}
    \label{fig:SketchPacificConv}
  \end{center}
\end{figure}

In fact, everywhere that there is a gradient in the windstress, there will either be a convergence or a divergence.  If the wind stress gradients are mostly north-south, then we can write:
\begin{equation}
    \overline{w}\ \delta y =  \overline{v}D|_{y+\delta y} - \overline{v}D|_y = - \frac{1}{\rho_0 f}\left(  \tau_x^w(y+\delta y) - \tau_x^w(y) \right)
\end{equation}
or if we allow $\delta y$ to become very small:
\begin{equation}
    w = -\frac{1}{\rho_0 f} \frac{d \tau_x^W}{d y}
\end{equation}
So in the N. Pacific, the distance between 15 and 45 N is 3300 km, so $\frac{1}{\rho_0 f}\frac{d\tau_x^W}{d y} \approx 10^{-6}\ \mathrm{ms^{-1}}$, or close to 0.1 m per day.  Integrated over the basin this represents $30\times10^{6}\ \mathrm{m^3\,s^{-1}}$.

Note that the full Ekman vertical velocity is given by the \Wikiref{curl (mathematics)} of the  wind vector, so winds that spin in a clockwise direction tend to drive water towards the center, and winds that spin counter clockwise pull water from the center:
\begin{equation}
    \mathrm{curl} \tau^W = \frac{d \tau^W_y}{d x}-\frac{d \tau^W_x}{d y}
\end{equation}

\clearpage
\section{Exercises}

\paragraph{Coriolis to west}
Using the logic above, and a sketch, argue that shooting a hockey puck towards a target to the west (i.e. a tangential shot against the direction of rotation) will lead to a Coriolis force to the north (or towards the axis of rotation).

\paragraph{Calculate Ekman convergence and divergence} Suppose we have a wind from the west that increases to the north so that at 30 N it blows at $0.1\ \mathrm{N\,m^2}$.  At a point 50 km north it blows at  $0.2\ \mathrm{N\,m^2}$, and 100 km north it blows at  $0.3\ \mathrm{N\,m^2}$. What is the average rate of Ekman ``pumping'' between the wind measurements (in m/s)? Sketch.

\paragraph{Where is the Ekman pumping in N. Pacific?}

Based on the figure below, where is there Ekman pumping and suction (by latitude) in the North Pacific? How well does this correspond to the shape of the isohalines of the upper ocean in the salinity section below?

\includegraphics{figs/Coriolis/PacificWindsSm}
\includegraphics{figs/WaterMasses/P15_CTDSAL_all_1000}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End: