\documentclass[]{article}

\usepackage{caption,subcaption,graphicx,float,url,amsmath,amssymb,amsthm,tocloft,cancel,mathrsfs}
\usepackage[toc,nonumberlist]{glossaries}
\usepackage{glossaries-extra,thmtools,gensymb,braket,bm,tensor}
\usepackage[toc,page]{appendix}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[toc,page]{appendix}
\newcommand\numberthis{\addtocounter{equation}{1}\tag{\theequation}}
\newcommand{\Lagr}{\mathscr{L}}
\newtheorem{thm}{Theorem}
\newtheorem{defn}[thm]{Definition}
\newtheorem{cor}[thm]{Corollary}
\newtheorem{lemma}[thm]{Lemma}
\graphicspath{{figs/}}
\widowpenalty10000
\clubpenalty10000
\setcounter{tocdepth}{2}

%opening
\title{Theoretical Minimum\\Cosmology}
\author{Simon Crase (compiler)\\simon@greenweaves.nz}

\begin{document}

\maketitle

\begin{abstract}
	These are my notes for the \emph{Cosmology}\cite{susskind2013cosmology} lectures from Leonard Susskind's \emph{Theoretical Minimum} series\cite{susskind2007theoretical}.

	Disclaimer: I have created these notes as an aide-m\'emoire for my own use; if you find them useful, you are welcome, but I'd appreciate hearing from you. They are not intended
	as a substitute for listening to the lectures. The intellectual property for all material derived from the lectures belongs, of course, to Professor Susskind; any mistakes, however, are my own.

	The notes were created using TexStudio\cite{TexStudio}, which I recommend for compiling them to a PDF, and the bibliography was created using JabRef\cite{Jabref}.
\end{abstract}

\tableofcontents
\listoffigures
\listoftables
\listoftheorems

\section{The expanding (Newtonian) universe}

Cosmology as a science dates back to the discovery of the microwave radiation from the Big Bang in the 1960s. Before that Cosmology was more like natural science--classifying oddities. Thinking of the universe as a physical system, as a system to study mathematically, with a set of physical principles and a set of equations is relatively new.

We will study the universe as a physical system.

First observation: the universe is isotropic. So it should be homogeneous.

There are around $10^{11}$ galaxies within view of Earth.

\subsection{How would Newton have handled expanding universe?}

\begin{figure}[H]
	\begin{center}
		\caption{We start by introducing a set of coordinates.}
		\begin{subfigure}[t]{0.45\textwidth}
			\caption{Chose grid so that galaxies are at grid points, so grid moves with galaxies. Galaxies move coherently (from observation).}\label{fig:cosmo-1-grid}
			\includegraphics[width=0.8\textwidth]{cosmo-1-grid}
		\end{subfigure}
		\;
		\begin{subfigure}[t]{0.45\textwidth}
			\caption{Define distance: $D_{ab}=a(t) \Delta x_{ab}$, where scale parameter $a$ may or may not be a constant.}\label{fig:cosmo-1-distance}
			\includegraphics[width=0.8\textwidth]{cosmo-1-distance}
		\end{subfigure}
	\end{center}
\end{figure}

 A more general formula for distance between two galaxies $a$ and $b$ is:

 \begin{align*}
 	D_{ab}=&a(t) \sqrt{\Delta_{ab} x^2 + \Delta_{ab} y^2 + \Delta_{ab} z^2} \text{, now calculate velocity}\\
 	V_{ab}=&\dot{a} \Delta x_{ab} \text{, neglecting $y$ and $z$}\\
 	\frac{V_{ab}}{D_{ab}} =& \frac{\dot{a}}{a} \text{, independent of choice of galaxies!}\\
 	=& H \text{, Hubble constant at a given time.}\\
 	V =& H D \text{, Hubble's Law.}
\end{align*}

\subsection{How much mass is in $\Delta x \Delta y \Delta z$?}

Let $M$ denote the mass within $\Delta x \Delta y \Delta z$: $M = \nu \Delta x \Delta y \Delta z$, and $V$ denote the volume.

\begin{align*}
	V =& a^3 \Delta x \Delta y \Delta z\\
	M =& \nu \Delta x \Delta y \Delta z \text{, constant}\\
	\rho =& \frac{\nu}{a^3}
\end{align*}

\url{https://youtu.be/P-medYaqVak?t=1170}

\section{Matter and radiation dominated universes}
\section{Geometries of space: flat, spherical, hyperbolic}
\section{Cosmological Thermodynamics}
\section{Vacuum energy}
\section{Dark matter and allocation of energy density}
\section{Temperature History of the Universe}
\section{Baryogenesis}
\section{Inflation}
\section{Inhomogeneities and quantum fluctuations}

\bibliographystyle{unsrt}
\addcontentsline{toc}{section}{Bibliography}
\raggedright
\bibliography{tm}

\end{document}


\input{../../assignment-header}

%========================================================================
\title{Assignment 08:  Pendulum Swing-Up}
\date{Assigned:  March 16  ---  Due:  March 28 at 11:55pm}
\author{Optimal Control for Robotics}
%========================================================================
\begin{document}
\maketitle
%=================================================

\section*{Introduction}

In this assignment you will compute the optimal swing-up trajectory for a simple
pendulum using multiple shooting. We will keep things simple:
use Euler's method on a uniform time grid,
with one simulation step per multiple shooting segment.

\section*{Deliverables}

Implement the function \texttt{simplePendulumOptimBvp} using the template provided.

\section*{Write-Up:}

There is no separate write-up for this assignment. Instead,
please include the total time you spent working on this assignment in
the comments near the top of your implementation of\\
\texttt{simplePendulumOptimBvp}.
Be sure to clearly organize and document your code.

\section*{Comments}

In this assignment I provide a simple script to run your optimization,
along with an encrypted version of the solution.
You should run this script with a variety of different parameters to learn
more about how trajectory optimization behaves with different inputs.
Once you've done that, then start working on implementing your code.

\par If you run out of time and your assignment does not work correctly,
then add a notes section to the comments in the top of \texttt{simplePendulumOptimBvp}.
You should clearly state that the function does not work, why you think it does
not work, and your best guess about how to go about debugging it if you had time.

\par The file \texttt{simplePendulumOptimBvpSoln.p} provides you with a correct
implementation of multiple shooting, against which you can compare your code.
This implementation is not guarenteed to obtain the \quotes{true} solution to
the trajectory optimization problem. If FMINCON converges, then the resulting
trajectory is an approximation of a locally optimal trajectory.

\par One final note: \texttt{simplePendulumOptimBvpSoln.p} uses a very simple
initialization routine. If you are clever you can probably beat it: either by
finding the same solution more quickly or by avoiding a local minima.
For example, the optimal solution might require three swings,
but \texttt{simplePendulumOptimBvpSoln.p} only manages to find a solution with
one swing (which has a higher objective function value).

%=================================================
\end{document}


\section{Performance}

It is notoriously hard to test the performance of garbage collection
algorithms.  Nevertheless, we would like to get some idea of the time
the various phases take.  To that end, we created a few test cases
which (together with some educated guesses) will give us some ballpark
figures with respect to performance of our method, at least as
compared to other methods.

For our tests, we chose a heap size of $2^{19}$ words of $8$ bytes
each.  This heap size was not chosen randomly.  It was chosen so that
the entire heap would fit in the cache memory of the computer used for
the tests (x86-64, 1.6GHz, GNU/Linux, SBCL), and research
\cite{Marlow:2011:MGC:2076022.1993482} indicates that a nursery this
size is a good choice.

Thus, we have the following definitions valid for all the tests:

{\small\begin{verbatim}
(defparameter *size* (ash 1 19))

(defparameter *heap* (make-array *size*))

(defparameter *bitmap*
  (make-array *size* :element-type 'bit))
\end{verbatim}}

\subsection{Phase 1, marking}

We did not attempt to test the marking phase, because it is no
different from the marking phase of any other algorithm.

\subsection{Phase 2, compaction}

To test the compaction phase, we wrote the following compaction
program:

{\small\begin{verbatim}
(defun move (heap bitmap)
  (declare (type (simple-vector #.*size*) heap)
           (type (array bit (#.*size*)) bitmap)
           (optimize (speed 3) (safety 0) (debug 0)))
  (let* ((d (position 0 bitmap))
         (s (position 1 bitmap :start d)))
    (declare (type (integer 0 #.*size*) d s))
    (loop until (= s #.*size*)
          do (setf (aref heap d) (aref heap s))
             (incf d)
             (incf s)
             (loop until (or (= s #.*size*)
                             (= (sbit bitmap s) 1))
                   do (incf s)))))
\end{verbatim}}

The hypothesis here is that the time consumed by the compaction phase
is determined by two elements:

\begin{enumerate}
\item A constant term that has to do with scanning the bitmap.
\item A term proportional to the number of elements that have to be
  moved.
\end{enumerate}

To test the hypothesis, we executed the function \texttt{move} with
three different bitmaps as shown in the following table:

\begin{tabular}{|r|r|}
\hline
Element count & CPU time (ms)\\
\hline
$1$ & $0.8$ \\
$2^{18}$ & $1.1$\\
$2^{19}-1$ & $1.4$\\
\hline
\end{tabular}

To avoid measuring the performance of the function \texttt{position},
the first line in the table was executed with a bitmap where the
element with index $1$ was equal to $1$.  The second line in the table
was executed with a bitmap where every other element was $1$.  The
third line in the table was executed with a bitmap where only element
$0$ was $0$.  In all cases, the test was run in a loop with $1000$
iterations.  The loop was the argument of the \texttt{time} macro, and
the right column of the table shows the result returned by
\texttt{time} divided by $1000$.

The hypothesis above seems confirmed where scanning the bitmap seems
to take around $0.8$ms and moving an element seems to take around
$1$ns per element.

Now, in a typical collection cycle, at least around half of the
elements of the heap would be dead (if not, objects would be
\emph{promoted}).  Furthermore, since objects that are allocated
together die together, the bitmap will contain long runs of $0$s and
long runs of $1$s, and because objects either die young or survive a
long time, the bitmap will typically start with a run of $1$s.  A
slightly more clever implementation of the bitmap would then test $64$
bits at a time rather than individual bits like we do in our test.  In
such an implementation, the time for managing the bitmap would be
negligible.  In a typical collection cycle, significantly fewer than
half of the elements would have to be moved.  The expected compaction
time would therefore be closer to $0.3$ms.

\subsection{Phase 3, building the table}

In order to test the performance of building the table, we used the
following function:

{\small\begin{verbatim}
(defun build-table (heap bitmap start)
  (let ((acc 0)
        (end start))
    (declare (type fixnum acc start end)
             (type (simple-vector #.*size*) heap)
             (type (simple-array bit (#.*size*)) bitmap)
             (optimize (speed 3) (safety 0) (debug 0)))
    (loop for address of-type fixnum from 0
          for prev of-type bit = 1 then bit
          for bit of-type bit across bitmap
          do (when (= bit 0)
               (when (= prev 1)
                 (setf (svref heap end) address)
                 (setf (svref heap (1+ end)) acc)
                 (incf end 2))
               (incf acc))
          finally (when (= prev 1)
                    (setf (svref heap end) address)
                    (setf (svref heap (1+ end)) acc)
                    (incf end 2))
                  (return end))))
\end{verbatim}}

In that program, the parameter \texttt{start} contains the total
number of live words in the heap, which is the same as the start index
into the heap where the table should be built.

As for the compaction phase, the hypothesis here is that the time
consumed by the compaction phase is determined by two elements:

\begin{enumerate}
\item A constant term that has to do with scanning the bitmap.
\item A term proportional to the number of entries in the table to be
  built, which is the same as the number of zones of dead elements.
\end{enumerate}

We further hypothesize that the performance of this phase is
independent of the \emph{size} of the zones of dead elements.

We start by testing the last hypothesis.  For that, we run the
function \texttt{build-table} above with two different bitmaps, both
containing $2^{10}$ zones of dead objects and the same number of zones
of live objects.  In the first test, the size of each live zone is a
single word.  In the second case, the size of each live zone is $2^8$
words.  The result of this test is shown in the following table:

\begin{tabular}{|r|r|}
\hline
Live zone size & CPU time (ms)\\
\hline
$1$ & $1.0$\\
$2^8$ & $1.0$\\
\hline
\end{tabular}

As we can see, the hypothesis that the sizes of the zones do not
influence the performance of this phase is confirmed.

Next we tested the function \texttt{build-table} for different sizes
of the table to be built.  The size of each live zone was a single
word.  The following table shows the result:

\begin{tabular}{|r|r|}
\hline
Table size & CPU time (ms)\\
\hline
$2^{1}$ & $1.0$\\
$2^{9}$ & $1.0$\\
$2^{11}$ & $1.0$\\
$2^{13}$ & $1.1$\\
$2^{15}$ & $1.4$\\
$2^{17}$ & $1.2$\\
\hline
\end{tabular}

It appears that the time to build the table is entirely dominated by
the time to traverse the bitmap.  We have no explanation as to why the
time does not increase monotonically with the table size, but we have
run this test numerous times and we still get this surprising result.
It appears that this anomaly does not manifest itself on computers
other than ours.

With improved bitmap management code, we believe that building the
table will typically take about the same as compacting the heap, i.e.,
around $0.3$ms.

\subsection{Phase 3, adjusting pointers}

In the worst case, every live pointer must be updated, and each such
pointer requires $n$ iterations to search the break table, where $n$
is the depth of that table.  The number of live pointers is the
difference of the size of the heap and the number of words required
for the break table.  In the worst case, the number of words required
for the break table is as small as possible (which is $2^{n+1}$), so
that the number of live pointers is $2^N - 2^{n+1}$.  The total amount
of work required can therefore be expressed as $n \cdot (2^N -
2^{n+1})$.  When $N = 19$, this function has a maximum for $n = 15$.

We can get an upper bound on the time to adjust the pointers by
creating a heap with roughly $2^{19} - 2^{16}$ words containing random
values in some interval $[0,K]$ followed by a break table with
$2^{16}$ words, with addresses in the interval $[0,K+1]$.

In order to search the break table, we used the following binary
search function:

{\small\begin{verbatim}
(defun binary-search (heap first last address)
  (declare (type (simple-vector #.*size*) heap)
           (type (integer 0 (#.*size*))
                 first last address)
           (optimize (speed 3) (debug 0) (safety 0)))
  (let ((f first)
        (l last))
    (declare (type (integer 0 (#.*size*)) f l))
    (loop until (= (- l f) 2)
          do (let* ((mid (1+ (ash (ash (+ l f) -2) 1)))
                    (elt (svref heap mid)))
               (declare (type fixnum elt))
               (if (< address elt)
                   (setf l mid)
                   (setf f mid))))
    (svref heap (1+ f))))
\end{verbatim}}

The following function was used to adjust the pointers of the heap:

{\small\begin{verbatim}
(defun adjust ()
  (declare (optimize (speed 3) (debug 0) (safety 0)))
  (loop with w = (worst-frontier 19)
        with l = (1- (expt 2 19))
        with g = *g*
        for i from 0 below w
        do (incf (aref *g* i)
                 (binary-search g w l i))))
\end{verbatim}}

On our computer, this function takes approximately $30$ms to run,
again on a heap containing $2^{19}$ words.  While this number may seem
high, we should keep in mind that it is pessimistic in several
different ways:

\begin{itemize}
\item This test assumes that almost $90\%$ of the heap is live.  When
  there are this many live objects, they will be \emph{promoted}, so a
  typical number should be significantly less than $50\%$.
\item It assumes that there are $2^{15}$ dead zones, which would make
  each zone very small.  A more typical situation is that dead zones
  are significantly larger, so that there are significantly fewer dead
  zones.
\item It assumes that every live word in the heap contains a pointer
  that needs to be adjusted.  More typically, many words contain
  immediate data, or pointers to older generations, and these pointers
  do not need to be adjusted.
\end{itemize}

Furthermore, there is a simple optimization that we think can
eliminate a large fraction of the table searches.  By keeping the
result ($a_i$, $d_i$, and $a_{i+1}$) between iterations and comparing
the pointer to be adjusted to these values, we think that in many
cases we will get a hit, simply because it is likely that two
consecutive pointers belong to the same live zone and refer to objects
in that zone.

An educated guess, then, is that a typical value for this phase would
be closer to $5$ms.

\subsection{Overall performance}

Taken together, the performance figures in the previous section
suggest that the entire compaction phase of our suggested
\emph{mark-and-compact} sliding collector would take around $6$ms on a
fairly modest desktop, with an absolute worst-case of around $30$ms.

Including the mark phase, there are two possibilities:

\begin{itemize}
\item Compaction dominates over marking.  Then the figures given above
  are reasonable estimates of the total time for the entire
  collection.
\item Marking dominates over compaction.  Then the figures given above
  are sufficiently good that our method is no more expensive than
  others such as mark-and-sweep and copying.
\end{itemize}

In addition, we should not forget the reasons for choosing a
sliding collector in the first place:

\begin{itemize}
\item Compared to a copying collector, only half the space is
  required, or, alternatively, the space available in the nursery is
  twice as big.  When this is the case, the number of required
  collections decreases, a situation which gives more time for
  short-lived objects to die, thereby decreasing the total amount of
  work spent in garbage collection.
\item Also compared to a copying collector, the relative ages of
  objects are kept intact, which decreases the risk of promoting
  objects that will die soon after a collection.
\item Compaction eliminates the potential problem of fragmentation.
\end{itemize}

All these factor contribute to decreasing the total cost of garbage
collection, although measuring the exact impact is of course very
hard.

It is not advisable to use our algorithm on the global heap, say a
heap that is three orders of magnitude larger than the one used in
our tests.  Even multiplying our timing results by a thousand would
imply a significant pause, but since the break table requires a
binary search, the time to search it would be proportional to the
logarithm of the size of the heap, penalizing performance even more.
To manage a global heap, we would recommend the technique by
Kermany et al \cite{Kermany:2006:CCI:1133981.1134023}, which has a
greater overhead, but has the advantage of being incremental,
concurrent, and parallel, thus effectively eliminating significant
pauses.

One might want to speculate about the performance of our algorithm
when the size of the heap is significantly larger than the size of the
cache, but much smaller than the global heap.  Such a situation might
be desirable if it turns out that the cost of cache misses is
compensated by the fact that more time between collections will allow
more objects to die.  For that reason, we re-ran our tests with a heap
that is $4$ times the size of the heap used for the previous tests.  We
observed a factor $10$ slowdown compared to the tests reported in this
section.  In order for this configuration to be advantageous, the
number of objects that die between two garbage-collection cycles has
to increase substantially, and whether this is the case depends on the
type of application being executed.  Only benchmarks in a final system
or simulations with real allocation traces can determine whether this
is the case.  Note, however, that even though overall throughput
\emph{might} improve with a larger heap, the time for an invocation of
the garbage collector becomes closer to what is tolerable for many
so-called \emph{soft real time} applications such as generation of
sound or images in real time.


\documentclass[]{article}

\usepackage{graphicx}
\usepackage{longtable}
\usepackage{hyperref}
\usepackage{color}
\usepackage{soul}
\usepackage{amsmath}

\DeclareRobustCommand{\hlcyan}[1]{{\sethlcolor{cyan}\hl{#1}}}
\DeclareRobustCommand{\hlgreen}[1]{{\sethlcolor{green}\hl{#1}}}
\DeclareRobustCommand{\hlred}[1]{{\sethlcolor{red}\hl{#1}}}
\DeclareRobustCommand{\hlyellow}[1]{{\sethlcolor{yellow}\hl{#1}}}
\DeclareRobustCommand{\hlorange}[1]{{\sethlcolor{orange}\hl{#1}}}

%opening
\title{MiT Single Variable Calculus, Fall 2010 \\ Pending Questions}
\author{Fakhir}

\begin{document}

\maketitle

\url{https://ocw.mit.edu/courses/mathematics/18-01sc-single-variable-calculus-fall-2010/}\\~\\

Questions that I had in my mind while doing this course: \\~\\

\section{Differentiation}
\subsection{Session 1, 2}
\begin{enumerate}
	\item How do we cut a cone using a plane so that it gives us the hyperbola $1/x$ (because this particular curve has a weird skewed shape. Pretty hard to imagine the resulting cross-section unless the cones are themselves skewed somehow.)

	\item The ``difference quotient'' formula for differentiation has delta x or dx in the denominator where $dx \rightarrow 0$. We can't just put dx to 0 ``INITIALLY'' because that will give us $0/0$. But after simplifying the formula (and some cancellations), as a very last step we plug 0 to $dx$ and everything works out magically ! So What is happening here? Is the simplification process changing anything? (even though it shouldn't)
\end{enumerate}

\subsection{Session 3}

\begin{enumerate}
	\item The geometric interpretation of derivative is that its the "slope of a line". Are there any other interpretations?
	Yes:
	\begin{enumerate}
		\item Rate of change (average change)
		\item ...
	\end{enumerate}

\end{enumerate}

\subsection{Session 5}

\begin{enumerate}
	\item Stupid question: Why do we study limits in "calculus"? Shouldn't this be a part of say functions or real analysis or something?

	\item Is the ``limit'' a ``linear operator''? i.e:
	$$\lim_{x\to\infty} \left(f(x)+g(x)\right) = \lim_{x\to\infty} f(x) + \lim_{x\to\infty} g(x)$$

	\item \textbf{My Comment:} Most of the proofs we do in math appear to be ``trial and error''. We keep on discarding things that do not work and accept things that do work (at least appear to). But once we reach at the end, the result may give an insight to something totally new, something that we were not even looking for initially, a novel insight into the reality (its like traveling to the unknown place through a worm hole or suddenly being illuminated by the divine).

	\item I didn't quite understand what he means by ``one sided limit'' in the MiT solutions file. In some places (Q4 and Q5) he has used ``no need to use one sided limit here because..'', not too sure what he is saying. I thought one sided limit means the special case when either $\lim_{x\to k^{+}}$ exists or $\lim_{x\to k^{-}}$ but not both. Am I missing something here?

	\item Show that $\sqrt{x}$ is a continuous function specially at $x=0$ (beware: it has one sided limit).

	\item Show that $|\sin{x}|$ is a continuous function specially at $x=0$
\end{enumerate}

\subsection{Session 7}
\begin{enumerate}
	\item What actually the trigonometric functions signify? i.e the $\sin$ function takes an angle and returns a real number. What does that number exactly represent? Geometric interpretation of trigonometric functions?
\end{enumerate}

\subsection{Session 8}
\begin{enumerate}
	\item \textbf{Clip 1:} He proved the limit $\lim_{\Delta x\to0}\frac{\sin(\Delta x)}{\Delta x}= 1$ by drawing just a figure and then ``conjecturing'' the conclusion. No rigorous mathematical steps that lead to the final answer. In the notes pdf he also substitutes a bunch of values for $\delta x$ in order to ``convince'' us about the answer.

	\item \textbf{Clip 2:} Similarly, $\lim_{\Delta x\to0}\frac{1-\cos(\Delta x)}{\Delta x}= 0$ proved by mere geometric ``intuition''. \\

	\hlgreen{Answer to above two points:} Just search the web online such as \href{https://math.stackexchange.com/questions/75130/how-to-prove-that-lim-limits-x-to0-frac-sin-xx-1}{here}.\\

	But now as opposed to the methods in previous sections he is trying to compute the limit very near to zero BUT NOT AT zero. So the behavior of both the numerator and denominator are being studied as $\theta$ gets very small BUT NOT EXACTLY ZERO. In previous methods we somehow tried to compute the limits exactly at $\Delta x\to 0$ by somehow trying to cancel out $\Delta x$. This means many varying methods to compute the limits, I'm confused !!!

	\item \textbf{Clip 4:} \hlyellow{Need to review this proof again. I was unable to understand how he proved the congruent $\theta$. I didn't understand either of the methods but specially didn't get that rotation/translation argument.}\\

	\underline{To ask:} The Geometric proofs do not appear to be as ``rigorous'' as analytical ones. Specially in this proof no matter how small you get, $\Delta x$ can become very very tiny but will never be 0. So how on earth are we deducing exact limits from approximations?

\end{enumerate}

\end{document}

mit-ocw/[in progress] Single Variable Calculus, 2010/Penging Questions/Pending Questions.tex


\documentclass[11pt]{article}

\usepackage{common}
\title{HW0: Sample Assignment}
\author{Alexander Rush \\ srush@seas.harvard.edu \and Sam Wiseman \\ swiseman@seas.harvard.edu }
\begin{document}

\maketitle{}
\section{Introduction}

This note lays out our expectation for a homework submission in
\textit{CS287: Statistical Natural Language Processing}. While you do
not have to follow this template to the letter, we do expect that
write-ups have a very clear structure and cover all the elements
described in this note. With this in mind, the burden is on the
presenter to demonstrate why deviations from the standard are
necessary.

All write-ups should include a short introduction. In this section you
should summarize the underlying problem in high-level language and
describe the extensions that you have decided to propose in your
implementation. When you describe these extensions you should
carefully cite the papers of interest. For instance, it will often be
useful to cite the work seen in class
\citep{murphy2012machine}. Alternatively, you can also cite papers
inline, for instance the work of \citet{berger1996maximum}.

\section{Problem Description}

In general, homeworks will be specified using informal
language. As part of the assignment, we expect you to write-out a
definition of the problem and your model in formal language. For this
class, we will use the following notation:

\begin{itemize}
\item $\boldb, \boldm$;  bold letters for vectors.
\item $\boldB, \boldM$;  bold capital letters for matrices.
\item $\mcB, \mcM$;  script-case for sets.
\item $b_i, x_i$; lower case for scalars or indexing into vectors.
\end{itemize}

For instance in natural language processing, it is common to use
discrete sets like $\mcV$ for the vocabulary of the language, or $\mcT$ for a
tag set of the language.  We might also want one-hot vectors
representing words. These will be of the type
$\boldv \in \{0,1\}^{|\mcV|}$. In a note, it is crucial to define the
types of all variables that are introduced. The problem description is the
right place to do this.

% NLP is also
% full of sequences. For instance sentences, $w_1, \ldots, w_N$, where
% here $N$ is a constant length and $w_i \in \mcV$ for all
% $i \in \{1, \ldots N\}$. If we pretend sentences are all the same
% length, we can have scoring function over sentences,
% $s : \mcV^N \mapsto \reals$.  One might be defined as:

% \[ s(w_1, \ldots, w_N) = \sum_{i = 1}^N p(w_i | w_{i-2}, w_{i-1}), \]

% \noindent where $p$ is the bigram probability, which we will cover later in the class.

\section{Model and Algorithms}

Here you specify the model itself. This section should formally
describe the model used to solve the task proposed in the previous
section. This section should try to avoid introducing new vocabulary
or notation, when possible use the notation from the previous section.
Feel free to use the notation from class, but try to make the note
understandable as a standalone piece of text.

This section is also a great place to include other material that
describes the underlying structure and choices of your model, for
instance here are some example tables and algorithms from full
research papers:

\begin{itemize}
\item diagrams of your model,

  \begin{center}
    \includegraphics[width=0.4\textwidth]{network}
  \end{center}
\item feature tables,

  \begin{center}
    \begin{tabular}{@{}lll@{}}
      \toprule
      &\multicolumn{2}{c}{Mention Features  } \\
      & Feature & Value Set\\
      \midrule
      & Mention Head & $\mcV$ \\
      & Mention First Word & $\mcV$ \\
      & Mention Last Word & $\mcV$ \\
      & Word Preceding Mention & $\mcV$ \\
      & Word Following Mention & $\mcV$\\
      & \# Words in Mention & $\{1, 2, \ldots \}$ \\
      & Mention Type & $\mathcal{T}$ \\
      \bottomrule
    \end{tabular}
  \end{center}

\item pseudo-code,

  \begin{algorithmic}[1]
    \Procedure{Linearize}{$x_1\ldots x_N$, $K$, $g$}
    \State{$B_0 \gets \langle (\langle \rangle, \{1, \ldots, N\}, 0, \boldh_0, \mathbf{0})  \rangle$}
    \For{$m = 0, \ldots, M-1$ }
    \For{$k = 1, \ldots, |B_m|$}
    \For{$i \in \mcR$}
    \State{$(y, \mcR, s, \boldh) \gets \mathrm{copy}(B_m^{(k)})$}
    \For{word $w$ in phrase $x_i$}
    \State{$y \gets y $ append $w$ }
    \State{$s \gets s + \log q(w, \boldh) $ }
    \State{$\boldh \gets \delta(w, \boldh)$}
    \EndFor{}
    \State{$B_{m+|w_i|} \gets B_{m+|w_i|} + (y, \mcR - i, s,   \boldh)$}
    \State{keep top-$K$ of $B_{m+|w_i|}$ by $f(x, y) + g(\mcR)$}
    \EndFor{}
    \EndFor{}
    \EndFor{}
    \State{\Return{$B_{M}^{(k)}$}}
    \EndProcedure{}
  \end{algorithmic}

\end{itemize}

\section{Experiments}

Finally we end with the experimental section. Each assignment will make clear the main experiments and baselines that you should run. For these experiments you should present a main results table. Here we give a sample Table~\ref{tab:results}. In addition to these results you should describe in words what the table shows and the relative performance of the models.

Besides the main results we will also ask you to present other results
comparing particular aspects of the models. For instance, for word
embedding experiments, we may ask you to show a chart of the projected
word vectors. This experiment will lead to something like
Figure~\ref{fig:clusters}. This should also be described within the
body of the text itself.

\begin{table}[h]
\centering
\begin{tabular}{llr}
 \toprule
 Model &  & Acc. \\
 \midrule
 \textsc{Baseline 1} & & 0.45\\
 \textsc{Baseline 2} & & 2.59 \\
 \textsc{Model 1} & & 10.59  \\
 \textsc{Model 2} & &13.42 \\
 \textsc{Model 3} & & 7.49\\
 \bottomrule
\end{tabular}
\caption{\label{tab:results} Table with the main results.}
\end{table}

\begin{figure}
  \centering
  \includegraphics[width=6cm]{cluster_viz}
  \caption{\label{fig:clusters} Sample qualitative chart.}
\end{figure}

\section{Conclusion}

End the write-up with a very short recap of the main experiments and the main results. Describe any challenges you may have faced, and what could have been improved in the model.

\bibliographystyle{apalike}
\bibliography{writeup}

\end{document}