\documentclass{article}
\newsavebox{\oldepsilon}
\savebox{\oldepsilon}{\ensuremath{\epsilon}}
\usepackage[minionint,mathlf,textlf]{MinionPro} % To gussy up a bit
\renewcommand*{\epsilon}{\usebox{\oldepsilon}}
\usepackage[margin=1in]{geometry}
\usepackage{graphicx} % For .eps inclusion
%\usepackage{indentfirst} % Controls indentation
\usepackage[compact]{titlesec} % For regulating spacing before section titles
\usepackage{adjustbox} % For vertically-aligned side-by-side minipages
\usepackage{array, amsmath,  mhchem}
\usepackage[hidelinks]{hyperref}
\usepackage{courier, subcaption}
\usepackage{multirow, enumerate}
\usepackage[autolinebreaks,framed,numbered]{mcode}

\usepackage{float}
\restylefloat{table}

\pagenumbering{gobble}
\setlength\parindent{0 cm}
\renewcommand{\arraystretch}{1.2}
\begin{document}
\large

MCB 135 Problem Set 5 \hfill Due Monday, March 9, 2015 at 2:30 PM

\section*{Problem 1: Working with proteomics data (50 points)}

In this problem, you will explore the protein-protein interaction network recently described by \linebreak \href{http://www.cell.com/abstract/S0092-8674\%2814\%2901422-6}{Rolland et al. (2014)}. The interactions are provided in tab-delimited format on the course \linebreak website (\mcode{rolland.tsv}). Each line describes one interaction: the two columns give the \href{http://www.ncbi.nlm.nih.gov/gene}{NCBI \linebreak Entrez gene identification numbers} for the two proteins participating in the interaction.

\begin{enumerate}[a)]
\setlength{\itemsep}{0pt}
\item Determine the number of interactions in which each protein participates. (The MATLAB \linebreak functions \mcode{unique()} and \mcode{histcounts()} may be useful.) Find the average number of interactions per protein, $\mu$.
\item Plot the distribution of interactions per protein -- i.e., the probability that a randomly-chosen protein participates in $x$ interactions -- on log-log axes.
\end{enumerate}
Under the hypothesis that all possible edges in a network occur with equal probability, the number of edges per node is expected to follow a Poisson distribution.
\begin{enumerate}[a)]
\setlength{\itemsep}{0pt}
\setcounter{enumi}{2}
\item Plot a Poisson distribution with parameter $\mu$ on the same axes as the true distribution of interaction counts. In MATLAB, this can be done with the function \mcode{poisspdf()}.
\end{enumerate}
An alternative ``preferential attachment" model holds that new edges are connected preferentially to nodes that already have many edges. In the resulting ``scale-free" network, the distribution of edges per node follows a power law.
\begin{enumerate}[a)]
\setlength{\itemsep}{0pt}
\setcounter{enumi}{3}
\item  Fit a power law to the data and plot it on the same axes as in parts (b) and (c). In MATLAB, this can be done with the function \mcode{fit(x,y,'power1')}.
\item Which distribution -- Poisson or power law -- is a better fit to the Rolland et al. data? Explain why this distribution be expected in light of how protein-protein interactions evolve.
\item How many self-interactions are reported in this data set? How many self-interactions would you have expected given the number of proteins and the total number of interactions? (You may assume that all edges are equally likely, even if that assumption does not match your results from part e.)
\item Provide a biological justification for any discrepancy between the two values found in part f.
\end{enumerate}

\section*{Problem 2: Autorepression with a time delay (50 points)}

In lecture, we studied an autoregulating transcriptional repressor to find its protein abundance $p(t)$ as a function of an input, $x(t)$, that drives the gene's transcription:

\begin{center}\includegraphics[width=0.4 \textwidth]{autorepression.pdf}\end{center}

We now consider a modified version of this model in which there is a delay $\tau$ between mRNA production and translation\footnote{In eukaryotes, transcriptional elongation, mRNA processing, and nuclear export introduce delays (usually on the order of minutes) between the initiation of transcription and the initiation of translation.}. This delay is reflected in the modified rate equations:
\begin{eqnarray}
\begin{aligned}
\frac{d m}{dt} & = & c_m a(t) - \gamma_m m(t)\\
\frac{d p}{d t} & = & c_p m(t-\tau) - \gamma_p p(t) \label{eqn:rateequations}
\end{aligned}
\end{eqnarray}

\noindent where $a(t)$, the mRNA's net expression level, is $x(t)-k p(t)$ (the activating input minus the strength of the negative autoregulation).

\begin{enumerate}[a)]
\setlength{\itemsep}{0pt}
\item Show using the definition of the Laplace transform that for a function $f(t)$ which is non-zero only for positive $t$,
\[ \mathcal{L}\left[f(t-\tau)\right] = e^{-s\tau} \mathcal{L}\left[f(t)\right] = e^{-s\tau} \tilde{f}(s)\]
\item Transform rate equations (\ref{eqn:rateequations}) and simplify to find an expression for the forward transfer \linebreak  function $G(s) =\tilde{p}(s)/\tilde{a}(s)$. Assume that $m(0)=p(0)=0$.
\item Find an expression for the closed loop transfer function $\tilde{p}(s)/\tilde{x}(s)$ in terms of $k$ and $G(s)$.
\end{enumerate}
The autorepression system is unstable if any poles of the closed loop transfer function have a positive real part.
\begin{enumerate}[a)]
\setlength{\itemsep}{0pt}
\setcounter{enumi}{3}
\item Show that the only poles of the closed loop transfer function which could have a positive real part are the roots of $1 + kG(s)$.
\end{enumerate}
The result in part (d) implies that the system will be unstable if any roots of $1 + kG(s)$ have a positive real part. The Nyquist stability criterion can be used to determine whether any such roots exist, but in order to apply it, we must first know the number of poles of $1 + kG(s)$ with a positive real part.
\begin{enumerate}[a)]
\setlength{\itemsep}{0pt}
\setcounter{enumi}{4}
\item Demonstrate that the number of poles of $1 + kG(s)$ with positive real part is zero.
\end{enumerate}
The absence of poles with positive real part implies that the number of zeros of $1 + kG(s)$ with positive real part is equal to the number of times that the Nyquist plot of $1 + kG(s)$ encloses the origin clockwise. Equivalently, the system is stable if the Nyquist plot of $G(s)$ does not enclose the point $-1/k$.

\begin{enumerate}[a)]
\setlength{\itemsep}{0pt}
\setcounter{enumi}{5}
\item Show that $G(s) \to 0$ as $|s| \to \infty$. [The Nyquist plot is therefore well approximated by $G(s=i\omega)$ for $\omega \in (-n, n)$ with sufficiently large $n$: we can ignore other points on the Nyquist contour.]
\item Use MATLAB to plot $G(i\omega)$ on the complex plane for $\omega \in (-100, 100)$. Use $c_i=\gamma_i=1$ and two values of $\tau$: 0 and 1.
\item Show using your $\tau=1$ Nyquist plot that there exists a threshold gain $k^*$ above which the system is unstable. (Though you will not show it explicitly, this is true for all $\tau,c_i, \gamma_i>0$.) Confirm from the $\tau=0$ Nyquist plot that the system is stable for all $k$ when there is no time delay.
\item Approximately what value of the repressor's gain $k$ would be ideal when $\tau=c_i=\gamma_i=1$? (That is, what value of $k$ minimizes the deviation of $p(t)$ from zero in this case?)
\item The repressor's gain $k$ can be adjusted through mutation and natural selection. Describe two types of mutations that could alter the value of $k$.
\end{enumerate}

\end{document}

problem sets/ps5/tex/mcb 135 problem set 5.tex


\documentclass{article}

\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amsfonts}
\usepackage{tikz}
\usepackage{physics}
\usepackage{amssymb}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}

\usetikzlibrary{automata,positioning}

% Basic Document Settings
%

\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\pagestyle{fancy}
\lhead{\hmwkAuthorName}
\chead{\hmwkClass\ : \hmwkTitle}
\rhead{\firstxmark}
\lfoot{\lastxmark}
\cfoot{\thepage}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

\setlength\parindent{0pt}

%
% Create Problem Sections
%
\newcommand{\be}{\begin{equation}}
\newcommand{\ee}{\end{equation}}
\newcommand{\bes}{\begin{equation*}}
\newcommand{\ees}{\end{equation*}}
\newcommand{\bea}{\begin{flalign*}}
\newcommand{\eea}{\end{flalign*}}

\newcommand{\enterProblemHeader}[1]{
    \nobreak\extramarks{}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
}

\newcommand{\exitProblemHeader}[1]{
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \stepcounter{#1}
    \nobreak\extramarks{Problem \arabic{#1}}{}\nobreak{}
}

\setcounter{secnumdepth}{0}
\newcounter{partCounter}
\newcounter{homeworkProblemCounter}
\setcounter{homeworkProblemCounter}{1}
\nobreak\extramarks{Problem \arabic{homeworkProblemCounter}}{}\nobreak{}

%
% Homework Problem Environment
%
% This environment takes an optional argument. When given, it will adjust the
% problem counter. This is useful for when the problems given for your
% assignment aren't sequential. See the last 3 problems of this template for an
% example.
%
\newenvironment{homeworkProblem}[1][-1]{
    \ifnum#1>0
        \setcounter{homeworkProblemCounter}{#1}
    \fi
    \section{Problem \arabic{homeworkProblemCounter}}
    \setcounter{partCounter}{1}
    \enterProblemHeader{homeworkProblemCounter}
}{
    \exitProblemHeader{homeworkProblemCounter}
}

%
% Homework Details
%   - Title
%   - Due date
%   - Class
%   - Section/Time
%   - Instructor
%   - Author
%

\newcommand{\hmwkTitle}{Assignment\ \#6}
\newcommand{\hmwkDueDate}{Due 16th November 2018}
\newcommand{\hmwkClass}{Classical Mechanics}
\newcommand{\hmwkClassTime}{}
\newcommand{\hmwkClassInstructor}{Prof.Manas Kulkarni}
\newcommand{\hmwkAuthorName}{\textbf{Aditya Vijaykumar}}

%
% Title Page
%

\title{
    %\vspace{2in}
    \textmd{\textbf{\hmwkClass:\ \hmwkTitle}}\\
    \normalsize\vspace{0.1in}\small{\hmwkDueDate\ }\\
%    \vspace{3in}
}

\author{\hmwkAuthorName}
\date{}

\renewcommand{\part}[1]{\textbf{\large Part \Alph{partCounter}}\stepcounter{partCounter}\\}

%
% Various Helper Commands
%

% Useful for algorithms
\newcommand{\alg}[1]{\textsc{\bfseries \footnotesize #1}}

% For derivatives
\newcommand{\deriv}[1]{\frac{\mathrm{d}}{\mathrm{d}x} (#1)}

% For partial derivatives
\newcommand{\pderiv}[2]{\frac{\partial}{\partial #1} (#2)}

% Integral dx
\newcommand{\dx}{\mathrm{d}x}

% Alias for the Solution section header
\newcommand{\solution}{\textbf{\large Solution}}

% Probability commands: Expectation, Variance, Covariance, Bias
\newcommand{\E}{\mathrm{E}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\Cov}{\mathrm{Cov}}
\newcommand{\Bias}{\mathrm{Bias}}

\begin{document}
\maketitle

\begin{homeworkProblem}[1]
	Liouville Theorem states that in a Hamiltonian system, the total phase space volume is constant in time.

	Let our system consist of $ N $ points $ (q_k, p_k) $ in a $ 2N $ dimensional phase space. The volume of the phase space is,
	\begin{equation*}
	V = \prod_{i} d q_i \qq{and} \tilde{V} = \prod_{i} d \tilde{q}_i
	\end{equation*}
	where the tilded coordinates represents the volume at a later time. We know from Hamilton's equations,
	\begin{equation*}
	\tilde{q}_i = q_i + \pdv{H}{p_i} dt \qq{and} \tilde{p}_i = p_i - \pdv{H}{q_i} dt
	\end{equation*}
	We know for a fact that the volume transformation is related as follows,
	\begin{align*}
	\tilde{V} &= \det(J) V \\
	J &= \mqty[\pdv{\tilde{q}_i}{q_j} & \pdv{\tilde{q}_i}{p_j} \\ \pdv{\tilde{p}_i}{q_j} & \pdv{\tilde{p}_i}{q_j}]\\
	J &= \mqty[1 + \pdv[2]{H}{q_j}{p_i} dt & \pdv[2]{H}{p_i} dt \\ -\pdv[2]{H}{q_j} dt & 1 - \pdv[2]{H}{p_j}{q_j} dt] \\
	\det(J) &= 1 + \order{dt^2}
	\end{align*}
	Hence upto first order, $ \tilde{V} = V$. Hence proved. (files for the next part are attached)
\end{homeworkProblem}

\begin{homeworkProblem}[2]
	Transformations of coordinates $ (q,p,t) \rightarrow (Q,P,t)$ which preserves the form of Hamilton's equations are called canonical transformations. So, by definition,
	\begin{equation*}
	\dot{p} = \pdv{H}{q} \qq{,} \dot{q} = - \pdv{H}{p} \qq{and} \dot{P} = \pdv{K}{Q} \qq{,} \dot{Q} = -\pdv{K}{P}
	\end{equation*}
	The definition also implies that,
	\begin{align*}
	\delta(p \dot{q} - H) = 0 &\qq{and} \delta(P \dot{Q} - K) = 0\\
	\lambda(p \dot{q} - H) &= P \dot{Q} - K + \dv{F}{t}
	\end{align*}
	We deal with the $ \lambda = 1 $ case. The $ \dv{F}{t} $ term comes from the fact that Lagrangians are not unique and we can always add a total time derivative term without changing the equations of motion. If the above condition is satisfied, the transformation $ (q,p,t) \rightarrow (Q,P,t)$ is guaranteed to be canonical, and the function $ F $ is called a generating function. We deal with four classes of generating functions case-by-case,
	\begin{itemize}
		\item $ F = F_1 (q,Q,t) $,
		\begin{equation*}
		p \dot{q} - H = P \dot{Q} - K + \dv{F_1}{t} = P \dot{Q} - K + \pdv{F_1}{q}\dot{q} + \pdv{F_1}{Q}\dot{Q} + \pdv{F_1}{t}
		\end{equation*}
		As $ q $ and $ Q $ are independent, the coefficients should vanishh independently, such that $ K = H + \pdv{F_1}{t} $. This implies,
		\begin{equation*}
		\pdv{F_1}{q} = p \qq{and} \pdv{F_1}{Q} = -P
		\end{equation*}

		\item $ F = F_2 (q,P,t) - QP$,
		\begin{equation*}
		p \dot{q} - H = P \dot{Q} - K + \dv{F_2}{t} - \dv{(QP)}{t} = P \dot{Q} - K + \pdv{F_2}{q}\dot{q} + \pdv{F_2}{P}\dot{P} + \pdv{F_2}{t} - P \dot{Q} - Q \dot{P}
		\end{equation*}
		\begin{equation*}
		\implies \pdv{F_2}{q} = p \qq{and} \pdv{F_2}{P} = Q
		\end{equation*}
		\item $ F = F_3 (p,Q,t) + qp$,
		\begin{equation*}
		p \dot{q} - H = P \dot{Q} - K + \dv{F_3}{t} + \dv{(qp)}{t} = P \dot{Q} - K + \pdv{F_3}{Q}\dot{Q} + \pdv{F_3}{p}\dot{p} + \pdv{F_3}{t} + p \dot{q} + q \dot{p}
		\end{equation*}
		\begin{equation*}
		\implies \pdv{F_3}{Q} = -P \qq{and} \pdv{F_2}{p} = -q
		\end{equation*}
		\item $ F = F_4 (p,P,t) + qp - QP$,
		\begin{equation*}
		p \dot{q} - H = P \dot{Q} - K + \dv{F_4}{t} + \dv{(qp - QP)}{t} = P \dot{Q} - K + \pdv{F_4}{P}\dot{P} + \pdv{F_4}{p}\dot{p} + \pdv{F_4}{t} + p \dot{q} + q \dot{p} - P \dot{Q} - Q \dot{P}
		\end{equation*}
		\begin{equation*}
		\implies \pdv{F_4}{P} = Q \qq{and} \pdv{F_4}{p} = -q
		\end{equation*}
	\end{itemize}

	\textbf{Part (b)}\\
	We first use the Poisson Bracket invariance approach. We are given,
	\begin{equation*}
	Q_1 = q_1 \qq{,} Q_2 = p_2 \qq{,} P_1 = p_1 - 2 p_2 \qq{,} P_2 = -2 q_1 - q_2
	\end{equation*}
	Consider $ \pb{Q_1}{Q_2} $,
	\begin{align*}
	\pb{Q_1}{Q_2} &= \sum_{i =1}^{2} \pdv{Q_1}{q_i} \pdv{Q_2}{p_i} - \pdv{Q_1}{p_i} \pdv{Q_2}{q_i} = 0 \\
	\pb{P_1}{P_2} &= \sum_{i =1}^{2} \pdv{P_1}{q_i} \pdv{P_2}{p_i} - \pdv{P_1}{p_i} \pdv{P_2}{q_i} =  - \pdv{P_1}{p_1} \pdv{P_2}{q_1} - \pdv{P_1}{p_2} \pdv{P_2}{q_2} = 2-2 = 0   \\
	\pb{Q_1}{P_2} &= \sum_{i =1}^{2} \pdv{Q_1}{q_i} \pdv{P_2}{p_i} - \pdv{Q_1}{p_i} \pdv{P_2}{q_i} =   \pdv{Q_1}{q_1} \pdv{P_2}{p_1} = 0 \\
	\pb{Q_2}{P_1} &= \sum_{i =1}^{2} \pdv{Q_2}{q_i} \pdv{P_1}{p_i} - \pdv{Q_2}{p_i} \pdv{P_1}{q_i} =  - \pdv{Q_2}{p_2} \pdv{P_1}{q_2} = 0\\
	\pb{Q_1}{P_1} &= \sum_{i =1}^{2} \pdv{Q_1}{q_i} \pdv{P_1}{p_i} - \pdv{Q_1}{p_i} \pdv{P_1}{q_i} =  \pdv{Q_1}{q_1} \pdv{P_1}{p_1}  = 1\\
	\pb{Q_2}{P_2} &= \sum_{i =1}^{2} \pdv{Q_2}{q_i} \pdv{P_2}{p_i} - \pdv{Q_2}{p_i} \pdv{P_2}{q_i} = - \pdv{Q_2}{p_2} \pdv{P_2}{q_2}  = 1
	\end{align*}
	Hence, as $ \pb{Q_i}{P_j} = \delta_{ij}, \pb{Q_i}{Q_j} = 0,  \pb{P_i}{P_j}  = 0$, the transformation is canonical. We now use the symplectic approach. If we denote $ X = \mqty[Q_1 & Q_2  & P_1 & P_2]^T, x = \mqty[q_1 & q_2  & p_1 & p_2]^T  $, then $ X = Mx $ where $ M $ is the transformation matrix. From the definitions of the $ X $, we can see that,
	\begin{equation*}
	M = \left(
	\begin{array}{cccc}
	1 & 0 & 0 & 0 \\
	0 & 0 &0 & 1 \\
	0 & 0 & 1 & -2 \\
	-2 & -1 & 0 & 0 \\
	\end{array}
	\right)
	\end{equation*}
	For the transformation to be a canonical transformation, $ M^T J M = J $, where,
	\begin{align*}
	J &= \left(
	\begin{array}{cccc}
	0 & 0 & 1 & 0 \\
	0 & 0 &0 & 1 \\
	-1 & 0 & 0 & 0 \\
	0 & -1 & 0 & 0 \\
	\end{array}
	\right)
	\end{align*}
	\begin{align*}
	M^T J M &= \left(
	\begin{array}{cccc}
	1 & 0 & 0 & 0 \\
	0 & 0 &0 & 1 \\
	0 & 0 & 1 & -2 \\
	-2 & -1 & 0 & 0 \\
	\end{array}
	\right) \left(
	\begin{array}{cccc}
	0 & 0 & 1 & 0 \\
	0 & 0 &0 & 1 \\
	-1 & 0 & 0 & 0 \\
	0 & -1 & 0 & 0 \\
	\end{array}
	\right) \left(
	\begin{array}{cccc}
	1 & 0 & 0 & 0 \\
	0 & 0 &0 & 1 \\
	0 & 0 & 1 & -2 \\
	-2 & -1 & 0 & 0 \\
	\end{array}
	\right) \\
	&= \left(
	\begin{array}{cccc}
	0 & 0 & 1 & 0 \\
	0 & -1 &0 & 0 \\
	-1 & 2 & 0 & 1 \\
	0 & 0 & -2 & 0 \\
	\end{array}
	\right)
	\left(
	\begin{array}{cccc}
	1 & 0 & 0 & 0 \\
	0 & 0 &0 & 1 \\
	0 & 0 & 1 & -2 \\
	-2 & -1 & 0 & 0 \\
	\end{array}
	\right) \\
	M^T J M &= \left(
	\begin{array}{cccc}
	0 & 0 & 1 & 0 \\
	0 & 0 &0 & 1 \\
	-1 & 0 & 0 & 0 \\
	0 & -1 & 0 & 0 \\
	\end{array}
	\right) = J
	\end{align*}
	Hence, it is a canonical transformation.\\
	\textbf{Part (c)}
	\begin{align*}
	l_i &= \epsilon_{i j k} x_j p_k \qq{using the Einstein summation convention}
	\end{align*}
	We now note the following,
	\begin{align*}
	\pb{l_i}{l_j} &= \epsilon_{i a b} \epsilon_{j m n} \pb{x_a p_b}{x_m p_n} \\
	&= \epsilon_{i a b} \epsilon_{j m n} \pb{x_a p_b}{x_m p_n}\\
	&= \epsilon_{i a b} \epsilon_{j m n} (\pb{x_a }{ p_n} x_m p_b + \pb{p_b}{x_m} x_a p_n)\\
	&= \epsilon_{i a b} \epsilon_{j m n} ( \delta_{an} x_m p_b - \delta_{bm} x_a p_n) \\
	&= \epsilon_{i n b} \epsilon_{j m n} x_m p_b - \epsilon_{i a m} \epsilon_{j m n}  x_a p_n \\
	&= - \epsilon_{i b n} \epsilon_{j m n} x_m p_b + \epsilon_{i m a} \epsilon_{j m n}  x_a p_n \\
	&= - (\delta_{ij} \delta_{bm} - \delta_{im} \delta_{j b}) x_m p_b + (\delta_{ij} \delta_{an} - \delta_{aj} \delta_{in}) x_a p_n \\
	&= - \delta_{ij}  x_b p_b  + x_i p_j + \delta_{ij}  x_a p_a  -  x_j p_i \\
	&= + x_i p_j -  x_j p_i \\
	\pb{l_i}{l_j} &= \epsilon_{i j k} l_k
	\end{align*}
	\begin{align*}
	\pb{x_i}{l_j} &= \epsilon_{j m n} \pb{x_i}{ x_m p_n}\\
	&= \epsilon_{j m n} x_m  \pb{x_i}{ p_n}\\
	&= \epsilon_{j m n} x_m  \delta_{in}\\
	\pb{x_i}{l_j} &= \epsilon_{i j m} x_m
	\end{align*}
	\begin{align*}
	\pb{p_i}{l_j} &= \epsilon_{j m n} \pb{p_i}{ x_m p_n}\\
	&= \epsilon_{j m n} p_n  \pb{p_i}{ x_m}\\
	&= -\epsilon_{j m n} p_n  \delta_{im}\\
	\pb{p_i}{l_j}&= \epsilon_{i j n} p_n
	\end{align*}
\end{homeworkProblem}

\begin{homeworkProblem}[3]
	We are given the Hamiltonian and generating function,
	\begin{equation*}
	H = \dfrac{p^2}{2} + \dfrac{\omega^2 x^2 }{2} + \alpha x^3 + \beta x p^2 \qq{and} \phi = xP + ax^2 P + bP^3
	\end{equation*}
	$ \phi = \phi (x,P) $. For $ \phi $ to be a canonical transformation,
	\begin{align*}
	\pdv{\phi}{x} = p &\qq{and} \pdv{\phi}{P} = Q\\
	\implies P + 2 a x P = p &\qq{and} x + ax^2 + 3b P^2 = Q \\
	\implies P \sqrt{-12 a b P^2+4 a Q+1} = p &\qq{and}\frac{\sqrt{-12 a b P^2+4 a Q+1}-1}{2 a} = x
	\end{align*}
	where we have only considered the $ x $ root with positive sign before the discriminant. Then,
	\begin{align*}
	K(Q, P) &= \frac{\alpha  \left(\sqrt{-12 a b P^2+4 a Q+1}-1\right)^3}{8 a^3}+\frac{\omega ^2 \left(\sqrt{-12 a b P^2+4 a Q+1}-1\right)^2}{8 a^2} \\
	&+\frac{\beta  P^2 \left(-12 a b P^2+4 a Q+1\right) \left(\sqrt{-12 a b P^2+4 a Q+1}-1\right)}{2 a}+ \frac{1}{2} P^2 \left(-12 a b P^2+4 a Q+1\right)
	\end{align*}
	Expanding the above upto third order, we have,
	\begin{align*}
	K(Q, P) &= Q^3 \left(P^2 \left(-30 a^2 b \omega ^2-2 a^2 \beta +36 \alpha  a b\right)-a \omega ^2+\alpha \right)+Q^2 \left(P^2 \left(9 a b \omega ^2+3 a \beta -9 \alpha  b\right)+\frac{\omega ^2}{2}\right)\\&+P^2 Q \left(2 a-3 b \omega ^2+\beta \right)+\frac{P^2}{2}
	\end{align*}
	As anharmonic terms of third order should not be present, we can see from above that,
	\begin{equation*}
	- a \omega^2 + \alpha = 0 \qq{and} 2 a-3 b \omega ^2+\beta = 0 \implies a = \dfrac{\alpha}{\omega^2} \qq{and} b = \dfrac{1}{3 \omega^2} \qty(\dfrac{2 \alpha}{\omega^2} + \beta)
	\end{equation*}
	Now we need to find $ \dot{x} $. From Hamilton's equation of motion we have,
	\begin{align*}
	\dot{x} = \pdv{H}{p} = p (1 + 2 \beta x) \implies p = \dfrac{\dot{x}}{1 + 2 \beta x} \\
	\dot{p} = -\dfrac{2 \beta \dot{x}^2 - \ddot{x} (1 + 2 \beta x) }{(1 + 2 \beta x)^2} = -\pdv{H}{x} = -\omega^2 x - 3 \alpha x^2 - \beta p^2\\
	\implies -{2 \beta \dot{x}^2 - \ddot{x} (1 + 2 \beta x) } = -(\omega^2 x - 3 \alpha x^2) (1 + 2 \beta x)^2 - \beta \dot{x}^2 \\
	\implies {\ddot{x} (1 + 2 \beta x) -\beta \dot{x}^2 } + (\omega^2 x - 3 \alpha x^2) (1 + 2 \beta x)^2 = 0
	\end{align*}
	$ x(t) $ will be given by the solution of this equation.

	\textbf{Part (b)}\\
	\begin{itemize}
		\item $ \phi(\va{r}, \va{P}) = (\va{r} \cdot \va{P}) + (\delta \va{a} \cdot \va{P}) $\\
		This looks like $ F_2 (q,P) $. From the results of Problem 2, we can then write,
		\begin{align*}
		\pdv{\Phi}{r} = p_r = P_r \qq{,} \pdv{\Phi}{\theta} = p_\theta =0 \qq{,} \pdv{\Phi}{\phi} = p_\phi = 0 \\
		\pdv{\Phi}{P_r} = Q_r =  r + \delta a_x\qq{,} \pdv{\Phi}{P_\theta} = Q_\theta = \delta a_\theta \qq{,} \pdv{\Phi}{P_\phi} = Q_\phi = \delta a_\phi
		\end{align*}
		as $  r + \delta a = Q $, it is evident that the transformation is a translation by constant, as the momentum remains the same but the coordinates get shifted by a constant amount.

		\item $ \Phi(\va{r}, \va{P}) = (\va{r} \cdot \va{P}) + (\va{\delta \psi} \cdot \va{r} \cross \va{P}) $ \\
		This looks like $ F_2 $ too. We have,
		\begin{equation*}
		p \cdot \delta \psi = \qty(\dv{\Phi}{r} = P + \pdv{(\va{\delta \psi} \cdot \va{r} \cross \va{P})}{r})\cdot \delta \psi =  P\cdot \delta \psi + \pdv{(\va{r} \cdot \va{P} \cross \va{\delta \psi})}{r} \cdot \delta \psi= P \delta \psi
		\end{equation*}
		\begin{equation*}
		Q = \pdv{\Phi}{p} = r +  r \delta \psi
		\end{equation*}
		The above transformations look like rotations in the phase plane.

		\item $ \Phi = qP + \delta \tau H(q,p,t) $\\
		This looks like $ F_2 $ again. We write,
		\begin{align*}
		\pdv{\Phi}{q} &= P + \delta \tau (-\dot{p}) = p \qq{and}\\
		 \pdv{\Phi}{P} &= q + \delta \tau \pdv{H}{P}\\
		  &=  q + \delta \tau \pdv{H}{P} \\
		  &=  q + \delta \tau \qty( \pdv{H}{p} \pdv{p}{P} + \pdv{H}{q} \pdv{q}{P} ) \\
		  &=  q + \delta \tau \qty( \dot{q} (1 - \delta \tau \dot{p}) )\\
		  Q & \approx q + \delta \tau \dot{q}
		\end{align*}
		\begin{equation*}
		\therefore Q \approx q + \delta \tau \dot{q} \qq{and} P \approx p +  \delta \tau \dot{p}
		\end{equation*}
		So the canonical transformation just corresponds to time translation by parameter $ d\tau $.

		\item  $  \Phi = \va{r} \cdot \va{P} + (r^2 + P^2)\delta a  $
		\begin{align*}
		\pdv{\Phi}{r} = P_r + 2 r \delta a \implies P_r = p_r - 2 r \delta a \\
		\pdv{\Phi}{P} = r + 2 P \delta a \implies Q = r + 2 P \delta a
		\end{align*}
		This is equivalent to rotation in the phase space by amount $ 2 \delta a $
	\end{itemize}
\end{homeworkProblem}

\begin{homeworkProblem}[4]
	We first note that,
	\begin{equation*}
	y = x^2 \implies \dot{y} = 2 x \dot{x}
	\end{equation*}
	and write down the Lagrangian and Hamiltonian of the system,
	\begin{align*}
	L &= \dfrac{m \dot{x}^2}{2} + \dfrac{m \dot{y}^2}{2} - mgy \\
	L &= \dfrac{m \dot{x}^2}{2} + {2m  x^2 \dot{x}^2 } - mgx^2\\
	\implies p = m \dot{x} + 4 mx^2 \dot{x} &\implies  \dot{x} = \dfrac{p}{m(1 + 4x^2)}
	\end{align*}
	Thus, we can write the Hamiltonian as,
	\begin{align*}
	H(x,p) &= \dfrac{p^2}{m(1 + 4x^2)} - \dfrac{m}{2}(1 + 4x^2)\dfrac{p^2}{m^2 (1 + 4x^2)^2} + mg x^2\\
	H(x,p) &= \dfrac{p^2}{2m(1 + 4x^2)} + mgx^2
	\end{align*}
	The Hamilton-Jacobi equation is given by,
	\begin{equation*}
	\dfrac{1}{2m(1 + 4x^2)}\qty(\pdv{S}{x})^2 + mgx^2 + \pdv{S}{t} = 0
	\end{equation*}
	Substituting $ S = W(x) - Et $, we get,
	\begin{align*}
	\dfrac{1}{2m(1 + 4x^2)}\qty(\dv{W}{x})^2 + &mgx^2 - E = 0
	\implies \dv{W}{x} = \sqrt{2m(E - mgx^2)(1 + 4x^2)}\\
	\implies S &= \int dx \sqrt{2m(E - mgx^2)(1 + 4x^2)} - Et
	\end{align*}
	We know that $ \pdv{S}{E} = \alpha t + \beta $ for constants $ \alpha $ and $ \beta $. Hence the equation of motion is,
	\begin{equation*}
	\sqrt{\dfrac{m(1+4x^2)}{2(E - mgx^2)}} - E = \alpha t + \beta
	\end{equation*}
	\textbf{Part (b)}\\
	We first note that,
	\begin{equation*}
	z = \dfrac{\xi^2 - \eta^2}{2} \qq{,} \rho = \eta \xi \qq{,} \psi = \phi \implies \dot{z} = \xi \dot{\xi} - \eta \dot{\eta} \qq{,} \dot{\rho} = \eta \dot{\xi} + \xi \dot{\eta} \qq{,} \dot{\phi} = \dot{\psi}
	\end{equation*}
	We first write down the Lagrangian and canonical momenta,
	\begin{align*}
	L &= \dfrac{m (\dot{\rho}^2 + \rho^2 \dot{\phi}^2 + \dot{z}^2)}{2} - \dfrac{k}{\sqrt{\rho^2 + z^2} } + Fz\\
	&= \dfrac{m ( \eta^2 \dot{\xi}^2 + \xi^2 \dot{\eta}^2+ 2 \eta \xi \dot{\eta} \dot{\xi} + \eta^2 \xi^2 \dot{\psi}^2 + \xi^2 \dot{\xi}^2 - 2 \xi \dot{\xi} \eta \dot{\eta} + \eta^2 \dot{\eta}^2 )}{2} - \dfrac{k}{\sqrt{\qty(\dfrac{\xi^2 - \eta^2}{2})^2 + \eta^2 \xi^2}} + F\dfrac{\xi^2 - \eta^2}{2}\\
	L&= m\dfrac{(\eta^2 + \xi^2)( \dot{\xi}^2 + \dot{\eta}^2)+ \eta^2 \xi^2 \dot{\psi}^2}{2} - \dfrac{2k}{\eta^2 + \xi^2} + F\dfrac{\xi^2 - \eta^2}{2}\\
	\implies &p_\xi = m (\eta^2 + \xi^2) \dot{\xi} \qq{,} p_\eta = m (\eta^2 + \xi^2) \dot{\eta} \qq{,} p_\psi = m \eta^2 \xi^2 \dot{\psi}\\
	\implies H &= \dfrac{p_\xi^2 + p_\eta^2}{2 m(\eta^2 + \xi^2)} + \dfrac{p_\psi^2}{2m \eta^2 \xi^2} +\dfrac{2k}{\eta^2 + \xi^2}  - F\dfrac{\xi^2 - \eta^2}{2}
	\end{align*}

	Let's apply the transformations given in the problem
	We can now write down the Hamilton-Jacobi equation as,
	\begin{equation*}
	\pdv{S}{t} +  \dfrac{1}{2 m(\eta^2 + \xi^2)} \qty[\qty(\pdv{S}{\xi})^2 + \qty(\pdv{S}{\eta})^2 ]+ \dfrac{1}{2m \eta^2 \xi^2} \qty(\pdv{S}{\psi})^2 + \dfrac{2k}{\eta^2 + \xi^2}  - F\dfrac{\xi^2 - \eta^2}{2} = 0
	\end{equation*}
	We now make the substitution $  S = S_1(\xi) + S_2(\eta) + S_3(\psi) - Et $. We then have,
	\begin{equation*}
	  \dfrac{1}{2 m(\eta^2 + \xi^2)} \qty[\qty(\dv{S_1}{\xi})^2 + \qty(\dv{S_2}{\eta})^2 ]+ \dfrac{1}{2m \eta^2 \xi^2} \qty(\dv{S_3}{\psi})^2 + \dfrac{2k}{\eta^2 + \xi^2}  - F\dfrac{\xi^2 - \eta^2}{2} = E
	\end{equation*}
	Out of the four terms above, we see that only the third terms depends on $ \psi $. As the RHS is a constant, the dependence on $ \psi $ also should vanish. This means,
	\begin{equation*}
	\qty(\dv{S_3}{\psi})^2 = \beta_1^2
	\end{equation*}
	Making the above substitution, multiplying the equation by $ 2m(\eta^2 + \xi^2) $, and collecting terms, we get,
	\begin{equation*}
	\qty[-2m \eta^2 E + \qty(\dv{S_2}{\eta})^2 + \dfrac{\beta_1^2}{ \eta^2} + Fm \eta^4] + \qty[-2m\xi^2 E + \qty(\pdv{S}{\xi})^2 + \dfrac{\beta_1^2}{ \xi^2} - Fm \xi^4] + 4mk = 0
	\end{equation*}
	We can see from the above form that the equation has become completely separable in the new coordinates.
\end{homeworkProblem}

\end{document}


\documentclass[letterpaper,10pt]{article}
\usepackage{graphicx,xspace}
\usepackage[utf8x]{inputenc}
\newcommand{\R}{\textsf{R}\xspace}

\usepackage{alltt}
\usepackage[left=1.5cm,top=1.5cm,bottom=1.5cm,right=1.5cm]{geometry}
\renewcommand{\abstractname}{Summary}
\newenvironment{rcode}{\begin{footnotesize}\begin{alltt}}{\end{alltt}\end{footnotesize}}
\newenvironment{rline}{\begin{small}\begin{ttfamily}}{\end{ttfamily}\end{small}}
%\setlength{\parskip}{10pt} \setlength{\parindent}{0mm}

\usepackage{/home/gavin/R/build/3.3-patched/share/texmf/tex/latex/Sweave}

\begin{document}
\title{Lab: Linear Regression}
\author{Gavin Simpson}
\date{Feb 2017}
\maketitle

\begin{abstract}
In this lab you will be introduced to fitting linear regression models using \R.
\end{abstract}

\section{Simple linear regression}
This is the simplest form of regression with a single response or dependent variable $y$ and one predictor or independent variables $x$. The linear regression model is defined by:
\begin{displaymath}
    y_i=\alpha+\beta x_i+\epsilon_i
\end{displaymath}
where $\alpha$ is a constant, the intercept, $\beta$ is the slope and $\epsilon_i$ is the error component.

In this part of the practical class we will model the response of stomatal density (number of stomata $\mathrm{mm}^{-2}$ leaf) of modern leaves of \textit{Salix herbacae} (dwarf willow) in relation to atmospheric $\mathrm{CO}_2$ concentrations (ppm by volume, ppmv). Data for 29 collections of leaves growing at different $\mathrm{CO}_2$ concentrations are in the text file \begin{rline}co2.txt\end{rline}.

You will perform a simple linear regression using \R. Load the data file \begin{rline}co2.csv\end{rline} into the R object `stomata`. Note that throughout it is \textbf{CO} --- \textbf{cee-oh} --- 2 and not \textbf{C0} --- \textbf{cee-zero} --- 2!

\begin{Schunk}
\begin{Sinput}
> stomata <- read.csv(file = "co2.csv")
> colnames(stomata)
> str(stomata)
\end{Sinput}
\end{Schunk}

The data contain 29 observations of two variables; \begin{rline}co2\end{rline} is the $\mathrm{CO}_2$ concentration and \begin{rline}sdensity\end{rline} is the stomatal density. The first thing we should do is plot the data:

\begin{Schunk}
\begin{Sinput}
> co2.exp <- expression(CO[2])
> plot(sdensity ~ co2, data = stomata, xlab = co2.exp, ylab = "Stomatal density")
\end{Sinput}
\end{Schunk}

\begin{rline}expression()\end{rline} creates a valid \R expression out of its arguments which \R then interprets rather than literally reproducing the contents as labels. So instead of the label being printed as CO[2], \R interprets the brackets as meaning display this as subscript and formats the text accordingly. \R has very powerful capabilities for displaying complex mathematical notation in plots. The code used to format this in a plot borrows heavily from \LaTeX; see \begin{rline}?plotmath\end{rline} for more information.

Linear models in \R are specified using the \begin{rline}lm()\end{rline} function and a standard formula notation that is used in a range of other R base functions and in many add-on packages:
\begin{displaymath}
    response \sim predictor_1 + predictor_2 \ldots + predictor_i
\end{displaymath}
To create a linear model of stomatal density as a function of $\mathrm{CO}_2$ type the following:

\begin{Schunk}
\begin{Sinput}
> sdens.lm <- lm(sdensity ~ co2, data = stomata)
> summary(sdens.lm)
\end{Sinput}
\begin{Soutput}
Call:
lm(formula = sdensity ~ co2, data = stomata)

Residuals:
    Min      1Q  Median      3Q     Max
-33.278  -8.566  -1.153  10.901  34.255

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept) 294.9904    33.2477   8.873 1.73e-09 ***
co2          -0.6467     0.1153  -5.608 5.99e-06 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 16.14 on 27 degrees of freedom
Multiple R-squared:  0.538,	Adjusted R-squared:  0.5209
F-statistic: 31.44 on 1 and 27 DF,  p-value: 5.985e-06
\end{Soutput}
\end{Schunk}

We then display the ANOVA table for the regression model we just specified. We used the \begin{rline}data\end{rline} argument of \begin{rline}resid()\end{rline} to tell R where to find the variables \begin{rline}sdensity\end{rline} and \begin{rline}co2\end{rline} instead of attaching the co2 data frame to the search path. This ensures that the variables being used are the ones you actually specify. The regression summary provides a succinct description of the model specified:
\begin{enumerate}
    \item The call, which describes the model being summarised,
	\item A description of the distribution of the residuals that allows one to quickly evaluate if these are normally distributed,
    \item Regression coefficients, standard errors and their associated t-values and significance levels. The stars provide a quick visual guide to the level of significance achieved by each coefficient,
    \item Finally the residual standard error, the multiple R-squared ($\mathrm{R}^2$), $\mathrm{R}^2$ adjusted for the degrees of freedom, the F statistic for the model and its associated p-value.
\end{enumerate}
To display the fitted line from the regression first re-plot the data (you can use the up-cursor to move back through the commands you have previously typed, so you do not need to retype everything):

\begin{Schunk}
\begin{Sinput}
> plot(sdensity ~ co2, data = stomata, xlab = co2.exp,
+      ylab = "Stomatal density")
> abline(sdens.lm, col = "red")           # plots the fitted line
\end{Sinput}
\end{Schunk}

\begin{rline}abline()\end{rline} is used for drawing lines on plots. It can be used to plot lines that fulfill many functions (we will use it to plot vertical lines on our diagnostics plots later), but when a linear model (of class \begin{rline}lm()\end{rline}) is used as an argument, \begin{rline}abline()\end{rline} extracts the slope and the intercept and plots a line accordingly.

To draw the 95\% confidence limits and prediction intervals we need to make use of the function \begin{rline}predict()\end{rline}. Enter the following code chunk:

\begin{Schunk}
\begin{Sinput}
> pred.frame <- data.frame(co2 = seq(min(stomata$co2), max(stomata$co2), length = 50))
> pp <- predict(sdens.lm, int = "p", newdata = pred.frame)
> pc <- predict(sdens.lm, int = "c", newdata = pred.frame)
> plot(sdensity ~ co2, data = stomata, xlab = co2.exp,
+      ylab = "Stomatal density", ylim = range(stomata$sdensity, pp))
> matlines(pred.frame$co2, pc, lty = c(1,2,2), col = "blue")
> matlines(pred.frame$co2, pp, lty = c(1,3,3), col = "red")
> identify(stomata$co2, stomata$sdensity)
\end{Sinput}
\end{Schunk}

\subsection*{Q and A}
\begin{enumerate}
\item Label points on the plot which look like potential outliers or which lie outside the prediction intervals. Remember, after executing the \texttt{identify()} funtion you need to click on the plot window to label points, then right click the plot window to finish. You cannot use the R console to enter further commands until you do so---as indicated by the there not being a prompt ``$>$'' displayed.
\end{enumerate}

This seems much more complicated than it really needs to be! But this illustrates a number of useful R commands. We want to predict stomatal density over the full range of $\mathrm{CO}_2$, not just the discrete $\mathrm{CO}_2$ values in our data. Also, when we plot lines using \begin{rline}plot(x, y, type = "l")\end{rline} for example, R plots lines between the data points which would result in a cats cradle if the data points are not in order.

First we create a new data frame to hold the $\mathrm{CO}_2$ values we wish to predict stomatal density for. We create a sequence of 50 equally-spaced values ranging from the minimum to the maximum value of $\mathrm{CO}_2$ in our data. We use \begin{rline}predict()\end{rline} to generate the prediction (\begin{rline}int = "p"\end{rline}) and confidence (\begin{rline}int = "c"\end{rline}) values, and instruct R to predict these values for our sequence of $\mathrm{CO}_2$ data. Next comes the familiar plot command where we have added the argument \begin{rline}ylim\end{rline} to make sure the full range of the prediction interval can be plotted. Finally we use \begin{rline}matlines()\end{rline} to plot multiple lines from the provided arguments. Effectively we use \begin{rline}matlines()\end{rline} to plot 3 lines with each call, plotting the confidence interval with dashed blue lines, the prediction interval with dotted red lines and the fitted line (which actually gets plotted twice) in red.`

\subsection{Regression diagnostics}
Once we have fitted our linear model, we must examine that model to check for the influence of outliers and whether the assumptions of linear least squares are met by our data. The summary output provided a simple look at the distribution of the residuals from the regression, but we can do much better with some graphical displays.

Firstly, we should plot the residuals of the fitted model to look
for patterns in the data or to assess whether the assumption of normally-distributed errors is maintained. The LOWESS smooth line shows the  patterns in the data.

\begin{Schunk}
\begin{Sinput}
> plot(fitted(sdens.lm), resid(sdens.lm), main = "Residuals vs Fitted",
+      ylab = "Residuals", xlab = "Fitted values")
> abline(h = 0, lty = "dashed", col = "grey")
> lines(lowess(fitted(sdens.lm), resid(sdens.lm)), col = "red")
\end{Sinput}
\end{Schunk}

Another way of visualising whether the residuals of the models follow a normal distribution is to plot a quantile-quantile or Q-Q plot. The \begin{rline}identify()\end{rline} function allow us to interactively label points in plots. Try the following:

\begin{Schunk}
\begin{Sinput}
> co2.resid <- resid(sdens.lm)
> co2.qq <- qqnorm(co2.resid)
> qqline(co2.resid)
> identify(co2.qq)
\end{Sinput}
\end{Schunk}

After executing the \begin{rline}identify()\end{rline} command above, click with the left mouse button on some of the points in the Q-Q plot. Label those points that deviate somewhat from the 1:1 line in the plot. To finish labelling points press the right mouse button.

The \begin{rline}resid()\end{rline} function is called an \textit{extractor function} in R. The \begin{rline}lm()\end{rline} model object contains a list of the residuals. We could access it directly by accessing the various components of the \texttt{sdens.lm} object. It is better practice, however, to use extractor functions to get at the parts of larger, more complex objects. Another example of an extractor function is \begin{rline}coef()\end{rline}, which can be used to extract the regression coefficients from an \begin{rline}lm()\end{rline} object.

To generate good predictive model that generalises well, it is important to determine whether the model is being unduly influenced by outlier values. We can look at a number of indicators that can allow us detect outliers that have a high degree of influence; hat values, DFBETAS and Cook's distance.

A useful guide for identifying outliers is to look for observations that have a hat value that is 2 or 3 times the average hat value. Enter the following commands and identify which leaves lie beyond these thresholds.

\begin{Schunk}
\begin{Sinput}
> plot(hatvalues(sdens.lm), type = "h", main = "Leverage: hat values",
+       xlab = "Observation", ylab = "Leverage")
> abline(h = c(2,3) * 2/29, lty = 2)
> identify(hatvalues(sdens.lm))
\end{Sinput}
\end{Schunk}

We use the \begin{rline}hatvalues()\end{rline} function to calculate the hat values which we then plot as \textit{histogram like} bars using the argument \begin{rline}type = "h"\end{rline}. \begin{rline}abline()\end{rline} is used to plot the horizontal lines at the given heights, \begin{rline}h\end{rline}.

We can also display the DFBETAS for the model. Here, you should identify those observations that lie at a distance from the main cluster of  points.

\begin{Schunk}
\begin{Sinput}
> plot(dfbetas(sdens.lm), type = "h", main = "Leverage: Dfbetas",
+       xlab = "Observation", ylab = "Dfbetas")
> abline(h = 0, col = "grey")
> identify(dfbetas(sdens.lm))
\end{Sinput}
\end{Schunk}

In a similar theme we can plot the Cook's distances for each observation. Again, identify those leaves that have a Cook's distance greater  than the threshold value.

\begin{Schunk}
\begin{Sinput}
> plot(cooks.distance(sdens.lm), type = "h", ylim = c(0, 1),
+       main = "Cook's distance", xlab = "Observation",
+       ylab = "Cook's distance")
> abline(h = 4/27, lty = 2) # 4/(n-k-1) n = 29, k = 1 parameter in model
> identify(cooks.distance(sdens.lm))
\end{Sinput}
\end{Schunk}

We use $4/27$ from the formulae $4/(n-k-1)$, where $n$ = 29, $k$ = 1 parameter model.

\subsection*{Q and A}
\begin{enumerate}
\item By now you should have enough information to answer the following question. Which of the leaves is a potential outlier?
\end{enumerate}

Having done all the hard work of generating regression diagnostic plots in R, it is worth noting that R comes with a plot method for linear models that produces some of the diagnostic plots we produced by hand. The following code produces a 2 x 2 grid of plots on the open device (window), draws the diagnostic plots and then resets the plotting region to 1 x 1.

\begin{Schunk}
\begin{Sinput}
> layout(matrix(1:4, ncol = 2))
> plot(sdens.lm, ask = FALSE)
> layout(1)
\end{Sinput}
\end{Schunk}

To find out more about the lower left plot, look at the help for \begin{rline}plot.lm()\end{rline} by typing \begin{rline}?plot.lm\end{rline} at the prompt.

In the \begin{rline}co2.txt\end{rline} data file one of the leaves (leaf 17, $\mathrm{CO}_2$ of 206 ppmv) is a full-glacial age sample matched against the Byrd ice-core $\mathrm{CO}_2$ measurements for 15 800--18 900 years BP. Generate another linear regression model of stomatal density against $\mathrm{CO}_2$ concentration, this time omitting leaf 17 from the analysis. Look to see if the intercepts and slopes differ from those obtained above, and take note of the standard errors of the regression coefficients.

\begin{Schunk}
\begin{Sinput}
> sdens.lm2 <- lm(sdensity ~ co2, data = stomata[-17,])
> summary(sdens.lm2)
\end{Sinput}
\end{Schunk}

Refer back to the instructions on how to plot the data with a fitted line if you wish to visually inspect the differences in the two models.

\section{Multiple linear regression}
So far we have seen how to fit simple linear regression models with a single predictor variable and a single response variable. In this section of the practical we will take a look at multiple regression models in R, where we have a single response variable with two or more predictor variables we wish to model.

The data we'll be using are from Loyn (1987), wherein 56 forest patches in southeastern Victoria, Australia, were selected and the abundance of birds within each patch was related to siz predictor variables:

\begin{enumerate}
\item \texttt{area}: patch area (ha)
\item \texttt{dist2Patch}: distance to the nearest patch (km)
\item \texttt{dist2LargePatch}: distance to the nearest large patch (km)
\item \texttt{grazing}: grazing stock on a 1--5 ordinal scale (light--heavy)
\item \texttt{altitude}: altitude of patch (m)
\item \texttt{yearIsolated}: year of isolation
\end{enumerate}

Three of these variables (\texttt{area}, \texttt{dist2Patch}, and \texttt{dist2LargePatch}) are higly skewed and hence we apply a $\log_{10}$ transformation.

You will use \R to explore these data and to fit a multiple linear regression model of the form:

\begin{displaymath}
    y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \ldots + \beta_mx_m
\end{displaymath}
where $x_1 \ldots x_m$ are our predictor variables and $\beta_0$, $\beta_1 \ldots \beta_m$ are parameters that we wish to estimate for combinations of $y$ and $x_m$.

Read in the data from \texttt{loyn.csv}, display it and plot a scatter-plot matrix of the variables using the following commands.

\begin{Schunk}
\begin{Sinput}
> birds <- read.csv("loyn.csv")
> birds
> plot(birds, gap = 0)
\end{Sinput}
\end{Schunk}

It is useful to have a numerical summary of the information shown in a scatter-plot matrix, and it is easy to generate a correlation matrix using the R function \begin{rline}cor()\end{rline}. \begin{rline}cor()\end{rline} does not generate p-values or significance values for the correlations however, and the associated function \begin{rline}cor.test()\end{rline} only works on two vectors, x and y at a time. Our function\footnote{The \begin{rline}corProb()\end{rline} function is based on code published on the R-Help mailing list by Bill Venables which has been modified to provide a print method.} \begin{rline}corProb()\end{rline} will generate the lower triangle of a correlation matrix with associated p-values shown in the upper triangle. Load the \begin{rline}corProb()\end{rline} function and generate the correlation matrix.

\begin{Schunk}
\begin{Sinput}
> source("corProb.R")
\end{Sinput}
\end{Schunk}

Then we use it by passing in the \begin{rline}birds\end{rline} data frame

\begin{Schunk}
\begin{Sinput}
> corProb(birds)
\end{Sinput}
\begin{Soutput}
Correlations are shown below the diagonal
P-values of the F-statistics are shown above the diagonal

                abundance area    yearIsolated dist2Patch dist2LargePatch
abundance        1.0000    0.0569  0.0001       0.0798     0.5230
area             0.2560    1.0000  0.9913       0.4267     0.8003
yearIsolated     0.5034   -0.0015  1.0000       0.4061     0.5415
dist2Patch       0.2361    0.1083  0.1132       1.0000     0.0172
dist2LargePatch  0.0872    0.0346 -0.0833       0.3172     1.0000
grazing         -0.6825   -0.3104 -0.6356      -0.2558    -0.0280
altitude         0.3858    0.3878  0.2327      -0.1101    -0.3060
                grazing altitude
abundance        0.0000  0.0033
area             0.0199  0.0032
yearIsolated     0.0000  0.0843
dist2Patch       0.0570  0.4192
dist2LargePatch  0.8376  0.0218
grazing          1.0000  0.0018
altitude        -0.4072  1.0000
\end{Soutput}
\end{Schunk}

Multiple linear regression models are fitted in R in the same way as the simple linear regression model you looked at earlier. To fit the full multiple regression model using all seven predictors use the following commands.

\begin{Schunk}
\begin{Sinput}
> birds.lm <- lm(abundance ~ log10(area) + log10(dist2Patch) + log10(dist2LargePatch) +
+                grazing + altitude + yearIsolated, data = birds)
\end{Sinput}
\end{Schunk}

Note that the \texttt{log10()} parts are applying the transformation to the variables as part of the model fitting. This transformation is part of the model fit now so when predicting for new data those data will automatically get the same transformation applied to them. Look at the model summary:

\begin{Schunk}
\begin{Sinput}
> summary(birds.lm)
\end{Sinput}
\begin{Soutput}
Call:
lm(formula = abundance ~ log10(area) + log10(dist2Patch) + log10(dist2LargePatch) +
    grazing + altitude + yearIsolated, data = birds)

Residuals:
     Min       1Q   Median       3Q      Max
-15.6506  -2.9390   0.5289   2.5353  15.2842

Coefficients:
                         Estimate Std. Error t value Pr(>|t|)
(Intercept)            -125.69725   91.69228  -1.371   0.1767
log10(area)               7.47023    1.46489   5.099 5.49e-06 ***
log10(dist2Patch)        -0.90696    2.67572  -0.339   0.7361
log10(dist2LargePatch)   -0.64842    2.12270  -0.305   0.7613
grazing                  -1.66774    0.92993  -1.793   0.0791 .
altitude                  0.01951    0.02396   0.814   0.4195
yearIsolated              0.07387    0.04520   1.634   0.1086
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 6.384 on 49 degrees of freedom
Multiple R-squared:  0.6849,	Adjusted R-squared:  0.6464
F-statistic: 17.75 on 6 and 49 DF,  p-value: 8.443e-11
\end{Soutput}
\end{Schunk}

Note the values for $\mathrm{R^2}$, adjusted-$\mathrm{R^2}$, $F$ and its $p$ for this, the full model.

\subsection*{Q and A}
\begin{enumerate}
\item What do these results suggest to you about the relationship between the predictor variables and the abundance of birds in forest patches?
\end{enumerate}

We can also produce an ANOVA table using R's \begin{rline}anova()\end{rline} function, which shows the marginal effect of adding each variable in turn to the model. Each variable is added to the model in the order that you specified in the right-hand side of the model equation or in the order they appear in the data frame or matrix from which the predictors were derived.

\begin{Schunk}
\begin{Sinput}
> anova(birds.lm)
\end{Sinput}
\begin{Soutput}
Analysis of Variance Table

Response: abundance
                       Df Sum Sq Mean Sq F value    Pr(>F)
log10(area)             1 3471.0  3471.0 85.1731 2.695e-12 ***
log10(dist2Patch)       1   65.5    65.5  1.6067 0.2109423
log10(dist2LargePatch)  1  136.5   136.5  3.3500 0.0732901 .
grazing                 1  543.6   543.6 13.3394 0.0006316 ***
altitude                1   15.7    15.7  0.3841 0.5382675
yearIsolated            1  108.8   108.8  2.6705 0.1086351
Residuals              49 1996.9    40.8
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
\end{Soutput}
\end{Schunk}

\subsection*{Q and A}
\begin{enumerate}
\item What do these results suggest to you about the relationship between the predictor variables and the abundance of birds in forest patches?
\item Which variables are candidates for removal?
\end{enumerate}

We can remove seemingly non-informative variables by updating the model fit and indicating which variables are to be removed from the model. Here we remove some variables and then generate the model summary and the ANOVA table coparing the full model (\texttt{birds.lm}) with the simplified model (\texttt{birds2.lm})

\begin{Schunk}
\begin{Sinput}
> birds2.lm <- update(birds.lm, . ~ . - log10(dist2Patch) - log10(dist2LargePatch) -
+                     altitude)
> summary(birds2.lm)
\end{Sinput}
\begin{Soutput}
Call:
lm(formula = abundance ~ log10(area) + grazing + yearIsolated,
    data = birds)

Residuals:
     Min       1Q   Median       3Q      Max
-14.5159  -3.8136   0.2027   3.1271  14.5542

Coefficients:
               Estimate Std. Error t value Pr(>|t|)
(Intercept)  -134.26065   86.39085  -1.554   0.1262
log10(area)     7.16617    1.27260   5.631 7.32e-07 ***
grazing        -1.90216    0.87447  -2.175   0.0342 *
yearIsolated    0.07835    0.04340   1.805   0.0768 .
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 6.311 on 52 degrees of freedom
Multiple R-squared:  0.6732,	Adjusted R-squared:  0.6544
F-statistic: 35.71 on 3 and 52 DF,  p-value: 1.135e-12
\end{Soutput}
\begin{Sinput}
> anova(birds.lm, birds2.lm)
\end{Sinput}
\begin{Soutput}
Analysis of Variance Table

Model 1: abundance ~ log10(area) + log10(dist2Patch) + log10(dist2LargePatch) +
    grazing + altitude + yearIsolated
Model 2: abundance ~ log10(area) + grazing + yearIsolated
  Res.Df    RSS Df Sum of Sq      F Pr(>F)
1     49 1996.8
2     52 2071.1 -3   -74.286 0.6076 0.6132
\end{Soutput}
\end{Schunk}

\subsection{Interaction terms}
Thus far we have only considered the unique contributions of each variable to the modelling of bird abundance. Perhaps the effect on bird abundance of one variable depends on the value(s) of one or more other variables in the model. We can account for such \textit{interactions} between variables via interaction terms. In the next code chunk we add 2-way (2nd order) and the 3-way (3rd-order) interactions between variables in the simplified model.

\begin{Schunk}
\begin{Sinput}
> birds3.lm <- update(birds2.lm, . ~ . + log10(area):grazing + log10(area):yearIsolated +
+                     grazing:yearIsolated + log10(area):grazing:yearIsolated)
> anova(birds2.lm, birds3.lm)
\end{Sinput}
\begin{Soutput}
Analysis of Variance Table

Model 1: abundance ~ log10(area) + grazing + yearIsolated
Model 2: abundance ~ log10(area) + grazing + yearIsolated + log10(area):grazing +
    log10(area):yearIsolated + grazing:yearIsolated + log10(area):grazing:yearIsolated
  Res.Df    RSS Df Sum of Sq      F   Pr(>F)
1     52 2071.1
2     48 1542.9  4    528.25 4.1085 0.006076 **
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
\end{Soutput}
\end{Schunk}

\subsection*{Q and A}
\begin{enumerate}
\item Do we need the 3-way interaction?
\end{enumerate}

What about the 2-way terms?

\begin{Schunk}
\begin{Sinput}
> birds4.lm <- update(birds3.lm, . ~ . - log10(area):grazing:yearIsolated)
> anova(birds3.lm, birds4.lm)
\end{Sinput}
\begin{Soutput}
Analysis of Variance Table

Model 1: abundance ~ log10(area) + grazing + yearIsolated + log10(area):grazing +
    log10(area):yearIsolated + grazing:yearIsolated + log10(area):grazing:yearIsolated
Model 2: abundance ~ log10(area) + grazing + yearIsolated + log10(area):grazing +
    log10(area):yearIsolated + grazing:yearIsolated
  Res.Df    RSS Df Sum of Sq      F Pr(>F)
1     48 1542.9
2     49 1546.4 -1   -3.4888 0.1085 0.7432
\end{Soutput}
\begin{Sinput}
> drop1(birds4.lm)
\end{Sinput}
\begin{Soutput}
Single term deletions

Model:
abundance ~ log10(area) + grazing + yearIsolated + log10(area):grazing +
    log10(area):yearIsolated + grazing:yearIsolated
                         Df Sum of Sq    RSS    AIC
<none>                                1546.4 199.83
log10(area):grazing       1    334.74 1881.1 208.80
log10(area):yearIsolated  1    239.74 1786.1 205.90
grazing:yearIsolated      1    272.21 1818.6 206.91
\end{Soutput}
\end{Schunk}

\subsection*{Q and A}
\begin{enumerate}
\item Do we need any of the 2-way interactions?
\end{enumerate}

\end{document}


% A (minimal) template for problem sets and solutions using the exam document class

% Organization:
%% Define new commands, macros, etc. in macros.tex
%% Anything that you would put before \begin{document} should go in prelude.tex

%% For multiple psets, each should get its own file to \input into main with a \section{}

\input{prelude}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{mathtools}
\usepackage{graphicx}
\usepackage{color}
\usepackage{float}

\def\del{\nabla}		% gradient
\def\div{{\nabla\cdot\,}}	% divergence
\def\curl{{\nabla\times}}	% curl
\def\laplace{{\nabla^2}}	% Laplacian
\def\<~{\stackrel{<}{\mbox{\scriptsize $\sim$}}}
\def\>~{\stackrel{>}{\mbox{\scriptsize $\sim$}}}
\def\eq#1{(\ref{#1})}
\def\half{\frac{1}{2}}
\def\fourth{\frac{1}{4}}

\DeclarePairedDelimiterX{\norm}[1]{\lVert}{\rVert}{#1}

\sloppy
\definecolor{lightgray}{gray}{0.5}
\setlength{\parindent}{0pt}

\begin{document}

\title{Numerical Methods for PDEs\\Homework 6}
\author{Francisco Jose Castillo Carrasco}
\date{\today}
\maketitle

\input{macros}

%% Content goes here

\section*{Problem 1}
\input{problem1}

\section*{Problem 2}
\input{problem2}

\section*{Problem 3}
\input{problem3}

\section*{Problem 4}
\input{problem4}

\section*{Problem 5}
\input{problem5}

\end{document}


\documentclass[10pt,a4paper]{article}

\usepackage{graphicx}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage[cm]{fullpage}
\usepackage{xcolor}
\usepackage[parfill]{parskip}
\usepackage{verbatim}
\usepackage{hyperref}

%%%
\usepackage{listings}
\lstset{
  basicstyle=\ttfamily,
  breaklines=true,
  frame=none,
  keepspaces=true,
  numberstyle=\small\color{black!80},
  xleftmargin=.0in,
  language=Prolog,
  deletekeywords={clause,not,var},
  commentstyle=\color{black!40},
  escapechar=|,
  aboveskip=\medskipamount,
  belowskip=\medskipamount,
  backgroundcolor=\color{black!20},
}
\newcommand{\inlinecode}[1]{\texttt{\small #1}}
%%%

\title{Homework assignment 2 -- Symbolic Systems I -- UvA, June 2020}
\author{\it Your Name Here}
\date{}

\begin{document}
\maketitle

\section*{Question 1}

Provide your answer to question 1 here.

Propositional logic formulas are typeset nicely as follows, for example:
\[ \begin{array}{r l}
  \varphi = &
    (x_1 \vee \neg x_2 \vee x_3)
    \wedge
    (x_2 \vee \neg x_1 \vee x_4)
    \wedge
    (x_3 \vee \neg x_4 \vee x_5)
    \wedge {} \\
  & (x_2 \vee x_3 \vee \neg x_4)
    \wedge
    (x_3 \vee \neg x_4 \vee \neg x_5)
    \wedge
    (x_4 \vee \neg x_5 \vee x_6) \\
\end{array} \]

\section*{Question 2}

Provide your answer to question 2 here.

If you want to include code (e.g., answer set programs), you can do it nicely as follows:

\begin{lstlisting}
a :- b, not c.
b :- not c.
\end{lstlisting}

\section*{Question 3}

Provide your answer to question 3 here.

If you want to use DL notation,
you can do that using, e.g., \verb|\sqsubseteq|, \verb|\sqcup|, etc%
\footnote{See, e.g., \url{https://oeis.org/wiki/List_of_LaTeX_mathematical_symbols\#Relation_operators}.}:
$C \sqsubseteq \neg (C \sqcap (D \sqcup E))$.

\section*{Question 4}

Provide your answer to question 4 here.

\section*{Question 5}

Provide your answer to question 5 here.

\end{document}


\documentclass[11 pt,A4]{article}
\usepackage{epsf,graphics,graphicx,fancyhdr,amsfonts,amsmath,color}
\font\BBb=msbm10
\usepackage{ifthen}

\setlength{\textwidth}{6.5 in}
\setlength{\oddsidemargin}{0cm}
\setlength{\evensidemargin}{0cm}
\setlength{\topmargin}{-0.5 in}
\setlength{\textheight}{9.0 in}
\parskip 0.1 in

 \pagestyle{plain}

\newcommand{\bc}{\begin{center}}
\newcommand{\ec}{\end{center}}
\newcommand{\bn}{\begin{enumerate}}
\newcommand{\en}{\end{enumerate}}
\newcommand{\bi}{\begin{itemize}}
\newcommand{\ei}{\end{itemize}}
\newcommand{\xbar}{\overline{x}}
\newcommand{\ybar}{\overline{y}}
\newcommand{\wh}{\widehat}
\newcommand{\ra}{\rightarrow}
\newcommand{\la}{\leftarrow}
\newcommand{\da}{\downarrow}
\newcommand{\nea}{\nearrow}
\newcommand{\sea}{\searrow}
\newcommand{\swa}{\swarrow}
\newcommand{\mc}{\multicolumn}
\newcommand{\vs}{\vspace{1cm}}
\newcommand{\vssm}{\vspace{0.5cm}}
\newcommand{\be}{\begin{eqnarray}}
\newcommand{\ee}{\end{eqnarray}}
\newcommand{\bes}{\begin{eqnarray*}}
\newcommand{\ees}{\end{eqnarray*}}

\newcommand{\tc}{\textcolor}
\newcommand{\tcb}{\textcolor{blue}}
\newcommand{\tcr}{\textcolor{red}}
\newcommand{\tcg}{\textcolor{green}}
\newcommand{\tcm}{\textcolor{magenta}}
\newcommand{\tcc}{\textcolor{cyan}}

\newcommand{\nt}{\tcr{\bf n}_t}
\newcommand{\nT}{{\bf n}_{1:T}}
\newcommand{\npt}{\tcr{\bf n}_{t-1}}
\newcommand{\yt}{\tcb{\bf y}_t}
\newcommand{\ypt}{\tcb{\bf y}_{t-1}}
\newcommand{\yT}{{\bf y}_{1:T}}
\newcommand{\gtheta}{\tcg{\theta}}

\newcommand{\solutions}{1}

%Stuff to make more compact
  \renewcommand{\baselinestretch}{1.0}
  \oddsidemargin -1.5cm \evensidemargin -1.5in
  \topmargin=-1.5cm \headheight=0pt
  \headsep 0.5cm \textwidth=19cm
  \textheight=25.0cm \columnsep 10pt \columnseprule 0pt \parindent 0pt
  \parskip 6.0pt
%  \usepackage{tweaklist}
%  \renewcommand{\itemhook}{
%    \setlength{\topsep}{-0pt}
%    \setlength{\itemsep}{6pt}
%    \setlength{\parsep}{6pt}
%  }
%  \renewcommand{\enumhook}{
%    \setlength{\topsep}{-0pt}
%    \setlength{\itemsep}{6pt}
%    \setlength{\parsep}{6pt}
%  }
%end of this stuff

\usepackage{color}

\pagestyle{myheadings}
\markright{Computing in Statistics \hspace{3.0 in}}

\begin{document}
\thispagestyle{empty}

\definecolor{purple}{rgb}{0.4,0.1,0.5}
\definecolor{lgray}{gray}{0.8}
\definecolor{lcyan}{cmyk}{0.4,0.2,0.1,0.1}
\definecolor{lred}{cmyk}{0.1,0.4,0.1,0.1}
\definecolor{lred2}{cmyk}{0.1,0.2,0.1,0.1}
\definecolor{lcyan2}{cmyk}{0.5,0.2,0.1,0.1}
\definecolor{lcyan3}{cmyk}{0.1,0.2,0.3,0.1}

\section*{MT 4113: Computing in Statistics, Practical 1, additional practice material}

This sheet contains additional material intended to give you some more practice with control structures, (a bit of) vectorization, and function writing.

\subsection*{Part 1 -- Control structures}

\bn

 \item Write R code that simulates a number from a random uniform distribution; if this number is more than or equal to 0.5 print out the word ``lucky'', and if less than 0.5 print out ``unlucky''.

 \item Here is some code
\begin{verbatim}
x <- seq(-10, 10, by = 0.01)
y <- x ^ 2
plot(x, y, type = 'l')
\end{verbatim}
Revise this code so that $y = x ^ 2$ when $x \geq 0$ and $y = 0$ when $x < 0$.

\item Write code to forecast the weather.  Your code should print one of three forecasts at random (with equal probablity of each occuring): either ``rainy'', ``hurricane'', or ``hailstorms''.  (Hint: if you want to use \texttt{switch()} then you should look at the help if the first argument to it will be an integer, as it works differently from the notes in this case.)

\item Extend the above code, so that it repeatedly forecasts the weather until you get a rainy forecast.  Print out the number of forecasts output before getting the first rainy forecast.

\item Edit the above code so it also stops if it reaches 4 forecasts without getting ``rainy''.  Have it print out an appropriate message (i.e., if it forecast ``rainy'' then as before, but if not a message saying it stopped after 4 tries).

\item Simulate a 4x3 matrix of random normal deviates (mean 0 sd 1).  Compute the sum of each row.

\en

\subsection*{Part 2 -- Functions}

\bn

\item Take the weather forecast code from question 3 of Part 1 and turn it into a function, \texttt{forecast} that has no arguments, but returns the forecast as a character string.  Once this function is defined, it should be possible to type \texttt{forecast()} and get a forecast.  I could save 10 forecasts with
\begin{verbatim}
n <- 10
forecasts <- numeric(n)
for(i in 1:n){
  forecasts[i] <- forecast()
}\end{verbatim}

\item Extend your \texttt{forecast} function so that it has one argument, \texttt{n}, which is the number of forecasts to produce.  Then you should be able to type \texttt{forecast(10)} or \texttt{forecast(n = 10)} and get a vector of 10 forecasts.

\item Extend the code from question 2 on the last tutorial to turn it into a function, \texttt{plot.quadratic(limits)} where \texttt{limits} is a vector of length 2 that gives the lower and upper limits for the x-axis.  Give the limits argument default values.  Add some error checking on the input if you like.

\item (Advanced optional question - only attempt if you found the previous ones easy, or if you like a challenge!)  Write a function \texttt{plot.function} so you can pass in the limits but also the type of univariate function you want to plot (e.g., $x^2$ or $\sin(x)$, etc).  It will then plot that function within the specified limits.  (Hint: in R, you can pass in functions as function arguments.)

\en

\end{document}


\documentclass{beamer}

\usepackage{default}
\usepackage[german]{babel}
\usepackage[utf8]{inputenc}                   % replace by the encoding you are using

\usetheme{Berlin}

\newcommand{\cindent}{\hskip20pt}

%Header Settings
%\setbeamertemplate{headline}{}

%Footer Settings
\setbeamertemplate{navigation symbols}{
	\usebeamerfont{footline}%
	\usebeamercolor[fg]{footline}%
	\hspace{1em}%
	\insertframenumber/\inserttotalframenumber
}

\title[Java]{Java - If-Else}
\author[W. Bombardelli]{William Bombardelli}
\institute[Schweizerschule Mexiko]
{
	\vskip 12pt
	Schweizerschule Mexiko, Ciudad de México, Mexico \\
	\texttt{\url{https://github.com/wbombardellis/java-unterricht}}
}
\date{30 October 2019}

\makeatletter
\hypersetup{
	pdftitle = {\@title}, pdfkeywords = {Java}, pdfauthor = {\@author}
}
\makeatother

\begin{document}
	\begin{frame}
		\titlepage
	\end{frame}

	\begin{frame}
		\frametitle{Organization}
		\tableofcontents
	\end{frame}

	%-------------------
	% If-Else
	%-------------------
	\section{If-Else}
	\begin{frame}
		\frametitle{If}
		\begin{itemize}
			\item Write a program that reads a double value and prints if it is positive.
		\end{itemize}
		\pause
		$if\ (a > 0)\ \{$\\
		\cindent $System.out.println(a\ + $ `` is positive." $);$\\
		$\}$\\
	\end{frame}

	\begin{frame}
		\frametitle{If-Else}
		\begin{itemize}
			\item Write a program that reads a double value and tells whether it is positive or not.
		\end{itemize}
		$if\ (a > 0)\ \{$\\
		\cindent $System.out.println(a\ + $ `` is positive." $);$\\
		$\}\ else\ \{$\\
		\cindent $System.out.println(a\ + $ `` is not positive." $);$\\
		$\}$\\
	\end{frame}

	\begin{frame}
		\frametitle{If-ElseIf-Else}
		\begin{itemize}
			\item Write a program that reads a double value and tells whether it is positive, negative or zero.
		\end{itemize}
		$if\ (a > 0)\ \{$\\
		\cindent $System.out.println(a\ + $ `` is positive." $);$\\
		$\}\ else\ if\ (a < 0) \ \{$\\
		\cindent $System.out.println(a\ + $ `` is negative." $);$\\
		$\}\ else\ \{$\\
		\cindent $System.out.println(a\ + $ `` is zero." $);$\\
		$\}$\\
	\end{frame}

	\begin{frame}
		\frametitle{Exercises}
		\begin{enumerate}
			\item Write a program that greets the user correctly according to the current time. The rule is the following:
			\begin{itemize}
				\item If current time is before 12a.m., then greet him/her with ``good morning".
				\item If current time is between 12a.m. and 16p.m., then greet him/her with ``good afternoon".
				\item If current time is after 16p.m, then greet him/her with ``good evening".
			\end{itemize}
		\end{enumerate}
	\end{frame}

	\begin{frame}
		\frametitle{Exercises}
		\begin{enumerate}[2]
			\item Extend your assignment's program further using If or If-Else.
		\end{enumerate}
	\end{frame}

	\begin{frame}
		\frametitle{If-ElseIf-Else Grammar Rules}
		$if\ (\langle \text{Boolean condition} \rangle)\ \{$\\
		\cindent $...$\\
		$\}\ [else\ if\ (\langle \text{Boolean condition} \rangle) \ \{$\\
		\cindent $...$\\
		$\}]\ [else\ \{$\\
		\cindent $...$\\
		$\}]$\\
	\end{frame}

	\begin{frame}
		\frametitle{Exercises}
		\begin{enumerate}
			\item Implement the following algorithm for finding real solutions of a quadratic polynomial $ax^2 + bx + c$ in Java. \footnote{Assume $a$ is not zero.}
			\begin{itemize}
				\item Read $a$, $b$ and $c$.
				\item Calculate $\Delta := b^2 - 4ac$
				\item If $\Delta \ge 0$ then print solutions $x_1$ and $x_2$
				\begin{itemize}
					\item $x_1 = \frac{-b + \sqrt{\Delta}}{2a}, x_2 = \frac{-b - \sqrt{\Delta}}{2a}$
				\end{itemize}
				\item Otherwise print that there is no real solutions for this polynomial.
			\end{itemize}
		\end{enumerate}
	\end{frame}

	\begin{frame}
		\frametitle{Exercises}
		\begin{enumerate}[2]
			\item Write a program that gets a positive integer number as input and writes whether that number is even or odd.
			\begin{itemize}
				\item Tip 1: An number $n \in \mathbb{N}$ is even if, and only if, $n$ divided by 2 gives remainder 0.
				\item Tip 2: In Java $a\ \%\ b$ results in the remainder of the whole division of $a$ by $b$.
				\item Also: Ensure that the input is positive. If it is not, then reject user's input.
			\end{itemize}
		\end{enumerate}
	\end{frame}

	\begin{frame}
	\frametitle{Exercises}
	\begin{enumerate}[3]
		\item Write a program that reads two numbers and a character representing the operation to perform over those numbers, and print the result accordingly. Example for an execution:\\
		\vskip20pt
		\texttt{Please provide the first number:}\\
		\texttt{2}\\
		\texttt{Please provide the operation: }\\
		\texttt{+}\\
		\texttt{Please provide the second number: }\\
		\texttt{3}\\
		\texttt{The result of 2 + 3 is 5}\\
	\end{enumerate}
	\end{frame}

	%-------------------
	% Summary
	%-------------------
	\section{Summary}

	\begin{frame}
		\frametitle{Summary}
		\begin{itemize}
			\item IF-Else allows you to control the flow of your program.
			\item Next Lesson: Boolean Algebra
		\end{itemize}
	\end{frame}

	\begin{frame}
		\frametitle{References}
		\begin{itemize}
			\item W3C Tutorial:
			\begin{itemize}
				\item \url{https://www.w3schools.com/java/java_conditions.asp}
			\end{itemize}
			\item Exercises: \url{https://www.w3schools.com/java/exercise.asp}
			\begin{itemize}
				\item Java If...Else
			\end{itemize}
		\end{itemize}

	\end{frame}

\end{document}


\documentclass[11pt]{article}

\usepackage{amsmath}
\usepackage{booktabs}
\usepackage{cleveref}
\usepackage{enumerate}
\usepackage{float}
\usepackage{graphicx}
\usepackage{listings}

\setlength\parindent{0pt}

\begin{document}

    \title{HPCSE II - Exercise 1}
    \author{Anian Ruoss}
    \maketitle

    All experiments were run on a node obtained by running the following
    command:
    \begin{lstlisting}[language=bash, basicstyle=\footnotesize]
bsub -n 24 -R fullnode -R "select[model=XeonE5_2680v3]" -Is bash
    \end{lstlisting}

    \subsection*{Task 4}
    \label{subsec:Task4}
    \begin{enumerate}[a)]
        \item See the first row in Table~\ref{table:OptGridCount}.
        \item The optimal number of grids is $6$ as illustrated
        in Table~\ref{table:OptGridCount}.
        \begin{table}[H]
            \caption{Iterations and running times for different numbers of
            grids for DownRelaxations $=1$ and UpRelaxations $=1$.}
            \begin{center}
                \input{./data/optimal_grid_count.tex}
            \end{center}
            \label{table:OptGridCount}
        \end{table}
        \item The optimal combination of UpRelaxations and DownRelaxations is
        determined in a greedy way.
        The optimal number of DownRelaxations in terms of running time is $3$
        as illustrated in Table~\ref{table:OptDownRelax}.
        Although, values of $5$ and $7$ produce less iterations, the time per
        iteration is significantly higher.
        \begin{table}[H]
            \caption{Iterations and running times for different numbers of
            DownRelaxations given GridCount $= 6$ and UpRelaxations $= 1$.}
            \begin{center}
                \input{./data/optimal_down_relaxations.tex}
            \end{center}
            \label{table:OptDownRelax}
        \end{table}
        The optimal number of UpRelaxations is $1$ as illustrated in
        Table~\ref{table:OptUpRelax}.
        \begin{table}[H]
            \caption{Iterations and running times for different numbers of
            UpRelaxations with GridCount $= 6$ and DownRelaxations $= 3$.}
            \begin{center}
                \input{./data/optimal_up_relaxations.tex}
            \end{center}
            \label{table:OptUpRelax}
        \end{table}
        \item The optimal configuration with respect to the results from
        tables~\ref{table:OptGridCount},~\ref{table:OptDownRelax},
        and~\ref{table:OptUpRelax} is given by GridCount $= 6$,
        DownRelaxations $= 3$, and UpRelaxations $= 1$.
        To find the global minimum one would have to run a grid search on all
        three parameters.
        However, since the exercise asked for a greedy approach and since
        the local minimum performed fairly well, the grid search was omitted.
        \item The solution reaches its equilibrium state for low frequencies
        faster on coarser grids as every cell covers more area (it is also
        cheaper to compute a full pass on coarse grids).
        Larger frequencies can only be resolved on finer grids and thus
        the solution is alternatingly computed on finer and coarser grids to
        increase convergence speed while maintaining a small discretization
        error.
        \item Using more DownRelaxations increases the number of iterations
        in an alternating manner (the TA could not provide an explanation
        for this phenomenon during the exercise class).
        Furthermore, the running time decreases slightly from $1$ to $3$
        because at every grid level the solution can converge to its
        equilibrium state before passing on its values to the next grid.
        However, using more relaxation steps doesn't lead to a better
        convergence as it has already converged on the grid with fewer
        relaxations and thus just increases the time spent on every grid.
        The number of UpRelaxations doesn't seem to affect the iterations at
        all and just increases the time spent at every grid.
    \end{enumerate}

    \subsection*{Task 5}
    \label{subsec:Task5}

    \begin{enumerate}[a)]
        \item \textit{Loop Interchange} can be applied to a prevent bad
        memory access patterns, i.e.\ swapping the $i$- and $j$-loops.
        \textit{Loop Fusion} can be applied to increase data locality, i.e.\
        performing the computations of two subsequent for-loops that rely on
        the same data in a single for-loop instead to reduce cache misses.
        \item Every grid should be allocated in its entirety before moving
        on to the next one as this prevents cache thrashing.
        This is an example where \textit{Loop Fusion} should not be
        applied as the different for-loops operate on different data.
        \item As smoothing on the $0$-th grid takes the most time, the
        maximum possible speedup can be obtained by optimizing the
        \textbf{applyJacobi} method.
        Thus, to determine the effect of \textit{Loop Blocking}, the
        for-loops in \textbf{applyJacobi} can be replaced with the blocked
        for-loops:
\begin{lstlisting}[language=c++, basicstyle=\tiny]
for (int ii = 1; ii < g[l].N - 1; ii += B) {
  for (int jj = 1; jj < g[l].N - 1; jj += B) {
    for (int i = ii; i < std::min(ii + B, (int)g[l].N - 1); ++i) {
      for (int j = jj; j < std::min(jj + B, (int)g[l].N - 1); ++j) {
        g[l].U[i][j] =
            (g[l].Un[i - 1][j] + g[l].Un[i + 1][j] + g[l].Un[i][j - 1] +
             g[l].Un[i][j + 1] + g[l].f[i][j] * g_l_h_squared) *
            0.25;
      }
    }
  }
}
\end{lstlisting}
        However, even when trying different block sizes
        $B \in \left\{ 8, 16, 32, 64, 128 \right\}$ (larger block
        sizes are not applicable due to the size of the largest grid), the
        running time does not decrease.
        In fact, since the grid sizes are so small that enough data can fit
        into the cache, the overhead induced by the two additional for-loops
        causes the program to run slower.
        For this reason, \textit{Loop Blocking} was not used for the final
        implementation.
    \end{enumerate}

    \subsection*{Task 6}
    \label{subsec:Task6}
    \begin{enumerate}[a)]
        \item \begin{itemize}
                  \item Division operator replaced with multiplication of
                  inverse.
                  \item Multiplication by $1$ removed.
                  \item $pow\left(x, 2\right)$ replaced with $x \cdot x$.
                  \item Constants pre-computed and moved out of loops.
                  \item Expressions simplified with associativity.
                  \item Copy of array avoided by pointer swap.
        \end{itemize}
    \end{enumerate}

    \subsection*{Task 7}
    \label{subsec:Task7}
    To generate the optimization reports, the following line in the Makefile
    has to be changed to:
    \begin{lstlisting}[language=bash, basicstyle=\tiny]
CFLAGS = -O3 -D NOALIAS -D NOFUNCCALL -qopt-report=3 -qopt-report-phase=vec
    \end{lstlisting}

    \begin{enumerate}[a)]
        \item Most loops cannot be vectorized due to vector dependence.
        This means that the compiler cannot know if the data accessed in one
        iteration are also changed in another iteration (which would case a
        race condition if vectorized).
        \item This problem can be fixed by adding
        \lstinline[language=c++, basicstyle=\footnotesize]{#pragma ivdep}
        in front of the loop if the vectors are in fact independent (which
        they are as the results are computed from and stored into different
        vectors).
        \item SIMD operations start working at the beginning of a block of
        memory and thus if the arrays are not aligned with the block starts
        the operations need to gather data from different blocks which is
        very inefficient.
        During the exercise class the following grids allocation was
        recommended:
\begin{lstlisting}[language=c++, basicstyle=\scriptsize]
g[i].U = (double**) _mm_malloc (sizeof(double*) * g[i].N, 16);
for (int j = 0; j < g[i].N; j++) {
    g[i].U[j] = (double*) _mm_malloc (sizeof(double) * g[i].N, 16);
}
\end{lstlisting}
        However, even in combination with
        \lstinline[language=c++, basicstyle=\footnotesize]{#pragma vector aligned}
        this allocation slowed down the implementation (probably because it
        only aligns pointers and not the underlying data).
        Hence, the original allocation was used for the final implementation.
    \end{enumerate}

    \begin{table}[H]
        \caption{Iterations and running times for the three problems after
        all (beneficial) optimizations have been performed.}
        \begin{center}
            \input{./data/optimal_problem.tex}
        \end{center}
        \label{table:FinalTimes}
    \end{table}

\end{document}


\documentclass[12pt]{article}

% packages
\usepackage{enumitem} % enumerations
\usepackage{fancyhdr} % header
\usepackage[a4paper, margin=2.5cm]{geometry} % customize page
\usepackage[colorlinks=true]{hyperref} % links
\usepackage{xcolor} % colors

% header
\pagestyle{fancy}
\fancyhead[L]{\textbf{Exercise Sheet 4: Clustering and Outlier Detection}}
\fancyhead[R]{DS 1 -- Winter 21/22}

% colors and commands
\definecolor{kitgreen}{HTML}{00876C}
\definecolor{kitblue}{HTML}{4664AA}

\hypersetup{allcolors=kitblue}

\newcommand{\code}[1]{\textcolor{kitgreen}{\texttt{#1}}}
\newcommand{\taskname}[1]{\textcolor{kitblue}{\textbf{[#1]}}}

\begin{document}

\section*{Setup}

Compared to the previous exercise sheets, you should install \code{pydataset}.
If you have used the \code{requirements.txt} from ILIAS to set up your environment, you already have this dependency available.
We will also work with \code{matplotlib}, \code{numpy}, \code{pandas}, \code{scikit-learn}, and \code{seaborn} again.

\section*{Task: Clustering and Outlier Detection}

The aim of this exercise is to apply clustering and outlier-detection approaches.
We work with the \code{faithful} dataset, which you can obtain from package \code{pydataset} with \code{data('faithful')}.
The \href{https://www.rdocumentation.org/packages/datasets/versions/3.6.2/topics/faithful}{dataset} contains eruption times and (between-eruption) waiting times for the \href{https://en.wikipedia.org/wiki/Old_Faithful}{geyser Old Faithful}.

\begin{enumerate}[label=\alph*), left=0pt, itemsep=12pt]
	\item
	\taskname{Visualization}
	Load the data and normalize it with the help of \code{scale()} from \code{sklearn.preprocessing}.
	Create a scatter plot of the data points and create histograms of their distribution.
	In particular, also use two-dimensional histograms, which are available in \code{matplotlib} as well as \code{seaborn}.
	\newline
	Where are potential clusters?
	Which points might be outliers?
	Why have we normalized the data beforehand?
	\item
	\taskname{K-Means}
	Train an instance of \code{KMeans} from \code{sklearn.cluster}.
	Visualize the results (e.g., by coloring the points according to their cluster membership in a scatter plot) for different values of \textit{k}.
	Use \code{silhouette\_score()} from \code{sklearn.metrics} to assess the quality of the k-means results.
	\newline
	Why does k-means produce these results?
	Is the silhouette coefficient a useful metric to assess clustering quality here?
	\item
	\taskname{DBSCAN}
	Train an instance of \code{DBSCAN} from \code{sklearn.cluster} and visualize the result as before.
	\newline
	Why do you obtain such a result?
	Can you leverage the silhouette coefficient to search for hyperparameter values that yield a better clustering result?
	\item
	\taskname{OPTICS}
	Train an instance of \code{OPTICS} from \code{sklearn.cluster} and create a reachability plot based on properties of the trained instance.
	\newline
	Can you use this plot to determine a good value for DBSCAN's hyperparameter $\epsilon$?
	\item
	\taskname{Outlier Detection}
	Detect outliers in the data with the help of \code{NearestNeighbors} from \code{sklearn.neighbors}, a distance-based approach, and \code{LocalOutlierFactor} from \code{sklearn.neighbors}, a density-based approach.
	Visualize the data points together with their outlier scores, e.g., by scaling the size and/or color of the points in a scatter plot accordingly.
	\newline
	For which data points do the results of the two approaches differ strongly, and why?
	\item
	\taskname{Subspace Outliers}
	Calculate the outlier scores for each feature of the dataset separately.
	Compare these results to your previous two-dimensional approach.
	\newline
	Are there points that only appear to be outliers in one subspace?
	Are there non-trivial outliers in the data?
\end{enumerate}

\end{document}


\documentclass{article} % For LaTeX2e
\usepackage{nips12submit_e,times}
%\documentstyle[nips12submit_09,times,art10]{article} % For LaTeX 2.09

\usepackage{amssymb}
\usepackage{amsmath}

\title{Chapter 2\\Outcomes, Events and Probability}

\author{
%Department of Computer and Information Science\\
%Temple University\\
%Philadelphia, PA 19122 \\
%\texttt{minxiao@temple.edu}
}

% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to \LaTeX{} to determine where to break
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
% puts 3 of 4 authors names on the first line, and the last on the second
% line, try using \AND instead of \And before the third author name.

\newcommand{\fix}{\marginpar{FIX}}
\newcommand{\new}{\marginpar{NEW}}

\nipsfinalcopy % Uncomment for camera-ready version

\begin{document}

\maketitle

\section{Summary}

\begin{itemize}
\item
\begin{itemize}
\item A {\em sample space} ($\Omega$) is a set of all the outcomes for a certain experiment.
\item A {\em event} is a subset of the sample space.
\end{itemize}
\item
\begin{itemize}
\item The {\em intersection} of event $A$ and event $B$ ($A \cap B$) is a set containing all the outcomes when both $A$ and $B$ occur.
\item The {\em union} of event $A$ and $B$  ($A \cup B$) is a set containing all the outcomes when $A$ or $B$ occurs.
\item The {\em complement} of event $A$, $A^c = \{\omega \in \Omega: \omega \notin A\}$, is a set containing all the outcomes when $A$ does not occur.
\end{itemize}
\item
\begin{itemize}
\item The event $A$ and event $B$ are {\em disjoint} or {\em mutually exclusive} if $A \cap B = \emptyset$.
\item The event $A$ implies the event $B$ if $A \subset B$.
\end{itemize}
\item {\em DeMorgan's Laws}\\
For any two events $A$ and $B$, we have
\begin{itemize}
\item $(A \cup B)^c = A^c \cap B^c$ and
\item $(A \cap B)^c = A^c \cup B^c$.
\end{itemize}
\item
\begin{itemize}
\item A {\em probability function} P on a finite sample space $\Omega$ assigns to each event $A$ in $\Omega$ a number $P(A)$ in [0,1] such that
\begin{itemize}
\item $P(\Omega) = 1$, and
\item $P(A\cup B) = P(A) + P(B)$ if A and B are disjoint.
\end{itemize}
The number P(A) is called the probability that A occurs.
\item A probability function on an infinite (or finite) sample space $\Omega$ assings to each event $A$ in $\Omega$ a number $P(A)$ in [0,1] such that
\begin{itemize}
\item $P(\Omega)$ = 1, and
\item $P\left(A_1 \cup A_2 \cup A_3 \cup \cdots \right) = P(A_1) + P(A_2) + P(A_3) + \cdots$ if $A_1, A_2, A_3, \ldots$ are disjoint events.
\end{itemize}
\end{itemize}
\item
\begin{itemize}
\item $P(A) = P(A \cap B) + P(A \cap B^c)$.
\item $P(A \cup B) = P(B) + P(A \cap B^c)$.
\item $P(A^c) = 1 - P(A)$.
\end{itemize}
\item The probability of union.\\
For any two events A and B we have \\$P(A \cup B) = P(A) + P(B) - P(A \cap B)$. \\
For any three events A and B and C we have \\$P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(A \cap C) - P(B \cap C) + P(A \cap B \cap C)$.
\item Products of sample spaces\\
When we perform an experiment $n$ times, then the correponding sample space is $\Omega = \Omega_1 \times \Omega_2 \times \cdots \Omega_n$, where $\Omega_i$ for $i=1, \ldots, n$ is a copy of the sample space of the original experiment. Moreover, the probability of the outcomes $(\omega_1, \omega_2, \ldots, \omega_n)$ is $P\left((\omega_1, \omega_2, \ldots, \omega_n)\right) = p_1 \cdot p_2 \cdots p_n$ if each $\omega_i$ has probability $p_i$.
\end{itemize}

\end{document}