\section{Margin Classification}


\section{Discussion}

	The inclusion of high-dimensional prediction techniques within the MICE framework has the potential
	to simplify the use of MI for social scientist.
	We studied bias and coverage of parameter estimates after imputation by seven high-dimensional data
	MI methods.
	Although extensive simulation studies had already been carried out by the researchers proposing these methods,
	no comparison study had been developed to assess their relative performance.
	Our research fills this gap and provides initial insights into applying such methods in social scientific research.
	In this section, we discuss the overall performance of the methods and we give recommendations for social scientists
	facing high-dimensional data imputation problems.

\paragraph{Methods that do not work well}

	We found that bridge is inadequate to deal with high-dimensional data imputation problems.
	In both the simulation and the resampling study the use of a fixed ridge penalty within the imputation
	algorithm manifested the same undesirable performance.
	The method worked well when many predictors were included in the imputation model, but the imputation
	task remained low dimensional.
	However, bridge led to extreme bias and unacceptable confidence interval coverage in all the high
	dimensional conditions.

	We have also confirmed that missForest, the high-dimensional data SI method, leads to low estimation
	bias, but results in severe confidence interval under-coverage of the true parameter values.
	Under-coverage coupled with unbiased estimates indicates that too little uncertainty is incorporated in
	the imputation procedure, which is to be expected from a single imputation approach.
	As a result, missForest should be avoided by social scientists with the goal of drawing inferential
	conclusions from their data analysis.

\paragraph{Methods that work best}

	IURR and MI-PCA were the two strongest performers.
	IURR excelled with the smallest estimation bias for item means, variances and regression coefficients.
	The method also produced small deviations from nominal coverage rates for these parameters.
	Furthermore, the large covariance estimation bias introduced by IURR in the high-dim-high-pm conditions
	only slightly exceeded the 10\% threshold and the CI coverage was just around 0.9.
	Comparatively, most of the other MI methods resulted in covariance PRBs larger than 20\%, and CICs
	well below 0.9.

	Compared to regular low dimensional MI, IURR does not require the imputer to make choices regarding which variables
	are relevant for the imputation procedure.
	The only additional decision required of the imputer is the number of folds for the cross-validation of lasso
	penalty.
	As a result, IURR is easy to specify and an extremely appealing method for imputation of large social scientific
	datasets.
	However, IURR is a relatively computationally intensive.
	If the number of variables with missing values is large, IURR might result in prohibitive imputation time.
	In such a scenario, a researcher might prefer to address imputation with the MI-PCA method.

	MI-PCA showed low bias and good coverage for both item means and covariances in experiments 1 and 2.
	Although it exhibited large bias of the item variances, the relationships between variables with missing values
	were always correctly estimated.
	It was the only method resulting in low bias and close-to-nominal CI coverage of the true covariance values,
	even in the high-dimensional conditions.
	Furthermore, it produced the lowest bias for the latent factor loadings.
	MI-PCA also resulted in low bias and CIC close to nominal rates for the focal regression coefficient
	in Experiment 3.
	Finally, when the CICs obtained with MI-PCA deviated significantly from nominal rates, they over-covered.
	This tendency is less worrisome than under-coverage as it leads to conservative, rather than liberal,
	inferential conclusions.
	As a result, this method is excellent approach for data analysts interested in testing theories on
	large social scientific datasets with missing values.

\paragraph{Methods with mixed results}
	DURR produced low bias and good CI coverage for item means, variances, and regression coefficients.
	However, compared to IURR, it suffered from greater performance deterioration when applied to
	high-dimensional data.
	As a result, our results suggest that DURR should not be preferred to IURR.

	There was little difference in performance between the use of CART and Random Forests as
	building blocks of the MICE algorithm.
	In line with what \cite{dooveEtAl:2014} found, when a difference was noticeable, it was in favor of the use
	of the simpler CART.
	Both MI-CART and MI-RF produced large covariance bias in experiments 1 and 2.
	Although, bias for means, variances, and regression coefficients was acceptable, it was usually larger
	than that obtained by other MI methods.
	Furthermore, in terms of CI coverage, they showed significant large under-coverage of most parameters in
	the high-dim-high-pm conditions.
	Although the non-parametric nature of these approaches elegantly avoids over-parametrization of imputation models,
	these methods are outperformed by IURR and MI-PCA.

	Blasso resulted in low item means and variances bias, even in the high-dimensional conditions.
	While the covariance bias was large in experiments 1 and 2, blasso performed well in the resampling study,
	where the overall biasing performance was similar to that of MI-OP.
	In terms of confidence interval coverage, blasso showed poor performance resulting in either CI
	under-coverage or CI over-coverage of true parameter values in almost all high-dimensional conditions.
	Furthermore, blasso did not fair particularly well in the recovery of the latent structure in our second
	experiment.
	Its factor loading PRBs were the highest among the MI methods.

	Theses mixed performances of blasso are also accompanied by a few obstacles to its application for social
	scientific research.
	Using \cite{hans:2010}'s Bayesian Lasso requires the specification of 6 hyper-parameters, which
	introduces more researcher degrees of freedom and demands a strong grasp of Bayesian statistics.
	Furthermore, the method has not currently been developed for multi-categorical data imputation,
	a common task in the social sciences.
	As a result, blasso is not recommended for imputation of large social scientific datasets.


\documentclass[a4paper,12pt]{article}
\usepackage{epsfig,latexsym,amsmath,amssymb,epic,eepic,psfrag,subfigure,float,euscript,array}
\usepackage[latin1]{inputenc}
\usepackage{standalone}
\usepackage{tikz,pgf,pgfplots}

\newenvironment{exercise}[1][Uppgift]{\begin{trivlist} \item[\hskip
    \labelsep {\stepcounter{exerctr}\bfseries #1
      \arabic{exerctr}}]}{\end{trivlist}\vspace{10mm}}

\newcounter{exerctr}
\newcounter{abcctr}[exerctr]

\newcommand{\abc}{\noindent\vspace{1mm}\\ {\bf
    \stepcounter{abcctr}(\alph{abcctr})\ }}
\newcommand{\bbm}{\begin{bmatrix}}
\newcommand{\ebm}{\end{bmatrix}}
\newcommand{\point}[1]{\hfill {\bf (#1p)}\\ \vspace{-5mm}}
\newcommand{\ctrb}{\EuScript{S}}
\newcommand{\Lap}{\mathcal{L}}
\newcommand{\obsv}{\EuScript{O}}
\newcommand{\realdel}[1]{\text{Re}\left\{#1\right\}}
\newcommand{\imagdel}{\text{Im}}
\newcommand{\bC}{\mathbb{C}}
\newcommand{\bR}{\mathbb{R}}
\newcommand{\bmpv}{\begin{minipage}[t]}
\newcommand{\bmps}{\begin{minipage}[t]{45mm}}
\newcommand{\bmpm}{\begin{minipage}[t]{90mm}}
\newcommand{\bmpl}{\begin{minipage}[t]{140mm}}
\newcommand{\emp}{\end{minipage}}
\newcommand*{\zethree}{\big(z - \mexp{-3h}\big)}
\newcommand*{\mexp}[1]{\ensuremath{\mathrm{e}^{#1}}}

\addtolength{\topmargin}{-1cm}
\textheight 23.5cm
%\oddsidemargin 0.61cm
%\evensidemargin 0.61cm

\def\OctaveG{tf([0.5 1], [1 0 -1])}

\title{Computerized control partial exam 1 (15\%)}
\author{Kjartan Halvorsen}

\begin{document}

\maketitle

\begin{description}
\item[Time] September 13 17:30
\item[Place] 4101
\item[Permitted aids] The single colored page with your own notes, table of Laplace transforms, calculator
\end{description}

All answers should be readable and well motivated (if nothing else is written). Solutions/motivations should be written on the provided spaces in this exam. Use the last page if more space is needed.

\begin{center}
{\Large Good luck!} \\
\end{center}

\begin{tabular}{|l|l|}
\hline
\multicolumn{2}{|l|}{\bmpl
Matricula and name
\vspace*{18mm}
\emp}\\
\hline

\end{tabular}

\clearpage

%-----------------------------------------------------------------
\subsection*{The system}
The dynamic model of a ship with input $u$ being the rudder angle and the output $y$ being the heading (see figure \ref{fig:tanker}) can be described as a continuous-time second order system with a pole in the origin
\[ G(s) = \frac{K}{s(s + a)}. \]
For fully loaded, large tankers this dynamics is often unstable, meaning that $a<0$.
\begin{figure}[h]
\begin{center}
\includegraphics[width=0.8\linewidth]{tanker}
\caption{Heading of a ship controlled by rudder input.}
\label{fig:tanker}
\end{center}
\end{figure}

Consider for this exam the normalized continuous-time model of the tanker
\[ G(s) = \frac{1}{s(s - 1)}. \]

\clearpage
\subsection*{Problem 1 (50p)}

The system is sampled with sampling interval $h$ using step-invariant (zero-order hold) sampling. \textbf{Circle the correct pulse-transfer function below, and show your calculations}
\begin{enumerate}
 \item \( H(z) = \frac{(1-\mexp{h} -h)z - \big((1-h)\mexp{h}-1\big)}{(z-1)(z-\mexp{h})}\)
 \item \( H(z) = \frac{(-1+\mexp{h} -h)z - \big((1-h)\mexp{h}-1\big)}{(z-1)(z-\mexp{2h})}\)
 \item \( H(z) = \frac{(-1+\mexp{h} -h)z - \big((1-h)\mexp{h}-1\big)}{(z-1)(z-\mexp{h})}\)
 \end{enumerate}

\noindent
\fbox{
\bmpl
{\bf Derivation:}\\
\vspace*{150mm}
\emp}

\clearpage
\subsection*{Problem 2 (20p)}
Assume that the sampling period is $h=0.2$. In figure \ref{fig:complex-plane} draw the poles (crosses) and zero (circle) for both the continuous-time transfer function $G(s)$ and the  discretized pulse-transfer function $H(z)$ you determined in Problem 1.
   \begin{figure}[h]
   \begin{center}
   \includegraphics[]{complex-plane}
   \caption{Problem 2: Plot the poles of the continuous-time system (on the left) and the poles and zero of the discrete-time system (on the right). Indicate (with arrows and/or colors) corresponding pairs of continuous-time and discrete-time poles.}
   \label{fig:complex-plane}
   \end{center}
   \end{figure}

\noindent
\fbox{
\bmpl
{\bf Calculations:}\\
\vspace*{90mm}
\emp}

\clearpage

\subsection*{Problem 3 (30p)}
The tanker is stabilized using error-feedback and a lead-compensator as in the figure below.
\begin{center}
  \includestandalone[width=0.5\linewidth]{feedback-lead}
\end{center}
The design is done in continuous-time giving the controller
\[ F(s) = 20 \frac{s+1}{s+6}. \]
Suggest a discretization method (give short motivation) and discretize the controller. Write the controller as a first order difference equation

\noindent
\fbox{
\bmpl
{\bf Solutions:}\\
\vspace*{140mm}
\emp}

\clearpage

\noindent
{\bf If necessary,} you can continue your solutions on this page. Mark clearly which problem the solution corresponds to.

%\end{document}

%*****************************************************************
%*****************************************************************
\newpage
\setcounter{page}{1}

\section*{Solutions}
\subsection*{Problem 1}
   First calculate the step-response of the continous-time system
   \[Y(s) = G(s)\frac{1}{s} = \frac{1}{s^2(s-1)} = \frac{1}{s-1} - \frac{1}{s} - \frac{1}{s^2}.\]
   The inverse Laplace-transform gives
   \[ y(t) = \mexp{t} - 1 - t\]
   Sampling this function gives
   \[ y(kh) = \mexp{kh} -1 - kh\]
   which has the Z-transform
   \[Y(z) = \frac{z}{z - \mexp{h}} - \frac{z}{z-1} - \frac{hz}{(z-1)^2}\]
   Dividing the z-transform of the system response to that of the input (the step) gives
   \begin{align*}
   H(z) &= \frac{Y(z)}{U(z)} = \frac{z-1}{z}Y(z) = \frac{z-1}{z-\mexp{h}} - 1 - \frac{h}{z-1}\\
        &= \frac{(z-1)^2 - (z-1)(z-\mexp{h}) - h(z-\mexp{h})}{(z-1)(z-\mexp{h})}\\
	&= \frac{(z-1)(z-1 -z + \mexp{h}) - hz + h\mexp{h}}{(z-1)(z-\mexp{h})}\\
        &= \frac{ (\mexp{h} - 1 -h)z - (\mexp{h}-1-h\mexp{h})}{(z-1)(z-\mexp{h})}\\
        &= \frac{ (\mexp{h} - 1 -h)z - \big( (1-h)\mexp{h}-1\big)}{(z-1)(z-\mexp{h})}\\
   \end{align*}

   The correct pulse transfer function is the third.

\subsection*{Problem 2}
The discrete-time poles are in $z=1$ and $z=\mexp{0.2} \approx 1.22$. The zero is  in
\[ z = - \frac{(1-h)\mexp{0.2} - 1}{\mexp{0.2} - 1 -h} \approx -1.07. \]
\begin{center}
\includegraphics[width=0.8\linewidth]{complex-plane-facit}
\end{center}

\subsection*{Problem 3}
Tustin's approximation should give reasonable performance, since it is a better approximation than backward or forward difference. It is also easy to apply. The approximation gives the pulse-transfer function
\[ F_d(z) = F(s)|_{s= \frac{2(z-1)}{h(z+1)}} = 20 \frac{ \frac{2(z-1)}{h(z+1)} + 1 }{ \frac{2(z-1)}{h(z+1)} + 6} = 20 \frac{2(z-1) + h(z+1)}{2(z-1) + 6h(z+1)} = 20 \frac{(2+h)z - (2-h)}{(2+6h)z - (2-6h)} \]
We get
\[ U(z) = F_d(z)E(z) = 20 \frac{(2+h)z - (2-h)}{(2+6h)z - (2-6h)} E(z) \]
or
\[ \big( (2+6h) z - (2-6h) \big) U(z) = 20 \big( (2+h)z - (2-h) \big) E(z). \]
Written as a difference equation
\[ (2+6h) u(kh+h) - (2-6h)u(kh) = 20 (2+h) e(kh+h) - 20 (2-h) e(kh)\]
or
\[ u(kh+h) = \frac{2-6h}{2+6h} u(kh) + \frac{20(2+h)}{2+6h} e(kh+h) - \frac{20(2-h)}{2+6h} e(kh). \]

\end{document}


\documentclass[a4paper,12pt]{article}
\usepackage[left=2.5cm,right=2.5cm,top=2.5cm,bottom=2.5cm]{geometry}
\usepackage{amsmath,amssymb,amsthm,algorithm,algorithmic,graphicx,yhmath,url,enumitem,lscape,hyperref}
\usepackage{wrapfig,subfigure}
\usepackage{lineno}

\newcounter{problem}
\newcounter{remark}
\newcounter{hint}
\newenvironment{remark}{\refstepcounter{remark} \vspace{0.1cm} \par \noindent {\bf Remark \arabic{remark}}}{\vspace{0.3cm}}
\newenvironment{hint}{\refstepcounter{hint} \vspace{0.1cm} \par \noindent {\bf Hint \arabic{hint}}}{\vspace{0.3cm}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\Rn}{\mathbb{R}^n}
\newcommand{\Rnn}{\mathbb{R}^{n \times n}}
\newcommand{\bes}{\begin{equation*}}
\newcommand{\ees}{\end{equation*}}
\newcommand{\be}{\begin{equation}}
\newcommand{\ee}{\end{equation}}
\newcommand{\bms}{\begin{multline*}}
\newcommand{\emults}{\end{multline*}}

% Matrices
\newcommand{\bbm}{\begin{bmatrix}}
\newcommand{\ebm}{\end{bmatrix}}
\newcommand{\bpm}{\begin{pmatrix}}
\newcommand{\epm}{\end{pmatrix}}

% Strecthing
\renewcommand{\arraystretch}{1.3}

\newcommand{\eps}{\epsilon}
\newcommand{\fl}{\text{fl}}
\newcommand{\Lp}{{L^p}}
\newcommand{\Ker}{\text{Ker}\,}
\newcommand{\loc}{{\text{loc}}}
\newcommand{\ccinf}{C_c^\infty}
\newcommand{\supp}{\text{supp}}
\newcommand{\dist}{\text{dist}}

% Gravitational acceleration
\newcommand{\bg}{\mathbf{g}}
% Friction force
\newcommand{\bFf}{\mathbf{F_f}}
% Gravitational force
\newcommand{\bFg}{\mathbf{F_g}}
% Force
\newcommand{\bF}{\mathbf{F}}
% Position, velocity, acceleration
\newcommand{\br}{\mathbf{r}}
\newcommand{\bv}{\mathbf{v}}
\newcommand{\ba}{\mathbf{a}}

% Jacobian
\newcommand{\bDF}{\mathbf{DF}}

\newtheorem{theorem}{Theorem}[section]
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{definition}[theorem]{Definition}
\title{Approximation of real functions and solution of non-trivial equations}
\author{Carl Christian Kjelgaard Mikkelsen}

\begin{document}

\pagenumbering{arabic}
% Turn off line numbering by commenting the following line.
% \linenumbers
\thispagestyle{empty}

\noindent
Ume\aa{} University \hfill Fall 2018 \\
Department of Computing Science\\

\vskip 2.5cm

\begin{center} {\Huge {\bf Project 2}}\\{\Large Approximation of real functions and solution of non-trivial equations}\end{center} \vskip 0.3cm
\begin{center}
  {\huge Scientific Computing}
  \vfill
  {\Large The deadline for this project can be found at: \href{http://www8.cs.umu.se/kurser/5DV005/HT18/planering.html}{http://www8.cs.umu.se/kurser/5DV005/HT18/planering.html}\\
  (Link \emph{Overview} on the course homepage.)}
\end{center}
  \vfill
  {% \large
\begin{itemize}
\item The submission should consist of:
  \begin{itemize}
  \item The complete report, including
    \begin{itemize}
    \item A front page with the following information:
      \begin{enumerate}
      \item Your {\bf name}.
      \item The {\bf course name}.
      \item Your {\bf username} at the Department of Computing
        Science.
      \item The {\bf project number}.
      \item The {\bf version} of the submission (in case of re-submissions).
      \end{enumerate}
    \end{itemize}
  \item An appendix with the source code.
  \item To simplify feedback, the main report (optionally excluding
    the appendix) must have {\bf numbered sections} and {\bf page
      numbers}.
  \end{itemize}
\item The submitted code must be Matlab-compatible. If you choose to
  work in Octave, verify that your code is Matlab-compatible before
  you submit your project.
\item If you write your report using \LaTeX, double-check that your
  references have been resolved correctly before you submit. ``Figure ??'' is useless to any reader.
\item Your report should be submitted as a pdf file uploaded via
  the\linebreak
  \href{https://webapps.cs.umu.se/labresults/v2/handin.php?courseid=337}{https://webapps.cs.umu.se/labresults/v2/handin.php?courseid=337}
  page, also available as the
\begin{center}
  \href{http://www8.cs.umu.se/kurser/5DV005/HT18/index2.html}{Submit/Check results}
\end{center} link at the bottom left of the course home page.
\item Furthermore, every scrap of MATLAB code developed by your team should be uploaded in a single zip file, {\tt P1Code.zip}.
\end{itemize}
  }
  \vfill

\newpage

\maketitle

\tableofcontents
\listoffigures

\section{Purpose} Let $I = [a,b] \subset \R$ and let $f: I \rightarrow \R$ denote a function. In this project we consider the problem of computing $f$ and solving the equation $f(x) = 0$. Frequently, but not universally, our knowledge of $f$ is limited to a table of function values, say,
\bes
\begin{array}{|c|c|c|c|c|} \hline
  x_0 & x_1 & x_2 & \dots & x_n \\ \hline
  f(x_0) & f(x_1) & f(x_2) & \dots & f(x_n) \\ \hline
\end{array}
\ees
We must construct accurate approximations of $f(t)$ for all $t \in [x_0,x_n]$ from this data. You will develop software which can compute accurate approximations and solve complicated equations accurately.

\section{The zero theorems} Much our work hinges on a set of theorems which assert the existence of a zero:
  \begin{itemize}
  \item Robust root finding for a continuous function $f : I \rightarrow \R$ requires a bracket $(a_0,b_0)$ such that $f(a_0)f(b_0) < 0$. By the intermediate value theorem there exists at least one zero $r \in (a_0,b_0)$.
  \item If $a_0 \in I$ and $b_0 \in I$ are zeros of a differentiable function $f : I \rightarrow \R$, then by Rolle's theorem. there exists $c$ between $a_0$ and $b_0$ such that
    \bes f'(c)=0.
    \ees
  \item  If $a_0 \in I$ and $b_0 \in I$ and $f : I \rightarrow \R$, is differentiable, then by the mean value theorem there exists $c$ between $a_0$ and $b_0$ such that
    \be
    f'(c)=\frac{f(b)-f(a)}{b-a}
    \ee
  \end{itemize}

  Assignment 1 ruthlessly exploited the intermediate value theorem. You will now develop a scripts which illustrates both Rolle's theorem and the mean value theorem.

  \begin{enumerate} \item
    Copy {\tt a2f1.m} into {\tt no2/work/MyZeroTheorem.m} and complete the script according to the specifications. The script is finished when it reproduces Figure~\ref{fig:MyZeroTheorems}.
  \end{enumerate}

  \begin{figure}
    \centering
    \includegraphics[width=10cm]{MyZeroTheorems.pdf} \caption[Illustration of central theorems]{A illustration of Rolle's theorem and the mean value theorem of differentiation.} \label{fig:MyZeroTheorems}
    \end{figure}

  \section{Approximation of derivatives}

  Let $f : I \rightarrow \R$ be a function which is infinitely often differentiable. Let $h > 0$. By Taylor's theorem we have
  \bes
  f(x + h) = f(x) + f'(x) h + \frac{1}{2} f''(x) h^2 + \dotsc, \frac{1}{p!} f^{(p)}(x) h^p + O(h^{p+1}), \quad h \rightarrow 0, \quad h > 0.
  \ees
  \begin{enumerate}
  \item Show that
    \be \label{equ:space-central}
    f'(x) = \frac{f(x+h) - f(x-h)}{2h} + O(h^2), \quad h \rightarrow 0, \quad h > 0.
    \ee
  \item Show that
    \be \label{equ:left-endpoint}
    f'(x) = \frac{-3 f(x) + 4 f(x+h) - f(x+2h)}{2h} + O(h^2), \quad  h \rightarrow 0, \quad h > 0.
    \ee
  \item Show that
    \be \label{equ:right-endpoint}
    f'(x) = \frac{f(x-2h)-4f(x-h)+3f(x)}{2h} + O(h^2), \quad  h \rightarrow 0, \quad h > 0.
    \ee
  \item Copy {\tt a2f2.m} into {\tt no2/work/MyDerivs.m} and complete the function according to the specification. It likely that {\tt MyDerivs} is working correctly when the minimal working example {\tt a2f3.m} regenerates Figure~\ref{fig:MyDerivs}.
  \item What evidence do you find to support the hypothesis that error committed by {\tt MyDerivs} is $O(h^2)$ as suggested by equations \eqref{equ:space-central}, \eqref{equ:left-endpoint}, \eqref{equ:right-endpoint}?
 \end{enumerate}

 \begin{figure}
    \centering
    \includegraphics[width=10cm]{MyDerivs.pdf} \caption[The error of finite difference approximations of the derivative]{The error of finite difference approximations of the first derivative decay rapidly when decreasing the stepsize $h$ between the sample points.} \label{fig:MyDerivs}
    \end{figure}

 \section{Hermite's piece-wise approximation}
 Consider the polynomials $p_0$, and $p_1$ given by
 \be
 p_0(t) = (1 + 2t)(1-t)^2, \qquad  p_1(t) = t^2 (3-2t) \\
 \ee
 as well as the polynomials $q_0$ and $q_1$ given by
 \be
 q_0(t) = t(1-t)^2, \qquad q_1(t) = t^2 (t-1).
 \ee

 \begin{enumerate}
 \item Show that
   \be
   p_0(0) = 1, \quad p_0(1) = 0, \quad p_0'(0) = 0, \quad p_0'(1) = 0.
   \ee
 \item Show that
   \be
   p_1(0) = 0, \quad p_1(1) = 1, \quad p_1'(0) = 0, \quad p_1'(1) = 0.
   \ee
  \item Show that
   \be
   q_0(0) = 0, \quad q_0(1) = 0, \quad q_0'(0) = 1, \quad q_0'(1) = 0.
   \ee
 \item Show that
   \be
   q_1(0) = 0, \quad q_1(1) = 0, \quad q_1'(0) = 0, \quad q_1'(1) = 1.
   \ee
 \end{enumerate}
 Consider a function $f : [a,b] \rightarrow \R$ such that $f$ is differentiable and $f'$ is continuous. Let $\phi : [a,b] \rightarrow \R$ denote the linear function which maps $a$ into $0$ and $b$ into $1$, i.e.,
 \be
 \phi(x) = \frac{x - a}{b-a}.
 \ee
 Hermite's approximation of $f : [a,b] \rightarrow \R$ is the polynomial $p : [a,b] \rightarrow \R$ given by
 \be
 p(x)  = f(a) p_0(\phi(x)) + f(b) p_1(\phi(x)) + f'(a) (b-a) q_0(\phi(x)) + f'(b) (b-a) q_1(\phi(x))
 \ee
 \begin{enumerate}[resume]
 \item Show that Hermite's approximation satisfies
   \be
   p(a) = f(a), \quad p(b) = f(b), \quad p'(a) = f'(a), \quad p'(b) = f'(b).
   \ee
 \end{enumerate}
 In short, Hermite's approximation reproduces the values of $f$ and $f'$ at the endpoints of the intervals. If we are given a list of points
 \bes
 a= x_0 < x_1 < x_2 < \dots < x_n = b
 \ees
 as well as the function values $f(x_j)$ and the derivatives $f'(x_j)$, then we can approximate $f$ using a piece-wise cubic polynomial $p : [a,b] \rightarrow \R$ given by
 \be
\forall x \in [x_{j-1},x_j] \: : \: p(x) = p_j(x)
 \ee
 where $p_j$ is Hermite's approximation of $f$ corresponding to the sub-interval $[x_{j-1}, x_j]$. By design, the function $p$ is differentiable and $p'$ is continuous.
\begin{enumerate}[resume]
 \item Copy {\tt a2f4.m} into {\tt no2/work/MyPiecewiseHermite} and complete the function according to the specification. It is likely that {\tt MyPiecewiseHermite} is working correctly when the minimal working example {\tt a2f5.m} generates Figure~\ref{fig:MyPiecewiseHermite}.

 \item What evidence do you find to support the conjecture that the error $f(x) - p(x)$ decays as $O(h^4)$?
 \end{enumerate}

 \begin{figure}
    \centering
    \includegraphics[width=10cm]{MyPiecewiseHermite.pdf} \caption[The error of the piecewise Hermite approximation]{The error of a piecewise Hermite approximation decays rapidly when we decrease the stepsize $h$ between the sample points.} \label{fig:MyPiecewiseHermite}
    \end{figure}

   \section{Event location for ordinary differential equations}

   The function {\tt range\_rkx} computes the position $(x(t),y(t))$ and velocity $(x'(t),y'(t))$ of an artillery shell by solving an ordinary differential equation
   \bes
   \gamma'(t) = f(t,\gamma(t))
   \ees
   with respect to the function $t \rightarrow \gamma(t)$ which satisfies
   \bes
   \gamma(t) = (x(t), y(t), x'(t), y'(t))^T.
   \ees
   The specific function $f$ is complicated and describes all the necessary physics. In practice, we are not particularly interested in the trajectory itself, rather we seek to solve nonlinear equation of the form
   \bes
   g(\gamma(t)) = 0,
   \ees
   where $g$ is a function such that the composition $g \circ \gamma$ is defined. For the sake of simplicity, we assume that $g$ is defined for all $z \in \R^4$, and write
   \bes
   g(z) = g(z_1, z_2, z_3, z_4).
   \ees
   The function $g$ is called an event function and we say that an event has occurred at time $t$ if $g(\gamma(t)) = 0$. Obviously, some events function are more interesting than others.

   \begin{itemize}
   \item The choice of $g(z) = z_2 - c$ corresponds to solving the nonlinear equation
     \be
     y(t) = c
     \ee
     This is equivalent to finding the time $t$ where the shell reaches the height $c$.
   \item If $x \rightarrow h(x)$ is a function which represents the height of the landscape, then the choice of $g(z) = z_2 - h(z_1)$ corresponds to solving the nonlinear equation
     \bes
     y(t) - h(x(t)) = 0.
     \ees
     This is equivalent to computing the time $t$ where the shell hits the ground.
   \end{itemize}
   By necessity, we are limited to computing a \textit{finite} set of approximations, say, $\gamma(t_j)$ for $j=0,1,\dots,m$. However, the use of Hermite's approximations allows us to extend our approximation to cover the entire interval $[t_0,t_m]$. This allows us to rapidly \textit{approximate} the solution(s) of an event equation.

   \begin{enumerate}
   \item Copy the minimal working example {\tt range\_rkx\_mwe2} into the file {\tt no2/work/MyEvent.m}. Extend {\tt MyEvent} to the point where it defines Hermite's approximation of both $t \rightarrow x(t)$ and $t \rightarrow y(t)$ and solves the non-linear equation $y(t) - h(x(t)) = 0$ where $x \rightarrow h(x) $ is the small hill defined by the script {\tt a2f6.m}, see Figure~\ref{fig:MyEvent}.
   \end{enumerate}

\begin{figure}
    \centering
    \includegraphics[width=10cm]{MyEvent.pdf} \caption[The point of impact between a shell and a simple landscape]{The point of impact between a shell and a simple landscape, i.e. a spherical hill.} \label{fig:MyEvent}
    \end{figure}

    \begin{remark} The computed solution of an event equation depends on the size of the time step used to approximate the trajectory in the first place. Estimating the error, i.e. the difference betweeen the computed solution and the real solution is a non-trival problem. It will take us a few weeks to develop the necessary skills.
   \end{remark}

   \bibliographystyle{plain}
\bibliography{../../../lecture-notes/refs}

\end{document}


\documentclass[xcolor=dvipsnames]{beamer}
%% Add [xcolor=dvipsnames] to use beamer colors

%------------------------------------

%% Boadilla theme is used
\mode<presentation> {
\usetheme{Boadilla}
\setbeamertemplate{navigation symbols}{}
\setbeamertemplate{footline}[frame number] %% For frame numbering
}

%------------------------------------

%% For using footnote w/o numbers
\makeatletter
\def\blfootnote{\gdef\@thefnmark{}\@footnotetext}
\makeatother

%------------------------------------

%% Add your favourite packages below
\usepackage{tikz} % Schematics
\usepackage{xcolor} % Color
\usepackage{amsmath} % Equations
\usepackage{braket} % BraKet notation
\usepackage{bm} % Bold equation
\usepackage{graphicx} % Images

%------------------------------------

%% Title
\title{\Large{\textbf{Title of your presentation}}}

%% Default title color is blue. To change color:
\setbeamercolor{title}{fg=Black, bg=Blue!20}
%% fg = foreground, bg = background, ! = opacity

%% Author
\author{\large{Your Name}}

%% Date (chosen today). Can be changed to a custom date
\date{\today}

%------------------------------------

%% Start the document
\begin{document}

%% Title page
\begin{frame}
\titlepage % Print the title page as the first slide
\end{frame}

%------------------------------------

%% Default frametitle color is blue. To change color:
\setbeamercolor{frametitle}{fg=Black, bg=Blue!20}

%------------------------------------
%% Start the frames/slides %%
%------------------------------------

%% Use section (and subsections) to organize
\section{Section Name}

\begin{frame}
\frametitle{Frame title}

We can add words, equations, images, etc.

%% Add an image
\centering
%\includegraphics[width=0.7\linewidth]{FilePath/PictureFile.format}

\end{frame}

%----------------------------------------

\begin{frame}
\frametitle{Frame with bullet points}

\begin{itemize}
\setlength \itemsep{1.5 em} % Space between bullets

\item<2-> This bullet appears\footnotemark[1]

\item<3-> And then this

\item<4-> And so on

\item<5-> If a sentence is too long, we can choose
\\ where to break our line % Using \\

\end{itemize}

\footnotetext[1]{\tiny{Some footnote text}}
\end{frame}

%----------------------------------------

\begin{frame}
\frametitle{Frames vs slides}

\only<1>{
Usually frame and slide are interchangeable.
}
\only<2>{
\center{But we can also have multiple slides in \textit{one} frame.}
}
\only<3>{
\center{We can \textbf{highlight} using \alert{red} or a {\color{RoyalPurple} specific} color}
}
\only<4>{
\center{We can also add color to the equations}

$$ \color{RoyalPurple}
H \ket{\Psi} = E \ket{\Psi}
$$
}
\end{frame}

\end{document}


\subsection{Aim and Objectives}

%The aim of the project is to investigate the current capabilities and future potential of artificial models for associative memory.
The aim of the project is to investigate the current capabilities of artificial models for associative memory.

The achieve this aim, the following objectives have been identified.

\begin{enumerate}
	\item Outline key models for associative memory from different fields of research (e.g. Hopfield, memristors, ...).
	% TODO: Remove in favour of "... to that of human potential" objective.
	\item Evaluate the current capabilities of the models for associative memory both in terms of their capacity to store correlated information and in terms of the applications in which they've been used.
	%\item Compare the capabilities of the ``state of the art'' models for associative memory to that of human potential.
	%\item Discuss future possibilities if models for associative memory were to reach human or post-human potential.
\end{enumerate}


\documentclass[20pt]{article} % USenglish for autoref
\usepackage[paper=screen, centering, papersize=20cm]{geometry}
\usepackage{cmap}		% to search and copy ligatures
\usepackage[utf8]{inputenc}	% for Linux computer and Mac
%\usepackage[latin1]{inputenc}	% für Windows computer
\usepackage[T1]{fontenc}	% to search for ligatures in the pdf
\usepackage[USenglish]{babel}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{enumerate}
\usepackage{amsthm}
\usepackage{listings}
\usepackage{graphicx}
\usepackage[]{algorithm2e}
%\usepackage{stmaryrd}		% for \mapsfrom
\usepackage{aliascnt}		% for Aliascounter so that autoref gives the thms the right names
\usepackage[pdfborder={0 0 0}]{hyperref}   % if you want to have math environment in captions you have to use \texorpdfstring{$x^2$}{x2}; without frame around the links
\usepackage[figure]{hypcap}		% to make autoref link to figures and not the captions of figures
%\usepackage{paralist}		% for compactitem
\usepackage{mathtools}		% for \coloneqq
\usepackage[]{todonotes}	% use option disable to disable all the todos
\usepackage{float}
\usepackage{lipsum}
\theoremstyle{break}
\newtheorem{definition}{Definition}[section]
\newtheorem{exa}[definition]{Example}
\newtheorem{cor}[definition]{Corollary}
\newtheorem{lem}[definition]{Lemma}
\newtheorem{conj}[definition]{Conjecture}
\newtheorem{quest}[definition]{Research question}
\newtheorem{thm}[definition]{Theorem}
\newtheorem{prop}[definition]{Proposition}
\newtheorem{rem}[definition]{Remark}

\renewcommand{\labelenumi}{(\roman{enumi})} % roman numbers in enumerations

\usepackage{graphicx,tikz,pgfplots} % Allows including images

\usetikzlibrary{calc,decorations.markings}
\usetikzlibrary{shapes,snakes}

\DeclareMathOperator{\Aff}{Aff}
\DeclareMathOperator{\der}{der}
\DeclareMathOperator{\Trans}{Trans}
\DeclareMathOperator{\dir}{dir}
\DeclareMathOperator{\Sing}{Sing}
\DeclareMathOperator{\Stab}{Stab}
\DeclareMathOperator{\id}{id}
\DeclareMathOperator{\im}{im}
\DeclareMathOperator{\NN}{\mathbb{N}}

%For L Cal and L Twidle:
\newcommand{\LC}{\mathcal{L}}
\newcommand{\LT}{\widetilde{\mathcal{L}}}

%For X bar
\newcommand{\XB}{\overline{X}}

%For the real and complex numbers
\newcommand{\RR}{\mathbb{R}}
\newcommand{\CC}{\mathbb{C}}

\newcommand{\remark}[2][]{\todo[color=green!50, #1]{#2}}

\title{Essential concepts covered so far}

\begin{document}
\maketitle

The following topics are the most important topics and will be covered in final exam. I try to present them in a way with minimal technical details and I hope this presentation would be more accessible.

  \newpage

  \section{Polynomial interpolation}

  $x_0, \dots, x_n$ distinct points on the domain of some function $f$.\\

  Two kinds of polynomial interpolations:
  \begin{enumerate}
  \item Lagrange interpolation: find a polynomial $p$ of degree at most $n$, such that $p(x_i)=f(x_i)$.
  \item Hermite interpolation: find a polynomial $p$ of degree at most $2n+1$, such that $p(x_i)=f(x_i)$, $p'(x_i)=f'(x_i)$.
  \end{enumerate}

  \newpage

  How to find interpolation polynomials:

  \begin{enumerate}
  \item Lagrange interpolation:
    \[p(x)=\sum_i\left( f(x_i)\cdot {\prod_{j\not=i}(x-x_j)\over \prod_{j\not=i}(x_i-x_j)}\right)\]
  \item Hermite interpolation:
 \[p(x)=\sum_i\left( f'(x_i)\cdot {(x-x_i)\prod_{j\not=i}(x-x_j)^2\over \prod_{j\not=i}(x_i-x_j)^2}\right. \]
  \[\left. +f(x_i)\cdot \left(1-(x-x_i)\sum_{j\not=i}{2\over x_i-x_j}\right)\cdot {\prod_{j\not=i}(x-x_j)^2\over \prod_{j\not=i}(x_i-x_j)^2} \right)\]
 \end{enumerate}

\newpage

Why are these formulas valid?

\begin{enumerate}
\item Lagrange interpolation: $p=\sum_if(x_i)p_i$, where
  \[p_i={\prod_{j\not=i}(x-x_j)\over \prod_{j\not=i}(x_i-x_j)}\]
  By calculation, we can verify that
  \[p_i(x_j)=\begin{cases}1 & i=j\\0 & i\not=j\end{cases}\]
  So $p(x_i)=f(x_i)p_i(x_i)+\sum_{j\not=i}f(x_j)p_j(x_i)=f(x_i)$.
\item Hermite interpolation: $p=\sum_if(x_i)p_i+\sum_jf'(x_i)q_i$.
  \[p_i=\left(1-(x-x_i)\sum_{j\not=i}{2\over x_i-x_j}\right)\cdot {\prod_{j\not=i}(x-x_j)^2\over \prod_{j\not=i}(x_i-x_j)^2}\]
  \[q_i={(x-x_i)\prod_{j\not=i}(x-x_j)^2\over \prod_{j\not=i}(x_i-x_j)^2}\]
  By calculation, we can verify that
  \[p_i(x_j)=\begin{cases}1 & i=j\\0 & i\not=j\end{cases}\]
  \[p_i'(x_j)=0\]
  \[q_i(x_j)=0\]
  \[q_i'(x_j)=\begin{cases}1 & i=j\\0 & i\not=j\end{cases}\]
\end{enumerate}

\newpage

\begin{thm}
\item Both Lagrange and Hermite interpolations are unique given $f$ and interpolation points.
\item If $p$ is the Lagrange interpolation of $f\in C^{n+1}$, $x, x_i\in [a, b]$, then there is $s\in [a, b]$:
  \[f(x)-p(x)={f^{(n+1)}(s)\prod_i(x-x_i)\over (n+1)!}\]
\item If $p$ is the Hermite interpolation of $f\in C^{2n+2}$, $x, x_i\in [a, b]$, then there is $s\in [a, b]$:
  \[f(x)-p(x)={f^{(2n+2)}(s)\prod_i(x-x_i)^2\over (2n+2)!}\]
\end{thm}

Intuition: If higher order derivative is zero, it's polynomial, hence error must be zero. So error should depend on higher order derivative. Just like Taylor series, higher order derivatives often come with higher degree polynomials.

\newpage

\section{Approximation theory}

\begin{itemize}
\item $L^\infty([a, b])$ norm: $\|f-g\|_{\infty}=\sup_{x\in [a, b]}|f(x)-g(x)|$.
\item $L^2([a, b])$ norm: $\|f-g\|_2=(\int_a^b(f-g)^2dx)^{1/2}$.
\item Weighted $L^2([a, b])$ norm: $\|f-g\|_2=(\int_a^bw(f-g)^2dx)^{1/2}$.
\end{itemize}

\newpage

Orthogonal polynomials:

\begin{definition}We call $\phi_j$, $j=0, 1, 2, \dots$ a system of {\bf orthogonal polynomials} with weight $w$, if
  \begin{enumerate}
   \item $\phi_j$ is of degree $j$.
   \item $\int_a^bw\phi_j\phi_idx$ is non-zero iff $i=j$.
  \end{enumerate}
\end{definition}

Orthogonal polynomials can be found via Gram-Schmidt applied to $\{1, x, x^2, x^3, \dots\}$.
  \[\phi_0=1\]
  \[\phi_j=x^j-\sum_{i=0}^{j-1}{\int_a^b wt^j\phi_i(t)dt\over \int_a^b w\phi_i^2(t)dt}\phi_i(x)\]

\newpage

$L^2$ best approximation (used in two problems in HW4 but less important in final exam):\\

Let $f, \psi_1, \dots, \psi_n\in L^2([a, b])$. Find a linear combination of $\psi_i$, $\sum_ia_i\psi_i$, such that $\|f-\sum_ia_i\psi_i\|_2$ is minimized.\\

Key idea: The best approximation $f^*=\sum_ia_i\psi_i$ is the {\bf orthogonal projection}, i.e. $\int_a^b\psi_i(f-f^*)dx=0$ for all $i$.

\newpage

\section{Numerical Integration (Chapter 7, 10)}

\subsection{Quadrature rule}

Question: Estimate $\int_a^bf(x)dx$.\\

Let $x_0, x_1, \dots, x_n$ be $n+1$ distinct points in $[a, b]$, then we can use the Lagrange interpolation polynomial to estimate $f$, and hence
\[\int_a^b f(x)dx\approx \sum_kw_kf(x_k)\]
  Where
  \[w_k=\int_a^b {\prod_{j\not=k}(x-x_j)\over \prod_{j\not=k}(x_k-x_j)} dx\]
To estimate the integration.\\

The points $x_i$ are called {\bf quadrature points}, and $w_i$ called {\bf quadrature weights}.

\newpage

\begin{itemize}
\item Newton-Cotes quadrature: quadrature points evenly spaces on $[a, b]$.
\item Gauss quadrature: quadrature points are roots of orthogonal polynomials.
\item Error bound via interpolation error bound.
\item Composite method: divide the interval, then use quadrature rule on subintervals.
\end{itemize}

\end{document}


\input{../../_common/preamble}

\title{Introduction to dimensionality reduction}

\begin{document}

\maketitle

\begin{frame}{Contents}
    \tableofcontents[hideallsubsections]
\end{frame}

\section{Dimensionality reduction}

\begin{frame}{Dimensionality reduction}
    \only<1>{%
        \begin{block}{Idea}
            \begin{itemize}
                \item Identify correlated columns
                \item Replace them with a new column that `encapsulates' the
                      others
            \end{itemize}
        \end{block}
        \vfill
        \begin{block}{Example}
            \begin{itemize}
                \item \{ car, cat, truck, van \}
                \item[$\rightarrow$] \{ cat, vehicle \}
            \end{itemize}
        \end{block}}
    \only<2>{%
        \begin{block}{Why?}
            \begin{itemize}
                \item `True' dimensionality is lower
                \item Too many correlated variables $\rightarrow$ collinearity
                \item Difficult to visualise
            \end{itemize}
        \end{block}
        \vfill
        \begin{block}{How?}
            \begin{itemize}
                \item Project onto a \alert{lower\hyp{}dimensional} space\ldots
                \item \ldots while \alert{retaining (most of) some property}
            \end{itemize}
        \end{block}}
\end{frame}

\section{Manifold learning}

\begin{frame}{Multidimensional scaling (MDS)}
    \only<1>{%
        \begin{block}{Aim}\vspace{-0.5em}
            \begin{itemize}
                \item Project onto a lower\hyp{}dimensional space\ldots
                \item \ldots while \alert{retaining most of the distance
                      structure}
            \end{itemize}
        \end{block}
        \begin{block}{Method}\vspace{-0.5em}
            \begin{itemize}
                \item Input: dissimilarity matrix (not necessarily a metric)
                \item Find a `close' representation (squared loss)
            \end{itemize}
        \end{block}
        \vfill
        \begin{block}{Limitations}\vspace{-0.5em}
            \begin{itemize}
                \item Somewhat slow (numerical optimisation)
                \item Embeddings are not necessarily unique or `optimal'
            \end{itemize}
        \end{block}}
    \only<2>{%
        \begin{center}
            \vfill
            \includegraphics[width=0.8\textwidth]{figures/mds_cities_1} \\[\bigskipamount]
            {\scriptsize%
             From Cutting et al.\ (2013)}
            \vfill
        \end{center}}
    \only<3>{%
        \begin{center}
            \vfill
            \includegraphics[width=0.8\textwidth]{figures/mds_cities_2} \\[\bigskipamount]
            {\scriptsize%
             From Cutting et al.\ (2013)}
            \vfill
        \end{center}}
\end{frame}

\section{PCA and PLS}

\begin{frame}{Principal component analysis (PCA)}
    \only<1>{%
        \begin{block}{Aim}\vspace{-0.5em}
            \begin{itemize}
                \item Project onto a lower\hyp{}dimensional space\ldots
                \item \ldots while \alert{retaining most of the correlation
                      structure}
            \end{itemize}
        \end{block}
        \begin{block}{Method}\vspace{-0.5em}
            \begin{itemize}
                \item Eigendecomposition of covariance/correlation matrix
                \item Typically using singular value decomposition (SVD)
            \end{itemize}
        \end{block}
        \vfill
        \begin{block}{Limitations}\vspace{-0.5em}
            \begin{itemize}
                \item \alert{Unsupervised} method
                      $\rightarrow$ outcome is disregarded
                \item PCs may not be explanatory of $\mat{Y}$
                      (noise\hyp{}driven)
            \end{itemize}
        \end{block}}
    \only<2>{%
        \begin{block}{Model}\vspace{-0.5em}
            \begin{itemize}
                \item Defined by the `direction' vectors $p_{i}$
                      (\alert{loadings})

                \item Loadings are oriented in such a way that the project data
                      $t_{i}$ (\alert{scores}) have maximum variance
            \end{itemize}
        \end{block}
        \vspace{-1em}
        \begin{center}
            \includegraphics[height=0.5\textheight]{figures/pca_decomposition} \\
            {\scriptsize%
             From \textit{Process Improvement Using Data}}
        \end{center}}
    \only<3>{%
        \vspace{1em}
        \begin{center}
            \includegraphics[width=\textwidth]{figures/pca_projection} \\[\bigskipamount]
            {\scriptsize%
             From Alex Williams' blog}
        \end{center}}
    \only<4>{%
        \vspace{1em}
        \begin{center}
            \includegraphics[width=\textwidth]{figures/pca_two_views} \\[\bigskipamount]
            {\scriptsize%
             From Alex Williams' blog}
        \end{center}}
\end{frame}

\begin{frame}{Partial least squares (PLS) regression}
    \only<1>{%
        \begin{center}
            \includegraphics[width=\textwidth]{figures/lr_vs_pcr} \\
            {\scriptsize%
             From \textit{Process Improvement Using Data}}
        \end{center}
        \begin{block}{Advantages}\vspace{-0.5em}
            \begin{itemize}
                \item Single\hyp{}step model
                \item Components capture variability in $\mat{X}$ \alert{and}
                      $\mat{Y}$
                \item[$\rightarrow$] Fewer components, more compact model
            \end{itemize}
        \end{block}}
    \only<2>{%
        \begin{center}
            \includegraphics[height=0.8\textheight]{figures/pls} \\
            {\scriptsize%
             From Böhm et al.\ (2013)}
        \end{center}}
\end{frame}

\end{document}


\section{Event Tree Data Importer}
\label{sec:ETdataImporter}

The \textbf{ETImporter} post-processor has been designed to import Event-Tree (ET) object into
RAVEN. Since several ET file formats are available, as of now only the OpenPSA format
(see https://open-psa.github.io/joomla1.5/index.php.html) is supported. As an example,
the OpenPSA format ET is shown below:

\begin{lstlisting}[style=XML,morekeywords={anAttribute},caption=ET in OpenPSA format., label=lst:ETModel_2]
<define-event-tree name="eventTree">
    <define-functional-event name="ACC"/>
    <define-functional-event name="LPI"/>
    <define-functional-event name="LPR"/>
    <define-sequence name="1"/>
    <define-sequence name="2"/>
    <define-sequence name="3"/>
    <define-sequence name="4"/>
    <initial-state>
        <fork functional-event="ACC">
            <path state="0">
                <fork functional-event="LPI">
                    <path state="0">
                        <fork functional-event="LPR">
                            <path state="0">
                                <sequence name="1"/>
                            </path>
                            <path state="+1">
                                <sequence name="2"/>
                            </path>
                        </fork>
                    </path>
                    <path state="+1">
                        <sequence name="3"/>
                    </path>
                </fork>
            </path>
            <path state="+1">
                <sequence name="4"/>
            </path>
        </fork>
    </initial-state>
</define-event-tree>
\end{lstlisting}

This is performed by saving the structure of the ET (from file) as a \textbf{PointSet}
(only \textbf{PointSet} are allowed), since an ET is a static Boolean logic structure. Each realization in the
\textbf{PointSet} represents a unique accident sequence of the ET, and the \textbf{PointSet} is structured as follows:
\begin{itemize}
  \item Input variables of the \textbf{PointSet} are the branching conditions of the ET. The value of each input variable can be:
  \begin{itemize}
    \item  0: event did occur (typically upper branch)
    \item  1: event did not occur (typically lower branch)
    \item -1: event is not queried (no branching occured)
  \end{itemize}
  \item Output variables of the \textbf{PointSet} are the ID of each branch of the ET (i.e., positive integers greater than 0)
\end{itemize}

\nb that the 0 or 1 values are specified in the \xmlNode{path state="0"} or \xmlNode{path state="1"} nodes in the ET OpenPSA file.

Provided this definition, the ET described in Listing~\ref{lst:ETModel_2} will be converted to \textbf{PointSet} that is characterized
by the following variables:
\begin{itemize}
	\item Input variables: statusACC, statusLPI, statusLPR
	\item Output variable: sequence
\end{itemize}
and the corresponding \textbf{PointSet} if the \xmlNode{expand} node is set to False is shown in Table~\ref{PointSetETExpandFalse}.
If \xmlNode{expand} set to True, the corresponding \textbf{PointSet} is shown in Table~\ref{PointSetETExpandTrue}.
\begin{table}[h]
    \centering
    \caption{PointSet generated by RAVEN by employing the ET Importer Post-Processor with \xmlNode{expand}
             set to False for the ET of Listing~\ref{lst:ETModel_2}.}
    \label{PointSetETExpandFalse}
	\begin{tabular}{c | c | c | c}
		\hline
		ACC & LPI & LPR & sequence \\
		\hline
		0.  &  0. &  0. & 1. \\
		0.  &  0. &  1. & 2. \\
		0.  &  1. & -1. & 3. \\
		1.  & -1. & -1. & 4. \\
		\hline
	\end{tabular}
\end{table}
\begin{table}[h]
    \centering
    \caption{PointSet generated by RAVEN by employing the ET Importer Post-Processor with \xmlNode{expand}
             set to True for the ET of Listing~\ref{lst:ETModel_2}.}
    \label{PointSetETExpandTrue}
	\begin{tabular}{c | c | c | c}
		\hline
		ACC & LPI & LPR & sequence \\
		\hline
		0.  &  0. &  0. & 1. \\
		0.  &  0. &  1. & 2. \\
		0.  &  1. &  0. & 3. \\
		0.  &  1. &  1. & 3. \\
		1.  &  0. &  0. & 4. \\
		1.  &  0. &  1. & 4. \\
		1.  &  1. &  0. & 4. \\
		1.  &  1. &  1. & 4. \\
		\hline
	\end{tabular}
\end{table}

The ETImporter PP supports also:
\begin{itemize}
  \item links to sub-trees
	      \nb If the ET is split in two or more ETs (and thus one file for each ET), then it is only required to list
	      all files in the Step. RAVEN automatically detect links among ETs and merge all of them into a single PointSet.
  \item by-pass branches
  \item symbolic definition of outcomes: typically outcomes are defined as either 0 (upper branch) or 1 (lower branch). If instead the ET uses the
    \textbf{success/failure} labels, then they are converted into 0/1 labels
    \nb If the branching condition is not binary or \textbf{success/failure}, then the ET Importer Post-Processor just follows
	   the numerical value of the \xmlNode{state} attribute of the \xmlNode{<path>} node in the ET OpenPSA file.
  \item symbolic/numerical definition of sequences: if the ET contains a symbolic sequence then a .xml file is generated.  This file contains
        the mapping between the sequences defined in the ET and the numerical IDs created by RAVEN. The file name is the concatenation of the ET name
        and ``\_mapping''. As an example the file ``eventTree\_mapping.xml'' generated by RAVEN:
        \begin{lstlisting}[style=XML]
            <map Tree="eventTree">
              <sequence ID="0">seq_1</sequence>
              <sequence ID="1">seq_2</sequence>
              <sequence ID="2">seq_3</sequence>
              <sequence ID="3">seq_4</sequence>
            </map>
        \end{lstlisting}
        contains the mapping of four sequences defined in the ET (seq\_1,seq\_2,seq\_3,seq\_4) with the IDs generated by RAVEN (0,1,2,3).
        Note that if the sequences defined in the ET are both numerical and symbolic then they are all mapped.
  \item The ET can contain a branch that is defined as a separate block in the \xmlNode{define-branch} node and it is
        replicated in the ET; in such case RAVEN automatically replicate such branch when generating the PointSet.
\end{itemize}
The \xmlNode{collect-formula} are not considered since this node is used to connect the Boolean formulae generated by the
Fault-Trees to the branch (i.e., fork) point.

%
\ppType{ETImporter}{ETImporter}
%
\begin{itemize}
  \item \xmlNode{fileFormat}, \xmlDesc{string, required field}, specifies the format of the file that contains the ET structure (supported format: OpenPSA).
  \item  \xmlNode{expand},\xmlDesc{bool, required parameter}, expand the ET branching conditions for all branches even if they are not queried
\end{itemize}

\textbf{Example:}
\begin{lstlisting}[style=XML,morekeywords={anAttribute},caption=ET Importer input example., label=lst:ET_PP_InputExample]
  <Files>
    <Input name="eventTreeTest" type="">eventTree.xml</Input>
  </Files>

  <Models>
    ...
    <PostProcessor name="ETimporter" subType="SR2ML.ETImporter">
      <fileFormat>OpenPSA</fileFormat>
      <expand>False</expand>
    </PostProcessor>
    ...
  </Models>

  <Steps>
    ...
    <PostProcess name="import">
      <Input   class="Files"        type=""                >eventTreeTest</Input>
      <Model   class="Models"       type="PostProcessor"   >ETimporter</Model>
      <Output  class="DataObjects"  type="PointSet"        >ET_PS</Output>
    </PostProcess>
    ...
  </Steps>

  <DataObjects>
    ...
    <PointSet name="ET_PS">
      <Input>ACC,LPI,LPR</Input>
      <Output>sequence</Output>
    </PointSet>
    ...
  </DataObjects>
\end{lstlisting}

\subsection{ET Importer Reference Tests}
\begin{itemize}
	\item test\_ETimporter.xml
	\item test\_ETimporterMultipleET.xml
	\item test\_ETimporterSymbolic.xml
	\item test\_ETimporter\_expand.xml
	\item test\_ETimporter\_DefineBranch.xml
	\item test\_ETimporter\_3branches.xml
	\item test\_ETimporter\_3branches\_NewNumbering.xml
	\item test\_ETimporter\_3branches\_NewNumbering\_expanded.xml.
\end{itemize}


\documentclass[]{article}

\usepackage{graphicx}
\usepackage{longtable}
\usepackage{hyperref}
\usepackage{color}
\usepackage{soul}
\usepackage{amsmath}

\DeclareRobustCommand{\hlcyan}[1]{{\sethlcolor{cyan}\hl{#1}}}
\DeclareRobustCommand{\hlgreen}[1]{{\sethlcolor{green}\hl{#1}}}
\DeclareRobustCommand{\hlred}[1]{{\sethlcolor{red}\hl{#1}}}
\DeclareRobustCommand{\hlyellow}[1]{{\sethlcolor{yellow}\hl{#1}}}
\DeclareRobustCommand{\hlorange}[1]{{\sethlcolor{orange}\hl{#1}}}

%opening
\title{Session 3}
\author{Fakhir}

\begin{document}

\maketitle

\section*{Solutions}

\begin{enumerate}
	\item $f'(1) = -10+60 = 50$ \\~\\
	The balance Feb will be more as compared to Jan because the rate of change i.e. balance difference per month is $+ve$.
	\item $f'(10) = -100+60 = -40$ \\~\\
	This shows that the balance in current month was less than the balance in the previous month. So no, the account balance does not continue to increase throughout the year.
	\item The account balance will continue to increase as long as the change in balances during consecutive months remains $+ve$. Through trial and error we find: \\
	$f'(5) = +10$ \\
	$f'(6) = 0$ \\
	and \\
	$f'(7) = -10$ \\~\\
	So the balance was greatest in the month of June. \\~\\

\end{enumerate}

Tip from MiT solutions: Also plug-in some values for $t$ to check the actual account balance i.e $f(t)$ in the above solutions to verify the answer.

\end{document}

mit-ocw/[in progress] Single Variable Calculus, 2010/my solutions/1. Differentiation/Session3.tex


\documentclass[12pt]{article}
\usepackage{amsfonts, amstext, amsmath, amsthm, amscd, amssymb, hyperref, float, longtable, fullpage}
\usepackage{epsfig, graphics, psfrag}
\usepackage{graphicx}
\usepackage{color}
\usepackage[font=small, labelfont=bf]{caption}
\usepackage{float}
\newtheorem{proposition}{Proposition}[section]
\newtheorem{theorem}{Theorem}[section]
\newtheorem{lemma}{Lemma}[section]
\newtheorem{corollary}{Corollary}[section]
\newtheorem{conjecture}{Conjecture}[section]
\newtheorem{claim}{Claim}[section]
\newtheorem{definition}{Definition}[section]
\let\olddefinition\definition
\renewcommand{\definition}{\olddefinition\normalfont}
\newtheorem{remark}{Remark}[section]
\let\oldremark\remark
\renewcommand{\remark}{\oldremark\normalfont}
\newtheorem{question}{Question}[section]
\let\oldquestion\question
\renewcommand{\question}{\oldquestion\normalfont}
\newtheorem{notation}{Notation}[section]
\let\oldnotation\notation
\renewcommand{\notation}{\oldnotation\normalfont}
\newtheorem{example}{Example}[section]
\let\oldexample\example
\renewcommand{\example}{\oldexample\normalfont}
\newtheorem{algorithm}{Algorithm}[section]
\let\oldalgortihm\algorithm
\renewcommand{\algorithm}{\oldalgorithm\normalfont}
\newtheorem{method}{Method}[section]
\let\oldmethod\method
\renewcommand{\method}{\oldmethod\normalfont}
\let\oldemptyset\emptyset
\let\emptyset\varnothing
\newcommand{\ndiv}{\hspace{-4pt}\not|\hspace{2pt}}

\setlength{\parindent}{0in}

% DO NOT MAKE ANY CHANGES TO ANYTHING ABOVE THIS LINE. This is the ``preamble'' of the document that sets everything up.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\title{{\bf{Math 3710W}} \\ Mathematical Modeling [Writing Portfolio]}

\author{Lixin Zheng}

\date{\today}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%START BELOW %%%%%%%%%%%%%%%%

\begin{document}

\maketitle
\section{Linear Difference Equations}

\subsection{Introduction}

Linear difference equations are used to model biological processes mathematically. There are fundamental steps in mathematical modelling that necessitate the understanding and implementation of difference equations. The phases include developing the mathematical model that describes biological processes (formulation), applying mathematical techniques to explain model behaviour (analysis), and analysing the effects of the analysis (interpretation). Having said that, biological processes are represented using difference equations.
Difference equations are relationships between quantities as they vary over discrete time and space intervals. The difference equations are the differences between successive values of a function of a discrete variable are used to prove mathematical equality.
We shall begin the study of difference equations by considering the first and simplest type of difference equation known as linear difference equations. The knowledge of linear difference equation can be fundamental. First, we give out some definitions of imperative terminologies and categorise different types of difference equations.Then we give some methods that could be used to solve linear difference equations. In the last section, we shall introduce and explain an example of linear difference equation referred to as Leslie matrix model.

\subsection{Notations and Definitions}

Now let us consider about difference equation, the changes in states of a system are modelled over discrete intervals. The letter t is used to denote the time variable in most cases. The length of time interval is denoted as $\delta{t}$ in which the the discrete time models are t= 0, $\delta{t}$, 2$\delta{t}$ and so on. $\delta{t}$ =1 for easy definition of notation.
Suppose we denote the system at t as x(t). The lowercase letters denote the real variables while the capital letters denote the vectors of real variables. For instance, the notation Xt or X(t) is used when X is a vector.

Let us give a precise definition of of first-order difference equation.

\begin{definition}
A difference equation of order k has the form
\end{definition}

\begin{equation}
f(x_{t+k}, x_{t+k-1}, …, x_{t+1}, x_{t}, t) =0 \tag{1.2.1}
\end{equation}

where f is dependent on $x_{t}$ and $x_{t+k}$ otherwise the order of the difference equation may be different from k. this is autonomous difference equation because f have nothing to do with t. otherwise it is called a non-autonomous equation.The assumption we will always lay in the difference equation is that the coefficients are real and the functions are real valued.

The difference equation is usually used is in the form of:
\begin{equation}
x_{t+k}+a_{1}x_{t+k-1}+....+a_{k-1}x_{t+1}+a_{k}x_{1}=b_{t}\tag{1.2.2}
\end{equation}

The order of the difference equation (1.2.2) is k. The function $b_{t}$ may depend on t except the state variables.

\begin{definition}
The difference equation (1.2.2) is said to be linear if:
\end{definition}

the coefficients $a_{j},j=1,...k$ constant or depend on t but independent of the state variables. otherwise it is said to be non-linear.it is also said to be homogeneous if the difference equation (1.2.2) is linear and $b_{t}=0$. Otherwise, it is said to be non-homogeneous.

\begin{definition}
If there is a k first-order difference equation, we can express it in the following form:
\begin{equation}
    x_{i}(t+1) = f_{i}(x_{1}(t), x_{2}(t),....x_{k}(t), t), i=1,2..k\tag{1.2.3}
\end{equation}
If the functions $f_{i}$ in the above equation is independent on t for i=1,2...k, we call it is autonomous.Otherwise, we can call it is non-autonomous.
\end{definition}

\begin{example}
Let us suppose such a difference equation: $x_{t+1} = ax_t^2+bx_{t-1}, a\neq0, b\neq0$. We call this equation is autonomous, second-order and non-linear.
\end{example}

\subsection{First-order difference equation}

a first-order, linear difference equation can be written in the form of:
\begin{equation}
    x_{t+1}=a_{t}x_{t}+b_{t}\tag{1.3.1}
\end{equation}
It is a typiacl difference equation. While the $b_t$ and $a_t$ is known, and $x_0$ is a Fixed value, there is some way to get a unique solution for it:

Let us consider about the $x_1$, that is :
\begin{equation}
    x_1 = a_0x_0 + b_0\tag{1.3.2}
\end{equation}
and it is the same for $x_{t+1} = a_tx_t + b_0$. Now we let $a_t = a$ and $b_t = b$, which can make the equation be simpler. This king of equation is called First order linear difference equation with constant coefficients. We can let the $t > 1$, in which way we can get a equation between $x{t+1}$ and $x_0$

\begin{equation}
    x_{t+1}=a^{t+1}x_{0}+b\sum_{i=0}^{t}c^{i}\tag{1.3.3}
\end{equation}
The above equation applies for constant coefficients $a_{t}=c$ and $b_{t}=b$.

one method that can be used to find the general solution when the coefficients are constant is demonstrated in the example below;

\begin{example}
let
\begin{equation}
    x_{t+1}=ax_{t}+b\tag{1.3.4}
\end{equation}
where a and b are constants. The solution of (1.3.4) include a general solution and a particular solution.Since b is a constant, so we can get that the particular solution of this equation is a constant. We assume the solution is $x_p=k$, in which the k is a constant. In order to find the k, we can let the $x_{t+1}$ the same as $x_t$, then equation (1.3.5) can be got:
\begin{equation}
    k=ak+b\tag{1.3.5}
\end{equation}
$k=b/(1-a)$ if a is not equal to 1.
\end{example}
\setcounter{method}{0}
\begin{method}
To solve the problem of a first-order homogeneous equation, we do the following
\begin{enumerate}
    \item Step 1 Determine one particular solution of the non-homogeneous equation
    \item Step 2 Find the general solution of the homogeneous equation
    \item Step 3 The sum of step 1 and step 2 gives the general solution.
\end{enumerate}
\end{method}

\begin{example}Solve
\begin{equation}\label{DE}
    x_t = c + bx_{i+1}\tag{1.3.6}
\end{equation}
\end{example}

\noindent{\bf Solution.}
non-homogeneous equation is ct = c. we then look for a particular equation which could be $x_t = F$ as the constant.
\begin{equation}
    F = c+bF\tag{1.3.7}
\end{equation}

The general solution is $x_t = Cb_t + c+b_F$

We have that

\begin{equation}
f(t, x_{t-1}) = bX_{t-1} + c\tag{1.3.8}
\end{equation}

We notice that c is a dependent variable while b is independent of t.\\
By method of successive calculation demonstrated in example 2.1, we could agree that the solution of the equation is:

\begin{equation*}
    x_t = b^t x_0 + \sum^t_{b^t-k} Ck\tag{1.3.9}
\end{equation*}
\\
\begin{example}
Now let us consider a practical problem which can be sloved by First-order difference equation:
Let us think about Pension issues, which we are all concerned. An old man deposits 200,000 dollars pension in the bank at the age of 60 with a monthly interest rate of $0.4\%$. Then He will take out 2,000 dollars a month as his living expenses, we can consider When will he run out of money:

This is a classical difference equation problem, so we can build a model by difference equation:
\begin{equation}
    A_{k+1} = aA_k - b\tag{1.3.10}
\end{equation}
where the $a = 1,04$ and $b = 2000$, and $A_0 = 200,000$. We can get the equation of $A_k$ that $A_n = a_nA_0 - b\frac{a_n - 1}{ra_n}$. For this difference equation, we need to let $A_n = 0$ and then $A_0 = \frac{b(a^n - 1)}{ra^n}$, where $r = 0.04$. We can use matlab to plot the figure:
\begin{figure}[htbp]
    \centering
    \includegraphics[scale=0.4]{difference_equation.png}
    \caption{The relationship between Time and Money}
    \label{fig:my_label}
\end{figure}
According to the figure, we can know that the old run out of his money when the Month is 128.
\end{example}

\subsection{Second-Order and Higher-Order Equations}
For Higher-Order Equations, there is a new method to slove the kth-order, non-homogeneous, linear difference equation. Let us consider the general solution, it is a function of k constants, which is uniquely decided by $x_0, x_1...$. It can also be used in homogeneous, second-order, linear difference equation with constant coefficients\\\\

If we have the difference equation,
\begin{equation}
    x_{t+2}+ax_{t+1}+bx_{t}=0\tag{1.4.1}
\end{equation}
given that a and b are constants. assume the solution has $x_{t}=\lambda^{t}$. Then we can get:
\begin{equation*}
    \lambda^{t+2}+a\lambda^{t+1}+b\lambda^{t}=0\tag{1.4.2}
\end{equation*}
\begin{equation*}
    \lambda^{t}(\lambda^{2}+a\lambda+b)=0\tag{1.4.3}
\end{equation*}

\begin{definition}
The equation $\lambda^2+a\lambda+b=0$ is called the characteristic equation and $\lambda^{2}+a\lambda+b$ is called the characteristic polynomial of the difference equation(2.4.1).\\\\
For equation (1.4.1) to be second order, b is not equal to 0. Therefore, none of the eigenvalues of the characteristic equation can be zero.\\\\
\end{definition}

\subsubsection{Case 1}
If the eigenvalues are real and distinct,$\lambda_1$ is not equal to $\lambda_2$.then the two linearly independent solutions are $x^{1}=\lambda_1$ and $x^2=\lambda^{t}_2$. The general solution is \\\\
\begin{equation}
    x_t=c_1\lambda^{t}_1+c_2\lambda^{t}_2\tag{1.4.1.1}
\end{equation}

\subsubsection{Case 2}
Suppose the eigenvalues are real and equal, $\lambda_1=\lambda_2$. Then the two linearly independent solutions are $x^1=\lambda^t_{1}$ and $x^2=t\lambda^t_{1}$. The general solution is\\\\
\begin{equation}
    x_t=c_1\lambda^t_{1}+c_2t\lambda^t_{1}\tag{1.4.1.2}
\end{equation}

The solutions $x^1$ and $x^2$ are linearly independent.

\begin{example}
Consider this third-order, homogeneous equation
\begin{equation*}
    x_{t+3}+x_{t+2}+x_{t+1}+x{t}=0\tag{1.4.1.3}
\end{equation*}

The equation satisfies $\lambda^3+\lambda^2+\lambda+1=0$ \\\\
This can also be factorised and written as $(\lambda+1)(\lambda^2+1)=0$\\\\
The general solution to the homogeneous equation is given by
\begin{equation*}
    x_t=c_1(-1)^1+c_2cos(t\pi/2)-c_3sin(t\pi/2)\tag{1.4.1.4}
\end{equation*}
The difference equation is non-homogeneous if
\begin{equation*}
    x_{t-3}+x_{t+2}+x_{t+1}+x_{t}=4t\tag{1.4.1.5}
\end{equation*}
we need to find the non-homogeneous equation in order to find the general solution. we assume the particular solution is a linear polynomial in t:\\\\
\begin{equation*}
    x_p=k_1t+k_2\tag{1.4.1.6}
\end{equation*}
Then we substitute the value of $x_p$ into the non-homogeneous difference equation\\\\
\begin{equation*}
    k_1(t+3)+k_2+k_1(t+2)+k_2+k_1(t+1)+k_2+k_1t+k_2=4t\tag{1.4.1.7}
\end{equation*}
when we simplify, we get;
\begin{equation*}
    4k_1t+6k_1+4k_2=4t\tag{1.4.1.8}
\end{equation*}
The coefficients are equated such that $4k_1=4$ and $6k_1+4k_2=0$. the values of k1 and k2 are $k_1=1$ and $k_2=-3/2$.The general solution is:
\begin{equation*}
    x_t=x_h+x_p=c_1(-1)^1+c_2cos(t\pi/2)+c_3sin(t\pi/2)+t-3/2\tag{1.4.1.9}
\end{equation*}

\end{example}
\subsection{Logistic Model}
Logistic Model is a classical ecological model, which is proposed famous ecologist Robert May. The difference equation of Logistic Model is as following:
\begin{equation}
    x_{k+1} = bx_k(1-x_k)\tag{1.5.1}
\end{equation}
First of all, we can let $x_{k+1} = x_k$ to calculate the balance point, which is $x_1^* = 0$ and $x_2^* = 1 - \frac{1}{b}$. There are three cases for it:

case1 : If $b < 1$, there is only one balance point $x_1^*$;

case2 : If $1 < b < 3$, $x_2^*$ will be the only one balance point;

case3 : If $b > 3$, there is no balance point;

Now there is a ecological example which is modelling with Logistic Model. There are two species in the same region. In the absence of other populations, each population can grow infinitely. The existence of the second population will reduce the growth rate of the first population, which is is proportional to the product of the number of the two species.

Then, we can build a model as following:
\begin{equation}
    \Delta x_n = k_1x_n - k_3x_ny_n\tag{1.5.2}
\end{equation}
\begin{equation}
    \Delta y_n = k_2y_n - k_4x_ny_n\tag{1.5.3}
\end{equation}
we assume $k_1 = 0.2, k_2 = 0.3, k_3 = 0.001, k_4 = 0.004$, then:
\begin{equation}
    \begin{cases}
    x_{n+1} = 1.2x_n - 0.001x_n y_n \\
    y_{n+1} = 1.3y_n - 0.002x_n y_n \\
    \end{cases}\tag{1.5.4}
\end{equation}
The balance point is (0, 0) and (150, 200), Then let us consider three case:

\subsubsection{case1}
    When the $x_0 = 151$ and $y_0 = 199$, which represents the initial number of two species. We can know in the figure how the number of them change:
    \begin{figure}[htbp]
    \centering
    \includegraphics[scale=0.5]{numbers_1.png}
    \caption{the changes when x0 = 151 and y0 = 199}
    \label{fig:my_label}
\end{figure}
In this case, we can see that the $x_t$ become more and more and $y_t$ become less and less.
\subsubsection{case2}
This case we consider that if $x_0 = 149$ and $y_0 = 201$, the figure is as following:
    \begin{figure}[H]
    \centering
    \includegraphics[scale=0.3]{numbers_2.png}
    \caption{the changes when x0 = 151 and y0 = 199}
    \label{fig:my_label}
\end{figure}
On the contract, the $x_t$ become less and less and $y_t$ become more and more.
\subsubsection{case3}
In this case, we will let $x_0 = 10$ and $y_0 = 10$. The same wat, let us look at the figure at first:
    \begin{figure}[H]
    \centering
    \includegraphics[scale=0.5]{numbers_1.png}
    \caption{the changes when x0 = 10 and y0 = 10}
    \label{fig:my_label}
\end{figure}
Now there is a interesting phenomenon. At first, the $x_t$ will increase and $y_t$ increase, too. After increasing for about 16 times, $x_t$ will decrease but $y_t$ increase faster.

Like the example 1.4, the code will be given on the last.
\subsection{Summary}

We have discussed and seen how to work with difference equations. The derivation of general solutions for non-homogeneous and homogeneous solutions has been discussed. The knowledge of difference equation is fundamental in non-linear difference equation, and therefore this part of the whole problem of discussion is very important to master and understand. The reader should not confuse the two terms and how they should be used in solving mathematical problems.
\subsection{Codes}
Example 1.4:

x0 = 200000; n = 200;

b = 2000; r = 0.004

k = (0:n)';

y1 = dai(x0, n, r, b)

round([k, y1'])

plot(k, y1', '*')

xlabel('Month')

ylabel('Money')

p = 128;

text(x(p),y(p),['(',num2str(x(p)),',',num2str(y(p)),')'],'color','b')

function x = dai(x0, n, r, b)

a = 1+r;

x = x0;

for k = 1:n

    x(k+1) = a*x(k)-b;

end

end

Logistic Mode:
k1 = 1.2;

k2 = 1.3;

k3 = 0.001;

k4 = 0.002;

x(1) = 10;

y(1) = 10;

for k = 1:1:25

    x(k+1) = k1*x(k) - k3*x(k)*y(k);

    y(k+1) = k2*y(k) - k4*x(k)*y(k);

end

figure;

t = 1:1:26;

plot(t,x,'LineWidth',2);

hold on;

plot(t,y,'LineWidth',2);

legend('x(t)','y(t)');

ylabel('x(t) and y(t)');

xlabel('t');

\begin{thebibliography}{4}
\bibitem{came}
https://en.wikipedia.org/wiki/Applied\_mathematics
\bibitem{came}
https://www.nature.com/subjects/applied-mathematics
\bibitem{came}
https://www.siam.org/Portals/0/Student\%20Programs/Thinking\%20of\%20a\%20Ca
reer/brochure.pdf
\bibitem{came}
https://en.wikipedia.org/wiki/Coronavirus\_disease\_2019
\bibitem{came}
LindaJ.S.Allen-AnIntroductiontoMathematicalBiology-Pearson(2006)
\end{thebibliography}
\end{document}


\section{Semantic Versioning}
\label{constitution_semver}

This document should be versioned according to the principles of Semantic Versioning. The version is given in $\text{MAJOR}.\text{MINOR}.\text{PATCH}$. When this document is updated in such a way that REV fundamentally changes how it works and would cause for immediate action, for example changing officers, the MAJOR number should be updated. If the way REV operates is changed, but requires no immediate action, the MINOR number should be changed. The PATCH number should be updated for grammatical errors or to make wording more clear. These numbers may exceed 9, so $5.13.0$ would be acceptable.


% !TEX TS-program = pdflatex
% !TEX encoding = UTF-8 Unicode

% This is a simple template for a LaTeX document using the "article" class.
% See "book", "report", "letter" for other types of document.

\documentclass[10pt,twocolumn]{article}

\usepackage[utf8]{inputenc} % set input encoding (not needed with XeLaTeX)
\usepackage{graphicx}
\usepackage{listings}
\usepackage{xcolor,colortbl}
\usepackage[section]{placeins}
\usepackage{amsthm}
\usepackage{mathtools}
\graphicspath{ {images/} }

%%% Examples of Article customizations
% These packages are optional, depending whether you want the features they provide.
% See the LaTeX Companion or other references for full information.

%%% PAGE DIMENSIONS
\usepackage{geometry} % to change the page dimensions
\geometry{a4paper} % or letterpaper (US) or a5paper or....
\geometry{margin=0.55in} % for example, change the margins to 2 inches all round
% \geometry{landscape} % set up the page for landscape
%   read geometry.pdf for detailed page layout information

\usepackage{graphicx} % support the \includegraphics command and options

% \usepackage[parfill]{parskip} % Activate to begin paragraphs with an empty line rather than an indent

%%% PACKAGES
\usepackage{booktabs} % for much better looking tables
\usepackage{array} % for better arrays (eg matrices) in maths
\usepackage{paralist} % very flexible & customisable lists (eg. enumerate/itemize, etc.)
\usepackage{verbatim} % adds environment for commenting out blocks of text & for better verbatim
\usepackage{subfig} % make it possible to include more than one captioned figure/table in a single float
\usepackage{indentfirst}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsthm}
% These packages are all incorporated in the memoir class to one degree or another...

%%% HEADERS & FOOTERS
\usepackage{fancyhdr} % This should be set AFTER setting up the page geometry
\pagestyle{fancy} % options: empty , plain , fancy
\renewcommand{\headrulewidth}{0pt} % customise the layout...
\lhead{}\chead{}\rhead{}
\lfoot{}\cfoot{\thepage}\rfoot{}

%%% SECTION TITLE APPEARANCE
\usepackage{sectsty}
\allsectionsfont{\sffamily\mdseries\upshape} % (See the fntguide.pdf for font help)
% (This matches ConTeXt defaults)

%%% ToC (table of contents) APPEARANCE
\usepackage[nottoc,notlof,notlot]{tocbibind} % Put the bibliography in the ToC
\usepackage[titles,subfigure]{tocloft} % Alter the style of the Table of Contents
\renewcommand{\cftsecfont}{\rmfamily\mdseries\upshape}
\renewcommand{\cftsecpagefont}{\rmfamily\mdseries\upshape} % No bold!
\setlength{\parindent}{0.5cm}

%%% END Article customizations

%%% The "real" document content comes below...

\title{On 6k $\pm$ 1 Primes in Goldbach Strong Conjecture}
\author{Marcin Barylski}
\date{\small{Published: February 18, 2018 \\ The last update: September 3, 2019}}

\definecolor{Gray}{gray}{0.85}
\definecolor{LightCyan}{rgb}{0.88,1,1}
\newcolumntype{a}{>{\columncolor{Gray}}c}
\newcolumntype{b}{>{\columncolor{white}}c}

\newtheorem{theorem}{Theorem}
\newtheorem{lemma}[theorem]{Lemma}

\newcommand\bigforall{\mbox{\huge $\mathsurround0pt\forall$}}
\newcommand\bigexists{\mbox{\huge $\mathsurround0pt\exists$}}

\begin{document}
\maketitle

\begin{abstract}
Goldbach strong conjecture, still unsolved, states that all even integers $n \textgreater 2$ can be expressed as the sum of two prime numbers (Goldbach partitions of $n$). Each prime $p \textgreater 3$ can be expressed as $6k \pm 1$. This work is devoted to studies of $6k \pm 1$ primes in Goldbach partitions and enhanced Goldbach strong conjecture with the lesser of twin primes of form $6k - 1$ used as a baseline.
\end{abstract}

\section{Introduction}

Goldbach strong conjecture ($GSC$, also called binary) asserts that all positive even integer $n$ $\geq$ 4 can be expressed as the sum of two prime numbers. This hypothesis, formulated by Goldbach in 1742 in letter to Euler \cite{goldbach1742} and then updated by Euler to the form above is one of the oldest and still unsolved problems in number theory. Empirical verification showed that it is true for all $n$ $\leq$ 4 x $10^{18}$ \cite{oliveira2012} \cite{oliveira2013}.\par
The expression of a given positive even number $n$ as a sum of two primes $p_1$ and $p_2$ is called a Goldbach Partition ($GP$) of $n$.  Let's denote this relation as $GSC(n, p_1, p_2)$. Then Goldbach strong conjecture can be written as (1):

\begin{equation}
\displaystyle\mathop{\bigforall}_{x \textgreater 1, x \in \mathbb{N}} \displaystyle\mathop{\bigexists}_{p_1, p_2 \in \mathbb{P}} GSC (2x, p_1, p_2)
\end{equation}

\section{6k $\pm$ 1  primes in GSC}

Every prime $p \textgreater 3$ can be written as $6k \pm 1$, where $k \in \mathbb{N}$ (lemma proved in \cite{barylski2018}). There are exactly two primes that are not of form $6k \pm 1$: $2$ and $3$. Prime $2$ is present in one partition only: $GSC(4, 2, 2)$, while prime $p = 3$ plays much important role in $GSC$ - it is the most frequent prime in the partitions for even $n \textless 10^6$ \cite{barylski2018}. \par
Let's exclude both primes $2$ and $3$ from a set of primes used to fulfill $GSC$. All remaining primes are of form $6k \pm 1$. It would not be possible to build neither $4$ nor $6$ nor $8$ from a sum of two such primes (because these numbers always have $GP$ with either $2$ or $3$: $GSC(4, 2, 2)$, $GSC(6, 3, 3)$, $GSC(8, 3, 5)$, $GSC(8, 5, 3)$), but situation is changing for bigger even numbers. Let $R(n)$ be a set of $GPs$ of $n$, while  $R_{6k \pm 1}(n)$ a set of $GPs$ of $n$ but without using primes $2$ and $3$ in any $GP$. As shown above: $R_{6k \pm 1}(4) = \emptyset$, $R_{6k \pm 1}(6) = \emptyset$, $R_{6k \pm 1}(8) = \emptyset$.

\begin{lemma}
$0 \leq \left\vert R(n)\right\vert - \left\vert R_{6k \pm 1}(n)\right\vert \leq 1$
\end{lemma}
\begin{proof}
There are exactly two primes $2$ and $3$ that are not of form $6k \pm 1$, where $k \in \mathbb{N}$. Let's analyze two cases: $n = 4$ and $n \textgreater 4$. For first case we have: $R(4) = (2,2)$, $R_{6k \pm 1}(4) = \emptyset$, thus $\left\vert R(4)\right\vert - \left\vert R_{6k \pm 1}(4)\right\vert = 1$ which fulfills the lemma. $2$ is not a part of any other $GP$. Let's take a look at even $n \textgreater 4$. There are $n$ for which $3$ is present in $GP$ (i.e. $R(22) = (3,19), (5,17), (11,11)$, $R_{6k \pm 1}(22) = (5,17), (11,11)$ ) or missing (i.e. $R(24) = R_{6k \pm 1}(24) = (5,19), (7,17), (11,13)$. $3$ can exist in at least one $GP$ for $n \textgreater 4$ because in $GSC(n, 3, p_1)$ we have just one way to express $p_1$: $p_1 = n - 3$. Thus for $n \textgreater 4$ we have that $\left\vert R(n)\right\vert - \left\vert R_{6k \pm 1}(n)\right\vert$ is either $0$ or $1$, and this fulfills the remaining part of the lemma.
\end{proof}

 Let $R_{6k + 1}(n)$ be a set of GPs of $n$ that both factors are primes of form $6k + 1$, and $R_{6k - 1}(n)$ be a set of GPs of $n$ that both factors are primes of form $6k - 1$. By definition $R_{6k + 1}(n) \subseteq R_{6k \pm 1}(n)$ and $R_{6k - 1}(n) \subseteq R_{6k \pm 1}(n)$.

\begin{lemma}
\begin{equation}
\displaystyle\mathop{\bigforall}_{n \in \mathbb{N}}
\left\vert R_{6k - 1}(6n) \right\vert = \left\vert R_{6k + 1}(6n) \right\vert = 0
\end{equation}
\end{lemma}
\begin{proof}
Every number of form $6n$, $n \in \mathbb{N}$, is divisible by both $2$ and $3$. Let's assume that $p_1$ is of form $6k_1-1$ and $p_2$ is of form $6k_2-1$ ($k_1, k_2 \in \mathbb{N}$). Then $s = p_1 + p_2 = 6k_1 - 1 + 6k_2 - 1 = 6(k_1 + k_2) - 2 = 2(3k_1 + 3k_2 - 1)$. $s$ is divisible by $2$ but is not divisible by $3$ because $3$ does not divide $3k_1 + 3k_2 - 1$. Similar reasoning can be done for a case when both $p_1$ and $p_2$ are of form $6k+1$. This means that $6n$ cannot be built from a sum of neither two primes of form $6k-1$ nor $6k+1$.
\end{proof}

\section{GSC broken down into three}

Original $GSC$ does not say anything particular about primes. Let's take a look at even numbers $n \textgreater 8$. Each such number can be expressed as either $3x$ or $3x+1$ or $3x+2$, where $x \in \mathbb{N}$. Calculations run for small $n$  show that original $GSC$ can be extended to a form (3):

\begin{equation}
\displaystyle\mathop{\bigforall}_{m \textgreater 4, m \in \mathbb{N}}
\left\{ \begin{array}{ll}
GSC(2m, p_{6k-1}, p_{6k+1}) & \textrm{if $m$ $mod$ $3 = 0$}\\
GSC(2m, p_{6k+1}, p_{6k+1}) & \textrm{if $m$ $mod$ $3 = 1$}\\
GSC(2m, p_{6k-1}, p_{6k-1}) & \textrm{if $m$ $mod$ $3 = 2$}\\
\end{array} \right.
\end{equation}
\\where
$p_{6k-1}$ is a prime of form ${6a-1}$ \cite{A007528} and $p_{6k+1}$ is a prime of form  ${6b+1}$ \cite{A002476} ($a, b \in \mathbb{N}$). Conjecture (3) uses limited set of prime numbers in $GSC$ - primes $2$ and $3$ are excluded.\par
Every twin prime pair different than $(3, 5)$ is of form ($6k - 1$, $6k + 1$), where $k \in \mathbb{N}$ \cite{barylski2018}. This gives a hint that yet stronger version of conjecture (3) is potentially possible. If we assume that $k$ is the same in all three conditions for the same $n$, and both $p_{6k-1}$ are the lesser of twin primes ($\mathbb{P_{LT}}$), then we can articulate the following  hypothesis (4):

\begin{equation}
\displaystyle\mathop{\bigexists}_{A \in \mathbb{N}}
\displaystyle\mathop{\bigforall}_{n \textgreater A, n \in \mathbb{N}}
\displaystyle\mathop{\bigexists}_{p_1, p_2 \in \mathbb{P_{LT}}}
GSC(6n-2, p_1, p_2)
\end{equation}
\\where
$A$ is a constant to be provided.

\begin{lemma}
If conjecture (4) is true, then we have a method to proof or invalidate $GSC$.
\end{lemma}
\begin{proof}
If both $p_1$ and $p_2$ in $GSC(6n-2, p_1, p_2)$ are the lesser of twin primes, then we have $GSC(6n, p_1+2, p_2)$ and $GSC(6n+2 , p_1+2, p_2+2)$. This is true because both $p_1+2$ and $p_2+2$ are the greater of twin primes. $GSC$ is formulated for even numbers $\textgreater 2$. If $n \in \mathbb{N}$, then numbers $6n-2$, $6n$ and $6n+2$ can build every even number $\textgreater 2$. Conjecture (4) starts from point $A+1$. If $A$ is finite, then we have a finite number of additional cases ($\leq A$) to verify against $GSC$.
\end{proof}

\section{Results of experiments}

Experiments were focused, firstly, on confirmation of conjecture (3) for bigger even numbers, secondly, on search for value of $A$ in conjecture (4), and thirdly, on looking for possible patterns between $R_{6k \pm 1}(n)$ and $R(n)$, and inside $R_{6k \pm 1}(n)$. \par

\begin{figure}[ht]
\centering
\captionsetup{justification=centering}
\includegraphics[width=9cm]{f_hypo_6kpm1}
\caption{ $R_{6k \pm 1}(n)$ ($4$ \textless $n$ \textless $2 \times 10^6$, $n = 2k, k \in \mathbb{N}$)}
\label{fig:6kmp1inpartitions}
\end{figure}

Conjecture (3) was confimed for $4 \leq m \leq 2 \times 10^6$ - this means that all even numbers $n$ that $8$ $\textless$ $n$ $\textless$ $4 \times 10^6$ have (3) fulfilled. Figure $\ref{fig:6kmp1inpartitions}$ depicts number of $GPs$ of even $n$ $\textgreater$ $4$ built from primes $p$ $\textgreater$ $3$. There is only one non-$6k \pm 1$-like prime which can be a member of such partition, $3$, but for s given $n$ it can be present in one $GP$ only (Lemma 1). This means that Figure $\ref{fig:6kmp1inpartitions}$ is very close to shape of original Goldbach's comet. \par
 \par

\begin{figure}[ht]
\centering
\captionsetup{justification=centering}
\includegraphics[width=9cm]{f_hypo_6km1_in_6n_4}
\caption{ Number of GPs  for $n$ with both primes that are the lesser of twin primes, with average values \\ ($n=4$ $mod$ $6$, $2$ \textless $n$ \textless $4 \times 10^5$, $n = 2k, k \in \mathbb{N}$)}
\label{fig:lessertwinprimesin6n4}
\end{figure}

\begin{figure}[ht]
\centering
\captionsetup{justification=centering}
\includegraphics[width=9cm]{f_lesser_to_all}
\caption{ Ratio of $\left\vert R_{LTP}(n) \right\vert$ to $\left\vert R(n) \right\vert$, with average values ($n=4$ $mod$ $6$, $2$ \textless $n$ \textless $4 \times 10^5$, $n = 2k, k \in \mathbb{N}$)}
\label{fig:lessertwinprimesin6n4_to_all}
\end{figure}

Calculations run for $1 \leq n \leq 4 \times 10^6$ confirmed that there are only 12 known cases when even number of form $6n-2 \textgreater 2$ is not a sum of two the lesser of twin primes: 4, 94, 400, 514, 784, 904, 1114, 1144, 1264, 1354, 3244, 4204. This sequence was submitted to OEIS database as OEIS A321221 \cite{A321221}. $A$ in conjecture (4) is taken from last term: if $6n-2=4204$, then $n=701$, thus $A = 701$. A321221 is a subset of sequence A007534 \cite{A007534} described in \cite{zwillinger1979}. $701$ is also the last term of related sequence \cite{A243956}.\par

Figure $\ref{fig:lessertwinprimesin6n4}$ illustrates number of $GPs$ of $n$ ($4$ mod $6$) with two primes that are the lesser of twin primes. Let $R_{LTP}(n)$ be a set of all partitions of $n$ where both primes are the lesser of twin primes. Figure $\ref{fig:lessertwinprimesin6n4_to_all}$ depicts ratio of number of elements of $R_{LTP}(n)$ to number of elements of $R(n)$. Obviously this ratio is $0$ only for $n$ from OEIS A321221.\par

 It has been computationally verified that the following even numbers of form $6n-2$ have just one partition with two primes that are the lesser of twin primes: 10, 16, 28, 40, 52, 64, 106, 124, 136, 172, 184, 226, 262, 304, 394, 412, 442, 484, 544, 556, 604, 634, 664, 682, 694, 724, 736, 754, 772, 802, 874, 934, 976, 994, 1012, 1984, 1174, 1204, 1324, 1384, 1414, 1534, 1564, 1594, 1606, 1744, 1786, 1852, 1864, 1996, 2074, 2164, 2584, 2674, 3052, 3424, 3502, 3844, 9844, 12742, 15124, 15814, 24094, 24532 - no further terms were found so far. Figure $\ref{fig:lessertwinprimesin6n4}$ demonstrates ascending trend of average number of partitions of $6n-2$. \par

 Figures $\ref{fig:hypo6kpm1c1}$ and $\ref{fig:hypo6kpm1c2}$ are devoted to differences between primes of form $6k-1$ and $6k+1$ in GPs. In general we can observe two cases: the first one with difference close to 0, and the second one - with difference either positive or negative (and with generally ascending trend for bigger $n$).

\begin{figure}[ht]
\centering
\captionsetup{justification=centering}
\includegraphics[width=9cm]{f_hypo_6kpm1_count1}
\caption{ Number of primes of form $6k-1$ in $R(n)$ - number of primes of form $6k+1$ in $R(n)$ ($2$ \textless $n$ \textless $4 \times 10^5$, $n = 2k, k \in \mathbb{N}$)}
\label{fig:hypo6kpm1c1}
\end{figure}

\begin{figure}[!ht]
\centering
\captionsetup{justification=centering}
\includegraphics[width=9cm]{f_hypo_6kpm1_count2}
\caption{ $\left\vert R_{6k - 1}(n) \right\vert - \left\vert R_{6k + 1}(n) \right\vert$ ($2$ \textless $n$ \textless $4 \times 10^5$, $n = 2k, k \in \mathbb{N}$)}
\label{fig:hypo6kpm1c2}
\end{figure}

\section{Summary and next steps}

Executed experiments confirmed that (3) is true for $4 \leq n \leq 4 \times 10^6$ and (4) with $A=701$ is true at least for $1 \leq n \leq 4 \times 10^6$. As a result this work led to more precise conjecture (5):

\begin{equation}
\displaystyle\mathop{\bigforall}_{n \textgreater 701, n \in \mathbb{N}}
\displaystyle\mathop{\bigexists}_{p_1, p_2 \in \mathbb{P_{LT}}}
GSC(6n-2, p_1, p_2)
\end{equation}

If (5) is true, then $GSC$ is true.\par
Furthermore, even if $3$ looks to be the most common prime in GPs, executed experiments revealed that all even $n > 8$ have at least one partition without prime $3$, with both primes of form $6k \pm 1$. This observation raised another open question which can be foundation of further research work: which primes can be skipped in $GSC$? Maybe prime set is much bigger than required to fullfil $GSC$?

\begin{thebibliography}{9}
\bibitem{goldbach1742}
  Christian Goldbach,
  \emph{On the margin of a letter to Leonard Euler},
  1742.
\bibitem{oliveira2012}
  Tomás Oliveira e Silva,
  \emph{Goldbach conjecture verification.}
  http://sweet.ua.pt/tos/goldbach.html,
  2012.
\bibitem{oliveira2013}
  Tomás Oliveira e Silva, Siegfried Herzog, and Silvio Pardi,
  \emph{Empirical verification of the even Goldbach conjecture and computation of prime gaps up to 4 $\times 10^{18}$.},
  Mathematics of Computation, vol. 83, no. 288, pp. 2033-2060,
  July 2014 (published electronically on November 18, 2013.
\bibitem{barylski2018}
  Marcin Barylski,
  \emph{Studies on Twin Primes in Goldbach Partitions of Even Numbers},
  http://tas-moto.org/research/TwinPrimesInGoldbachPartitions.pdf,
  2018.
\bibitem{A007528}
  OEIS Foundation Inc. (2018), The On-Line Encyclopedia of Integer Sequences, https://oeis.org/A007528. Primes of form 6n-1.
\bibitem{A002476}
  OEIS Foundation Inc. (2018), The On-Line Encyclopedia of Integer Sequences, https://oeis.org/A002476. Primes of the form 6m+1.
\bibitem{A321221}
  OEIS Foundation Inc. (2018), The On-Line Encyclopedia of Integer Sequences, https://oeis.org/A321221. Numbers of the form 6n-2 which are not a sum of two numbers that are the lesser of twin primes.
\bibitem{A243956}
  OEIS Foundation Inc. (2018), The On-Line Encyclopedia of Integer Sequences, https://oeis.org/A243956. Positive numbers n without a decomposition into a sum n = i+j such that 6i-1, 6i+1, 6j-1, 6j+1 are twin primes.
\bibitem{A007534}
  OEIS Foundation Inc. (2019), The On-Line Encyclopedia of Integer Sequences, https://oeis.org/A007534. Even numbers that are not the sum of a pair of twin primes.
\bibitem{zwillinger1979}
  Dan Zwillinger,
  \emph{A Goldbach Conjecture Using Twin Primes}, Math. Comp. 33, No.147, p.1071,
  1979.
\end{thebibliography}

\end{document}