% !TEX root = ../sel-report.tex
\section{What is a good requirement?}

\begin{mdframed}[outerlinewidth=0.5,roundcorner=5pt]
A requirement is a separately verifiable contractual statement
stating a need of the customer
\end{mdframed}

\begin{itemize}

\item The customer has certain needs and goals. However, those needs are likely to be vague or incomplete. If you build an actual product, it is hard to see how you (or your customer) would check that the product you built satisfies the customer's needs. What is needed is a \textbf{precise requirements document}.

\item A precise requirements document describes everything necessary to produce a safe and correct system---one that fulfills the needs of the customer---nothing more.

\item The requirements document must provide the software developer with \textbf{all} the information needed for the system to be built.

\item At the same time the specification must not over-constrain developers by venturing into design and implementation detail.

\item The requirements document thus specifies \emph{what} the system will do---not \emph{how} the system will do it!

\end{itemize}

\section{System Under Description}

Some stuff. Here is a png figure (see folder ``pics'')

\begin{figure}[htb]
\includegraphics[width=.4\textwidth]{pics/sud.png}
\includegraphics[width=.4\textwidth]{pics/sud.pdf}
\caption{PDF (right) displays better than jpeg or png (right)}
\label{fig:1}
\end{figure}

See Fig.~\ref{fig:1} for how to put an image in a figure environment. The ``htb'' says put the figure first here, then top of the page, then bottom, depending on layout algorithm.

\section{R-descriptions and E-descriptions}
\reqm{ENV}
{Although it is physically possible for the trip and keypad commands to occur at the same time, the system is hardwired to respond sequentially (i.e. there is an arbitrary choice which one is dealt with first). Thus all signals arrive sequentially.}
{Refer to where your mathematical model uses this}
\label{E1}

\reqm{REQ}
{If the system is armed then a trip signal from the motion detector shall result in the siren being sounded.\\}
{Refer to mathematical model}
\label{R2}

\subsection{Subsection heading}

Other stuff with inline maths $p \limp q$ and

\begin{equation}\label{eqn:1}
p \land q \lor r \limp z
\end{equation}

In (\ref{eqn:1}), we see how to write a formula.

Here is a verbatim environment with smaller text:

\begin{code}
=========================================
Use Case 1: uc1.txt: adding product types
=========================================
  report:      ok
  id:          0
  products:
  stock:
  orders:
  carts:
  order_state:
->add_type("nuts")
  report:      ok
  id:          0
  products:    nuts
  ..
\end{code}

Here is some formatted PVS
\begin{pvs}
gate: THEORY
BEGIN
  p, q, r: bool
  not_gate(x: bool): bool = NOT x
  and_gate(x:bool, y:bool): bool = x AND y
  or_gate(x:bool, y:bool): bool = x OR y
  check1: CONJECTURE and_gate(p, not_gate(q)) => (p AND NOT q)
  check2: CONJECTURE and_gate(p, not_gate(q)) =  (p AND NOT q)

  circuit_implementation(x:bool, y:bool): bool =
    and_gate(
              (or_gate(  and_gate(x, not_gate(y))
                       , and_gate(x,y))
             , y))

   and_conjecture: CONJECTURE
     and_gate(p,q) = circuit_implementation(p,q)

   or_conjecture:  CONJECTURE
     or_gate(p,q)  = circuit_implementation(p,q)
END gate
\end{pvs}

\begin{textbox}
Some text
\end{textbox}

\subsection{Signum Function}
Let's start with a simple example. In mathematics, the sign function or signum  (from \emph{signum}, Latin for ``sign") function $sgn(x)$ --- is a mathematical function that extracts the sign of a real number. In mathematical expressions the sign

\begin{figure}[ht]
\begin{mdframed}[outerlinewidth=0.5,roundcorner=5pt]
\begin{minipage}{.4\textwidth}
\[sgn(x:\Rl) =
\begin{cases}
-1, &\text{if $x < 0$;}\\
0, &\text{if $x=0$;}\\
1, &\text{if $x >0$.}
\end{cases}\]
\end{minipage}
\qquad\qquad\qquad
\begin{minipage}{.5\textwidth}
\begin{tabular}{|l|l|}
\hline
        & $sgn(x)$ \\ \hline
$x < 0$ & -1     \\ \hline
$x = 0$ & 0      \\ \hline
$x > 0$ & 1      \\ \hline
\end{tabular}
\end{minipage}
\end{mdframed}
\caption{\small (a) Standard mathematical definition of $sgn(x)$. (b) Function table definition}
\end{figure}

The normal mathematical definition and the table layout are identical in meaning. However, the table layout is easier on the reader. Note that both description methods are \textbf{complete} and \textbf{disjoint}.

\subsection*{Completeness}
The description of $sgn$ is complete because the function is defined for all possible inputs $x\in \Rl$.

\subsection*{Disjointness}
The description is disjoint because each row in the table describes a case that is independent of the other rows --- hence avoiding inconsistencies
--- where the same condition triggers different (and possibly contradictory) behaviours.

For example, in a computer controlling a robot, we would not want the press of a \emph{move} button to signall to a robot to move right and move left at the same time.

The listing below shows how to describe the function in PVS.\footnote{%
See \url{https://wiki.eecs.yorku.ca/project/sel-students/p:tutorials:pvs:start}.}

 \section{Bibliography}

 See this \cite{Lamsweerde09}, this\ cite{GS93} and this \cite{Spivey92}. Run bibtex and then latex three times.
\bibliographystyle{plain}
\bibliography{ref}


\chapter{Defining the problem}

\section{Iris flowers}
We are given sepal length, sepal width, petal length, petal width and our goal is to classify whether an iris flower is any of the following:

\begin{enumerate}
    \item Iris Setosa, see figure \ref{fig:setosa}. The image was taken from the Wikipedia article for Iris setosa~\cite{setosa}.
    \item Iris Versicolour, see figure \ref{fig:versicolour}. The image was taken from the Wikipedia article for Iris versicolour ~\cite{versicolor}.
    \item Iris Virginica, see figure \ref{fig:virginica}. The image was taken from the Wikipedia article for Iris Virginica ~\cite{virginica}.
\end{enumerate}
\begin{figure}
    \centering
    \includegraphics[scale=0.2]{setosa}
    \caption{Iris Setosa}
    \label{fig:setosa}
\end{figure}
\begin{figure}
    \centering
    \includegraphics[scale=0.2]{versicolour}
    \caption{Iris Versicolour}
    \label{fig:versicolour}
\end{figure}
\begin{figure}
    \centering
    \includegraphics{virginica}
    \caption{Iris Virginica}
    \label{fig:virginica}
\end{figure}

\section{History of the dataset}
The dataset was introduced by Ronald Fischer in 1936 in his paper "The use of multiple measurements in taxonomic problems" ~\cite{setosa}. It is currently the most popular dataset in the UCI Machine Learning repository ~\cite{uci}
\section{Attribute information}
The dataset contains the following attributes ~\cite{iris}:
\begin{enumerate}
    \item Sepal length in cm
    \item Sepal width in cm
    \item Petal length in cm
    \item Petal width in cm
    \item Class - one of Iris Setosa, Iris Versicolour, Iris Virginica
\end{enumerate}
\section{First look at the dataset}
The dataset contains 150 records, with 50 for each class. The attributes sepal length, sepal width, petal length and petal width vary between 0 and 8. There are no missing values.

\chapter{Analyze the data}
\section{Handling missing values}
Since there are no missing values, we do not need to take actions to handle the missing values
\section{Balancing classes}
The classes are perfectly balanced, so we do not need Kappa metrics or anything like this.
\section{Normalization and standardization}
All attribute types are numeric, but they seem to take on values in slightly different intervals. Normalization or standardization may be beneficial.
\section{Descriptive statistics}
Both sepallength and sepalwidth seem to be normally distributed - see fig \ref{fig:sepal}, which means that Logistic Regresson is a good candidate, since it assumes normal distribution of the attributes. There seems to be a good separation between classes for petallength and petalwidth - see fig \ref{fig:plot_matrix}, so we should expect that a model with high accuracy exists and since SVM tries to separate the data with a line, SVM is probably an interesting choice, as are trees. Since a point is usually surrounded by points of the same class, k-nearest neighbour and other lazy methods seem like a good candidate. If we always predict the majority class, we get an accuracy of 32\%, so we should definitely aim for more than that baseline.
\begin{figure}
    \centering
    \includegraphics[scale=0.8]{sepal}
    \caption{Sepal attributes distribution}
    \label{fig:sepal}
\end{figure}
\begin{figure}
    \centering
    \includegraphics[scale=0.6]{plot_matrix}
    \caption{Separations per attribute of the classes}
    \label{fig:plot_matrix}
\end{figure}

\chapter{Prepare data}
We will describe the different views of the dataset that we will use in future experiments.
\section{Raw view of the dataset}
This is the way we downloaded it from the UCI Machine Learning dataset repository
\section{Normalized view of the dataset}
This is the raw dataset after normalizing all attributes, so that all attributes are now between 0 and 1.
\section{Standardized view of the dataset}
This is the raw dataset after standardizing all attributes, so that the mean of all attributes is now 0 and the standart deviation is 1.
\section{Dataset after feature selection}
This is the raw dataset with only petallength and petalwidth. Visually, they seem to separate the classes very well. Also, after running CFS and evaluating on the subsets and using greedy stepwise for search method, we get a result that petallength and petalwidth should be selected.

\chapter{Evaluate algorithms}
\section{Initial experiment design}
On the datasets in previous chapter, we will run the following algorithms:
\begin{enumerate}
    \item Logistic regression with ridge parameter 1.0E-8 without conjugate gradient descent
    \item k-Nearest Neighbour with 3 neighbours without distance weighting
    \item Naive Bayes
    \item Multilayer Perceptron neural network with learning rate 0.3 and momentum 0.2
    \item Support vector machine using sequential minimal optimization for training and logistic regression for calibrator with polykernel and 0.001 tolerance parameter
    \item Decision tree - Pruned C4 decision tree
    \item Adaboost - with pruned C4 decision trees instead of decision stumps
    \item Random forest - to avoid potential overfitting of the decision tree
\end{enumerate}

We will use cross validation on 5 folds. Results are available in the Github Repository ~\cite{repo}. After running the experiment, we see no significant difference between the results in the different views of the dataset. We will therefore choose the dataset with only two attributes and run the expreriment again. We then see ~\cite{repo} that SVM gives the best result, followed by logistic regression and multilayer perceptron.
We will concentrate on tuning these three algorithms in the next chapter and selecting one of them

\chapter{Impove results}
\section{Experiment for improving result}
We will now run an experiment with slightly different parameters to find the optimal parameters and select a feasible model. The configurations is available in the Github reposiory ~\cite{repo}. After running the experiments, we see that the results are similar, but the SVM classifier is with highest accuracy 96.07\%, so we sill choose this algorithm.

\chapter{Present results}
The final Weka model is availble in the Github repository