\documentclass{article}
\usepackage{ws_template}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{a4wide}
\usepackage{graphicx}
\usepackage{booktabs}

% please submit the corresponding pdf by email to
% homework@class,brml.org, and write "homework sheet xx" in the
% title.  No more, no less!  (Instead of xx, however,
% put the decimal number of the homework sheet.)

% Please update the following line, only change XX to the homework
% sheet number
\title{homework sheet 04}

\author{
\name{Denys Sobchyshak}\\
\imat{03636581}\\
\email{denys.sobchyshak@gmail.com}
\And
\name{Sergey Zakharov} \\
\imat{03636642}\\
\email{ga39pad@mytum.de}
}

% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to \LaTeX{} to determine where to break
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
% puts 3 of 4 authors names on the first line, and the last on the second
% line, try using \AND instead of \And before the third author name.

\begin{document}
\maketitle

\section{Basic Probability}
\textbf{Problem 1:} \\
Let $ X_1 \sim N(\mu_1, \Sigma_1)$ and $ X_2 \sim N(\mu_2, \Sigma_2)$. We can form a Gaussian random vector R, which consists of the vectors $X_1$ and $X_2$:
\[ R = \binom{X_1}{X_2} \sim N \left(\binom{\mu_1}{\mu_2}, \begin{pmatrix} \Sigma_1 & 0\\ 0 & \Sigma_2  \end{pmatrix} \right)\]
The sum of $X_1$ and $X_2$ can be written as $S = IR$, where $I = [I I]$. That means that the sum is also a Gaussian which has a mean $\mu_1 + \mu_2$ and covariance $\Sigma_1 + \Sigma_2$.\\

\textbf{Problem 2:} \\
Suppose we have a fair coin. There are 3 random variables: $X_1$ and $X_2$ correspond to the 2 independent tosses of the coin; $X_3$ equals 1 if two tosses resulted in heads. The probability $P(X_i) = \frac{1}{2}$ for each $X_i$, where $i \in (1,2,3)$
All the variables are pairwise independent since: $P(X_i \cap X_j) = P(X_i)P(X_j) = \frac{1}{4}$, where $ i \neq j$.
But they are not mutually independent since: $P(X_1 \cap X_2 \cap X_3) =\frac{1}{4} \neq P(X_1)P(X_2)P(X_3) = \frac{1}{8}$\\

\textbf{Problem 3:} \\
For discrete case:
\begin{flalign*}
	&Var(X+Y)&\\
	&=\sum_{x}\sum_{y} (x+y)^2 P_{XY}(x,y)-(E(X+Y))^2 &\\
	&=\sum_{x}\sum_{y} x^2 P_{XY}(x,y)+\sum_{x}\sum_{y} 2xy2 P_{XY}(x,y)+\sum_{y}\sum_{x}y^2 P_{XY}(x,y)-
		(E(X))^2-2E(X)E(Y)-(E(Y))^2 &\\
	&=\sum_{x} x^2 P_{X}(x)-(E(X))^2+\sum_{y}y^2 P_{Y}(y)-(E(Y))^2+\sum_{x}\sum_{y} 2xy2 P_{XY}(x,y)-2E(X)E(Y) &\\
	&=E(X^2)-(E(X))^2+E(Y^2)-(E(Y))^2+2(E(XY)−E(X)E(Y)) &\\
	&=Var(X)+Var(Y)+2Cov(X,Y)
\end{flalign*}\\

\textbf{Problem 4:} \\
Correlation is defined as $\rho(X,Y)= \sum_{i=1}^{n} \frac{1}{n}\frac{x_i-\mu_x}{\sigma_x}\frac{y_i-\mu_y}{\sigma_y} =\frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}}$. To prove that it lies in the range $[-1, 1]$ we need to first find upper bound of it's absolute value. Applying Cauchy-Schwarz inequality we get that:\\
\[ |Cov(X,Y)|^2 \leq  Var(X)Var(Y) \]
\[ |Cov(X,Y)| \leq  \sqrt{Var(X)Var(Y)} \]
Using the result above in correlation formula we obtain:\\
\[ |\rho|=|\frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}}| \leq |\frac{\sqrt{Var(X)Var(Y)}}{\sqrt{Var(X)Var(Y)}}|=1 \]
One can notice that depending on divergence of the pairwise discrete values of random variables their product will result either in negative or positive value, which means that the variables either diverge from mean in the same maner (are larger or lower then mean together) or in the opposite one (one is larger while the other is smaller) which intuitevly describes correlation and also gives it either negative or positive value.\\

\textbf{Problem 5:} \\
From properties of distribution moments we can deduce:
\[ Var(Y)=Var(aX+b)=a^2Var(X) \]
\[ Cov(X,Y)=Cov(X,aX+b)=aCov(X,X)=a(Var(X))^2 \]
\[ \rho(X,Y)=\frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}}=\frac{a(Var(X))^2}{\sqrt{Var(X)a^2Var(X)}} \]
Now depending on the sign of $a$ our corellation will be equal to 1 or -1.\\

\textbf{Problem 6:} \\
\[p(X,Y) = \frac{cov[X,Y]}{\sqrt{var[X]var[Y]}} = \frac{E[XY] - E[X]E[Y]}{\sqrt{var[X]var[Y]}} \overset{!}{=} 0 \Leftrightarrow E[XY] = E[X]E[Y] \]
Y is uniformly distributed on [0,1] and we know that if $V \sim U(a,b)$ $E[V] = (a + b)/2$.\\
So $E[X] = (-1 + 1)/2 = 0$ and $E[Y] = (0 + 1)/2 = 1/2$.
Finally $E[XY] = \int^1_{-1}{x^3 dx} = 0$.\\
Therefore $E[XY] = E[X]E[Y] = 0 \Leftrightarrow p(X,Y) = 0$\\

\textbf{Problem 7:} \\
$ Cov[X,Y] = Cov[Y,X] = 0$ because $\rho(X,Y) = 0$. So, $\Sigma_z = \begin{pmatrix} \sigma^2_X & 0\\ 0 & \sigma^2_Y \end{pmatrix}$ and $\mu_Z = \binom{\mu_X}{\mu_Y}$ Then:
\begin{equation*}
	\begin{aligned}
p_Z(z) & = \frac{1}{2\pi \Sigma_Z} exp \left(\frac{-1}{2} (z-\mu_z)^T \Sigma^{-1}_Z(z-\mu_z)\right)\\
& = \frac{1}{2 \pi \sigma^2_X 2 \pi \sigma^2_Y} exp \left( \frac{-(x-\mu_X)^2}{2 \sigma^2_X} + \frac{-(y-\mu_Y)^2}{2 \sigma^2_Y} \right) \\
& = p_X(x)p_Y(y)
	\end{aligned}
\end{equation*}

\textbf{Problem 8:} \\
\[ H[X] = -\sum_x{ln \frac{1}{1/p(x)}p(x)} \leq -\ln \frac{1}{\sum_x{1/p(x)p(x)}} = \ln n\]
\[ D_{KL}(p||q) = \sum_x{p(x) \ln \frac{p(x)}{q(x)}} = -\sum_x{p(x) \ln \frac{q(x)}{p(x)}} \geq -\ln\sum_x{p(x) \frac{q(x)}{p(x)}}= -\ln \sum_xq(x) = -\ln(1) =0\]

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{listings}

\title{EDA132 Project 3 - Induction of Decision Trees}
\author{Axel Larsson (920515-0395), Lewis Belcher (900429-3552)}
\date{}

\usepackage[backend=biber]{biblatex}
\addbibresource{references.bib}
\usepackage{graphicx}
\usepackage{mathtools}
\usepackage[thinlines]{easytable}
\usepackage{placeins}

\begin{document}

\maketitle

\section{Introduction}
The objectives of the assignment are to implement the so called \textbf{ID3}\cite{quinlan} algorithm (Iterative Dichotomiser 3) and to write a basic parser for a subset of the Weka ARFF format\cite{weka}. Furthermore, at least one of three improvements should also be implemented, the possible choices are:
\begin{enumerate}
    \item Pruning
    \item Support for any number of nominal values for the attributes
    \item Support for integer attribute values
\end{enumerate}
We have chosen to implement improvement 2.

\subsection{Supervised Learning}
The main objective of this project is to do \textbf{inductive learning} which means learning a general function or rule from specific input-output pairs. In supervised learning the agent learns a function mapping the input to the output. The agent is given a \textbf{training set} of $N$ example input-output pairs: $(x_1,y_1), (x_2,y_2),...(x_N,y_N)$. The idea is that each $y_j$ was generated by an unknown function $y=f(x)$ which we wish to approximate with a \textbf{hypothesis} $h$. The $x$ and $y$ can be any value and does not have to be numbers.

To test the inducted hypothesis, $h$ we run it against a \textbf{test set} of new examples distinct from the original training set. The objective is to test how well $h$ generalises, i.e. how well it predicts the value of $y$ for new inputs $x$. When the output $y$ is restricted to a discrete set of values the learning problem is called \textbf{classification} and when it is not discrete it is called \textbf{regression}.

\subsection{Decision Trees}
A decision tree represents a function that takes as input a vector of attribute values and returns a single output value, i.e. a decision. There are two types of decision trees, \textbf{classification trees} and \textbf{regression trees}, analogous to the learning problems of classification and learning respectively. In this project we have focused on classification trees with discrete-valued inputs and outputs.

A decision tree performs a number of tests in order to reach a decision. Each internal vertex in the tree corresponds to a test of one of the input attributes, $A_i$ and the arcs from that vertex are labelled with the possible values for that attribute, $A_i = v_{ik}$. Each leaf in the tree specifies a possible outcome of the decision tree. An example for a decision tree consists of an $(\textbf{x}, y)$ pair where \textbf{x} is a vector of values for the input attributes and $y$ is a single output value for the goal predicate, i.e. the decision. If the tree is a Boolean decision tree the $y$ would be either "yes" or "no".

% http://randomresearchdata.blogspot.se/2013/12/tikz-simple-decision-tree-or-cart-tree.html %

\subsection{ID3}
The goal of inducing decision trees is to find a tree that is consistent with the examples and that is as small as possible. However, the search space makes it intractable to find the smallest consistent tree since it is impossible to search through $2^{2^2}$ trees in the general case. The idea of ID3 is to find a good approximate solution, i.e. a small consistent tree.

ID3 uses a greedy divide-and-conquer strategy to find the tree. It always tests the most important attribute first which then divides the tree up into smaller sub problems (divide step). Then the algorithm recursively solves the problem for the smaller sub problems and finally assembles a complete tree (the conquer step).

\subsection{Entropy}
The ID3 algorithm needs a measure of importance, a function to decide which attributes are good to split on. ID3 is designed to approximately minimise the depth of the final tree. A perfect attribute thus divides the examples into sets, each of which are either all positive or all negative and leaves of the tree. A bad attribute to split on would leave the tree with the same proportion of positive and negative examples.

In this project we use entropy to define this importance function. Entropy is a measure of uncertainty of a random variable. A random variable with one certain outcome has no uncertainty and thus its entropy is zero. We would gain nothing from observing this random variable because its outcome would already be fully known. This is analogous to a bad attribute to split on; if we choose an attribute that leaves the tree with the same proportion of different examples as before the split, we would not gain any information; we would not reduce the entropy.

The entropy of a random variable $V$ with values $v_k$, each with probability $P(v_k)$ is defined as:
\begin{displaymath}
H(V) = \sum_{k} P(v_k)log_2\frac{1}{P(v_k)} = -\sum_{k}P(v_k)log_2P(v_k)
\end{displaymath}

When dealing with Boolean random variables that are true with a probability $q$, we introduce:
\begin{displaymath}
B(q) = -(q\times log_2q+(1-q)\times log_2(1-q))
\end{displaymath}
Thus the entropy of the goal attribute on the whole set is:

\begin{displaymath}
H(Goal) = B\bigg(\frac{p}{p+n}\bigg)
\end{displaymath}
where $p$ and $n$ are the number of positive and negative examples, respectively.

To measure how much entropy a test on a single attribute $A$ gives us, we must calculate the remaining entropy after the attribute test. An attribute with $d$ distinct values will divide the training set into $d$ subsets, where each subset has $p_k$ positive and $n_k$ negative examples. We define the expected entropy remaining after an attribute test to be:

\begin{displaymath}
Remainder(A) = \sum_{k=1}^{d}\frac{p_k+n_k}{p+n}B\bigg(\frac{p_k}{p_k+n_k}\bigg)
\end{displaymath}
Now we can define the information gain, i.e. the importance of an attribute test on $A$ as the expected reduction in entropy:

\begin{displaymath}
Gain(A) = B\bigg(\frac{p}{p+n}\bigg) - Remainder(A)
\end{displaymath}

\subsection{Pruning}
Overfitting is the tendency of learning algorithms to seize on any pattern in the data, however little it actually means for generalisation. This causes the hypothesis to generalise badly. In order to combat overfitting in decision trees, we introduce $\chi^2$-pruning. The idea is to fully build the decision tree and then prune certain vertices that are deemed irrelevant because they only detect noise in the data.

Pruning works by looking at vertices that have only leaves as descendants, until each one has either been accepted or pruned. An important part of pruning is thus deciding which attributes are actually irrelevant and which are not. For deciding this we look at the information gain of the attribute. If it is close to zero, the attribute is deemed irrelevant. The question then becomes; how large a gain is enough to warrant inclusion in the tree and reject pruning?

To answer this question we use a significance test which begins with the null hypothesis, which in this case is the assumption that there is no underlying pattern in the data and the attribute should be pruned. Then the data is used to calculate the deviation from the absence of pattern. Furthermore, we like to calculate a confidence interval to be able to say that a deviation is statistically unlikely, e.g. a $5\%$ probability or less.

So, we calculate the probability that a sample of size $v=n+p$ deviates from the expected distribution of positive and negative examples. In each subset $p_k$ and $n_k$ we have the expected numbers $\hat{p}_k$ and $\hat{n}_k$ assuming the null hypothesis:

\begin{displaymath}
\hat{p}_k = p \times \frac{p_k+n_k}{p+n},\ \ \ \hat{n}_k = n \times \frac{p_k+n_k}{p+n}
\end{displaymath}

The total deviation is then given by:

\begin{displaymath}
\Delta = \sum_{k=1}^d\frac{(p_k-\hat{p}_k)^2}{\hat{p}_k} + \frac{(n_k-\hat{n}_k)^2}{\hat{n}_k}
\end{displaymath}

$\Delta$ is distributed according to the $\chi^2$-distribution with $v-1$ degrees of freedom which is a distribution of a sum of the squares of $v-1$ independent standard normal random variables. We can calculate $\Delta$ and compare the value with a table over $\chi^2$ and if the value is higher at a particular confidence level, we reject the null hypothesis at that level which means that we reject the notion that that particular attribute is irrelevant and should be pruned.

\subsection{Multiclass Decision Tree}

A multiclass decision tree can be implemented by introducing a multiclass importance function. To do this one must consider the extension from a binomial entropy to a generalised multiclass entropy where each class now contributes a proportionality:

\begin{align*}
    H(V) = -\sum_k P(v_k) \log_b P(v_k)
\end{align*}

\noindent where $b$ is the number of classes (i.e. 3 classes for the Iris database with classes: Iris-setosa, Iris-versicolor, and Iris-virginica).

Once this has been taken into consideration it is trivial to extend the importance function to take into account multiclass classification if we also pass knowledge of all classes to the importance function, the information gain remains as the total entropy for the entire dataset minus the expected entropy remaining after testing a given attribute.

\section{Implementation}
The program is written in Python 3 and is divided into two modules:
\begin{itemize}
    \item \texttt{parser}
    \item \texttt{tree}
\end{itemize}
Both resides in the \texttt{decision\_trees} package in the root of the project. The root is \texttt{/h/d5/f/dat11al1/decision-tree-induction} on the LTH computer system.

\subsection{ARFF Parser}
The parser for the Weka ARFF format is divided into two parts that are both found in the \texttt{parser} module in the \texttt{decision\_trees} package. There is a class, \texttt{Lexer} which does the lexical analysis, i.e. scanning to divide the input file into a stream of tokens as a Python generator. The \texttt{Parser} then does the syntactic analysis by reading the tokens from the generator provided by the \texttt{Lexer}. The parser is implemented as a simple recursive descent parser that supports a subset of the ARFF format. The output of the parsing is represented as a data structure described by the class \texttt{Data}, also in the \texttt{parser} module. This data structure holds the relation, the attributes and the examples read from the input ARFF file.

The ARFF format does not adequately describe which attribute is the goal predicate and thus we have adopted the convention that the last attribute is to be named "classification" and be our goal predicate.

\subsection{Usage}
The program is written as a software library more than as an executable stand-alone program. But there is a main method implemented in \texttt{tree.py} that prints the decision tree for the restaurant example. To execute it, invoke \texttt{python3 tree.py} while in \texttt{/h/d5/f/dat11al1/decision-tree-induction/decision\_trees/}. The last number printed is "1.0" and indicates the performance of the tree as the proportion correctly classified examples from the training set.

\subsection{Requirements}
The program is written in Python 3 and has two dependencies:
\begin{itemize}
    \item numpy
    \item scipy
\end{itemize}
Both dependencies are present on the student computer system at LTH.

The tools \texttt{virtualenv}, \texttt{setuptools} and \texttt{pip} can be used to install the program in a virtual environment, but this is entirely optional and is unfortunately not available on the student computer system at LTH.

\subsection{Testing}
There are two test modules:
\begin{itemize}
    \item \texttt{test\_parser}
    \item \texttt{test\_tree}
\end{itemize}
Each can be run by executing \texttt{python3 tests/<test-module.py>} from the project root (\texttt{/h/d5/f/dat11al1/decision-tree-induction}).

\section{Results}

Figure \ref{fig:learning-curve} shows the results of building decision trees (using the implementation explained above) with an increasing number of training examples. The performance is shown to converge towards 1 for large training set sizes (where all combinations of examples are more likely to be included).

The algorithm was also tested using a contact lenses dataset in the Weka format. And the implementation was found to perform perfectly when trained using the entire data (i.e. training set = test set).

\begin{figure}[h!]
    \centering
    \textbf{Test set performance as a function of training set size.}
    \includegraphics[scale=0.5]{restaurant_learning_curve.png}
    \caption{The dependence on training set size on the test set performance. Examples for the training set were generated via a bootstrap sampling method for 100 examples and averaged over 20 trails (as was done in \cite{Russell}).}
    \label{fig:learning-curve}
\end{figure}

\section{Discussion}

The implementation used in this project produces the same results as those seen in \cite{Russell} for the restaurant example, indicating that the algorithm performs as expected.

We also tried implemented $\chi^2$-pruning but unfortunately we came across problems when trying to prune the restaurant example decision tree. Our implementation suggested pruning the entire tree, which seemed a bit drastic. Therefore the decision was made to drop this feature from the program.

\printbibliography

\end{document}


\documentclass[11pt,a4paper,DIV=14]{scrartcl}
\usepackage[utf8]{inputenc}
\usepackage{fouriernc}
\usepackage[T1]{fontenc}
\usepackage[english]{babel}
\usepackage[hidelinks]{hyperref}
\usepackage{natbib}
\usepackage{url}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{trfsigns}
\usepackage{nicefrac}
\usepackage{graphicx}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{xcolor}
\usepackage{comment}
\usepackage{mdframed}
\usepackage{tikz}
\usepackage{verbatim}

%\usepackage{chngcntr}
%\counterwithout{figure}{subsection}
%\counterwithout{table}{subsection}

\numberwithin{equation}{subsection}
\numberwithin{figure}{subsection}

\bibliographystyle{dinat}

\newcommand\fsd{\mathrm{d}} %Ableitungsoperator
\renewcommand{\vec}[1]{\mathbf{#1}} %Vektor
\newcommand{\eq}[1]{Eq. (\ref{#1})}
\newcommand{\fig}[1]{Fig. \ref{#1}} %Zitat Abbildung
\newcommand{\red}{\textcolor{red}}
\newcommand{\fscom}[2][red]{\textcolor{#1}{#2}}

\usetikzlibrary{shapes.misc}
\tikzset{cross/.style={cross out, draw,
         minimum size=2*(#1-\pgflinewidth),
         inner sep=0pt, outer sep=0pt}}

\definecolor{CalcColor}{rgb}{0.0,0.0,0.5}
\newcommand{\ExCalcCol}[2][CalcColor]{\textcolor{#1}{#2}}
%\excludecomment{calc}
\includecomment{calc}

%##############################################################################
\title{Signal- und Systemtheorie\\
Übung\thanks{
This tutorial is provided as Open Educational Resource (OER), to be found at
\url{https://github.com/spatialaudio/signals-and-systems-exercises}
accompanying the OER lecture
\url{https://github.com/spatialaudio/signals-and-systems-lecture}.
%
Both are licensed under a) the Creative Commons Attribution 4.0 International
License for text and graphics and b) the MIT License for source code.
%
Please attribute material from the tutorial as \textit{Frank Schultz,
Continuous- and Discrete-Time Signals and Systems - A Tutorial Featuring
Computational Examples, University of Rostock} with
\texttt{main file, github URL, commit number and/or version tag, year}.
}
\\
\small Universität Rostock Vst.-Nr. 24015}
%
\author{Dr. Frank Schultz, Prof. Sascha Spors\\
\small Institut für Nachrichtentechnik (INT)\\
\small Fakultät für Informatik und Elektrotechnik (IEF)\\
\small Universität Rostock
}
%
\date{Sommersemester 2020, Version: \today}

%##############################################################################
\begin{document}
\setcounter{section}{2}  % UE 3 in our current sequence
\maketitle
\tableofcontents
\clearpage
\section{UE 3: Solving 2nd Order, Linear, Ordinary Differential Equation with
Constant Coefficients}
\subsection{Problem Statement}
\subsubsection{Electric RLC Circuit}
For a voltage source $u_Q(t)=x(t)$---system input in Volt---connected to a
series circuit consisting of
a resistor with resistance $R$,
an inductor with inductance $L$ and
a capacitor with capacitance $C$,
Kirchhoff's voltage law yields
\begin{align}
\label{eq:KirchhoffLaw}
u_Q(t) = u_R(t) + u_L(t) + u_C(t),
\end{align}
for the system output $y(t) = u_C(t)$ in Volt according to Fig. \ref{fig:lowpass}.
This constitutes a linear and time-invariant (LTI) system describable by impulse
response $h(t)$ and step response $h_\epsilon(t)$.
%
\begin{figure}[h]
\centering
\includegraphics[width=0.5\textwidth]{lowpass.png}
\caption{Electric RLC circuit with current $i(t)$, input voltage $x(t)$
and output voltage $y(t)$.}
\label{fig:lowpass}
\end{figure}

%##############################################################################
\subsubsection{Ordinary Differential Equation (ODE)}
There is one current $i(t)$ flowing in the RLC circuit under discussion.
The corresponding current-voltage laws for the components are
$u_R(t) = R\,i(t)$,
$u_L = L\,\frac{\mathrm{d}i(t)}{\mathrm{d} t}$ and
$u_C(t) = \frac{1}{C} \, \int i(t) \mathrm{d}t$.
With these \eq{eq:KirchhoffLaw} is rewritten as
%
\begin{align}
L C \cdot \ddot{u}_C(t) + R C \cdot \dot{u}_C(t) + u_C(t) = u_Q(t)
\end{align}
using
$\ddot{u}_C(t) = \frac{\mathrm{d}^2 u_C(t)}{\mathrm{d}t^2}$ and
$\dot{u}_C(t) = \frac{\mathrm{d} u_C(t)}{\mathrm{d}t}$
as temporal derivative notations.
%
With
the system input $x(t)$ and
the system output $y(t)$ this 2nd order, linear ODE with
constant coefficients is rewritten
\begin{align}
\label{eq:ODE_RLC}
L C \cdot \ddot{y}(t) + R C \cdot \dot{y}(t) + y(t) = x(t).
\end{align}

%##############################################################################
\subsubsection{Task}
The ODE in \eq{eq:ODE_RLC} with
$R = 3\,\nicefrac{\text{V}}{\text{A}}$,
$L=2\,\nicefrac{\text{V s}}{\text{A}}$,
$C=\frac{8}{25}\,\nicefrac{\text{A s}}{\text{V}}$
is to be solved for $t\geq 0$
\begin{itemize}
\item[a)] $y_\text{particular}(t)$ when $x(t)=\delta(t)$
\item[b)] $y(t) = y_\text{null}(t)+y_\text{particular}(t)$ when
$x(t)=1$ and initial conditions $\dot{y}(0) = 0$, $y(0)=0$
\item[c)] $y(t) = y_\text{null}(t)+y_\text{particular}(t)$ when $x(t)=0$
and initial conditions $\dot{y}(0) = 2$, $y(0)=1$
\item[d)] $y(t) = y_\text{null}(t)+y_\text{particular}(t)$ when $x(t)=1$
and initial conditions $\dot{y}(0) = 2$, $y(0)=1$
\item[e)] $y(t) = y_\text{null}(t)+y_\text{particular}(t)$ when $x(t)=\sin(t)$
and initial conditions $\dot{y}(0) = 0$, $y(0)=0$
\end{itemize}
denoting the Dirac-Delta impulse $\delta(t)$.
%and Heaviside step function
%$\epsilon(t)= \{0\,\,\text{for}\,\,t < 0;\,\,\, 1\,\,\text{for}\,\,t \geq 0\}$.
%
For Dirac impulse initial conditions refer to $t=0-$ since excitation is at $t=0$.
For other signals, initial conditions refer to the exact time instance $t=0$,
excitation signals start at right limit $t=0+$.
%
The task shall be performed with help of the \textbf{fundamental system of solutions}
and and with the help of the \textbf{Laplace transform}.

%##############################################################################
%##############################################################################
\subsection{Preliminaries for Fundamental System of Solutions}

%##############################################################################
\subsubsection{Helping Variables}
Before solving any of the requested tasks, it is good scientific and engineering
practice to adapt the ODE for convenient handling and result's interpretation.
Thus (by some experience dealing with such problems and reading text books, cf.
\cite{LangeSigSys1}, \cite{Goeldner1987},  \cite{Oppenheim1997}, \cite{Strang2014}),
we introduce
\begin{equation}
\omega_0^2 = \frac{1}{L C} \rightarrow \omega_0 = \frac{1}{\sqrt{L C}}
\end{equation}
(this is the \textbf{resonance frequency} of the system when no damping would occur)
and
\begin{equation}
\sigma_0 = \frac{1}{2}\frac{R}{L},
\qquad
D = \frac{\sigma_0}{\omega_0}.
\end{equation}
(these are related to the \textbf{damping quality} of the system).
%
Furthermore,
\begin{equation}
\omega_D^2 = \omega_0^2 (1-D^2) \qquad \rightarrow \qquad \omega_D = \omega_0
\sqrt{1-D^2}
\end{equation}
becomes convenient for the case $\sigma_0^2 - \omega_0^2 < 0$.
We will later experience the usefulness of these helping variables.
For now, it is worth to note that $\omega_0$ and $\sigma_0$ intentionally share
\textbf{similarities} with the variable $s=\sigma + \im \omega$ of the
\textbf{Laplace transform}.

By introducing the variables into \eq{eq:ODE_RLC}, we have
\begin{align}
\label{eq:ODE_sigma0}
\boxed{
\frac{\ddot{y}}{\omega_0^2} + \frac{2 \sigma_0}{\omega_0^2} \dot{y} + y = x
}\,,
\end{align}
and
\begin{align}
\label{eq:ODE_D}
\boxed{
\frac{\ddot{y}}{\omega_0^2} + \frac{2 D}{\omega_0} \dot{y} + y = x}\,,
\end{align}
when omitting the temporal dependence $t$ in $y$ and $x$ for brevity.
%
This generalisation allows for discussion of the ODE's characteristics detached
from the underlying physical representation and helps for communication between
engineers with different job specialisation.

%##############################################################################
\subsubsection{Specific Parameters}
With the given values for the electric elements
$R = 3\,\nicefrac{\text{V}}{\text{A}}$,
$L=2\,\nicefrac{\text{V s}}{\text{A}}$,
$C=\frac{8}{25}\,\nicefrac{\text{A s}}{\text{V}}$
we can rewrite \eq{eq:ODE_RLC}
\begin{align}
\boxed{
(\frac{16}{25} \cdot \text{s}^2) \, \ddot{y} + (\frac{24}{25} \cdot \text{s})
\, \dot{y} + y = x.
}
\end{align}
Note that the straight letter s here represents the unit seconds, \textbf{not the Laplace
variable}, which is typeset as italic $s$.
%
The above variables have the quantities
\begin{equation}
\omega_0^2 = \frac{25}{16} \cdot \frac{\text{rad}^2}{\text{s}^2}
\rightarrow \omega_0 = \frac{5}{4} \cdot \frac{\text{rad}}{\text{s}}
\end{equation}
and
\begin{equation}
\sigma_0 = \frac{3}{4}\cdot \frac{\text{1}}{\text{s}}
\qquad D = \frac{3}{5}
\end{equation}
and since $\sigma_0^2 - \omega_0^2 = -1 \cdot \frac{1}{\text{s}^2}< 0\cdot \frac{1}{\text{s}^2}$
\begin{equation}
\omega_D = 1 \cdot \frac{\text{rad}}{\text{s}}.
\end{equation}
%
The chosen values for $R$, $L$, $C$ lead to convenient quantities, which is on
purpose for the upcoming manual calculus. For practical problems we should not
expect such nice numbers.
%
In the remainder we omit the physical units in the calculus.

%##############################################################################
\subsubsection{Homogeneous Solution}
\label{Sec:FundamentalSet}
The present 2nd order ODE can be solved by means of the
fundamental set of solutions
considering the eigenfunctions
\begin{align}
y = \mathrm{e}^{\lambda t}, \quad \lambda\in\mathbb{C}
\end{align}
of the system. Note the (intentional) similarity to $\mathrm{e}^{s t}$,
which is the integral kernel of the Laplace transform.
%
Derivatives with respect to time $t$ become (this is the nice part, why this
works at all)
\begin{align}
y = \mathrm{e}^{\lambda t},\quad
\dot{y} = \lambda \cdot \mathrm{e}^{\lambda t},\quad
\ddot{y} = \lambda^2 \cdot \mathrm{e}^{\lambda t}.
\end{align}
%
We insert these in \eq{eq:ODE_sigma0} with $x=0$, to obtain the
so called \textbf{null solution} $y_\text{null}$, also known as
\textbf{homogeneous solution} $y_h$.
Thus,
\begin{align}
\frac{1}{\omega_0^2} (\lambda^2 \cdot \mathrm{e}^{\lambda t}) +
\frac{2 \sigma_0}{\omega_0^2} (\lambda \cdot \mathrm{e}^{\lambda t}) +
\mathrm{e}^{\lambda t} = 0,\nonumber\\
\label{eq:CharEq}
(\underbrace{\frac{\lambda^2}{\omega_0^2} +
\frac{2 \sigma_0 \lambda}{\omega_0^2} + 1}_\text{characteristic equation})
\cdot \mathrm{e}^{\lambda t} = 0.
\end{align}
%
We are not able to force the left side of this equation to zero with the $\mathrm{e}^{\lambda t}$
function.
Thus, we calculate the zeros of the term within brackets, which is well known as
\textbf{characteristic equation}.
%
The zeros of this 2nd order polynomial are
\begin{align}
\label{lambda12}
\lambda_{1,2} = -\sigma_0 \pm \sqrt{\sigma_0^2 - \omega_0^2}.
\end{align}
%
The homogeneous solution $y_h$ depends on the characteristics
of the term $\sigma_0^2 - \omega_0^2$.
Three cases need to be considered.

\paragraph{Case I for Homogeneous Solution (Kriechfall, starke Dämpfung if $\sigma_0<0$)}
For
\begin{align}
\sigma_0^2 - \omega_0^2 > 0
\end{align}
the Ansatz reads
\begin{align}
y_h=
A \mathrm{e}^{\lambda_1 t} + B \mathrm{e}^{\lambda_2 t}
=
\mathrm{e}^{-\sigma_0 t} [A \mathrm{e}^{+t\,\sqrt{\sigma_0^2 - \omega_0^2}}
+ B \mathrm{e}^{-t\,\sqrt{\sigma_0^2 - \omega_0^2}}]
\end{align}
with the two constants $A$ and $B$ to be defined by the initial conditions
$\dot{y}(t)$ and $y(t)$ for a given $t$ (very often $t=0$)
of the the complete solution
$y(t) = y_\text{null}(t)+y_\text{particular}(t)$.

\paragraph{Case II for Homogeneous Solution (Aperiodischer Grenzfall)}
For
\begin{align}
\sigma_0^2 - \omega_0^2 = 0
\end{align}
a double zero
\begin{align}
\lambda_{1} = \lambda_{2} = -\sigma_0
\end{align}
results for \eq{lambda12}.
Then the Ansatz reads
\begin{align}
y_h = \mathrm{e}^{-\sigma_0 t} [A t + B].
\end{align}

\paragraph{Case III for Homogeneous Solution (Schwingungsfall,
schwache Dämpfung if $\sigma_0<0$)}
\label{pg:caseIII}
For
\begin{align}
\sigma_0^2 - \omega_0^2 < 0,
\end{align}
the already introduced helping variable
\begin{align}
\omega_D^2 = \omega_0^2 - \sigma_0^2 > 0
\end{align}
rearranges \eq{lambda12} to
\begin{align}
\label{eq:lmb12_caseIII}
\lambda_{1,2} = -\sigma_0 \pm \sqrt{-\omega_D^2}
= -\sigma_0 \pm \mathrm{j}\,\omega_D,
\end{align}
yielding a complex conjugate zero pair.
Then the Ansatz reads (cf. case I)
\begin{align}
\label{eq:Ansatz_caseIIIkomplex}
\boxed{
y_h = A \mathrm{e}^{\lambda_1 t} + B \mathrm{e}^{\lambda_2 t}.
}
\end{align}
or (using other $A$, $B$)
\begin{align}
\label{eq:Ansatz_caseIIIsincos}
y_h = \mathrm{e}^{-\sigma_0 t}
\left[ A \sin(\omega_D t) + B \cos(\omega_D t)\right].
\end{align}
While \eq{eq:Ansatz_caseIIIkomplex} is more elegant to perform calculus,
\eq{eq:Ansatz_caseIIIsincos} directly reveals the \textbf{damped}
(parameter $\sigma_0$) sine and cosine \textbf{oscillations}
(with angular frequency $\omega_D$).
%
Case III covers the characteristics of the majority of practical systems
for signal processing.

%##############################################################################
\subsubsection{Inhomogeneous Solution}
Now, let the ODE be of form
\begin{align}
a \, \ddot{y} + b \, \dot{y} + c \, y = x.
\end{align}
with constant coefficients $a, b, c\in \mathbb{R}$.
%
There are different suitable approaches for solving such an ODE for
different inhomogeneities $x$.
We restrict our discussion to the most often asked cases.
We should check our math lecture notes for e.g. the method
\textit{variation of parameters / Variation der Konstanten} for a general
solution concept, cf. \cite{Burg2013}.

\paragraph{Polynomial Function}
The particular solution $y_p$ for the inhomogeneous ODE with
polynomial $x = P_n(t)$ of degree $n$
(for example $x = p_n t^n + p_{n-1} t^{n-1} + ... +p_0$)
requires the Ansatz
\begin{align}
y_p =
\begin{cases}
Q_n(t)&\quad a\neq 0, c\neq 0\\
t Q_n(t)&\quad a\neq 0, b\neq 0, c=0\\
t^2 Q_n(t)&\quad a\neq 0, b=0, c=0
\end{cases}
\end{align}
and solving for the coefficients $q_n, q_{n-1},...,q_0$ by comparing
coefficients in
\begin{align}
a \, \ddot{y}_p + b \, \dot{y}_p + c \, y_p = x.
\end{align}

\paragraph{Exponential Function (i.e. Eigenfunctions of the ODE)}
The particular solution $y_p$ for the inhomogeneous ODE with
$x = \e^{s_x t}$
requires the Ansatz
\begin{align}
y_p =
\begin{cases}
A \cdot \e^{s_x t}&\quad \text{if} \quad  s_x \notin \lambda_{1,2}\\
A t \cdot \e^{s_x t}&\quad \text{if} \quad s_x \in \lambda_{1 \text{ or } 2}\\
A t^2 \cdot \e^{s_x t}&\quad \text{if} \quad s_x \in \lambda_{1 \text{ and } 2}
\end{cases}
\end{align}
and solving for the coefficient $A$ by comparing coefficients.

\paragraph{Cosine / Sine Functions (i.e. Special Case of ODE's Eigenfunctions)}
\label{Sec:CosSineAnsatzInhomo}
The particular solution $y_p$ for the inhomogeneous ODE with
$x=\sin(\omega_x t)$ or $x=\cos(\omega_x t)$
requires the Ansatz
\begin{align}
y_p =
\begin{cases}
A \cdot \cos(\omega_x t) + B \cdot \sin(\omega_x t)&\quad \text{if}
\quad \im \omega_x \notin \lambda_{1,2}\\
A t \cdot \cos(\omega_x t) + B t \cdot \sin(\omega_x t)&\quad \text{if}
\quad \im \omega_x \in \lambda_{1,2}
\end{cases}
\end{align}
and solving for the coefficients $A, B$ by comparing coefficients.

%##############################################################################
\subsubsection{Full ODE Solution}
The full solution of an ODE is the superposition of the homogeneous solution
and inhomogeneous (particular) solution
\begin{align}
y(t) = y_\text{null}(t)+y_\text{particular}(t) \qquad \rightarrow \qquad
y = y_h + y_p
\end{align}
and then resolving for the initial conditions $\dot{y}(t)$ and $y(t)$ for a
given $t$ (very often $t=0$) to obtain the specific solution $y$.

%##############################################################################
%##############################################################################
\subsection{Solutions Using Fundamental System}

%##############################################################################
\subsubsection{Task a) with Fundamental System / Impulse Response}
We shall solve
\begin{align}
\frac{16}{25} \ddot{y} + \frac{24}{25} \dot{y} + y = \delta(t)
\end{align}
for the particular solution $y_p$ in the time domain.

%
A brilliant didactical approach tailored for engineers to solve this task is
found in \cite{Strang2014}.
%
In short, the \textbf{particular solution} $y_p(t)$ of this equation
with \textbf{Dirac impulse excitation} is named the \textbf{Green's function} $g(t)$.
%
When the null solutions of the ODE are known, the Green's function can be found
by variation of parameters.
Making use of the important sifting property
$\int\limits_{-\infty}^{+\infty} \delta(t-t_0) \cdot f(t) \, \mathrm{d} t \stackrel{\mathrm{def}}= f(t_0)$
becomes a vital part when doing this.
%
However, we will not go into detail here, cf. \cite[p.133]{Strang2014} instead.
%
The important part of the story is:
Once $g(t)$ is known,
the \textbf{particular solution} $y_p$ \textbf{for any other inhomogeneity}
$x(t)$ can be derived with the \textbf{convolution} operation
$y_p(t) = x(t) * g(t)$.
%
This constitutes a fundamental (if not the most important) theorem for ODEs.
%
For our discussed ODEs in signal and
system theory (where we only handle time dependence), we call the Green's
function $g(t)$ typically the \textbf{impulse response} $h(t)$ of the
\textbf{LTI system} and use the convolution $y_p(t) = x(t) * h(t)$.

\textbf{Important detail}: Actually the problem
$\frac{16}{25} \ddot{h}(t) + \frac{24}{25} \dot{h}(t) + h(t) = \delta(t),\,\,\,
\dot{h}(0-)=0,\,\,\,h(0-)=0$
solves for the impulse response, where we need the \textbf{left sided}
limit $t=0-$, since the Dirac impulse excites at $t=0$.
%
This proof is a little bit off-topic due to lack of time.
%
Instead, we can use the particular solution only by keeping in
mind that the system was in rest before excitation.

Said that, we can make our life even more convenient, since according to
\cite[p.97]{Strang2014},
the \textbf{Green's function is also found by the null solution}
(instead of Dirac excitation)
\textbf{and new specific initial conditions}.
In general for a 2nd order ODE this means
%\begin{align}
$a \, \ddot{y} + b \, \dot{y} + c \, y = 0,
\quad y(0)=0,
\quad \dot{y}(0)=\frac{1}{a}$,
%\end{align}
and thus for our specified example
\begin{align}
\frac{16}{25} \ddot{y} + \frac{24}{25} \dot{y} + y = 0,
\quad y(0)=0,
\quad \dot{y}(0)=\frac{25}{16}
\end{align}
for $t\geq 0$.
%
Feel free to explore the solution by usage of \url{https://www.wolframalpha.com}
with the input \verb|16/25*y’’(t)+24/25*y’(t)+y(t)=0, y(0)=0, y'(0)=25/16|
or with other tools that can handle symbolic math on a computer.
%
We are going to calculate this manually in the following.
%

\paragraph{Homogeneous Solution}
\label{Sec:TaskaHomo}
First, we need the homogeneous solution of the ODE for this subtask.
%
We will also need this in general, so it is anyway a good idea to calculate
this first.
%
According to Sec. \ref{Sec:FundamentalSet} the characteristic equation for the
ODE is \eq{lambda12}
\begin{align}
\lambda_{1,2} = -\sigma_0 \pm \sqrt{\sigma_0^2 - \omega_0^2}.
\end{align}
and with the chosen quantities, we get
\begin{align}
\lambda_{1,2} = -\frac{3}{4} \pm \im,
\end{align}
i.e. a complex conjugate zero pair with intentionally very simple numbers.
Thus, we deal with \textbf{case III} (page \pageref{pg:caseIII})
for the homogeneous solution, this is the case of \textbf{damped oscillation}.
The corresponding zero pair is depicted in Fig.~\ref{fig:sketch_lambda_plane},
left.
%
Repeating \eq{eq:lmb12_caseIII}
$\lambda_{1,2} = -\sigma_0 \pm \mathrm{j}\omega_D$,
we identify $\sigma_0 = \frac{3}{4}$ and $\omega_D = 1$.
Repeating \eq{eq:Ansatz_caseIIIkomplex}
$y_h = A \mathrm{e}^{\lambda_1 t} + B \mathrm{e}^{\lambda_2 t}$,
we therefore have to deal with
\begin{align}
y_h = A \, \mathrm{e}^{(-\frac{3}{4}+\im)\,t} + B \, \mathrm{e}^{(-\frac{3}{4}-\im)\,t},
\end{align}
and since $y_p=0$
\begin{align}
y = y_h + y_p = A \, \mathrm{e}^{(-\frac{3}{4}+\im)\,t} + B \, \mathrm{e}^{(-\frac{3}{4}-\im)\,t}.
\end{align}

\paragraph{Initial Conditions}
\label{Sec:TaskaInitCond}
The initial conditions $y(0)=0$ and $\dot{y}(0)=\nicefrac{25}{16}$ require to find
the derivative
\begin{align}
\dot{y} =
(-\frac{3}{4}+\im) A \, \mathrm{e}^{(-\frac{3}{4}+\im)\,t} +
(-\frac{3}{4}-\im) B \, \mathrm{e}^{(-\frac{3}{4}-\im)\,t}.
\end{align}
%
The ease of performing this derivation makes the Ansatz \eq{eq:Ansatz_caseIIIkomplex}
more elegant than \eq{eq:Ansatz_caseIIIsincos}.
%
However, we now have slightly more effort to find the coefficients $A$ and $B$,
but it is actually more enlightening, how our final result evolves.
%
Applying the initial conditions yields
%
\begin{align}
y(0) = 0 = A + B
\qquad
\dot{y}(0) = \nicefrac{25}{16} =
(-\frac{3}{4}+\im) \, A + (-\frac{3}{4}-\im) \, B
\end{align}
Thus, comparing coefficients and using one Euler identity
$\sin(x) = \frac{\e^{\im x}-\e^{-\im x}}{2\im}$
\begin{align}
A = \frac{\nicefrac{25}{16}}{2\im}\qquad B = -\frac{\nicefrac{25}{16}}{2\im}
\quad\rightarrow\quad
y = y_h + y_p =
 \frac{\nicefrac{25}{16}}{2\im} \, \mathrm{e}^{(-\frac{3}{4}+\im)\,t}
-\frac{\nicefrac{25}{16}}{2\im} \, \mathrm{e}^{(-\frac{3}{4}-\im)\,t},
\end{align}
yields the \textbf{impulse response} of the 2nd order system / ODE under discussion
\begin{align}
\boxed{
h = y_\text{Task a} = \nicefrac{25}{16} \cdot \mathrm{e}^{-\frac{3}{4} t} \sin(t) \qquad t\geq0
}\,.
\end{align}

%##############################################################################
\subsubsection{Task b) with Fundamental System / Step Response}
We shall solve
\begin{align}
\frac{16}{25} \ddot{y} + \frac{24}{25} \dot{y} + y = 1, \quad
\dot{y}(0) = 0,\quad y(0)=0
\end{align}
for $y$ with the Ansatz of fundamental set of solutions.

\paragraph{Inhomogeneous Solution}
The source term $x(t\geq 0+)=1$ exhibits unit
amplitude over time. This can be stated as the most simple polynomial $p_0=1$.
Thus, the polynomial Ansatz $y_p = q_0$ leads to
\begin{align}
\frac{16}{25} \ddot{y}_p + \frac{24}{25} \dot{y}_p + y_p = 1
\rightarrow q_0 = 1 \rightarrow  y_p = 1.
\end{align}

\paragraph{Specific Solution with Initial Conditions}
The superposition of the homogeneous (this timewe use the sin/cos-Ansatz
\eq{eq:Ansatz_caseIIIsincos} for demonstration purpose) and the inhomogeneous
solution is
\begin{align}
\label{eq:SpecificyTaskb}
y = y_h + y_p = \underbrace{
\mathrm{e}^{-\frac{3}{4} t} \cdot
\left[ A \sin(t) + B  \cos(t)\right]}_{y_h} +\underbrace{1}_{y_p}.
\end{align}
The initial condition $\dot{y}(0) = 0$ again requires to find the derivative
\begin{align}
\dot{y}
=
-\frac{3}{4}\mathrm{e}^{-\frac{3}{4} t} \cdot
\left[ A \sin(t) + B \cos(t)\right]
+
\mathrm{e}^{-\frac{3}{4} t} \cdot
\left[ A \cos(t)  - B \sin(t)\right].
\end{align}
The initial condition $\dot{y}(0) = 0$ yields
\begin{align}
0 = -\frac{3}{4}\cdot B + A \rightarrow A = \frac{3}{4}\cdot B.
\end{align}
The initial condition ${y}(0) = 0$ yields
\begin{align}
0 = B  + 1 \rightarrow B = -1.
\end{align}
The resulting coefficients
\begin{align}
A = -\frac{3}{4}\qquad B = -1
\end{align}
are inserted into \eq{eq:SpecificyTaskb}
\begin{align}
\boxed{
h_\epsilon = y_\text{Task b} =
1 - \mathrm{e}^{-\frac{3}{4} t} \cdot
\left(\frac{3}{4} \sin(t) + \cos(t)\right) \qquad t \geq 0
}\,,
\end{align}
giving the final result for task b) with $h_\epsilon(t\to\infty) = 1$.

In Fig.~\ref{fig:step_response_parts} the solution $y_\text{Task b}$ is depicted
as the thick, non-dashed orange graph. It results from the superposition of the
functions
i. unit amplitude (black),
ii. exponentially damped, negative sine (green, diamonds),
iii. exponentially damped, negative cosine (red, stars).
In grey colour, the pure sine and cosine functions with negative amplitude,
as well as the exponential
damping with different initial amplitude are indicated.

\begin{figure}[h!]
\centering
\includegraphics[width=0.75\textwidth]{step_response_parts}
\caption{Response of task b for ODE under discussion.}
\label{fig:step_response_parts}
\end{figure}

Feel free to double check this solution at Wolfram Alpha with input
\begin{verbatim}
16/25*y''(t)+24/25*y'(t)+y(t)=1,       y'(0)=0, y(0)=0
16/25*y''(t)+24/25*y'(t)+y(t)=Step[t], y'(0)=0, y(0)=0
\end{verbatim}

%##############################################################################
\subsubsection{Task a) Revisited with Fundamental System / Impulse Response}
%
The Heaviside step function
\begin{equation}
\epsilon(t) =
\begin{cases}
  0 & t\leq 0-\\
  1 & t\geq 0+
\end{cases}
\end{equation}
and
the Dirac impulse $\delta(t)$ are related by
\begin{align}
\dot{\epsilon}(t) = \delta(t).
\end{align}
In fact, in task a and b we just derived the impulse response $h$ and step response
$h_\epsilon$ of the ODE, respectively.
%
We can also relate the step and impulse response by
\begin{align}
\dot{h_\epsilon}(t) = h(t).
\end{align}
%
The \textbf{temporal derivative of the step response} $h_\epsilon=y_\text{Task b}$
\textbf{yields the impulse response} $h$.
Thus, for the step response
\begin{align}
h_\epsilon = y_\text{Task b} =
1 - \mathrm{e}^{-\frac{3}{4} t} \cdot
\left( \frac{3}{4} \sin(t) + \cos(t)\right)
\end{align}
the derivative is the impulse response
\begin{align}
h = \dot{h_\epsilon} = \dot{y}_\text{Task b} =
 - (-\frac{3}{4})\,\mathrm{e}^{-\frac{3}{4} t} \cdot
\left( \frac{3}{4} \sin(t) + \cos(t)\right)
- \mathrm{e}^{-\frac{3}{4} t} \cdot
\left( \frac{3}{4} \cos(t) - \sin(t)\right).
\end{align}
Rearranging yields the expected result identical to task a
\begin{align}
\boxed{
h = y_\text{Task a} = \frac{25}{16} \mathrm{e}^{-\frac{3}{4} t} \sin(t) \qquad t\geq 0
}\,.
\end{align}
%
In Fig. \ref{fig:impulse_step_response} the impulse response is depicted as
the dashed blue graph, whereas the step response is plotted in thick
orange again.
The impulse response is a weighted and exponentially damped sine function
with $h(t\to\infty) = 0$.

\begin{figure}[h!]
\centering
\includegraphics[width=0.75\textwidth]{impulse_step_response}
\caption{Impulse response (task a, blue, dotted) and step response (task b, orange, thick)
of the ODE under discussion.}
\label{fig:impulse_step_response}
\end{figure}

Feel free to verify that Wolfram Alpha returns the particular solution to the
input
\begin{verbatim}
16/25*y''(t)+24/25*y'(t)+y(t)=DiracDelta[t]
\end{verbatim}
precisely identical to the manually calculated impulse response.

%##############################################################################
\subsubsection{Task c) with Fundamental System}
Now, we shall solve
\begin{align}
\frac{16}{25} \ddot{y} + \frac{24}{25} \dot{y} + y = 0,
\quad \dot{y}(0) = 2,\quad y(0)=1
\end{align}
for $y$.
%
Since $y_p=0$, we can immediately adapt the solution of Sec.
\ref{Sec:TaskaHomo} and \ref{Sec:TaskaInitCond} as
%
\begin{align}
y_h + y_p = y =& A \, \mathrm{e}^{(-\frac{3}{4}+\im)\,t} + B \, \mathrm{e}^{(-\frac{3}{4}-\im)\,t}\\
\dot{y} =&
(-\frac{3}{4}+\im) A \, \mathrm{e}^{(-\frac{3}{4}+\im)\,t} +
(-\frac{3}{4}-\im) B \, \mathrm{e}^{(-\frac{3}{4}-\im)\,t}
\end{align}
%
Applying the initial conditions yields
%
\begin{align}
\dot{y}(0) = 2 =
(-\frac{3}{4}+\im) A+
(-\frac{3}{4}-\im) B
\qquad
y(0) = 1 = A + B.
\end{align}
Thus, the coefficients become
\begin{align}
A = \frac{1}{2} + \frac{11}{8\im}\qquad B = 1-A = \frac{1}{2} - \frac{11}{8\im}.
\end{align}
%
Inserting these and applying the Euler identities
$\cos(x) = \frac{\e^{\im x}+\e^{-\im x}}{2}$,
$\sin(x) = \frac{\e^{\im x}-\e^{-\im x}}{2\im}$
yields
\begin{align}
\boxed{
y_\text{Task c} = \mathrm{e}^{-\frac{3}{4} t} \cdot
\left( \frac{11}{4} \sin(t) + \cos(t)\right) \qquad t\geq 0
}\,.
\end{align}
In Fig.~\ref{fig:initial_conditions_response_parts} the brown graph depicts the
response $y_\text{Task c}$, which is decaying to zero since the L and C are
just discharging. As typical for such ODEs, again
weighted and exponentially damped cosine (red, stars) and sine (green, diamonds)
function are superimposed.

\begin{figure}[h!]
\centering
\includegraphics[width=0.75\textwidth]{initial_conditions_response_parts}
\caption{Response of task c for the ODE with initial conditions but no external input.}
\label{fig:initial_conditions_response_parts}
\end{figure}

Feel free to double check this solution at Wolfram Alpha with input
\begin{verbatim}
16/25*y''(t)+24/25*y'(t)+y(t)=0, y'(0)=2, y(0)=1
\end{verbatim}

%##############################################################################
\subsubsection{Task d) with Fundamental System}
In this task we shall solve
\begin{align}
\frac{16}{25} \ddot{y} + \frac{24}{25} \dot{y} + y = 1, \quad
\dot{y}(0) = 2,\quad y(0)=1
\end{align}
for $y$.
%
This task is the \textbf{superposition} of the solution from \textbf{task b} (exterior source
but vanishing initial conditions) and the solution of \textbf{task c} (no exterior
source, but initial conditions). Thus, we should expect
\begin{align}
&y_\text{Task d} =
y_\text{Task b} +
y_\text{Task c} = 1 + 2\mathrm{e}^{-\frac{3}{4} t} \sin(t) = \nonumber\\
&\left[1 - \mathrm{e}^{-\frac{3}{4} t} \cdot
\left( \frac{3}{4} \sin(t) + \cos(t)\right)\right]
+\left[\mathrm{e}^{-\frac{3}{4} t} \cdot
\left( \frac{11}{4} \sin(t) + \cos(t)\right)\right].
\end{align}
%
In order to reassure this result, we can start with \eq{eq:SpecificyTaskb} from
task b and rearrange this for the differing initial conditions:
The superposition of the homogeneous and inhomogeneous solution is
\begin{align}
y = y_h + y_p = \underbrace{
\mathrm{e}^{-\frac{3}{4} t} \cdot
\left( A \sin(t) + B \cos(t)\right)}_{y_h} + \underbrace{1}_{y_p}
\end{align}
Again, derivation with respect to time is
\begin{align}
\dot{y}
=
-\frac{3}{4}\,\mathrm{e}^{-\frac{3}{4} t} \cdot
\left( A \sin(t) + B \cos(t)\right)
+
\mathrm{e}^{-\frac{3}{4} t} \cdot
\left( A \cos(t)  - B \sin(t)\right).
\end{align}
The initial condition $\dot{y}(0) = 2$ yields
\begin{align}
2 = -\frac{3}{4}\cdot B + A \rightarrow A = 2 +  \frac{3}{4}\cdot B.
\end{align}
The initial condition ${y}(0) = 1$ yields
\begin{align}
1 = B  + 1 \rightarrow B = 0.
\end{align}
Thus,
\begin{align}
A = 2\qquad B = 0
\end{align}
inserted
\begin{align}
\boxed{
y_\text{Task d} =
1+\mathrm{e}^{-\frac{3}{4} t}
\, 2 \sin(t)\qquad t \geq 0
}
\end{align}
leads to the final result for task d) as expected.
%
\begin{figure}[b!]
\centering
\includegraphics[width=0.75\textwidth]{response_full}
\caption{Response of task d for the ODE with initial conditions and step input.}
\label{fig:response_full}
\end{figure}
%
In Fig.~\ref{fig:response_full} the response $y_\text{Task d} $ is depicted as
magenta graph. This is the result of the superposition of the Heaviside step
function and an exponentially damped, weighted sine function.
The final result asymptotically converges to unit amplitude.
%
This is expected: once the initial conditions are vanished for $t\to\infty$
the external excitation of the step function is the dominant part for the system
response.
%
We can see this in the equations: $y_\text{Task d}(t\to\infty) = y_\text{Task b}(t\to\infty)$

Feel free to double check this solution at Wolfram Alpha with input
\begin{verbatim}
16/25*y''(t)+24/25*y'(t)+y(t)=1, y'(0)=2, y(0)=1
\end{verbatim}

%##############################################################################
\subsubsection{Task e) with Fundamental System}
In this task we shall solve
\begin{align}
\label{eq:InHomoODE_sin}
\frac{16}{25} \ddot{y} + \frac{24}{25} \dot{y} + y = \sin(t), \quad
\dot{y}(0) = 0,\quad y(0)=0
\end{align}
for $y$.
%
We already know the homogeneous solution
$y_h = \mathrm{e}^{-\frac{3}{4} t} \cdot
\left[ A \sin(t) + B \cos(t)\right]$.
%
The inhomogeneity in this case can be solved with the Ansatz (cf.
Sec. \ref{Sec:CosSineAnsatzInhomo})
\begin{align}
y_p = C \sin(t) + D \cos(t).
\end{align}
Inserting this into \eq{eq:InHomoODE_sin}, the
coefficients are solved to
\begin{align}
C = \frac{25}{73}\qquad D = -\frac{200}{219}.
\end{align}
%
The solution then becomes
\begin{align}
y = y_h + y_p = \underbrace{\mathrm{e}^{-\frac{3}{4} t} \cdot
\left[ A \sin(t) + B \cos(t)\right]}_{\text{damped solution} \, y_h}+
\underbrace{\frac{25}{73} \sin(t) - \frac{200}{219} \cos(t)}_{\text{oscillating solution} \, y_p}.
\end{align}
By considering the initial conditions the remaining unknown coefficients solve to
\begin{align}
A = \frac{25}{73}\qquad B = \frac{200}{219}
\end{align}
giving the final result
\begin{align}
\boxed{
y_\text{Task e} = \frac{25}{73} \mathrm{e}^{-\frac{3}{4} t} \sin(t) +
\frac{200}{219} \mathrm{e}^{-\frac{3}{4} t} \cos(t) +
\frac{25}{73} \sin(t) -
\frac{200}{219} \cos(t) \qquad t\geq 0
}\, ,
\end{align}
%
\begin{figure}[h!]
\centering
\includegraphics[width=0.75\textwidth]{sine_excitation_response}
\caption{Response of task e for the ODE with zero initial conditions and
$\sin(t)$ input.}
\label{fig:sine_excitation_response}
\end{figure}

The response is depicted in Fig.~\ref{fig:sine_excitation_response} in
thick/blue.
It results from the superposition of i. weighted and \textbf{exponentially damped}
sine and cosine functions (yellow and green graphs) and ii. weighted but \textbf{undamped}
sine and cosine functions (brown and purple graphs).

\subsubsection{Steady State of the System}
%
In this example it is important to realise that the initial conditions
(exponentially damped sine and cosine) contribute
to the system response only up to approximately $t=2\pi$
seconds, i.e. in the first period $T=\frac{2\pi}{\omega_D}=2\pi$ of the
excitation's frequency here.
%
For times $t\gg2\pi$ seconds the system response is determined by the
undamped sine and cosine functions, yielding a system response of
\begin{align}
\label{eq:steadystate_timedomain}
y(t\gg 2\pi, t\to\infty) = \frac{25}{3 \sqrt{73}}\cdot
\sin\left(t-\tan^{-1}
\left[\frac{\nicefrac{200}{219}}{\nicefrac{25}{73}}\right]\right),
\end{align}
depicted as the red dotted graph in Fig.~\ref{fig:sine_excitation_response}.
This constitutes the so called \textbf{steady-state} of the system response
for $t\to\infty$.
For that, we see that the excitation signal $x=\sin(t)$ is altered in amplitude
(very subtle damping of about 0.975) and
a time delay (in terms of phase: about -70 degrees or about $-0.386 \pi$ in
radian), cf. the black line indicator in the figure.
Note that $\frac{\nicefrac{200}{219}}{\nicefrac{25}{73}} = \nicefrac{8}{3}$ again
results in simple numbers for the chosen example. We will rather not see such
for real practical problems.

The two parameters in the
\textbf{steady state}---\textbf{the amplitude and the phase offset}---and
their specific quantities are very important for the
interpretation of ODEs as LTI systems. Each excitation frequency has its own
quantity set, yielding frequency dependent system characteristics.
We will deal with it in detail when discussing ODEs in Laplace domain.
%
The so called Bode plot or \textbf{Bode diagram} is one still important tool
to visualize amplitude and phase over frequency in approximated manner.

Feel free to double check this solution at Wolfram Alpha with input
\begin{verbatim}
16/25*y''(t)+24/25*y'(t)+y(t)=Sin[t], y'(0)=0, y(0)=0
\end{verbatim}

%##############################################################################
\subsection{Preliminaries for Laplace Transform}
We should now study the problem with the help of the Laplace transform.
For the chosen ODE and input signals the calculus effort is about the same,
however for more complicated cases, the approach via Laplace transform
is often the more convenient approach for the solution.
%
For instance, Laplace transform is more powerful, when the \textbf{steady state}
system response for oscillations \textbf{at many different frequencies} is
searched for.
%
Instead of solving each individual problem with the approach that we took before,
the problem is conveniently solved by means of the \textbf{transfer function},
which stores amplitude and phase offset for each arbitrary oscillation frequency.
%
We will discover this, when solving tasks a) to e) with the help of the Laplace
transform. The solutions must be of course identical to these we have found so far.

To obtain the time function the inverse Laplace function is needed.
%
We do not need to use the complex integral and application of the
\textbf{residue theorem}
here, but rather make use of \textbf{correspondence tables for Laplace transforms}.
%
The English and German textbooks
\cite{Strang2014,GirodRabensteinStenger2007,Girod2001,Oppenheim1997,Fliege1991,
Goeldner1987,LangeSigSys1,Wunsch1972}
can be considered classics and cover the Laplace transform very well for our
purpose.

%##############################################################################
\subsubsection{Laplace Transform Fundamentals}
The Laplace transform pair reads
\begin{align}
Y(s) = \mathcal{L}\{y(t)\}= \int\limits_{-\infty}^{\infty} y(t) \e^{-s t}
\mathrm{d} t\qquad
y(t) = \mathcal{L}^{-1}\{Y(s)\}= \frac{1}{2\pi\im}\int
\limits_{\sigma-\im \infty}^{\sigma+\im\infty} Y(s) \e^{+s t} \mathrm{d} s.
\end{align}
We use the operators
$y(t) \quad \laplace \quad Y(s)$ and
$Y(s) = \mathcal{L}\{y(t)\}$
to indicate forward Laplace transform.
We use the operators
$Y(s) \quad \Laplace \quad y(t)$ and
$y(t) = \mathcal{L}^{-1}\{Y(s)\}$
to indicate inverse Laplace transform.

To make the Laplace transform helpful for solving ODEs, we first reconsider
that the Laplace transform is linear, i.e. it preserves scaling and
superposition in both signal domains
\begin{align}
A y_1(t) + B y_2(t) \quad \laplace \quad A Y_1(s) + B Y_2(s).
\end{align}
The second, fundamental important characteristics of the Laplace transform is
\begin{align}
\mathcal{L}\{\frac{\mathrm{d}^n y(t)}{\mathrm{d} t^n}\}=
s^n Y(s) - \sum_{k=1}^{n} s^{n-k} \cdot
\frac{\mathrm{d}^{k-1} y(t)}{\mathrm{d} t^{k-1}}\bigg|_{t=0},
\end{align}
The temporal derivation operation will be transformed to a multiplication
operation with Laplace variable $s$ (\textbf{differential equations} essentially \textbf{become
algebraic equations}, that is the key idea and a special case of
so called operator theory, we have seen a similar concept already for the
characteristic polynomial).
%
For typical systems up to 2nd order, we will need
\begin{align}
\label{eq:Laplace0thd}
\boxed{\mathcal{L}\{{y(t)}\} = Y(s)},
\end{align}
\begin{align}
\label{eq:Laplace1std}
\mathcal{L}\{\frac{\mathrm{d} y(t)}{\mathrm{d} t}\}=
s \cdot Y(s) - \sum_{k=1}^{1} s^{1-k} \cdot
\frac{\mathrm{d}^{k-1} y(t)}{\mathrm{d} t^{k-1}}\bigg|_{t=0}
= \boxed{\mathcal{L}\{\dot{y}\} = s \cdot Y(s) - y(0)}
\end{align}
and
\begin{align}
\label{eq:Laplace2ndd}
\mathcal{L}\{\frac{\mathrm{d}^2 y(t)}{\mathrm{d} t^2}\}=
s^2 \cdot Y(s) - \sum_{k=1}^{2} s^{2-k} \cdot
\frac{\mathrm{d}^{k-1} y(t)}{\mathrm{d} t^{k-1}}\bigg|_{t=0}
= \boxed{\mathcal{L}\{\ddot{y}\} = s^2 \cdot Y(s) - s \cdot y(0) - \dot{y}(0)}\,.
\end{align}

Note the \textbf{application of the initial conditions} in these transform rules!

%##############################################################################
\subsubsection{Algorithm for Solving ODEs via Laplace Transform}

Solving an ODE problem via the indirect way of Laplace transform is then as follows
\begin{itemize}
\item[1.] transform the ODE equation to the Laplace domain by applying linearity and
derivative rules
\item[2.] insert the initial conditions of $y(0)$  and $\dot{y}(0)$
(if they are zero, actually the additional terms immediately become zero,
this is very nice)
\item[3.] find the Laplace transform $X(s)$ of inhomogeneity $x(t)$ and insert
it (note that we are most often interested in $x(t)=\delta(t)$,
$x(t)=\epsilon(t)$, $x(t)=\exp(s_1 t)$ with $s_1\in\mathbb{C}$
as they tell us the most interesting things of the ODE)
\item[4.] reorganize the transformed equation for $Y(s)$
\item[5.] perform \textbf{inverse transformation} $y(t) = \mathcal{L}^{-1}\{Y(s)\}$
by means of either \textbf{partial fraction decomposition} and usage of
\textbf{transform tables}
or if this fails by means of the residue theorem...if this even fails we have a
very delicate problem at hand.
\footnote{For many systems we deal with in engineering,
someone very likely calculated the inverse Laplace transform already and we
might find it in formularies, list of integrals and textbooks.
So before trying to solve the integral of the inverse Laplace transform
manually, try to find the tabulated solution.
Though, very often the solution must be tweaked to fit into the tabulated
transforms.
Anyway, deriving solutions with residue theorem by our own is a
good exercise and enlightening.}
\end{itemize}

We aim at \textbf{right-sided}, and more specifically \textbf{causal systems and signals}.
Thus the region of convergence (ROC) is placed right of the most right pole of a
Laplace transform's function.
%
A stable LTI system requires poles only left of the $\Im(s)$-axis, i.e. only in
the left half of the $s$-plane.

%##############################################################################
\subsubsection{Laplace Transform of the 2nd Order Linear ODE with Constant Coefficients}
The ODE under discussion is still \eq{eq:ODE_sigma0}
\begin{align}
\frac{\ddot{y}}{\omega_0^2} + \frac{2 \sigma_0}{\omega_0^2} \dot{y} + y = x
\end{align}
for which task a) to e) are to be solved by means of Laplace transform.
The general Ansatz valid for all tasks is sketched here.
%
First, considering linearity (scaling, superposition) yields
\begin{align}
\frac{1}{\omega_0^2} \ddot{y} +
\frac{2 \sigma_0}{\omega_0^2} \dot{y} + y = x
\quad \laplace \quad
\frac{1}{\omega_0^2} \mathcal{L}\{\ddot{y}\} +
\frac{2 \sigma_0}{\omega_0^2} \mathcal{L}\{\dot{y}\} + \mathcal{L}\{y\} =
\mathcal{L}\{x\}.
\end{align}
%
Second, the temporal derivative rules \eq{eq:Laplace0thd}, \eq{eq:Laplace1std}
and \eq{eq:Laplace2ndd} are applied to functions $y$ and $x$ yielding
\begin{align}
\frac{1}{\omega_0^2} \left[ s^2 \cdot Y(s) - s \cdot y(0) - \dot{y}(0)\right]  +
\frac{2 \sigma_0}{\omega_0^2} \left[ s \cdot Y(s) - y(0) \right] + Y(s) = X(s).
\end{align}
%
Rearranging for $Y(s)$ yields (we must savely distinguish between $Y$, $y$ and $\dot{y}$
in that equation)
\begin{align}
Y(s) = \frac{X(s)}{\frac{1}{\omega_0^2} s^2 +
\frac{2 \sigma_0}{\omega_0^2} s + 1}
+ \frac{\frac{1}{\omega_0^2} s \cdot y(0) + \frac{2 \sigma_0}{\omega_0^2} \cdot y(0) +
\frac{1}{\omega_0^2} \cdot \dot{y}(0)}{\frac{1}
{\omega_0^2} s^2 + \frac{2 \sigma_0}{\omega_0^2} s + 1}.
\end{align}
%
It is now important to realise (and shows the links in a beautiful way)
that the \textbf{characteristic equation} (\ref{eq:CharEq})
appears as the \textbf{denominator in the Laplace function}, we only have to change
variable $\lambda\rightarrow s$.
Therefore, discussions of the characteristic equation in terms of its zeros become
discussions in terms of poles in the Laplace domain,
cf. Fig.~\ref{fig:sketch_lambda_plane} left vs. right.

\begin{figure}[h!]
\captionsetup[subfigure]{font=footnotesize}
\centering
\subcaptionbox{\textbf{Zeros} in complex $\lambda$-plane of ODE's characteristic
equation \eq{eq:CharEq}.}[.4\textwidth]{%
\begin{tikzpicture}
\draw [-latex] (-2,0) -- (3.5,0) node [right]  {$\Re(\lambda)$};
\draw [-latex] (0,-2) -- (0,2) node [above] {$\Im(\lambda)$};
\draw[dashed] (0,0) -- node[pos=0.7, right] {$\omega_0$}(135:2) node[solid, fill=white, circle, draw=blue, ultra thick] {};
\draw[dashed] (0,0) -- node[pos=0.6, above right] {$ $}(-135:2) node[solid, fill=white, circle, draw=blue, ultra thick] {};
\draw[dashed] (-1.4142,0) -- (-1.4142,+1.4142);
\draw[dashed] (-1.4142,-1.4142) -- (-1.4142,-0.5) node [left, above] {$-\sigma_0$};
\draw[dashed] (-1.4142,+1.4142) -- (0,+1.4142) node [right] {$+\omega_D$};
\draw[dashed] (-1.4142,-1.4142) -- (0,-1.4142) node [right] {$-\omega_D$};
\draw[black, -latex] (-0.75,0.75) arc (135:180:1);
\node at (1.9,1) {$\omega_D = \omega_0 \sqrt{1-D^2}$};
\node at (1.3,0.5) {$\sigma_0 = D \omega_0$};
\node at (-0.7,0.25) {$\alpha$};
\node at (1.4,-0.25) {$\cos \alpha = D$};
\end{tikzpicture}}
\subcaptionbox{\textbf{Poles} in complex $s$-plane of LTI system's transfer function, cf. Sec. \ref{sec:TransferFunction}.}[.4\textwidth]{%
\begin{tikzpicture}
\draw [-latex] (-2,0) -- (3.5,0) node [right]  {$\Re(s)$};
\draw [-latex] (0,-2) -- (0,2) node [above] {$\Im(s)$};
\draw[dashed] (0,0) -- node[pos=0.7, right] {$\omega_0$}(135:2) node[solid, fill=white, cross=6pt, draw=blue, ultra thick] {};
\draw[dashed] (0,0) -- node[pos=0.6, above right] {$ $}(-135:2) node[solid, fill=white, cross=6pt, draw=blue, ultra thick] {};
\draw[dashed] (-1.4142,0) -- (-1.4142,+1.4142);
\draw[dashed] (-1.4142,-1.4142) -- (-1.4142,-0.5) node [left, above] {$-\sigma_0$};
\draw[dashed] (-1.4142,+1.4142) -- (0,+1.4142) node [right] {$+\omega_D$};
\draw[dashed] (-1.4142,-1.4142) -- (0,-1.4142) node [right] {$-\omega_D$};
\draw[black, -latex] (-0.75,0.75) arc (135:180:1);
\node at (1.9,1) {$\omega_D = \omega_0 \sqrt{1-D^2}$};
\node at (1.3,0.5) {$\sigma_0 = D \omega_0$};
\node at (-0.7,0.25) {$\alpha$};
\node at (1.4,-0.25) {$\cos \alpha = D$};
\end{tikzpicture}}
\caption{Sketch of ODE's homogeneous solution case III with a complex conjugate
solution.}
\label{fig:sketch_lambda_plane}
\end{figure}

%##############################################################################
\subsubsection{Specific Parameters}
For our chosen quantities
$\omega_0=\frac{5}{4}$,
$\sigma_0 = \frac{3}{4}$,
$\omega_D=1$
the ODE reads
\begin{align}
\frac{16}{25} \ddot{y} + \frac{24}{25} \dot{y} + y = x.
\end{align}
Some tasks in this exercise require the initial conditions
$\dot{y}(0)=0$ and $y(0)=0$.
Then the Laplace transform becomes
\begin{align}
\boxed{
Y(s) = \frac{X(s)}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}}\,.
\end{align}
Other tasks require initial conditions $\dot{y}(0)=2$ and $y(0)=1$.
Then the Laplace transform reads
\begin{align}
\boxed{
Y(s) = \frac{X(s)}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}+
\frac{\frac{16}{25} s + \frac{56}{25}}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}}\,.
\end{align}

We require the following input signals for this exercise
\begin{itemize}
  \item $x(t) = \delta(t) \quad \laplace \quad X(s) = 1$
  \quad ROC: $s \in \mathbb{C}$
  \item $x(t) = \epsilon(t) \quad \laplace \quad X(s) = \frac{1}{s}$
  \quad ROC: $\Re(s)>0$
  \item $x(t) =  \sin(\omega_D t) \epsilon(t) \quad \laplace \quad
   X(s) = \frac{\omega_D}{s^2 + \omega_D^2}$
   \quad ROC: $\Re(s)>0$ with $\omega_D=1$
\end{itemize}

%##############################################################################
%##############################################################################
\subsection{Solutions Using Laplace Transform}

%##############################################################################
\subsubsection{Task a) with Laplace Transform / Impulse Response}
\label{sec:TransferFunction}
We shall find the inverse Laplace transform
$y(t) = \mathcal{L}^{-1}\{Y(s)\}$ for
\begin{align}
Y(s) = \frac{1}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}
\quad \text{ROC}: \Re(s) > -\frac{3}{4}.
\end{align}
Since the \textbf{Dirac Delta impulse} is the \textbf{input signal} here,
$Y(s)$ is here and only here by definition equivalent with the
\textbf{transfer function} $H(s)$, for which the \textbf{fundamental theorem}
\begin{align}
\boxed{y(t) = x(t)*h(t) \quad \laplace \quad Y(s) = X(s) \cdot H(s)}
\end{align}
connects time and Laplace domain.
%
The inverse Laplace transform can be conveniently solved by carefully
rearranging
$Y(s)$ to
\begin{align}
H(s) = Y(s) = \frac{\frac{25}{16}}{s^2 + \frac{24}{16} s + \frac{25}{16}}=
\frac{25}{16} \cdot \frac{1}{(s + \frac{3}{4})^2 + 1^2}
\end{align}
corresponding to a damped sine function (cf. Appendix) as the LTI system's
impulse response
\begin{align}
\boxed{h(t) = y_\text{Task a}(t) = \frac{25}{16} \e^{-\frac{3}{4} t} \sin(t)
\cdot \epsilon(t)}\,.
\end{align}

Feel free to double check this solution at Wolfram Alpha with input
\begin{verbatim}
InverseLaplaceTransform[1/(16/25*s^2+24/25*s+1)]
\end{verbatim}

We should make ourselves comfortable with the Laplace transform pairs of sine, cosine
functions and its exponentially damped versions in the Appendix.
These will often appear as solutions for ODEs under discussion.

%##############################################################################
\subsubsection{Task b) with Laplace Transform / Step Response}
We shall find the inverse Laplace transform $y(t) = \mathcal{L}^{-1}\{Y(s)\}$
for
\begin{align}
Y(s) = \frac{\frac{1}{s}}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}
\quad \text{ROC}: \Re(s) > 0.
\end{align}
%
We solve this problem by means of partial fraction decomposition.
We have a single pole in the origin $s_{\infty,1}=0$ and a complex conjugate
pole pair $s_{\infty,2,3}=-\frac{3}{4}\pm \im$.
Thus, the Ansatz is
\begin{align}
Y(s) = \frac{1}{s}
\cdot \frac{1}{\frac{16}{25} s^2 + \frac{24}{25} s + 1} =
\frac{A}{s-s_{\infty,1}} + \frac{Bs+C}{(s-s_{\infty,2})(s-s_{\infty,3})}.
\end{align}
Rearranging yields
\begin{align}
1 =
A \cdot (\frac{16}{25} s^2 + \frac{24}{25} s + 1) +
(B s + C) \cdot s.
\end{align}
Comparison of coefficients for $s^2, s^1, s^0$ yields
\begin{align}
  A = 1\quad B = -\frac{16}{25} \quad C = -\frac{24}{25}.
\end{align}
Inserting these into the Ansatz leads to
\begin{align}
Y(s) =
\frac{1}{s} - \left[\frac{\frac{16}{25} s + \frac{24}{25}}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}\right]=
\frac{1}{s} - \left[\frac{s + \frac{3}{2}}{s^2 + \frac{3}{2} s + \frac{25}{16}}\right]=
\frac{1}{s} - \left[\frac{s + \frac{3}{2}}{(s + \frac{3}{4})^2 + 1^2}\right].
\end{align}
Now, the denominator of the second fraction looks well for sine or cosine related
Laplace transforms.
Another rearranging step (this is the part, where some experience
is helpful to bring fractions to a suitable form)
\begin{align}
Y(s) =
\frac{1}{s} - \left[\frac{s + \frac{3}{4}}{(s + \frac{3}{4})^2 + 1^2} +
\frac{\frac{3}{4}}{(s+\frac{3}{4})^2 + 1^2}\right]
\end{align}
splits the term in the bracket.
Individual inverse Laplace transform of each fraction is allowed due to
linearity. Thus, we get
\begin{align}
\boxed{y(t)_\text{Task b} =
\left[1
- \e^{-\frac{3}{4} t} \cos(t)
- \frac{3}{4}\e^{-\frac{3}{4} t} \sin(t) \right] \cdot \epsilon(t)}\,.
\end{align}

Feel free to double check this solution at Wolfram Alpha with input
\begin{verbatim}
InverseLaplaceTransform[(1/s)/(16/25*s^2+24/25*s+1)]
\end{verbatim}

%##############################################################################
\subsubsection{Task c) with Laplace Transform}
We shall find the inverse Laplace transform $y(t) = \mathcal{L}^{-1}\{Y(s)\}$
for
\begin{align}
Y(s) =
\frac{\frac{16}{25} s + \frac{56}{25}}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}
\quad \text{ROC}: \Re(s) > -\frac{3}{4}.
\end{align}
This particular problem becomes solvable by rearranging the fraction to
\begin{align}
Y(s) =
\frac{s + \frac{7}{2}}{s^2 + \frac{3}{2} s + \frac{25}{16}} =
\frac{s + \frac{7}{2}}{(s + \frac{3}{4})^2 + 1^2}
=\frac{s + \frac{3}{4}}{(s + \frac{3}{4})^2 + 1^2}
+\frac{\frac{11}{4}}{(s + \frac{3}{4})^2 + 1^2}.
\end{align}
This results in exponentially damped cosine and sine functions
\begin{align}
\boxed{y(t)_\text{Task c} =
\left[\e^{-\frac{3}{4} t} \cos(t)
+\frac{11}{4}\e^{-\frac{3}{4} t} \sin(t) \right] \cdot \epsilon(t)}\,.
\end{align}

Feel free to double check this solution at Wolfram Alpha with input
\begin{verbatim}
InverseLaplaceTransform[(16/25*s+56/25)/(16/25*s^2+24/25*s+1)]
\end{verbatim}

%##############################################################################
\subsubsection{Task d) with Laplace Transform}
We shall find the inverse Laplace transform $y(t) = \mathcal{L}^{-1}\{Y(s)\}$
for
\begin{align}
Y(s) = \frac{\frac{1}{s}}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}+
\frac{\frac{16}{25} s + \frac{56}{25}}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}
\quad \text{ROC}: \Re(s) > 0.
\end{align}
Straightforward superposition is just to be done here:
We already have calculated the individual inverse Laplace transforms in task b
and c. So the solution becomes
\begin{align}
\boxed{y(t)_\text{Task d} = y(t)_\text{Task b} + y(t)_\text{Task c}
= \left[1+\mathrm{e}^{-\frac{3}{4} t} \, 2 \sin(t) \right] \cdot \epsilon(t)
}\,.
\end{align}
We have observed this characteristics in the time domain as well.

Feel free to double check this solution at Wolfram Alpha with input
\begin{verbatim}
InverseLaplaceTransform[(1/s+16/25*s+56/25)/(16/25*s^2+24/25*s+1)]
\end{verbatim}

%##############################################################################
\subsubsection{Task e) with Laplace Transform}
We shall find the inverse Laplace transform $y(t) = \mathcal{L}^{-1}\{Y(s)\}$
for
\begin{align}
Y(s) = \frac{\frac{1}{s^2 + 1^2}}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}
\quad \text{ROC}: \Re(s) > 0.
\end{align}
Here we deal with two complex conjugate pole pairs (the first one from system
characteristic, the second from the $sin(t)$-input signal)
\begin{align}
s_{\infty,1,2} = -\frac{3}{4} \pm \im \qquad s_{\infty,3,4} = \pm \im.
\end{align}
We again try to solve with partial fraction decomposition, here with the Ansatz
\begin{align}
Y(s) = \frac{A s + B}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}+
\frac{C s + D}{s^2+1}.
\end{align}
Hence,
\begin{align}
1=
(A s + B) \cdot (s^2+1)+
(C s + D) \cdot (\frac{16}{25} s^2 + \frac{24}{25} s + 1).
\end{align}
Manual comparison of coefficients becomes more and more inconvenient with growing
numbers of coefficients.
Although, in this example a matrix inversion is more calculus effort than plain
comparison of coefficients, we should keep in mind that we can set up a system
of linear equations in matrix notation
%wolfram alpha:
%inv[[[0, 1, 0, 1], [10/13, 10/13, 1, 1], [250/137, 125/137, 274/137, 1], [750/241, 250/241, 723/241, 1]]]*[1, 5/13, 25/137, 25/241]
% \begin{align}
% \underbrace{
% \begin{pmatrix}
% 0 & 1 & 0 & 1\\
% \nicefrac{10}{13} & \nicefrac{10}{13} & 1 & 1\\
% \nicefrac{250}{137} & \nicefrac{125}{137} & \nicefrac{274}{137} & 1\\
% \nicefrac{750}{241} & \nicefrac{250}{241} & \nicefrac{723}{241} & 1
% \end{pmatrix}
% }_{\mathbf{M}}
% \cdot
% \underbrace{
% \begin{pmatrix}
% A \\
% B\\
% C\\
% D
% \end{pmatrix}
% }_{\mathbf{u}}
% =
% \underbrace{
% \begin{pmatrix}
% 1\\
% \nicefrac{5}{13}\\
% \nicefrac{25}{137}\\
% \nicefrac{25}{241}
% \end{pmatrix}
% }_{\mathbf{v}}
% \end{align}
% when inserting $s=0,1,2,3$ (included in the ROC) to obtain four equations.
%
%
%
%inv[[[1, 0, 16/25, 0], [0, 1, 24/25, 16/25], [1, 0, 1, 24/25], [0, 1, 0, 1]]]*[0, 0, 0, 1]
\begin{align}
\underbrace{
\begin{pmatrix}
1 & 0 & \nicefrac{16}{25} & 0\\
0 & 1 & \nicefrac{24}{25} & \nicefrac{16}{25}\\
1 & 0 & 1 & \nicefrac{24}{25}\\
0 & 1 & 0 & 1
\end{pmatrix}
}_{\mathbf{M}}
\cdot
\underbrace{
\begin{pmatrix}
A \\
B\\
C\\
D
\end{pmatrix}
}_{\mathbf{u}}
=
\underbrace{
\begin{pmatrix}
0\\
0\\
0\\
1
\end{pmatrix}
}_{\mathbf{v}}
.
\end{align}
%
The matrix $\mathbf{M}$ is full rank (if we do not believe this, we should assure
us. However, it is worth to note that the system response is unique, thus the defined
linear system of equations must be also unique,
requiring that square $\mathbf{M}$ is invertible, which in fact it is).
%
Thus, solving $\mathbf{u} = \mathbf{M}^{-1} \cdot \mathbf{v}$ (in later practice
we might leave this job for a computer, such as e.g. input to Wolfram Alpha
\begin{verbatim}
inv[[[1, 0, 16/25, 0], [0, 1, 24/25, 16/25], [1, 0, 1, 24/25], [0, 1, 0, 1]]]*[0, 0, 0, 1]
\end{verbatim}
) yields the coefficients
\begin{align}
\label{eq:coeffABCDsinLaplace}
  A = \frac{128}{219}
  \quad B = \frac{48}{73}
  \quad C = -\frac{200}{219}
  \quad D = \frac{25}{73}.
\end{align}
%
%
Inserting these into the Ansatz results in
\begin{align}
Y(s) = \frac{\frac{128}{219} s +
\frac{48}{73}}{\frac{16}{25} s^2 +
\frac{24}{25} s + 1} +
\frac{-\frac{200}{219} s +
\frac{25}{73}}{s^2+1}.
\end{align}
This again needs to be reformulated such that \textbf{Laplace transforms of
exponentially damped and undamped sine/cosine functions}
can be revealed (this is a typical exam task).
%
The last fraction is straightforward with regard to that:
\begin{align}
Y(s) = \frac{\frac{128}{219} s + \frac{48}{73}}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}
-\frac{200}{219}\frac{s}{s^2+1^2}
+\frac{25}{73}\frac{1}{s^2+1^2}.
\end{align}
%
The first fraction is similar to the approaches of tasks a) - d) and can be
rearranged as
\begin{align}
Y(s) = &
\frac{\frac{200}{219} s + \frac{75}{73}}{s^2 + \frac{3}{2} s + \frac{25}{16}}
-\frac{200}{219}\frac{s}{s^2+1^2}
+\frac{25}{73}\frac{1}{s^2+1^2}\\
&
\frac{\frac{200}{219} s + \frac{75}{73}}{(s + \frac{3}{4})^2 + 1^2}
-\frac{200}{219}\frac{s}{s^2+1^2}
+\frac{25}{73}\frac{1}{s^2+1^2}\\
&
\frac{200}{219} \frac{s + \frac{3}{4}}{(s + \frac{3}{4})^2 + 1^2}+
\frac{\frac{25}{73}}{(s + \frac{3}{4})^2 + 1^2}
-\frac{200}{219}\frac{s}{s^2+1^2}
+\frac{25}{73}\frac{1}{s^2+1^2}.
\end{align}
This term now can be conveniently transformed back into time domain,
fraction by fraction due to the linearity of the Laplace transform
\begin{align}
\boxed{
  y_\text{Task e}(t) =
  \left[ \frac{200}{219} \e^{-\frac{3}{4} t} \cos(t) +
  \frac{25}{73} \e^{-\frac{3}{4} t} \sin(t) -
  \frac{200}{219} \cos(t) +
  \frac{25}{73} \sin(t) \right] \cdot \epsilon(t)}\,.
\end{align}

Feel free to double check this solution at Wolfram Alpha with input
\begin{verbatim}
InverseLaplaceTransform[(1/(s^2+1^2))/(16/25*s^2+24/25*s+1)]
\end{verbatim}

%##############################################################################
\subsubsection{Transfer Function and Frequency Response of the System}
%
With \eq{eq:steadystate_timedomain}
\begin{align}
y(t\gg 2\pi, t\to\infty) = \frac{25}{3 \sqrt{73}}\cdot
\sin\left(t-\tan^{-1}
\left[\frac{\nicefrac{200}{219}}{\nicefrac{25}{73}}\right]\right),
\end{align}
we explored the \textbf{steady state} of the system giving
amplitude and phase offset for $\omega=1$.
%
A convenient way to calculate the steady state for many (all) $\omega$, is to evaluate
the \textbf{Laplace transform}, i.e. \textbf{the transfer function} of the system,
see taks a)
\begin{align}
H(s) = \frac{1}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}
\end{align}
along the
$\im\omega$-axis of the $s$-domain
\begin{align}
H(\im\omega)
= \frac{1}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}\bigg|_{s=\im\omega}
= \frac{1}{\frac{16}{25} (\im\omega)^2 + \frac{24}{25} \im\omega + 1}
= \frac{1}{ (1 -\frac{16}{25} \omega^2) + \im (\frac{24}{25} \omega)}.
\end{align}
This constitutes the \textbf{Fourier transform} $\mathcal{F}\{h(t)\}$
of the impulse response $h(t)$ and is known as
\textbf{frequency response} $H(\im\omega)$ of the system.
%
Typically, $H(\im\omega)$ is complex-valued (only in rare, special cases
it is real-valued), thus it is meaningful to specify the magnitude and the phase
of the complex number $H(\im\omega)$ at each $\omega$.
%
This is depicted in Fig. \ref{fig:frequency_response_mag_phase}: on top magnitude
$|H(\im\omega)|$ over angular frequency $\omega$, at the bottom plot normalized
phase $\angle H(\im\omega) / \pi$ over $\omega$.
%
%We see in the magnitude plot that, lower frequencies ($\omega<1$) can pass,
%while higher frequencies $\omega>3$ are attenuated.
%
%The system characteristics is therefore called \textbf{low-pass} filter or
%\textbf{high-cut} filter.

If we set up $\omega=1$ in the frequency response
\begin{align}
H(\im(\omega=1))
= \frac{1}{ (1 -\frac{16}{25} (1)^2) + \im (\frac{24}{25} (1))}
= \frac{1}{ \frac{9}{25} + \im \frac{24}{25}}
= \frac{\frac{9}{25} - \im \frac{24}{25}}{ \frac{81}{625} + \frac{576}{625}}
\end{align}
the complex number $H(\im(\omega=1)) = \frac{25}{73} - \im \frac{200}{219}$ results.
%
We are familiar with these numbers as they appeared somehow in task e), cf.
$C$ and $D$ in \eq{eq:coeffABCDsinLaplace}.
%
The resulting magnitude and phase
\begin{equation}
|H(\im(\omega=1))| = \frac{25}{3 \sqrt{73}}
\qquad
\angle H(\im(\omega=1)) = -\tan^{-1}
\left[\frac{\nicefrac{200}{219}}{\nicefrac{25}{73}}\right]
= -\tan^{-1}\left[\frac{8}{3}\right]
\end{equation}
are precisely identical with \eq{eq:steadystate_timedomain},
proving that the shown concepts are consistent.
%
Thus, evaluating the frequency response give us information about the steady
state of a system.
%
\begin{figure}[h!]
\centering
\includegraphics[width=0.75\textwidth]{frequency_response_mag_phase}
\caption{Frequency response for the ODE with Laplace function
$H(s) = \left[ \frac{16}{25} s^2 + \frac{24}{25} s + 1 \right]^{-1}$.}
\label{fig:frequency_response_mag_phase}
\end{figure}

%##############################################################################
%##############################################################################
\clearpage
\subsection*{Appendix A}

%##############################################################################
\subsubsection*{Important Laplace Transform Pairs for Right-Sided Signals}
The Dirac Delta Impulse pair reads
\begin{align}
\boxed{\delta(t) \quad \laplace \quad 1 \quad \text{ROC}: \mathbb{C}}\,.
\end{align}
The Heaviside step function pair reads
\begin{align}
\boxed{\epsilon(t) \quad \laplace \quad \frac{1}{s} \quad \text{ROC}: \Re(s)>0}\,.
\end{align}
With the modulation theorem
\begin{align}
\e^{s_0 t} x(t) \quad \laplace \quad X(s-s_0) \quad \{s \, | \, s-\Re(s_0) \in \text{ROC}(X)\}.
\end{align}
we get (this notation conveniently indicates a pole at $s_0$)
\begin{align}
\boxed{\e^{s_0 t} \epsilon(t)\quad \laplace \quad \frac{1}{s-s_0} \quad \text{ROC}: \Re(s)>\Re(s_0)}\,,
\end{align}
and with changed sign (this notation conveniently represents right-sided,
damped signals for $\Re(s_0)>0$ due to a pole at $-s_0$)
\begin{align}
\boxed{\e^{-s_0 t} \epsilon(t)\quad \laplace \quad \frac{1}{s+s_0} \quad \text{ROC}: \Re(s)>\Re(-s_0)}\,.
\end{align}
%
For the special case of $s_0 = \im \omega_0$, the addition of both versions
yields
\begin{align}
(\e^{+\im \omega_0 t} + \e^{-\im \omega_0 t})  \epsilon(t)\quad &\laplace \quad \frac{1}{s-\im \omega_0} + \frac{1}{s+\im \omega_0} \quad \text{ROC}: \Re(s)>0\\
(\e^{+\im \omega_0 t} + \e^{-\im \omega_0 t})  \epsilon(t)\quad &\laplace \quad \frac{(s+\im \omega_0) + (s-\im \omega_0)}{(s-\im \omega_0)(s+\im \omega_0)}\quad \text{ROC}: \Re(s)>0\\
(\e^{+\im \omega_0 t} + \e^{-\im \omega_0 t})  \epsilon(t)\quad &\laplace \quad \frac{2 s}{s^2 + \omega_0^2}\quad \text{ROC}: \Re(s)>0
\end{align}
\begin{align}
\boxed{\cos(\omega_0 t) \epsilon(t)\quad \laplace \quad \frac{s}{s^2 + \omega_0^2}\quad \text{ROC}: \Re(s)>0}\,,
\end{align}
for which the last is due to the Euler identity $\cos(\omega_0 t) = \frac{1}{2} (\e^{+\im \omega_0 t}+\e^{-\im \omega_0 t})$.
%
For $\sin(\omega_0 t) = \frac{1}{2 \im} (\e^{+\im \omega_0 t}-\e^{-\im \omega_0 t})$ a similar
procedure yields
\begin{align}
\boxed{
\sin(\omega_0 t)\epsilon(t)\quad \laplace \quad \frac{\omega_0}{s^2 + \omega_0^2} \quad \text{ROC}: \Re(s)>0}\,.
\end{align}
%
Modulation theorem leads to (note that $s_0 \in \mathbb{C}$, we often encounter $s_0 > 0, \in \mathbb{R}$ in ODEs)
\begin{align}
\boxed{\e^{-s_0 t} \cos(\omega_0) \epsilon(t)\quad \laplace \quad \frac{(s+s_0)}{(s+s_0)^2 + \omega_0^2}\quad \text{ROC}: \Re(s)>\Re(-s_0)}\,,
\end{align}
and
\begin{align}
\boxed{\e^{-s_0 t} \sin(\omega_0) \epsilon(t)\quad \laplace \quad \frac{\omega_0}{(s+s_0)^2 + \omega_0^2}\quad \text{ROC}: \Re(s)>\Re(-s_0)}\,.
\end{align}

%##############################################################################
%##############################################################################
\newpage
\subsection*{Appendix B}
TBD the big picture ODE 1st/2nd, cf. \cite[p.117]{Strang2014}

\bibliography{literatur}
\end{document}


\chapter{Linear time invariant CT systems}

\section{System types}

A system is an interconncted set of components or sub-systems. Mathematically a system is a transformation between one or more signals, a rule that maps functions to functions.

\begin{itemize}
\item single input - single output (SISO) system.
  \begin{center}
    \begin{tikzpicture}[auto, node distance=3cm,>=latex']
      \node [input, name=input] {};
      \node [block, right of=input] (system) {System $T$};
      \node [output, right of=system] (output) {};

      \draw [draw,->] (input) -- node {Input $x$} (system);
      \draw [->] (system) -- node {Output $y$} (output);
    \end{tikzpicture}
  \end{center}
\item single input - multiple output (SIMO) system
  \begin{center}
    \begin{tikzpicture}[auto, node distance=3cm,>=latex']
      \node [input, name=input] {};
      \node [block, right of=input] (system) {System $T$};
      \node [output, right of=system] (output) {};

      \draw [draw,->] (input) -- node {Input $x$} (system);
      \draw [->] (system) -- node {Output $\vec y$} (output);
    \end{tikzpicture}
  \end{center}
\item general case, multiple input - multiple output (MIMO)
  \begin{center}
    \begin{tikzpicture}[auto, node distance=3cm,>=latex']
      \node [input, name=input] {};
      \node [block, right of=input] (system) {System $T$};
      \node [output, right of=system] (output) {};

      \draw [draw,->] (input) -- node {Input $\vec x$} (system);
      \draw [->] (system) -- node {Output $\vec y$} (output);
    \end{tikzpicture}
  \end{center}
\end{itemize}

We will focus on single input - single output, CT and DT systems.

\begin{itemize}
\item If both input and output are CT signals, it is a CT system.
  \begin{center}
    \begin{tikzpicture}[auto, node distance=4cm,>=latex']
      \node [input, name=input] {};
      \node [block, right of=input] (system) {System $T$};
      \node [output, right of=system] (output) {};

      \draw [draw,->] (input) -- node {Input $x(t)$} (system);
      \draw [->] (system) -- node {Output $y(t)$} (output);
    \end{tikzpicture}
  \end{center}
\item If both input and output are DT signals, it is a DT system.
  \begin{center}
    \begin{tikzpicture}[auto, node distance=4cm,>=latex']
      \node [input, name=input] {};
      \node [block, right of=input] (system) {System $T$};
      \node [output, right of=system] (output) {};

      \draw [draw,->] (input) -- node {Input $x[n]$} (system);
      \draw [->] (system) -- node {Output $y[n]$} (output);
    \end{tikzpicture}
  \end{center}
\item If input and output are not both CT or DT signals, it is a hybrid CT-DT system.
  \begin{center}
    \begin{tikzpicture}[auto, node distance=4cm,>=latex']
      \node [input, name=input] {};
      \node [block, right of=input] (system) {System $T$};
      \node [output, right of=system] (output) {};

      \draw [draw,->] (input) -- node {Input $x[n]$} (system);
      \draw [->] (system) -- node {Output $y(t)$} (output);
    \end{tikzpicture}
  \end{center}
  \begin{center}
    \begin{tikzpicture}[auto, node distance=4cm,>=latex']
      \node [input, name=input] {};
      \node [block, right of=input] (system) {System $T$};
      \node [output, right of=system] (output) {};

      \draw [draw,->] (input) -- node {Input $x(t)$} (system);
      \draw [->] (system) -- node {Output $y[n]$} (output);
    \end{tikzpicture}
  \end{center}
\end{itemize}

As a shorthand notation for the graphical description above we can use $x \mapsto y$. A system maps  a function $x$ to a function $y$:

\begin{itemize}
\item CT system
  \[
  x(t) \mapsto y(t)
  \]

\item DT system
  \[
  x[n] \mapsto y[n]
  \]
\item Hybrid CT-DT system
  \[
  x[n] \mapsto y(t)
  \]
  \begin{center}
    or
  \end{center}
  \[
  x(t) \mapsto y[n]
  \]
\end{itemize}

When a system has no input, the system is {\it autonomous}. An autonomous system just produces output: $\mapsto y$.

\begin{center}
  \begin{tikzpicture}[auto, node distance=4cm,>=latex']
    \node [input, name=input] {};
    \node [block, right of=input] (system) {System $T$};
    \node [output, right of=system] (output) {};

    \draw [->] (system) -- node {Output $y$} (output);
  \end{tikzpicture}
\end{center}

We can think of an autonomous system as a function generator, producing signals for use.

\section{CT system representations}

We can mathematically represent, or model, systems multiple ways.

\begin{itemize}
\item purely mathematically - in time domain we will use
  \begin{itemize}
  \item for CT systems: linear, constant coefficient differential equations. e.g.
    \[
    y^{\prime\prime} + ay^\prime + by = x
    \]

  \item for DT systems: linear, constant coefficient difference equation, e.g.
    \[
    y[n] = a y[n-1] + b y[n-2] + x[n]
    \]
  \end{itemize}
  or
  \begin{itemize}
  \item for CT systems: CT impulse response
  \item for DT systems: DT impulse response
  \end{itemize}
\item purely mathematically - in frequency domain we will use
  \begin{itemize}
  \item frequency response
  \item transfer function (complex frequency, covered in ECE 3704)
  \end{itemize}
\item graphically, using a mixture of math and block diagrams
\end{itemize}

Mathematical models:
\begin{itemize}
\item provide abstraction, removing (often) irrelevant detail.
\item can be more or less detailed, an {\it internal} v.s. {\it external} (block box) description
\item are not unique with respect to instantiation (implementation)
\item are limited to the regime they were designed for
\end{itemize}

\begin{example}[RC Circuit]
  Consider the RC circuit. It is a single input - single output system. We will be able to represent it mathematically or graphically and internally or externally.

  \begin{tabular}{ccc}
    & Graphical & Symbolic\\
    & & \\
    External &
    \begin{tikzpicture}[auto, node distance=3cm,>=latex',, scale=0.6, every node/.style={transform shape}]
      \node [input, name=input] {};
      \node [block, right of=input] (system) {$h(t)$};
      \node [output, right of=system] (output) {};

      \draw [draw,->] (input) -- node {$x(t)$} (system);
      \draw [->] (system) -- node {$y(t)$} (output);
    \end{tikzpicture}
    & $y(t) = h(t)*x(t)$\\
    Internal &
    \begin{circuitikz}[american voltages, scale=0.6, every node/.style={transform shape}]
      \draw
      (0,2) to[R, l=$R$] (3,2)
      (3,2) to[C, l=$C$] (3,0)
      (0,2) to[V, l_=$x(t)$] (0,0)
      (0,0) to[short] (3,0)
      (3,2) to[short, -o] (5,2)
      (3,0) to[short, -o] (5,0)
      (5,2) to[open, v=$y(t)$] (5,0);
    \end{circuitikz}

    & $y^\prime + \frac{1}{RC} y = \frac{1}{RC} x(t)$\\
  \end{tabular}
\end{example}

It does not matter what the underlying system implementation is. For example, consider a mechanical system, described by a second-order ODE:

\begin{center}
  \includegraphics[scale=0.6]{graphics/mechanical_system.pdf}
\end{center}

\begin{tabular}{ll}
  $y$ = position & $M$ = mass\\
  $y^\prime$ = velocity & $K$ = spring constant\\
  $y^{\prime\prime}$ = acceleration & $B$ = coefficient of friction\\
\end{tabular}

\[
y^{\prime\prime} + \frac{B}{M} y^\prime + \frac{K}{M}y = \frac{1}{M}f(t)
\]

Compare this to the parallel RLC circuit, described by the second-order ODE:

\begin{center}
  \begin{circuitikz}[american voltages]
    \draw
    (0,0) to[american current source, l=$f(t)$] (0,2)
    (2,0) to[R, l=$R$] (2,2)
    (4,0) to[L, l=$L$] (4,2)
    (6,0) to[C, l=$C$] (6,2)
    (0,0) to[short, -o] (8,0)
    (0,2) to[short, -o] (8,2)
    (8,2) to[open, v=$y(t)$] (8,0);
  \end{circuitikz}
\end{center}

\begin{tabular}{ll}
  $y$ = voltage & $R$ = resistance\\
  $Cy^\prime$ = capacitor current & $L$ = inductance\\
  & $C$ = capacitance\\
\end{tabular}

\[
y^{\prime\prime} + \frac{1}{RC} y^\prime + \frac{1}{LC}y = \frac{1}{LC}f(t)
\]

Comparing these systems, if $R = \frac{1}{B}$, $L = \frac{1}{K}$, and $C = M$, they are mathematically identical.

\section{System properties and classification}

Choosing the right kind of system model is important. Here are some important properties that allow us to broadly classify systems.
\begin{itemize}
\item Memory
\item Invertability
\item Causality
\item Stability
\item Time-invariance
\item Linearity
\end{itemize}

Let's define each it turn.

\subsection{Memory}
The output of a system with memory depends on previous or future inputs and is said to be {\it dynamic}. Otherwise the system is memoryless or {\it instantaneous}, and the output $y(t)$ at time $t$ depends only on $x(t)$.
For example in CT:
\[
y(t) = 2x(t)
\]
is a memoryless system, while
\[
y(t) = \int\limits_{-\infty}^{t} x(\tau) \; dt
\]
has memory.

\subsection{Invertability}

A system is invertable if there exists a system that when placed in series with the original recovers the input.
\[
x(t) \mapsto{T} y(t) \mapsto{T^{-1}} x(t)
\]
where $T^{-1}$ is the inverse system of $T$. For example, consider a system
\[
x(t) \mapsto y(t) = \int\limits_{-\infty}^t x(\tau) \; d\tau
\]
and a system
\[
y(t) \mapsto z(t) = \frac{dy}{dt}
\]
The combination in series $x(t) \mapsto y(t) \mapsto z(t) = x(t)$, i.e. the derivative undoes the integral.

\subsection{Causality}
A CT system is causal if the output at time $t$ depends on the input for time values at or before $t$:
\[
y(t) \;\text{depends on}\; x(\tau) \;\text{for} \; \tau \leq t
\]
All physical CT systems are causal, even if all continuous systems are not (e.g. continuous 2D images $f(u,v)$, have no "before" and "after").

For example, consider a CT system whose impulse response is $h(t) = e^{-t^2}$. This implies the system produces output \emph{before} (i.e. for $t < 0$) the impulse is applied at $t=0$, somehow anticipating the arrival of the impulse. Barring time-travel, this is physically impossible.

\subsection{Stability}

A CT system is (BIBO) stable if applying a bounded-input
\[
x(t) < \infty \; \forall \; t
\]
results in a bounded-output $x(t) \mapsto y(t)$ and
\[
y(t) < \infty \; \forall \; t
\]
Note, bounded in practice is limited by the physical situation, e.g. positive and negative rails in a physical circuit.

For example, a CT system described by the LCCDE
\[
\frac{dy}{dt}(t) - 2y(t) = x(t)
\]
is unstable because the solution $y(t)$ will have one term of the form $Ce^{2t}$, for most non-zero inputs $x(t)$ or any non-zero initial condition, that grows unbounded as time increases.

\subsection{Time-invariance}
A CT system is time-invariant if, given
\[
x(t) \mapsto y(t)
\]
then a time-shift of the input leads to the same time-shift in the output
\[
x(t-\tau) \mapsto y(t-\tau)
\]

An important example is a CT system described by a LCCDE, e.g.
\[
\frac{dy}{dt}(t) + y(t) = x(t)
\]
but non-zero auxillary conditions at some $t_0$, $y(t_0) = y_0$. Such systems will have a term in it's solution that depends on $y_0$. However if I time shift the input, the term that depends on $y_0$ does not shift (since it is achored to $t_0$) and the total output does not shift identically with the input. Thus the system cannot be time-invariant.

\subsection{Linearity}

A CT system is linear if the output due to a sum of scaled individual inputs is the same as the scaled sum of the individual outputs with respect to those inputs. In other words given
\[
x_1(t) \mapsto y_1(t) \;\text{and}\; x_2(t) \mapsto y_2(t)
\]
then
\[
a x_1(t) + b x_2(t) \mapsto a y_1(t) + b y_2(t)
\]
for constants $a$ and $b$.
Note this property extends to sums of arbitrary signals, e.g. if
\[
x_i(t) \mapsto y_i(t) \; \forall\; i \in [1 \cdots N]
\]
then given $N$ constants $a_i$, if the system is linear
\[
\sum\limits_{i = 1}^N a_i x_i(t) \mapsto \sum\limits_{i = 1}^N a_i y_i(t)
\]
This is a very important property, called {\it superposition}, and it simplifies the analysis of systems greatly.

Similar to time-invariance an important non-linear system is that is described by a LCCDE with non-zero auxillary conditions at some $t_0$, $y(t_0) = y_0$. Again such systems will have a term in it's solution that depends on $y_0$. Given two inputs, each individual response will have that term in it, so thier sum has double that term. However the response due to the sum of the inputs would again only have one and the sum of the responses would not be the same as the response of the sum. Such a system cannot be linear.

\section{Stable LTI Systems}

The remainder of this course is about stable, linear, time-invariant (LTI) systems. As we have seen in CT such systems can be described by a LCCDE with zero auxillary (initial) conditions (the system is \emph{at rest}).

We have seen previously how to find the impulse response, $h(t)$, of such systems. We now note some relationships between the impulse response and the system properties described above.

\begin{itemize}
\item If a system is memoryless then $h(t) = C \delta(t)$ for some constant $C$.
\item If a system is causal then  $h(t) = 0$ for $t < 0$.
\item If a system is BIBO stable then
  \[
  \int\limits_{-\infty}^{\infty} |h(t)| \; dt < \infty
  \]
\end{itemize}


\clearpage
\subsubsection{Pre-Test Loop} % (fold)
\label{sub:pre_test_loop}

The Pre-Test Loop is a looping statement that allows code to be run 0 or times. The loop checks the condition at the start, and if the condition is True the loop's body is executed. At the end of the loops body the computer jumps back to the condition, checking it again to determine if the loop's body should execute again. If the condition is False when it is checked the loop jumps ends, and control jumps to the next statement in the code.

\begin{figure}[h]
   \centering
   \includegraphics[width=\textwidth]{./topics/control-flow/diagrams/PreTestLoop}
   \caption{The Pre-Test Loop checks the condition, then runs the loop's body}
   \label{fig:looping-pre-test}
\end{figure}

\mynote{
\begin{itemize}
  \item A pre-test loop is an \textbf{action}, creating a loop in the code's sequence of instructions.
  \item The standard pre-test loop is the \textbf{while statement}.
  \item A pre-test loop allows instructions to be run 0 or more times.
  \item The condition is checked when the loop's code is entered, and it is checked again at the end of each loop.
\end{itemize}
}

% subsection post_test_loop (end)