\documentclass{article}
\title{REDUCE-MathML Interface}
\author{Luis Alvarez-Sobreviela \\ alvarez@zib.de \\ \\
	Konrad--Zuse--Zentrum f\"ur Informationstechnik Berlin \\
        Takustra\ss e 7 \\ D-14195 Berlin-Dahlem}
\date{August 9, 1999}

\begin{document}

\maketitle

\section{Introduction}

MathML is intended to facilitate the use and re-use of mathematical and
scientific content on the Web, and for other applications such as computer
algebra systems. For this reason we believe MathML is an important step
for the scientific community considering the widespread use of the
Internet. We found necessary to include REDUCE in this trend, and
developed the MathML-REDUCE interface.

The MathML interface for REDUCE provides an easy to use series of
commands, allowing it to evaluate and output MathML. This manual is
therefore intended to give the user a good guideline on how to make of it a
proper use.
\\

The principal features of this package can be resumed as:
\begin{itemize}
\item Evaluation of MathML code. Allows REDUCE to parse MathML expressions
and evaluate them.
\item Generation of MathML compliant code. Provides the printing of REDUCE
expressions in MathML source code, to be used directly in web page
production.

\item Generation of MathML code surrounded by HTML $<$embed$>$$<$/embed$>$
tags. This is useful for directly including MathML into a webpage. It can
then easily be rendered on any browser with the IBM Tech Explorer plug-in
\footnote{Currently available at
http://www.software.ibm.com/network/techexplorer/}.

\end{itemize}

We assume that the reader is familiar with MathML 1.0. If not, the
specification. is available at: \\

{\tt http://www.w3.org/Math/ }

\section{Getting Started}

\subsection{Loading}

The MathML-REDUCE interface package is under the name {\tt mathml}, and so is
loaded by supplying {\tt load mathml;}.

\subsection{Switches}

There are three switches which can be used alternatively and
incrementally.  These are {\bf mathml}, {\bf web} and {\bf both}. There
use can be described as follows:

\begin{description}
\item[{\tt mathml}:] All output will be printed in MathML.
\item[{\tt both}:] All output will be printed in both MathML and normal
REDUCE.

\item[{\tt web}:] All output will be printed within an HTML $<embed>$ tag.
This is for direct use in an HTML web page when using IBM Tech Explorer.
\end{description}

MathML has often been said to be too verbose. If {\bf both} is on, an easy
interpretation of the results is possible, improving MathML readability.

\subsection{Entering MathML}

The MathML-REDUCE interface gives the user various ways of providing
input. This can be done via a file containing MathML or by writing MathML
directly in the prompt. \\

\subsubsection{Reading MathML from a File: {\tt MML}}

When reading from a file the command {\bf mml} is used. {\bf mml} takes as
argument the name of the file containing the MathML. \\

{\tt mml}(FILE:{\it string}):{\it expression}

\paragraph{Example:} As long as the file given contains valid MathML, no
errors should be produced.

{\tt 1:  mml "ex.mml";}

\subsubsection{Reading MathML from Prompt: {\tt PARSEML}}

By using the function {\bf parseml} it is possible to introduce a series of
valid mathml tokens to the prompt. {\bf parseml} takes no arguments,
although once it is called it will prompt you to enter mathml tags starting
with {\tt $<$mathml$>$} and ending with {\tt $<$/mathml$>$}. It then returns
an expression resulting from evaluating the input. \\

\paragraph{Example:} Here is an extract of a REDUCE session where {\tt
parseml()} is used:\\

{\tt 2: parseml();

2: <math>

2: <apply><plus/>

2: <cn>3</cn>

2: <cn>5</cn>

2: </apply>

2: </math>
\\

\hspace*{1cm}  8 \\

3: } \\

If for some reason you do not want to continue typing in at the prompt, and
cancel what has already been typed, it is possible to exit by calling
control {\tt C} or control {\tt D}.

Although it is simpler to edit a file and then use {\tt mml}, we still
added {\bf parseml} for completeness.

\section{The Evaluation of MathML}

MathML is evaluated by the interface always before outputting any results.
Just in the same way as REDUCE normally does. The program works in the same
way as the {\tt algebraic} mode. Undefined variables remain undefined,
and it is possible to use all normal REDUCE switches and packages.

\paragraph{Example:}

\noindent The following MathML:\\
\\
{\tt
\hspace*{0mm}<math>

\hspace*{1mm}<reln><gt/>

\hspace*{6mm}<ci>x</ci>

\hspace*{6mm}<ci>y</ci>

\hspace*{1mm}</reln>
\\
\hspace*{0mm}</math>}
\\
\\
will evaluate to {\tt gt(x,y)}. This only states that x is greater than
y.
\\
The interesting characteristic, is that we can set the undefined values of
x and y.
\\
Suppose we enter the following:
\\

{\tt 5: x:=3; y:=2;
\\

x := 3
\\

y := 2}
\\
\\
If we once again enter and evaluate the above piece of MathML we will have as
result:
\\

{\tt t}
\\
\\
because it is true that 3 is greater than 2. It is important to note that
it is also possible to set only one of the two variables x or y above, say
{\tt y:=4}. The expression will then not evaluate completely, and we will
have:  \\

{\tt gt(x,4)}
\\
\\
When one of the switches is on, the MathML output will be:
\\
\\
{\tt
\hspace*{1mm}<math>\\
\hspace*{5mm}   <reln><gt/>\\
\hspace*{9mm}      <ci>x</ci>\\
\hspace*{9mm}      <cn type="integer">4</cn>\\
\hspace*{5mm}   </reln>\\
\hspace*{1mm}</math>\\}

Hence, it is possible when dealing with a MathML formula representation in
which there are a set of undefined variables to set each variable to the
desired value and evaluate it. Let us consider a second example to make sure
everything is clear.

\paragraph{Example:}

Let the file {\tt ex.mml} contain the following mathml:
\\
\\
{\tt               <math>\\
\hspace*{2mm}       <apply><int/>\\
\hspace*{5mm}        <bvar>\\
\hspace*{9mm}         <ci>x</ci>\\
\hspace*{5mm}        </bvar>\\
\hspace*{5mm}        <apply><fn><ci>F</ci><fn>\\
\hspace*{9mm}         <ci>x</ci>\\
\hspace*{5mm}        </apply>\\
\hspace*{2mm}       </apply>\\
                  </math>}  \\
\\
If we do the following:\\

{\tt 1: mml "ex.mml";}\\
\\
This is what we get:
\\
\\
{\tt int(f(x),x);}\\
\\
It is clear that this has remained unevaluated. We can now set the function
{\tt f(x)} as follows:\\

{\tt for all x let f(x)=2*x**2;}\\
\\
If we then enter '{\tt mml "ex.mml"}' once again, we will have the
following result:\\

{\Large \( \frac {2*x^{3}}{3}\)}
\\
\\
Hence the MathML-REDUCE interface allows the user to set a value to a
variable in order to manipulate the evaluation of the MathML, without needing
to edit the MathML itself.

\subsection{Using Boolean Values}

Boolean values are not defined in MathML, despite their importance in REDUCE.
To get around this problem, we can set a variable's value to a boolean value,
and when evaluating, the MathML-REDUCE interface will use the boolean value
of the variable.
\\
Suppose we want to evaluate the following expression:
\\
\\
{\tt and(t,nil);}
\\
\\
Then all we do, is we create a file with the following MathML:
\\
\\
{\tt          $<$mathml$>$\\
\hspace*{2mm} $<$apply$>$$<$and/$>$\\
\hspace*{5mm}         $<$ci$>$ a $<$/ci$>$\\
\hspace*{5mm}         $<$ci$>$ b $<$/ci$>$\\
\hspace*{2mm}      $<$/apply$>$\\
        $<$/mathml$>$\\}
\\
And before evaluating it, we set {\tt a} to {\tt true} and {\tt b} to {\tt
nil}. When evaluating the MathML, it will produce the same result as the
equivalent REDUCE expression.

\section{Interpretation of Error Messages}

The MathML-REDUCE interface has a set of error messages which aim to help the
user understand and correct any invalid MathML. Because there can exist many
different causes of errors, such error messages should be considered merely
as advice. Here we shall consider the most important error messages.

\paragraph{Missing tag:}

Many MathML tags go by pairs, such as {\tt $<$apply$>$$<$/apply$>$,
$<$reln$>$$<$/reln$>$}, {\tt $<$ci$>$$<$/ci$>$}, etc\ldots. In the
case
where the ending tag is missed out, or misspelled, it is very probable that an
error of this type will be thrown.

\paragraph{Ambiguous or Erroneous Use of {\tt $<$apply$>$}}

This error message defines an undefined error. When this error message
appears, it is not very clear to the interface where exactly lies the error.
Probable causes are a misuse of the {\tt $<$apply$>$$<$/apply$>$} tags, or a
mispelling of the tag preceding the {\tt $<$apply$>$}. However, other types
of errors may cause this error message to appear.

Tags following an {\tt $<$apply$>$} tag may be misspelled without causing an
error message, but they will be considered as operators by REDUCE and
therefore evaluate to some unexpected expression.

\paragraph{Syntax Errors}

It is possible that the input MathML is not syntactically correct. In such
a situation, the error will be spotted, and in some cases a solution might be
presented. There are a variety of syntax errors messages, but relying on
their advice might not always be helpful.
\\

Despite the verbose nature of the error messages and their recommended
solutions, we do recommend that in most situations reference to the MathML
specification is made.

\section{Limitations of the Interface}

Not all aspects of MathML have been perfectly fitted into the interface.
There are still some problems unsolved in the present version of the
interface:

\begin{itemize}
\item MathML Presentation Markup is not supported. The
interface will treat every presentation tag as an unknown tag, or a REDUCE
operator. We found presentation markup not prioritary when dealing with
computer algebra systems, although in the future, parsing of presentation
markup within content markup shall be supported.
\item Certain MathML tags
do not play an important role in the REDUCE environment. Such tags will not
evaluate or affect in anyway the interface's behaviour. They will be parsed
correctly, although their action will be ignored. These tags are:
\begin{enumerate}
{\tt
\item $<$interval$><$/interval$>$
\item $<$inverse$><$/inverse$>$
\item $<$condition$><$/condition$>$
\item $<$compose$><$/compose$>$
\item $<$ident$><$/ident$>$
\item $<$forall/$>$
\item $<$exists/$>$}
\end{enumerate}

Although {\tt $<$condition$><$/condition$>$} and {\tt
$<$interval$><$/interval$>$} tags are supported when used within the
following tags:
\begin{enumerate}
{\tt
\item $<$int/$>$
\item $<$limit/$>$
\item $<$sum/$>$
\item $<$product/$>$}
\end{enumerate}
\item The {\tt $<$declare$>$} construct takes one or two arguments. It sets
the first argument to the value of the second. In the case where the second
argument is a vector or a matrix, an obscure error message is produced. It is
clearly something which must be fixed in the future.\\ \item The program
throws an error when it encounters {\tt nil} between {\tt $<$ci$><$/ci$>$}
tags. It is not possible to use boolean values directly.  Please refer to the
above subsection treating this matter.
\end{itemize}

\section{Including MathML into a Webpage}

We shall not get into the details of including MathML in the design of a
web page, but it is important for completeness, to give a quick overview.
One can include MathML in two ways:

\begin{itemize}

\item By introducing MathML inside $<$math$>$$<$/math$>$ tags in any HTML
page.

\item By having MathML being embeded inside HTML $<$embed$>$$<$/embed$>$
tags.

\end{itemize}

When one has the {\bf mathml} switch on, MathML is generated inside
$<$math$>$$<$/math$>$ tags. Browsers such as WebEQ \footnote{Available at
http://www.webeq.com}can read this type of encoding.

\paragraph{Example:}
\begin{verbatim}

<title>MathML example</title>

Example of MathML

The following is MathML directly encoded within this homepage:

<math>
   <apply><int>
      <bvar>
         <ci>x</ci>
      </bvar>
      <apply><power/>
         <ci>x</ci>
         <apply><power/>
            <ci>x</ci>
            <cn type="integer">2</cn>
         </apply>
      </apply>
   </apply>
</math>

On some browsers such as WebEQ, the MathML is rendered correctly.

\end{verbatim}

But most people on the web use either Netscape or Microsoft Internet
Explorer, which do not at the moment support complete MathML rendering.

What must be done in cases where Netscape or Microsoft Internet Explorer
are used, is to make available the IBM Tech Explorer Plug-in, and include
the MathML inside the web page surrounded by $<$embed$>$$<$/embed$>$ tags.
This is easily done with the REDUCE-MathML interface by having the {\bf
web} switch on. Here is an example of how to do this:

\paragraph{Example}

\begin{verbatim}

<title>MathML example</title>

Example of MathML

The following is MathML directly encoded within this homepage:

<EMBED TYPE="text/mathml" MMLDATA="<math><apply><int><bvar><ci>x</ci></bvar>
<apply><power/><ci>x</ci><apply><power/><ci>x</ci><cn type="integer">
2 </cn></apply></apply></apply></math>"
HEIGHT=300 WIDTH=200>

On all browsers with the Tech Explorer Plug-in, this is rendered
correctly.

\end{verbatim}

Another way to embed MathML in a web page document is by having a
separate file contain MathML (usually a file with {\bf .mml} extension)
and to embed it inside a page with the following code:

\begin{verbatim}

<EMBED TYPE="text/mathml" src="mathml.mml" HEIGHT=300 WIDTH=200>

\end{verbatim}

\section{Examples}

We would like to present a series of examples which will illustrate the
possibilities of the interface.

\paragraph{Example 1} Type in the following and observe the resulting
expression:

\begin{verbatim}
23: on mathml;

24: solve({z=x*a+1},{z,x});
\end{verbatim}
\paragraph{Example 2}Have a file {\tt ex2.mml} containing the following
MathML source code:\\
\\
{\tt                 $<$mathml$>$\\
\hspace*{1mm}      $<$apply$>$$<$sum/$>$\\
\hspace*{5mm}       $<$bvar$>$\\
\hspace*{9mm}         $<$ci$>$x$<$/ci$>$\\
\hspace*{5mm}       $<$/bvar$>$\\
\hspace*{5mm}       $<$apply$>$$<$fn$>$$<$ci$>$F$<$/ci$>$$<$fn$>$\\
\hspace*{9mm}         $<$ci$>$x$<$/ci$>$\\
\hspace*{5mm}       $<$/apply$>$\\
\hspace*{1mm}      $<$/apply$>$\\
                  $<$/mathml$>$\\}
\\
and type:\\
\\
{\tt mml "ex2.mml"}

\paragraph{Example 3} This example illustrates how practical the switch
{\bf both} can be for interpreting verbose MathML.
Introduce the following MathML source into a file, say {\tt ex3.mml}\\
\\
{\tt $<$mathml$>$\\
\hspace*{1mm}  $<$apply$>$$<$int/$>$\\
\hspace*{5mm}    $<$bvar$>$\\
\hspace*{9mm}      $<$ci$>$x$<$/ci$>$\\
\hspace*{5mm}    $<$/bvar$>$\\
\hspace*{5mm}    $<$apply$>$$<$sin/$>$\\
\hspace*{9mm}      $<$apply$>$$<$log/$>$\\
\hspace*{13mm}       $<$ci$>$x$<$/ci$>$\\
\hspace*{9mm}      $<$/apply$>$\\
\hspace*{5mm}    $<$/apply$>$\\
\hspace*{1mm}  $<$/apply$>$\\
$<$/mathml$>$ \\}
\\
then do the following:

\begin{verbatim}

2: on both;

3: mml "ml";

\end{verbatim}

\section{An overview of how the Interface Works}

The interface is primarily built in two parts. A first one which parses
and evaluates MathML, and a second one which parses REDUCE's algebraic
expressions and prints them out in MathML format. Both parts work by
recursive parsing, using Top-Down Recursive Descent parsing with one token
look ahead.

The BNF description of the MathML grammar is to be defined informally in
APPENDIX E of the current MathML specification. It is with this document that
we have developed the MathML parser. The MathML parser evaluates all
that is possible and returns a valid REDUCE algebraic expression. When {\bf
mathml} or {\bf both} are on, this algebraic expression is fed into the
second part of the program which parses these expressions and transforms them
back into MathML.

The MathML generator parses through the algebraic expression produced by
either REDUCE itself or the MathML parser. It works in a very
similar way as the MathML parser. It is simpler, since no evaluation is
involved. All the generated code is MathML compliant. It is important to note
that the MathML code generator sometimes introduces Presentation Markup tags,
and other tags which are not understood by the MathML parser of the
interface\footnote{The set of tags not understood by the MathML parser are
detailed in section {\bf Limitations}.}.

\end{document}


\section{Forward Sampling Strategies}
\label{sec:forwardSamplingStrategies}
In order to perform UQ and dynamic
probabilistic risk assessment (DPRA),
a sampling strategy needs to be employed. The sampling strategy
perturbs the input space (domain of the uncertainties) to explore
the response of a complex system in relation to selected FOMs.

The most widely used strategies to perform UQ and PRA are generally
collected in RAVEN as \textit{\textbf{Forward}} samplers. \textit{\textbf{Forward}} samplers include
all the strategies that simply perform the sampling of the input space.  These strategies sample
without exploiting, through learning approaches,
the information made available from the outcomes of evaluation previously performed (adaptive sampling) and the
common system evolution (patterns) that different sampled calculations can generate in the phase space (Dynamic Event Tree).

As mentioned in Section~\ref{susub:OverviewSamplers}, RAVEN has
several different \textit{\textbf{Forward}} samplers:
\begin{itemize}
  \item \textit{Monte-Carlo}
  \item \textit{Grid-based}
  \item \textit{Stratified} and its specialization named \textit{Latin Hyper Cube}.
\end{itemize}
In addition, RAVEN posses advanced \textit{\textbf{Forward}} sampling strategies that:
\begin{itemize}
  \item Build a grid in the input space selecting evaluation points
    based on characteristic quadratures as part of stochastic collocation
    for generalized polynomial chaos method (\textit{Sparse
    Grid Collocation} sampler);
  \item Use high-density model reduction (HDMR) a.k.a. Sobol
    decomposition to approximate a function as the sum of
    interactions with increasing complexity (\textit{Sobol} sampler).
\end{itemize}
In the following subsections, we provide examples of input files
in RAVEN using the method, with explanatory commentary.
%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%  MONTE-CARLO %%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Monte-Carlo sampling through RAVEN}
\label{sub:MCexample}
The Monte-Carlo method is one of the most-used methodologies in several mathematic disciplines. In this section,
we will explain the techniques for employing this methodology in RAVEN, and we recommend the user to read the
theory manual to explore the theory of the method.
The goals of this section are about learning how to:
 \begin{enumerate}
   \item Set up a simple Monte-Carlo sampling for perturbing the input space of a driven code
   \item Load the outputs of the code into RAVEN DataObjects (HistorySets and PointSets)
   \item Print the contents of DataObjects to file
   \item Generate plots of the sampling results.
\end{enumerate}
In order to accomplish these tasks, the following RAVEN \textbf{Entities} (XML blocks in the RAVEN input file) are needed:
\begin{enumerate}
   \item \textbf{\textit{RunInfo}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingMontecarlo.xml}{RunInfo}
     As discussed in Section~\ref{sub:EntitiesAndFlow}, the \textit{RunInfo} \textbf{Entity} sets up the analysis
     that the user wants to perform. In this specific case, two steps (\xmlNode{Sequence}) are sequentially run
     using 12 processors (\xmlNode{batchSize}). This means that
     12 instances of the driven code are run simultaneously. Every time a simulation ends, a new one is launched.
     Note that the \xmlNode{JobName} is not required, but is useful in identifying the input file.
   \item \textbf{\textit{Files}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingMontecarlo.xml}{Files}
     Since the driven code uses a single input file, in this section the original input is identified and
     named. As detailed in the user manual, the attribute \xmlAttr{name} represents the alias that is used in
     all the other input blocks in order to refer to this file.
   \item \textbf{\textit{Models}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingMontecarlo.xml}{Models}
     The Model used in this example is the
     \textbf{AnalyticalBateman} external model, which is defined in section \ref{sec:analyticalbateman}.  This model uses a keyword-based input file
     and exports its output file in a CSV format, so the \textit{GenericCode} interface is used.
   \item \textbf{\textit{Distributions}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingMontecarlo.xml}{Distributions}
     In the \xmlNode{Distributions} block, the stochastic model for the uncertainties treated by the
     \xmlNode{Sampler} is defined. In this case two distributions are defined:
     \begin{itemize}
      \item $sigma \sim \mathbb{U}(1,10)$, used to model the uncertainties
        associated with the \textit{sigma} Model variables
      \item  $decayConstant \sim \mathbb{U}(0.5e-8,1e-8)$,  used to
        model the uncertainties associated with the Model variable \textit{decay constants}.  Note that the
        same distribution can be re-used for multiple input variables, while still keeping those variables
        independent.
     \end{itemize}
   \item \textbf{\textit{Samplers}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingMontecarlo.xml}{Samplers}
      To employ the Monte-Carlo sampling strategy ($100$ samples), a
      \xmlNode{MonteCarlo} node needs to be defined.  The
      Monte-Carlo method is employed on $eight$ model variables, each of which are listed by name
      and are associated with a distribution.
      Note that all the \textit{decay-\*} and
      \textit{sigma-\*} variables are associated with the distributions
      $decayConstant$ and $sigma$, respectively.
   \item \textbf{\textit{DataObjects}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingMontecarlo.xml}{DataObjects}
      In this block, two \textit{DataObjects} are defined to store results: 1) a PointSet named
      ``samples'', 2) a HistorySet named ``histories''.
      Note that in the \xmlNode{Input} node all the uncertainties
      perturbed through the Monte-Carlo strategy are listed. By this, any
      realization in the input space is linked in the DataObject to the outputs listed in the
      \xmlNode{Output} node.
   \item \textbf{\textit{OutStreams}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingMontecarlo.xml}{OutStreams}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %figure histories
 \begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ForwardSamplingStrategies/gold/RunDir/MonteCarlo/1-historyPlot_line-line-line-line.png}
  \caption{Plot of the histories generated by the Monte Carlo sampling for variables $A,B,C,D$.}
  \label{fig:historiesMCPlotLine}
 \end{figure}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 To see the results of the simulation, \xmlNode{OutStreams} are included in the input.
  In this block, both OutStream types are used:
  \begin{itemize}
    \item \textit{Print}:
     \begin{itemize}
       \item ``samples'' connected with the \textit{DataObjects} \textbf{Entity} ``samples''
                (\xmlNode{source})
       \item ``histories'' connected with the \textit{DataObjects} \textbf{Entity} ``histories'' (\xmlNode{source})
     \end{itemize}
     Note that in RAVEN, multiple entities can have the same name, as it takes a class, a type, and a name to
     uniquely identify a RAVEN object.
      When the two OutStream objects are used, all the information contained in the
      linked \textit{DataObjects} are going
    to be exported in CSV files (\xmlNode{type}).
    \item \textit{Plot}:
    \begin{itemize}
      \item ``historiesPlot'' connected with the  \textit{DataObjects}
      \textbf{Entity} ``samples''.  This plot will draw the final state of the
      variables $A,B,C,D$ with respect to the input variables $sigma$(s)
      and $decay$(s) .
      \item ``samplesPlot3D'' connected with the
      \textit{DataObjects} \textbf{Entity} ``histories''. This plot will draw the
      evolution of the variables $A,B,C,D$.
    \end{itemize}
     Note that both plots use gridded subplots. Four plots
     are placed in each of the figures.
  \end{itemize}
   \item \textbf{\textit{Steps}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingMontecarlo.xml}{Steps}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %figure samples
 \begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ForwardSamplingStrategies/gold/RunDir/MonteCarlo/1-samplesPlot3D_scatter-scatter-scatter-scatter.png}
  \caption{Plot of the samples generated by the MC sampling for variables $A,B,C,D$.}
  \label{fig:samplesMCPlotLine}
 \end{figure}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
   Once all the other entities are defined in the RAVEN input file, they must be combined in the
   \xmlNode{Steps} block, which dictates the workflow of RAVEN.  For this case,
   two \xmlNode{Steps} are defined:
   \begin{itemize}
     \item \xmlNode{MultiRun} ``sample'', used to run the multiple
     instances of the driven code and
     collect the outputs in the two \textit{DataObjects}. As it can be
     seen, the \xmlNode{Sampler} is specified to communicate to the
     \textit{Step} that the driven code needs to
     be perturbed through the Monte-Carlo sampling.
     \item  \xmlNode{IOStep} named ``writeHistories'', used to 1) dump
     the ``histories'' and ``samples'' \textit{DataObjects}
     \textbf{Entity} to a CSV file and 2) plot the data in the EPS file.
   \end{itemize}
\end{enumerate}
 Figures~\ref{fig:historiesMCPlotLine} and ~\ref{fig:samplesMCPlotLine} show the report
 generated by RAVEN of the evolution of the
 variables $A,B,C,D$ and their final values, respectively.
%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%          GRID          %%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Grid sampling through RAVEN}
\label{sub:Gridexample}
The Grid sampling method (also known as Full Factorial Design of Experiment) represents one of the simplest methodologies that can be employed in order to explore the interaction of multiple random variables with respect
selected FOMs.
The goal of this section is to show how to:
 \begin{enumerate}
   \item Set up a simple Grid sampling for performing a parametric analysis of a driven code
   \item Load the outputs of the code into the RAVEN DataObjects system
   \item Print out what contained in the DataObjects
   \item Generate basic plots of the code result.
\end{enumerate}
In order to accomplish these tasks, the following RAVEN \textbf{Entities} (XML blocks in the input files) are required:
\begin{enumerate}
   \item \textbf{\textit{RunInfo}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingGrid.xml}{RunInfo}
   As shown in Section~\ref{sub:EntitiesAndFlow}, the \textit{RunInfo} \textbf{Entity} is intended to set up the desired analysis.
   In this specific case, two steps (\xmlNode{Sequence}) are sequentially run
   using 12 processors (\xmlNode{batchSize}). This means that
   12 instances of the driven code are  run simultaneously.
   Every time a simulation ends, a new one is launched.
   \item \textbf{\textit{Files}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingGrid.xml}{Files}
   Since the driven code uses a single input file, in this section the original input is placed. As described in the user manual~\cite{RAVENuserManual}
   the attribute  \xmlAttr{name} represents the alias that is used in all the other input blocks in order to refer to this file.
   \item \textbf{\textit{Models}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingGrid.xml}{Models}
 The Model here is represented by the
 \textbf{AnalyticalBateman}, which already dumps its output file in a
 CSV format (standard format that RAVEN can read). For this reason,
 the \textit{GenericCode} interface is used.
   \item \textbf{\textit{Distributions}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingGrid.xml}{Distributions}
  In the Distributions XML section, the stochastic model for the
  uncertainties  treated by the Grid sampling are reported. In
  this case two distributions are defined:
  \begin{itemize}
    \item $sigma \sim \mathbb{U}(1,10)$, used to model the uncertainties
    associated with  the Model \textit{sigma}(s)
    \item  $decayConstant \sim \mathbb{U}(0.5e-8,1e-8)$,  used to
    model the uncertainties
    associated with  the Model \textit{decay constants}.
  \end{itemize}
   \item \textbf{\textit{Samplers}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingGrid.xml}{Samplers}
  To employ the Grid sampling strategy, a
  \xmlNode{Grid} node needs to be specified. As shown above, in each variable section, the  \xmlNode{grid} is defined.
  The number of samples finally requested are equal to $n_{samples} = \prod_{i=1}^{n} n_{steps_{i}+1} = 256$.
  Note that, for each variable, can be defined either in probability (CDF) or in absolute value.
   \item \textbf{\textit{DataObjects}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingGrid.xml}{DataObjects}
  Int this block, two \textit{DataObjects} are defined: 1) PointSet named
  ``samples'', and 2) HistorySet named ``histories''.
  In the \xmlNode{Input} node all the variables
  perturbed through the Grid strategy are listed. In this way, any
  realization in the input space is linked to the outputs listed in  the
  \xmlNode{Output} node.
   \item \textbf{\textit{OutStreams}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingGrid.xml}{OutStreams}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %figure histories
 \begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ForwardSamplingStrategies/gold/RunDir/Grid/1-historyPlot_line-line-line-line.png}
  \caption{Plot of the histories generated by the Grid sampling for variables $A,B,C,D$.}
  \label{fig:historiesGridPlotLine}
 \end{figure}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  In this block, both the Out-Stream types are constructed:
  \begin{itemize}
    \item \textit{Print}:
     \begin{itemize}
       \item named ``samples'' connected with the \textit{DataObjects} \textbf{Entity} ``samples''
                (\xmlNode{source})
       \item named ``histories'' connected with the \textit{DataObjects} \textbf{Entity} ``histories'' (\xmlNode{source}).
     \end{itemize}
      When these objects get used, all the information contained in the
      linked  \textit{DataObjects} are going
    to be exported in CSV files (\xmlNode{type}).
    \item \textit{Plot}:
    \begin{itemize}
      \item named ``historiesPlot'' connected with the  \textit{DataObjects}
      \textbf{Entity} ``samples''.  This plot will draw the final state of the
      variables $A,B,C,D$ with respect to the input variables $sigma$(s)
      and $decay$(s).
      \item named ``samplesPlot3D'' connected with the
      \textit{DataObjects} \textbf{Entity} ``histories''. This plot will draw the
      evolution of the variables $A,B,C,D$.
    \end{itemize}
     As it can be noticed, both plots are of type \textit{SubPlot}. Four plots
     are placed in each of the figures.
  \end{itemize}
   \item \textbf{\textit{Steps}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingGrid.xml}{Steps}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %figure samples
 \begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ForwardSamplingStrategies/gold/RunDir/Grid/1-samplesPlot3D_scatter-scatter-scatter-scatter.png}
  \caption{Plot of the samples generated by the Grid sampling for variables $A,B,C,D$.}
  \label{fig:samplesGridPlotLine}
 \end{figure}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
   Finally, all the previously defined \textbf{Entities} can be combined in
   the \xmlNode{Steps} block. As inferable,
   two \xmlNode{Steps} have been inputted:
   \begin{itemize}
     \item \xmlNode{MultiRun} named ``sample'', is used to run the multiple
     instances of the code and
     collect the outputs in the two \textit{DataObjects}. As it can be
     seen, the \xmlNode{Sampler} is inputted to communicate to the
     \textit{Step} that the driven code needs to
     be perturbed through the Grid sampling
     \item  \xmlNode{IOStep} named ``writeHistories'', used to 1) dump
     the ``histories'' and ``samples'' \textit{DataObjects}
     \textbf{Entity} in a CSV file and 2) plot the data in the EPS file and
     on the screen.
   \end{itemize}
\end{enumerate}
 Figures~\ref{fig:historiesGridPlotLine} and ~\ref{fig:samplesGridPlotLine}  report the evolution of the
 variables $A,B,C,D$ and their final values, respectively.

%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%    STRATIFIED    %%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Stratified sampling through RAVEN}
\label{sub:Stratifiedexample}
The Stratified sampling is a class of methods that relies on the assumption that the input space (i.e.,uncertainties)
can be separated in regions (strata) based on similarity of the response of the system for input set within the
same strata. Following this assumption, the most rewarding (in terms of computational cost vs. knowledge gain)
sampling strategy would be to place one sample for each region. In this way, the same information is not
collected more than once and all the prototypical behavior are sampled at least once. In
Figure~\ref{fig:StratifiedSamplingExample}, the Stratified sampling approach is exemplified.
 \begin{figure}[h!]
  \centering
  \includegraphics[scale=0.55]{pics/StratifiedSamplingExample.png}
  \caption{Example of Stratified sampling approach.}
  \label{fig:StratifiedSamplingExample}
 \end{figure}
\\The goal of this section is to show how to:
 \begin{enumerate}
   \item Set up a simple Stratified sampling in order to perform a parametric analysis on a driven code
   \item Load the outputs of the code into the RAVEN DataObjects system
   \item Print out what contained in the DataObjects
   \item Generate basic plots of the code result.
\end{enumerate}
To accomplish these tasks, the following RAVEN \textbf{Entities} (XML blocks in the input files) are defined:
\begin{enumerate}
   \item \textbf{\textit{RunInfo}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingStratified.xml}{RunInfo}
   As reported in Section~\ref{sub:EntitiesAndFlow}, the \textit{RunInfo} \textbf{Entity} is intended to set up the analysis
   that the user wants to perform. In this specific case, two steps (\xmlNode{Sequence}) are  run
   using 12 processors (\xmlNode{batchSize}). This means that
   12 instances of the driven code are run simultaneously.
   Every time a simulation ends, a new one is launched.
   \item \textbf{\textit{Files}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingStratified.xml}{Files}
   Since the considered code uses a single input file, in this section the original input is placed.
   The attribute  \xmlAttr{name} represents the alias that is going to be used in all the other input blocks in order to refer to this file.
   \item \textbf{\textit{Models}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingStratified.xml}{Models}
 The Model here is represented by the
 \textbf{AnalyticalBateman}, which already dumps its output file in a
 CSV format (standard format that RAVEN can read). For this reason,
 the \textit{GenericCode} interface is used.
   \item \textbf{\textit{Distributions}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingStratified.xml}{Distributions}
  In the Distributions XML section, the stochastic model for the
  uncertainties  treated by the Stratified sampling are reported. In
  this case two distributions are defined:
  \begin{itemize}
    \item $sigma \sim \mathbb{U}(1,10)$, used to model the uncertainties
    associated with  the Model \textit{sigma}(s)
    \item  $decayConstant \sim \mathbb{U}(0.5e-8,1e-8)$,  used to
    model the uncertainties
    associated with  the Model \textit{decay constants}.
  \end{itemize}
   \item \textbf{\textit{Samplers}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingStratified.xml}{Samplers}
  To employ the Stratified sampling strategy, a
  \xmlNode{Stratified} node needs to be specified. In each variable section, the  \xmlNode{grid} is defined.
  It is important to mention that the number of \xmlAttr{steps} needs to be the same for each of the variables,
  since, as reported in previous section, the Stratified sampling strategy it discretizes the domain in strata.
  The number of samples finally requested is equal to $n_{samples} = n_{steps} = 100$.
  If the grid for each variables is defined in CDF and of  \xmlAttr{type} = ``equal'', the Stratified
  sampling corresponds to the well-known Latin Hyper Cube sampling.
   \item \textbf{\textit{DataObjects}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingStratified.xml}{DataObjects}
  In this block, two \textit{DataObjects} are defined: 1) PointSet named
  ``samples'', 2) HistorySet named ``histories''.
  In the \xmlNode{Input} node all the variables
  perturbed through the Stratified strategy are listed. In this way, any
  realization in the input space is linked to the outputs listed in  the
  \xmlNode{Output} node.
   \item \textbf{\textit{OutStreams}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingStratified.xml}{OutStreams}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %figure histories
 \begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ForwardSamplingStrategies/gold/RunDir/Stratified/1-historyPlot_line-line-line-line.png}
  \caption{Plot of the histories generated by the Stratified sampling for variables $A,B,C,D$.}
  \label{fig:historiesStratifiedPlotLine}
 \end{figure}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  In this block, both the Out-Stream types are constructed:
  \begin{itemize}
    \item \textit{Print}:
     \begin{itemize}
       \item named ``samples'' connected with the \textit{DataObjects} \textbf{Entity} ``samples''
                (\xmlNode{source})
       \item named ``histories'' connected with the \textit{DataObjects} \textbf{Entity} ``histories'' (\xmlNode{source}).
     \end{itemize}
      When these objects get used, all the information contained in the
      linked  \textit{DataObjects} are going
    to be exported in CSV files (\xmlNode{type}).
    \item \textit{Plot}:
    \begin{itemize}
      \item named ``historiesPlot'' connected with the  \textit{DataObjects}
      \textbf{Entity} ``samples''.  This plot will draw the final state of the
      variables $A,B,C,D$ with respect to the input variables $sigma$(s)
      and $decay$(s)
      \item named ``samplesPlot3D'' connected with the
      \textit{DataObjects} \textbf{Entity} ``histories''. This plot will draw the
      evolution of the variables $A,B,C,D$.
    \end{itemize}
     As it can be noticed, both plots are of type \textit{SubPlot}. Four plots
     are going to be placed in each of the figures.
  \end{itemize}
   \item \textbf{\textit{Steps}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingStratified.xml}{Steps}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %figure samples
 \begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ForwardSamplingStrategies/gold/RunDir/Stratified/1-samplesPlot3D_scatter-scatter-scatter-scatter.png}
  \caption{Plot of the samples generated by the Stratified sampling for variables $A,B,C,D$.}
  \label{fig:samplesStratifiedPlotLine}
 \end{figure}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
   Finally, all the previously defined \textbf{Entities} can be combined in
   the \xmlNode{Steps} block. As inferable,
   two \xmlNode{Steps} have been inputted:
   \begin{itemize}
     \item \xmlNode{MultiRun} named ``sample'', used to run the multiple
     instances of the driven code and
     collect the outputs in the two \textit{DataObjects}. As it can be
     seen, the \xmlNode{Sampler} is inputted to communicate to the
     \textit{Step} that the driven code needs to
     be perturbed through the Stratified sampling.
     \item  \xmlNode{IOStep} named ``writeHistories'', used to 1) dump
     the ``histories'' and ``samples'' \textit{DataObjects}
     \textbf{Entity} in a CSV file and 2) plot the data in the EPS file and
     on the screen.
   \end{itemize}
\end{enumerate}
 As previously mentioned, Figures~\ref{fig:historiesStratifiedPlotLine} and ~\ref{fig:samplesStratifiedPlotLine}  report the evolution of the
 variables $A,B,C,D$ and their final values, respectively.

%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Sparse Grid Collocation sampling through RAVEN}
\label{sub:SGcsamplingExample}
The Sparse Grid Collocation sampler represents an advanced methodology to perform Uncertainty Quantification. They aim
to explore the input space leveraging the information contained in the associated probability density functions. It builds on generic Grid sampling by selecting evaluation points based on characteristic quadratures as part of stochastic collocation for generalized polynomial chaos uncertainty quantification. In collocation an N-D grid is constructed, with each uncertain variable providing an axis. Along each axis, the points of evaluation correspond to quadrature points necessary to integrate polynomials. In the simplest (and most naive) case, a N-D tensor product of all possible combinations of points from each dimension’s quadrature is constructed as sampling points. The number of necessary samples can be reduced by employing Smolyak-like sparse grid algorithms, which use reduced combinations of polynomial orders to reduce the necessary sampling space.
\\The goals of this section are about learning how to:
 \begin{enumerate}
   \item Set up a Sparse Grid Collocation sampling for the construction of a suitable surrogate model of a driven code
   \item Construct a GaussPolynomialRom surrogate model (training stage)
   \item Use the constructed GaussPolynomialRom surrogate model instead of the driven code.
\end{enumerate}
To accomplish these tasks, the following RAVEN \textbf{Entities} (XML blocks in the input files) need to be defined:
\begin{enumerate}
   \item \textbf{\textit{RunInfo}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingSparseGrid.xml}{RunInfo}
   As reported in section~\ref{sub:EntitiesAndFlow}, the \textit{RunInfo} \textbf{Entity} is intended to set up the analysis
   that the user wants to perform. In this specific case, six steps (\xmlNode{Sequence}) are going to be sequentially run
   using twelve processors (\xmlNode{batchSize}).  The first two steps build the ROM
   (\xmlString{sample}, \xmlString{train}), the next two validate
   the ROM against the original Code Model (\xmlString{validateModel}, \xmlString{validateROM}),
   and the last two produce plots and print data (\xmlString{output\_print}, \xmlString{output\_plot}).
   \item \textbf{\textit{Files}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingSparseGrid.xml}{Files}
   Since the driven code uses a single input file, in this section the original input is placed. As detailed in the user manual
   the attribute  \xmlAttr{name} represents the alias that is going to be used in all the other input blocks in order to refer to this file.
   \item \textbf{\textit{Models}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingSparseGrid.xml}{Models}
 The goal of this example is the generation of a \text{GaussPolynomialRom}
 for subsequent usage instead of the original code.  In addition to the previously explained Code model,
 the ROM of type \textit{GaussPolynomialRom} is specified here. The ROM is generated through a Sparse Grid
 Collocation sampling strategy. All 4 targets $A,B,C,D$ are modeled through this ROM as function
 of the uncertain $sigma$ and $decay$ parameters.  Note that the \xmlNode{Interpolation} nodes are not
 required, but are included for the sake of demonstration.
   \item \textbf{\textit{Distributions}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingSparseGrid.xml}{Distributions}
  In the Distributions XML section, the stochastic model for the
  uncertainties  treated by the Sparse Grid Collocation sampling are reported. In
  this case eight distributions are defined:
  \begin{itemize}
    \item $sigmaA \sim \mathbb{U}(6.9,8.1)$, used to model the uncertainty
    associated with  the Model \textit{sigma-A}
    \item $sigmaB \sim \mathbb{U}(3.9,5.1)$, used to model the uncertainty
    associated with  the Model \textit{sigma-B}
    \item $sigmaC \sim \mathbb{U}(1.9,3.1)$, used to model the uncertainty
    associated with  the Model \textit{sigma-C}
    \item $sigmaD \sim \mathbb{U}(0.9,1.1)$, used to model the uncertainty
    associated with  the Model \textit{sigma-D}
    \item  $decayConstantA \sim \mathbb{U}(3.8e-9,5.2e-9)$,  used to
    model the uncertainty
    associated with  the Model \textit{decay-A}
    \item  $decayConstantB \sim \mathbb{U}(5.8e-9,7.2e-9)$,  used to
    model the uncertainty
    associated with  the Model \textit{decay-B}
    \item  $decayConstantC \sim \mathbb{U}(6.8e-9,8.2e-9)$,  used to
    model the uncertainty
    associated with  the Model \textit{decay-C}
    \item  $decayConstantD \sim \mathbb{U}(7.8e-9,9.2e-9)$,  used to
    model the uncertainty
    associated with  the Model \textit{decay-D}.
  \end{itemize}
   \item \textbf{\textit{Samplers}}:
     \xmlExample[rom,ROM,SparseGridCollocation,MonteCarlo]{framework/user_guide/ForwardSamplingStrategies/forwardSamplingSparseGrid.xml}{Samplers}
  In order to employ the Sparse Grid Collocation sampling strategy, a
  \xmlNode{SparseGridCollocation} node needs to be defined.
  As can be
  seen from above, each variable is associated with a different distribution,
  defined in the  \xmlNode{Distributions} block.
  In addition, the \textit{GaussPolynomialRom} \xmlNode{ROM} is linked to the \xmlNode{SparseGridCollocation}
  sampler.  Because this sampler is used exclusively to build the ROM, some of the parameters of the ROM are
  needed by the sampler, and this connection makes that communication possible.  The setting of this ROM
  (e.g. polynomial order, Index set method, etc.) determines how the Stochastic Collocation Method is
  employed.

  Additionally, a \xmlNode{MonteCarlo} sampler is set up for validating the ROM against the original Code.  The random
  number generation seed (\xmlNode{initialSeed}) is specified and set to reset on each use
  (\xmlNode{reseedEachIteration}) so that the Monte Carlo sampler can be used to compare the ROM against the
  original model.  We use twenty samples (\xmlNode{limit}) to sample the ROM and the model, and then print and
  plot both data sets to compare them.
   \item \textbf{\textit{DataObjects}}:
     \xmlExample[rom,SparseGridCollocation]{framework/user_guide/ForwardSamplingStrategies/forwardSamplingSparseGrid.xml}{DataObjects}
  In this block, four \textit{DataObjects} are defined:
  1) a PointSet named ``inputPlaceholder'' used as a placeholder input for the ROM sampling step,
  2) a HistorySet named ``histories'' used to collect the samples needed to train the ROM,
  3) a PointSet named ``samplesModel'' to store the Code responses from Monte Carlo samples, and
  4) a PointSet named ``samplesROM'' to store the ROM responses from Monte Carlo samples.
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %figure histories
 \begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ForwardSamplingStrategies/gold/RunDir/SparseGrid/1-historyPlot_line-line-line-line.png}
  \caption{Plot of the training samples generated by the SparseGridCollocation sampling for variables $A,B,C,D$.}
  \label{fig:historiesSparseGridPlotLine}
 \end{figure}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
   \item \textbf{\textit{OutStreams}}:
     \xmlExample{framework/user_guide/ForwardSamplingStrategies/forwardSamplingSparseGrid.xml}{OutStreams}
  In this block, the following Out-Stream types are constructed:
  \begin{itemize}
    \item \textit{Print}:
     \begin{itemize}
       \item named ``samplesModel'' connected with the \textit{DataObjects} \textbf{Entity} ``samplesModel''
                (\xmlNode{source})
       \item named ``samplesROM'' connected with the \textit{DataObjects} \textbf{Entity} ``samplesROM''
                (\xmlNode{source})
       \item named ``histories'' connected with the \textit{DataObjects} \textbf{Entity} ``histories'' (\xmlNode{source})
       \item named ``rom\_output'' connected with the \textit{ROM} \textbf{Entity} ``rom'' (\xmlNode{source}).
     \end{itemize}
      When these objects get used, all the information contained in the
      linked  \textit{DataObjects} are going
    to be exported in ether CSV files for DataObjects or XML files for ROMs (\xmlNode{type}).
    \item \textit{Plot}:
    \begin{itemize}
      \item named ``historyPlot'' connected with the  \textit{DataObjects}
      \textbf{Entity} ``histories''.  This plot will draw the time-dependent state of the
      variables $A,B,C,D$ with respect to the input variables $sigma$(s)
      and $decay$(s)
      \item named ``samplesModelPlot3D'' connected with the
      \textit{DataObjects} \textbf{Entity} ``samplesModel''. This plot will draw the
      variables $A,B,C,D$ as Monte Carlo sampled on the Code.
      \item named ``samplesROMPlot3D'' connected with the
      \textit{DataObjects} \textbf{Entity} ``samplesROM''. This plot will draw the
      variables $A,B,C,D$ as Monte Carlo sampled on the ROM.
    \end{itemize}
     As it can be noticed, both plots are of type \textit{SubPlot}. Four plots
     are going to be placed in each of the figures.
  \end{itemize}
   \item \textbf{\textit{Steps}}:
     \xmlExample[rom,SparseGridCollocation]{framework/user_guide/ForwardSamplingStrategies/forwardSamplingSparseGrid.xml}{Steps}
  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %figure samples
 \begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ForwardSamplingStrategies/gold/RunDir/SparseGrid/1-samplesModelPlot3D_scatter-scatter-scatter-scatter.png}
  \caption{Plot of validation samples generated by Monte Carlo sampling on the Code for variables $A,B,C,D$.}
  \label{fig:samplesSparseGridPlotModel}
 \end{figure}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
   Finally, the previously defined \textbf{Entities} can be combined in
   the \xmlNode{Steps} block.
   The following \xmlNode{Steps} have been defined:
   \begin{itemize}
     \item \xmlNode{MultiRun} named ``sample'', used to run the training
     samples of the driven code and
     collect the outputs in the \textit{DataObjects}.
     The \xmlNode{Sampler} is specified to communicate to the
     \textit{Step} that the driven code needs to
     be sampled through the Sparse Grid Collocation sampling strategy;
     \item \xmlNode{RomTrainer} named ``train'', used to train (i.e.,
     construct) the GaussPolynomial ROM. This step is essential if the
     user want to use the ROM in later steps;
     \item \xmlNode{MultiRun} named ``sampleModel'', used to run the
     Monte Carlo perturbed samples of the original Model for use in verification.  The results are
     collected in the \textit{samplesModel} \textit{DataObjects}.
     \item \xmlNode{MultiRun} named ``sampleROM'', used to run the
     Monte Carlo perturbed samples of the previously constructed ROM for use in verificaiton.  The results are
     collected in the \textit{samplesROM} \textit{DataObjects}.
     \item  \xmlNode{IOStep} named ``output\_print'', used to dump
     the ``histories'', ``samplesModel'' and ``samplesROM'' \textit{DataObjects}
     \textbf{Entity} in a CSV file,
     \item  \xmlNode{IOStep} named ``output\_plot'', used to
     plot the data and store it in the PNG file and
     on the screen.
   \end{itemize}
\end{enumerate}
 As previously mentioned, Figure~\ref{fig:historiesSparseGridPlotLine}
 shows the evolution of the outputs $A,B,C,D$ under uncertainties.
 Figures~\ref{fig:samplesSparseGridPlotModel} and
 \ref{fig:samplesROMSparseGridPlotROM} show the final responses
 of the sampling employed using the driven code and the ROM,
 respectively. As it can be seen, the constructed ROM can accurately
 represent the response of the driven code. This example shows the
 potential of reduced order modeling, in general, and of the
 \textit{GaussPolynomialRom}, in particular.

  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %figure samples
 \begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ForwardSamplingStrategies/gold/RunDir/SparseGrid/1-samplesROMPlot3D_scatter-scatter-scatter-scatter.png}
  \caption{Plot of validation samples generated by Monte Carlo sampling on the ROM for variables $A,B,C,D$.}
  \label{fig:samplesROMSparseGridPlot}
 \end{figure}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


\documentclass[twoside]{MATH77}
\usepackage{multicol}
\usepackage[fleqn,reqno,centertags]{amsmath}
\begin{document}
\hyphenation{DCKDER}
\begmath 8.3 Check Code for Computing Derivatives

\silentfootnote{$^\copyright$1997 Calif. Inst. of Technology, \thisyear \ Math \`a la Carte, Inc.}

\subsection{Purpose}

Subroutine DCKDER checks the mutual consistency of code for computing values
of a (possibly vector valued) function with code for computing first
derivatives $(e.g.$, gradient vector or Jacobian matrix) of the function. In
particular this is expected to be useful for persons using the software of
Chapters 8.2, 9.2, and~9.3.

\subsection{Usage}

\subsubsection{Program Prototype, Double Precision}
\begin{description}
\item[INTEGER]  \ {\bf MODE, M, N, LDFJAC, IMAX, JMAX}

\item[DOUBLE PRECISION]  {\bf X}($\geq $N){\bf , FVEC}($\geq $M){\bf ,\\ FJAC%
}(LDFJAC,$\geq $N){\bf , TSTMAX,\\ TEST}(LDFJAC,$\geq $N)
\end{description}
\begin{tabbing}
\hspace{.2in}\=Assign values to M, N, and LDFJAC.\\

\>Set X() to the value of the vector ${\bf x}$ at which\\
\>\ \ \ \ \=consistency is to be tested.\\

\>Compute FJAC(,) as the M $\times $ N Jacobian matrix\\
\>\>of first partial derivatives of FVEC with respect\\
\>\>to ${\bf x}$, evaluated at X().\\

\>MODE = 1\\

10\>continue
\end{tabbing}\vspace{-8pt}
\begin{center}
\fbox{\begin{tabular}{@{\bf }c}
CALL DCKDER(MODE, M, N, X,\\
FVEC, FJAC, LDFJAC, TEST,\\
IMAX, JMAX,TSTMAX)\\
\end{tabular}}
\end{center}
\begin{tabbing}
\hspace{.2in}\=if(MODE .eq. 2) then\\

\>\ \ \ \ \=Compute FVEC() as an M-vector of function\\
\>\>\ \ \ \ \=values evaluated at X().\\

\>\>go to 10\\

\>end if\\

\>Here the process is completed. Results are in\\
\>TEST(,), IMAX, JMAX, and TSTMAX.
\end{tabbing}
\subsubsection{Argument Definitions}

\begin{description}
\item[MODE]  \ [inout] On initial entry set MODE = 1. DCKDER will return a
number of times with MODE = 2. The calling code should compute FVEC() as a
function of X() and call DCKDER again, not altering MODE. When DCKDER is
finished, it returns with MODE = 3.

\item[M]  \ [in] Number of terms in FVEC() and number of rows of data in
FJAC(,).

\item[N]  \ [in] Number of terms in X() and number of columns of data in
FJAC(,).

\item[X()]  \ [inout] Initially must contain the vector ${\bf x}$ around
which the testing will be done. Contains perturbed ${\bf x}$ on each return
with MODE = 2. On final return with MODE = 3, contains the original ${\bf x}$
exactly restored.

\item[FVEC()]  \ [in] On entries with MODE = 2, the user stores function
values in FVEC(), $i.e.$, FVEC($i$) $=f_i.$

\item[FJAC(,)]  \ [in] On the initial entry with MODE = 1, the user stores
the Jacobian matrix in FJAC(,), $i.e.$, FJAC$(i,j)=\partial f_i/\partial
x_j.$

\item[LDFJAC]  \ [in] Declared first dimension of the arrays FJAC(,) and
TEST(,). Require LDFJAC $\geq $ M.

\item[TEST(,)]  \ [inout] Array with the same dimensions as FJAC(). On final
return with MODE = 3, TEST$(i,j)$ contains the consistency measure for FJAC$%
(i,j)$ for all $i=1$, ..., M, and $j=1$, ..., N. This quantity is computed
as the signed difference: FJAC$(i,j)$ minus a central finite difference
approximation to $\partial f_i/\partial x_j$. The user does not need to
store anything in TEST(,) before the initial entry with MODE = 1. On
intermediate returns with MODE = 2, TEST(,) contains saved intermediate
quantities, and thus the user must not alter the contents of TEST(,) on
these returns.

\item[IMAX, JMAX, TSTMAX]  \ [inout] On final return with MODE = 3, these
quantities are set so that

TSTMAX $=abs($TEST(IMAX, JMAX$))=\max \{abs(\text{TEST}(i,j)):i=1$, ..., M; $%
j=1$, ..., N$\}$. The user does not need to store anything in these
variables before the initial entry with MODE = 1. On intermediate returns
with MODE = 2, these variables contain saved intermediate quantities, and
thus the user must not alter their contents on these returns.
\end{description}

\subsubsection{Modifications for Single Precision}

For single precision usage change the DOUBLE PRECISION statements to REAL
and change the name DCKDER to SCKDER. It is recommended that one use the
double precision rather than the single precision version of this package
for better reliability, except possibly on computers such as the Cray Y/MP
that have precision of about 14~decimal places in single precision.

\subsection{Examples and Remarks}

The program DRDCKDER illustrates the use of DCKDER. Results are shown in
ODDCKDER. This example was run using double precision IEEE arithmetic which
has precision of $\epsilon \approx 0.22 \times 10^{-15}$. If third derivatives
have about the same magnitude as function values, and the relative error in
function evaluations is about machine precision, then the magnitude of
entries of TEST(,) should be about $\epsilon ^{2/3}$ times the magnitude
of the function values. For example in our sample case we have $|f_1| =
0.0646$ and so we would expect the magnitude of terms in the first row of
TEST(,) to be about $(0.37 \times 10^{-10}) \times 0.0646 \approx 0.24 \times
10^{-11}$, with the stated assumptions on the size of third derivatives and
the error in function evaluations. If the relative error in the computation
of function values is larger than the machine precision, or the magnitudes
of third derivatives are larger than the magnitudes of the function values,
the values in TEST(,) will be larger as discussed in Section D, even when
the code being tested is correct.

\subsection{Functional Description}

Let ${\bf x}$ denote the vector given initially in X(). Assume FJAC(,)
contains the $m \times n$ Jacobian matrix of $\partial f_i / \partial x_j$
evaluated at ${\bf x}$. Let $\epsilon $ denote the
machine precision and $\omega $ denote the underflow limit. These are given
by D1MACH(4) and D1MACH(1) respectively (R1MACH() in single precision)
(See Chapter~19.1).
Define $\alpha = (3\epsilon )^{1/3}$ and $\sigma = \max (10^5\omega
/\alpha $, $\epsilon ^2)$. For each value of $j$ from 1 to $n$, compute $%
h_j = \alpha x_j$ if $|x_j| > \sigma $, or $h_j = \alpha \sigma $ if $0 <
|x_j| \leq \sigma $, or $h_j = \alpha $ if $x_j = 0$. Let ${\bf e}_j$
denote the $n$-vector that is all zeros except for the $j^{th}$ component
which is one.

For each $i$ and $j$ compute%
\begin{equation*}
\hspace{-12pt}\text{TEST}(i,j)=\text{FJAC}(i,j)-
\frac{f_i({\bf x}+h_j{\bf e}_j)-f_i({\bf x}-h_j{\bf e}_j)}{2h_j}
\end{equation*}
The error in this central difference approximation to the derivative $%
\partial f_i/\partial x_j$ is%
\begin{equation*}
h_j^2M_3/6+\delta /h_j
\end{equation*}
where $M_3$ denotes the magnitude of $\partial ^3f_i/\partial ^3x_j$
evaluated at some point on the line segment from ${\bf x}-h_j{\bf e}_j$ to $%
{\bf x}+h_j{\bf e}_j$, and $\delta $ is a bound on the error in computing $%
f_i$. The optimal step length to minimize this error estimate is%
\begin{equation*}
(3\delta /M_3)^{1/3}
\end{equation*}
If we assume $M_3\approx |f_i|$ and $\delta \approx \epsilon |f_i|$, so $%
\delta /M_3\approx \epsilon $, throughout the relevant interval, then the
optimal step length is $(3\epsilon )^{1/3}$ and the error bound is $\frac
12(3\epsilon )^{2/3}|f_i|\approx 1.04\epsilon ^{2/3}|f_i|$. These
formulas, and useful insights into the computational use of finite
differences, are given in Section~8.6 of \cite{Gill:1987:PO}.

\bibliography{math77}
\bibliographystyle{math77}

\subsection{Error Procedures and Restrictions}

Require MODE = 1 or 2 on any entry to DCKDER. If not, DCKDER will call the
error message processor of Chapter~19.2 with an error level of 0 and return
with MODE unchanged.

\subsection{Supporting Information}

The source language is ANSI Fortran~77.

\begin{tabular}{@{\bf}l@{\hspace{5pt}}l}
\bf Entry & \hspace{.35in} {\bf Required Files}\vspace{2pt} \\
DCKDER & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, DCKDER, ERFIN, ERMSG, IERM1, IERV1\rule[-5pt]{0pt}{8pt}}\\
SCKDER & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, ERFIN, ERMSG, IERM1, IERV1, SCKDER}\\
\end{tabular}

Designed and programmed by C. L. Lawson and F. T. Krogh, JPL, 1991.

\begcode

\medskip\
\lstset{language=[77]Fortran,showstringspaces=false}
\lstset{xleftmargin=.8in}

\centerline{\bf \large DRDCKDER}\vspace{5pt}
\lstinputlisting{\codeloc{dckder}}
\newpage
\vspace{20pt}\centerline{\bf \large ODDCKDER}\vspace{10pt}
\lstset{language={}}
\lstinputlisting{\outputloc{dckder}}
\end{document}


\documentclass{article}
\usepackage{verbatim}
\setcounter{secnumdepth}{0}

\begin{document}
   \centering
   \Huge
   \textsc{The Standard Definition of the Definite Integral Tutorial}
   \normalsize

   \section{Summation Theorems}

    \section{Standard Definition of an Integral}

    \begin{equation}
    	\int_a^b f(x) \mathrm{d} x = \lim_{i \to \infty} \sum_{i=1}^n f(x_i^*) \Delta x
    \end{equation}

    \section{Explanation}
        \subsection{Example 1}
        \subsection{Example 2}
    \section{Homework Assignment}
\end{document}


\chapter{An Alternative Means of Regularization for Domain Adaptation Problems}

\section*{Introduction}
Previously, we explored applications of entropic regularization for the Wasserstein distance (Sinkhorn) in domain adaptation. In particular, we derived a new generalization bound for target error with respect to the Sinkhorn divergence. However, for domain adaptation, entropic regularization may not be the ideal route to take.

When we prescribe a divergence measure for domain adaptation, the scenario we seek to penalize against is when the source $S$ and target $T$ distributions are not identical. However, with the entropic regularization, one is penalizing against $S$ and $T$ being independent. In this chapter, we propose an alternative regularization that may be more suitable for domain adaptation problems.

If $S$ and $T$ are identical in distribution, which is the ideal setting in machine learning, then there exists an identity mapping between the two. Thus, it makes sense to use a regularization that penalizes the deviation between our transport map and the identity map.

\newpage

\section*{Existence and Uniqueness of an Optimal Transport Map}
Our approach relies on Brenier's theorem, which concerns the existence and uniqueness of optimal maps.

\begin{theorem}{Brenier's Theorem}

	Let $\mu$ and $\nu$ be absolutely continuous probability measures on $\mathbb{R}^d$ with respect to the Lebesgue measure and $\nu$ has bounded support. There exists a convex function $\phi: \mathbb{R}^d\to \mathbb{R}$ such that its gradient, $\nabla \phi$, is the optimal transport map from $\mu$ to $\nu$.

\end{theorem}

We call $\phi$ a Kantorovich potential.

\begin{corollary}{Existence of Solution to Monge Problem}

Under the above assumptions, $\nabla \phi$ uniquely solves the Monge problem.

\[
\int_X \abs{x-\nabla\phi(x)}^2\,d\mu(x) = \int_{T_{\#}\mu=\nu}\abs{x-T(x)}^2\,d\mu(x)
\]

\end{corollary}

\section*{Convex Analysis Prerequisites}

Before we continue, we introduce some concepts from convex analysis that will be needed going forward.

\begin{definition}
	Let $f:\,X\to \mathbb{R}\cup +\infty$ be a lower semicontinuous function. The Fenchel-Legendre transform $f^{*}: X^{*}\to \mathbb{R}\cup +\infty$ is defined by
	$$
	f^{*}(x^{*}) = \sup_{x\in X} [\ang*{x^{*},x}-f(x)]
	$$

	This $f^{*}$ is always convex.
\end{definition}

\begin{definition}
	The subdifferential of a lower semi-continuous convex function $\phi$ at $x\in \textrm{dom} \phi$ is defined by

	$$
	\partial \phi(x) = \{ x^{*}\in X^{*}: \phi(y)-\phi(x)\geq \ang*{x^{*},y-x} \}
	$$
\end{definition}

\begin{corollary}
	Let $f:\,X\to \mathbb{R}\cup \{+\infty\}$ be a convex function. Then for any $x\in \textrm{int dom }f$,

	$$ \partial f(x)\neq \emptyset.$$
\end{corollary}

\begin{theorem}[Fenchel-Young inequality]
	$$f(x)+f^{*}(x^{*})\geq \ang*{x^{*},x}$$
\end{theorem}

\begin{corollary}[Fenchel-Young equality]
	$$f(x)+f^{*}(x^{*})= \ang*{x^{*},x}$$ iff
	$$x^{*}\in \partial f(x).$$
\end{corollary}

\begin{theorem}
	Let $v(y) = \inf_x [f(x)+g(x+y)].$ The Fenchel primal problem is
	$$p = v(0) = \inf_x [f(x)+g(x)].$$

	The dual problem is
	$$d = v^{**}(0)=\sup_{y^{*}} [-f^{*}(y^{*})-g^{*}(-y^{*})].$$
\end{theorem}

\begin{proof}
	Calculate $v^{*}(-y^{*}) = \sup_{x,y}[\ang*{-y^{*},y}-f(x)-g(x+y)].$

	Let $u = x+y$, so we have
	\begin{align*}
	v^{*}(-y^{*}) &= \sup_{x,u} \ang*{-y^{*},u-x} - f(x) - g(u) \\
	&= \sup_x [\ang*{y^{*},x}-f(x)]+\sup_u [\ang*{-y^{*},u}-g(u)] \\
	&= f^{*}(y^{*}) + g^{*}(-y^{*}).
	\end{align*}

	Thus,
	$$d = v^{**}(0) = \sup_{-y^{*}}[0 - v^{*}(-y^{*})] = \sup_{-y^{*}}[-f^{*}(y^{*})-g^{*}(-y^{*})].$$

	Weak duality $p\geq d$ follows immediately. Strong duality $p=d$ requires $\partial v(0)\neq \emptyset$, i.e.
	$$0\in \textrm{int dom }v = \textrm{int}[\textrm{dom } g-\textrm{dom} f].$$
\end{proof}

\begin{proof}[Strong duality criterion]
	Here we prove $0\in \textrm{int dom }v = \textrm{int}[\textrm{dom } g-\textrm{dom} f]$ implies $\partial v(0)\neq \emptyset$. Let $-y^{*}\in \partial v(0)$ we have
	$$
	f(x)+g(x+y)\geq v(y)\geq p-\ang*{y^{*},y}
	$$
	Letting $u=x+y$, we have
	$$p\leq f(x)-\ang*{y^{*},x}+g(u)+\ang*{y^{*},u}
	$$ and taking infimum with respect to $x,u$, we have
	$$
	p\leq -f^{*}(y^{*})-g^{*}(-y^{*})\leq d\leq p.
	$$
	This concludes the proof.
\end{proof}

\section*{Derivation of Primal Formulation}

The goal is to find the primal formulation for the following optimization problem:
$$ \inf_{u,v} \left[ \int u \, ds + \int v \, dt + \lambda (\norm{u}_H^2 + \norm{v}_H^2)\right],\, \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - u(x) - v(y)\leq c(x,y). $$ The arguments that follow are similar to those used to prove Kantorovich duality.

Let $$\phi_c = \{ (u,v) \in C(X)\times C(Y): \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - u(x) - v(y)\leq c(x,y)\}$$.
%$E^{*} = M(X\times Y)$ is the set of regular Radon measures with total variation norm.

In the next few lines, $z \in E = C(X\times Y)$ is the set of continuous functions on $X\times Y$ with sup. norm and $\pi\in E^{*} = M(X\times Y)$ is the set of regular Radon measures with total variation norm.

$$f(z) = \begin{cases}
\lambda( \left\|u\right\|_H^2 + \left\|v\right\|_H^2) + \int u\,ds + \int v\,dt, \, z(x,y)=u(x)+v(y) \\
+\infty, \, \textrm{ otherwise.}
\end{cases}$$

$$g(z) = \begin{cases}
0, \, \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - z(x,y)\leq c(x,y) \\
+\infty, \, \textrm{ otherwise.}
\end{cases}$$

$$f(z)+g(z) = \begin{cases}
\int_X u\,ds + \int_Y v\,dt + \lambda(\left\|u\right\|_H^2 + \left\|v\right\|_H^2), \, \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - u(x) - v(y)\leq c(x,y) \\
\infty, \textrm{ otherwise.}
\end{cases}$$

$$\inf_{z\in E} [f(z)+g(z)] = \inf_{(u,v)\in \Phi_c} \left\{ \int_X u\,ds + \int_Y v\,dt + \lambda(\left\|u\right\|_H^2 + \left\|v\right\|_H^2) \right\}$$

\begin{align*}
g^{*}(-\pi) &= \sup_{z\in E} \left[ -\int_{X\times Y} z\,d\pi - g(z)\right] \\
&= \sup_{z\in E} \left[ - \int_{X\times Y} z\,d\pi :\, \frac{1}{2}\norm{x}^2 + \frac{1}{2}\norm{y}^2 - c(x,y)\leq z(x,y) \right] \\
&= \begin{cases}
\int_{X\times Y} \left[c(x,y) - \frac{1}{2}\norm{x}^2 - \frac{1}{2}\norm{y}^2\right] \,d\pi,\, \pi\in M_{+}(X\times Y) \\
+\infty, \, \textrm{otherwise}
\end{cases}
\end{align*}

\begin{align*}
f^{*}(\pi) &= \sup_{z\in E} \left[ \int_{X\times Y} z\,d\pi - f(z)\right] \\
&= \sup_{z\in E} \left[ \int_{X\times Y} z(x,y)\,d\pi(x,y) - \int_X u\,ds - \int_Y v\,dt - \lambda(\left\|u\right\|_H^2 + \left\|v\right\|_H^2) :\, z= u\oplus v \right] \\
&= \begin{cases}
- \lambda(\left\|u\right\|_H^2 + \left\|v\right\|_H^2), \, \pi\in \Pi(s,t) \\
-\inf_u (\int u\,ds - \int u\,d\pi + \lambda \norm{u}_H^2) - \inf_v (\int v\,dt - \int v\,d\pi + \lambda \norm{v}_H^2), \, \textrm{otherwise}
\end{cases}
\end{align*}

When finite, the last line (when $\pi$ is not a coupling) can be rewritten as
$$\frac{\sup_{\norm{u}=1} (\int u\,ds - \int u\,d\pi)^2}{2\lambda} + \frac{\sup_{\norm{v}=1}(\int v\,dt - \int v\,d\pi)^2}{2\lambda}$$ if finite. Otherwise, $f^{*}(\pi)=+\infty$.

The dual problem is $\sup_{\pi\in E^{*}} \left[ -f^{*}(\pi)-g^{*}(-\pi)\right] $, which, from the above, is
$$ \sup_{\pi\in E^{*}} \left\lbrace - \int_{X\times Y} c\,d\pi - \int_X \varphi\,d\mu - \int_Y \psi\,d\nu + \int_{X\times Y} (\varphi(x)+\psi(y))\,d\pi + \lambda(\norm{\varphi}_H^2+\norm{\psi}_H^2)\right\rbrace . $$

\subsection*{What does the dual of f look like?}

If the previous claim holds,

$$
f^{*}(\pi) = \frac{1}{4\lambda} \left(Q(s,\pi_1) + Q(t,\pi_2)\right)
$$ where $$Q(s,\pi_1) = \int k(x,y)\,ds(x) ds(y) + \int k(x,y)\,d\pi_1(x) d\pi_1(y) - 2 \int k(x,y)\,ds(x)d\pi_1(y)$$ and similarly for $Q(t,\pi_2)$.

Then our dual problem is
\begin{align*}
\sup_{\pi\in E^{*}}(-f^{*}(\pi) - g^{*}(-\pi)) &= \sup_{\pi\in M_{+}} \left[ -\frac{1}{4\lambda} (Q(s,\pi_1)+Q(t,\pi_2)) - \int \left( c(x,y) - \frac{1}{2}\norm{x}^2 - \frac{1}{2}\norm{y}^2 \right)\,d\pi \right] \\
&= \inf_{\pi\in M_{+}} \left[ \frac{1}{4\lambda} (Q(s,\pi_1)+Q(t,\pi_2)) + \int \left( c(x,y) - \frac{1}{2}\norm{x}^2 - \frac{1}{2}\norm{y}^2 \right)\,d\pi \right]
\end{align*}

If $c(x,y)=\frac{1}{2}\norm{x-y}^2$, then $c(x,y)-\frac{1}{2}\norm{x}^2 - \frac{1}{2}\norm{y}^2 = x^{T}y$, which seems more tractable to work with. The question now is how to minimize
$$
\frac{1}{4\lambda} (Q(s,\pi_1)+Q(t,\pi_2)) + \int x^{T}y\,d\pi(x,y).
$$

\subsection*{Inner product space of signed measures}
Define the inner product with respect to signed measures $m,n$ on $\mathcal{X}$ as $$\ang{m,n} = \int \tilde{k}(u,v)\,dm(u)\,dn(v).$$

and

$$\norm{s\otimes \pi_2 - \pi}^2 = Q(s,\pi_1) = \norm{s-\pi_1}^2.$$

If we're looking at $\mathcal{X}^2$, then we have
$$
\ang{m,n} = \int \tilde{k}((u_1,u_2),(v_1,v_2))\,dm(u_1,u_2)\,dn(v_1,v_2)
$$

and

$$\norm{s\otimes \pi_2 - \pi}^2 = \int \tilde{k}((u_1,u_2),(v_1,v_2))\,d(s(u_1)\pi_2(u_2)-\pi(v_1,v_2))$$

Then our problem reduces to showing $$\inf_m \left\lbrace \ang{m,s-\pi_1} + \lambda \ang{m,m}\right\rbrace  = -\frac{1}{4\lambda} \ang{s-\pi_1,s-\pi_1}.$$

\begin{proof}
	\begin{align*}
	\ang{m,s-\pi_1} + \lambda\ang{m,m} &= \lambda(\ang{m,m}+\ang{m,\frac{s-\pi_1}{\lambda}}) \\
	&= \lambda (\norm{m + \frac{s-\pi_1}{2\lambda}}^2 - \norm{\frac{s-\pi_1}{2\lambda}}^2)
	\end{align*}

	Choosing $m = \frac{\pi_1 - s}{2\lambda}$, we obtain the desired minimum.
\end{proof}
\subsection*{Discrete setting}

What if we take $\pi_1$ and $\pi_2$ to be discrete measures? Then we can represent these by $\pi_1 = \Pi 1$ and $\pi_2^T=1^{T}\Pi$, i.e. taking the row and column marginals of $\Pi$.

Our optimization problem then becomes $$\inf_{\Pi} \frac{1}{4\lambda} (\norm{s-\Pi 1}^2+\norm{t-\Pi^{T} 1}^2) + tr(A^{T}\Pi).$$

\section*{Alternative Optimization Problem}
$f(\Pi,\lambda)=\dfrac{(\Pi\vec{1}-s)^{T}K(\Pi\vec{1}-s)+(\Pi^{T}\vec{1}-t)^{T}K(\Pi^{T}\vec{1}-t)}{4\tau}-Tr[\Pi K]+\lambda(1-\vec{1}^{T}\Pi1)-v_{1}E_{1}^{T}\Pi E_{1}-v_{2}E_{2}^{T}\Pi E_{1}-v_{3}E_{1}^{T}\Pi E_{2}-v_{4}E_{2}^{T}\Pi E_{2}$

where $E_{1}=\left(\begin{array}{c}
1\\
0
\end{array}\right)$ and $E_{2}=\left(\begin{array}{c}
0\\
1
\end{array}\right)$ (assuming this is the 2D setting).

$\frac{df}{d\Pi}=2K(\Pi\vec{1}-s)\vec{1}^{T}+2K(\Pi^{T}\vec{1}-t)\vec{1}^{T}-K-\lambda\vec{1}\vec{1^{T}}=2K[(\Pi+\Pi^{T})\vec{1}-(s+t)]\vec{1}^{T}-K-\lambda11^{T}-v_{1}\left(\begin{array}{cc}
1 & 0\\
0 & 0
\end{array}\right)-v_{2}\left(\begin{array}{cc}
0 & 0\\
1 & 0
\end{array}\right)-v_{3}\left(\begin{array}{cc}
0 & 1\\
0 & 0
\end{array}\right)-v_{4}\left(\begin{array}{cc}
0 & 0\\
0 & 1
\end{array}\right)$

If $K=I$, then by KKT we have

$\frac{\pi_{11}-\pi_{22}-(s_{1}+t_{1})+1}{2\tau}-(\lambda+v_{1}+1)=0$

$\frac{\pi_{22}-\pi_{11}-(s_{2}+t_{2})+1}{2\tau}-(\lambda+v_{2})=0$

$\frac{\pi_{11}-\pi_{22}-(s_{1}+t_{1})+1}{2\tau}-(\lambda+v_{3})=0$

$\frac{\pi_{22}-\pi_{11}-(s_{2}+t_{2})+1}{2\tau}-(\lambda+v_{4}+1)=0$

And it follows that

$v_{1}+1=v_{3}$

$v_{4}+1=v_{2}$

and also

$\frac{1}{2\tau}-1=v_{1}+v_{2}+2\lambda=v_{3}+v_{4}+2\lambda$.

If $\pi_{11},\pi_{22}=\frac{s_{1}+t_{1}}{2},\frac{s_{2}+t_{2}}{2}$,
then $\pi_{11}+\pi_{22}=1$ and thus $\pi_{12}=\pi_{21}=0$. Letting
$v_{1}=v_{4}=0$ and $v_{2}=v_{3}=1$, this satisfies slackness constraints.
The previous discrete optimization


\documentclass[t]{beamer}
\usetheme{Copenhagen}
\setbeamertemplate{headline}{} % remove toc from headers
\beamertemplatenavigationsymbolsempty

\usepackage{amsmath, tikz, pgfplots, tcolorbox, xcolor, marvosym}
\pgfplotsset{compat = 1.16}

\tikzstyle{input} = [circle, text centered, radius = 1cm, draw = black]
\tikzstyle{function} = [rectangle, text centered, minimum width = 2cm, minimum height = 1cm, draw = black]

\title{Intro to Functions}
\author{}
\date{}

\AtBeginSection[]
{
  \begin{frame}
    \frametitle{Objectives}
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\begin{frame}
\maketitle
\end{frame}

\section{Determine if a relation is a function.}

\begin{frame}{Relations and Functions}
\begin{tcolorbox}[colback=red!15!white, colframe=red!60!black, title=Relations]
A \textbf{relation} is a set of ordered pairs.
\end{tcolorbox}
\vspace{1cm}	\pause

\begin{tcolorbox}[colback=red!15!white, colframe=red!60!black, title=Domain]
The \textbf{domain} is the set of all input values (usually $x$) of a relation.
\end{tcolorbox}
\end{frame}

\begin{frame}{Relations and Functions}
\begin{tcolorbox}[colback=red!15!white, colframe=red!60!black, title=Range]
The \textbf{range} is the set of all output values (usually $y$) of a relation.
\end{tcolorbox}
\vspace{1cm}	\pause
\begin{tcolorbox}[colback=red!15!white, colframe=red!60!black, title=Function]
A \textbf{function} is a relation is which each element of the domain has only 1 element in the range.
\end{tcolorbox}
\end{frame}

\begin{frame}{Example 1}
Determine whether each relation represents a function. For those that do, state the domain and range.	\newline\\
(a) \quad $\{(1,5), \, (2, 5), \, (3, 7), \, (4, 8)\}$	\newline\\
\onslide<2->{All $x$-coordinates are different.\quad}
\onslide<3->{Is a function.}	\newline\\
\onslide<4->{Domain: 1, 2, 3, 4} \newline\\
\onslide<5->{Range: 5, 7, 8}
\end{frame}

\begin{frame}{Example 1}
(b) \quad $\{(5,1), \, (5,2), \, (7,3), \, (8,4)\}$	\newline\\
\onslide<2->{$x$-coordinates are not all different. \quad}
\onslide<3->{Is \textbf{not} a function.}
\end{frame}

\begin{frame}{Vertical Line Test}
It is also possible to determine if a relation is a function visually by using the \alert{vertical line test}:	\newline\\	\pause

\begin{tcolorbox}[colback=white!50!green, title=\textbf{Vertical Line Test}]
If every vertical line drawn hits the graph \underline{\textbf{at most once}}, then the relation is a function.
\end{tcolorbox}
\end{frame}

\begin{frame}{Example 1a Passes V.L.T.}
\begin{center}
\begin{tikzpicture}
\begin{axis}[
	axis lines = middle,
	grid=both,
	xmin = 0, xmax = 5.5,
	ymin = 0, ymax = 10.5
]
\addplot[color=blue, mark = *, only marks] coordinates {(1,5) (2,5) (3,7) (4,8)};
\end{axis}
\end{tikzpicture}
\end{center}
\end{frame}

\begin{frame}{Example 1b Fails V.L.T.}
\begin{center}
\begin{tikzpicture}
\begin{axis}[
	axis lines = middle,
	grid=both, minor tick num = 1,
	xmin = 0, xmax = 10.5,
	ymin = 0, ymax = 5.5
]
\addplot[color=blue, mark = *, only marks] coordinates {(5,1) (5,2) (7,3) (8,4)};
\end{axis}
\end{tikzpicture}
\end{center}
\end{frame}

\begin{frame}{Example 2}
Determine whether the graph of each represents a function.	\newline\\
(a) \newline\\
\begin{minipage}{0.5\textwidth}
\begin{tikzpicture}[domain = -1.5:1.5]
	\draw [<->] (-2,0) -- (2,0);
	\node at (2,0) [anchor = west] {\tiny $x$};
	\draw [<->] (0,-2) -- (0,2);
	\node at (0,2) [anchor = west] {\tiny $y$};
	\draw [<->, color = blue, very thick] plot(\x, {-1/2*\x-1/2});
\end{tikzpicture}
\end{minipage}
\begin{minipage}{0.4\textwidth}
\onslide<2->{Is a function}
\end{minipage}
\end{frame}

\begin{frame}{Example 2}
(b) \newline\\
\begin{minipage}{0.5\textwidth}
\begin{tikzpicture}[domain = -1.5:1.5]
	\draw [<->] (-2,0) -- (2,0);
	\node at (2,0) [anchor = west] {\tiny $x$};
	\draw [<->] (0,-2) -- (0,2);
	\node at (0,2) [anchor = west] {\tiny $y$};
	\draw [<->, color = blue, very thick] plot(\x, {-1/2*\x*\x+1/2});
\end{tikzpicture}
\end{minipage}
\begin{minipage}{0.4\textwidth}
\onslide<2->{Is a function}
\end{minipage}
\end{frame}

\begin{frame}{Example 2}
(c) \newline\\
\begin{minipage}{0.5\textwidth}
\begin{tikzpicture}
	\draw [<->] (-2,0) -- (2,0);
	\node at (2,0) [anchor = west] {\tiny $x$};
	\draw [<->] (0,-2) -- (0,2);
	\node at (0,2) [anchor = west] {\tiny $y$};
	\draw [color = blue, very thick] (0,0) ellipse (1.5 and 0.75);
\end{tikzpicture}
\end{minipage}
\begin{minipage}{0.4\textwidth}
\onslide<2->{Is not a function}
\end{minipage}
\end{frame}

\section{Evaluate a function using function notation.}

\begin{frame}{Functions}
Think of a function as a \alert{machine}. \newline\\	\pause
You give the function (machine) a value (input), it will process that value, and then return a value back to you (output). \newline\\	\pause

For instance, if you input 10 into the $x^2$ function, it will return $10^2$, or 100:	\newline\\
\begin{center}
\begin{tikzpicture}[node distance = 2.5 cm]
\node (inputVal) [input, color=blue] {\color{blue}\textbf{10}};
\node (func) [function, right of = inputVal] {$x^2$};
\node (outputVal) [input, right of = func] {\color{red}\textbf{100}};

\draw [->, >=stealth, thick, line width = 1.5] (inputVal) -- (func);
\draw [->, >=stealth, thick, line width = 1.5] (func) -- (outputVal);
\end{tikzpicture}
\end{center}
\end{frame}

\begin{frame}{Function Notation}
A function can be described using \alert{function notation}. \newline\\ \pause
$f(x)$ represents the value of the function when the value of $x$ is substituted into it.	\newline\\	\pause
We can use other notations for functions including, but not limited to
\[ g(x) \quad h(x), \quad f(n) \quad f\left(\text{\Smiley}\right) \]
\pause \newline\\
When we substitute a value for the variable and evaluate it, that is called {\color{blue}\textbf{evaluating the function}}.
\end{frame}

\begin{frame}{Example 3}
Evaluate $f(2), \, f(-2), \, \text{and } f(0)$ for each.	\newline\\
(a) \quad $f(x) = 2x+3$
\begin{align*}
\onslide<2->{f(2) &= 2(2) + 3} \\
\onslide<3->{&= 7} \\[10pt]
\onslide<4->{f(-2) &= 2(-2) + 3} \\
\onslide<5->{&= -1} \\[10pt]
\onslide<6->{f(0) &= 2(0) + 3} \\
\onslide<7->{&= 3}
\end{align*}
\end{frame}

\begin{frame}{Example 3}
Evaluate $f(2), \, f(-2), \, \text{and } f(0)$ for each.	\newline\\
(b) \quad $f(x) = 3x^2-1$
\begin{align*}
\onslide<2->{f(2) &= 3(2)^2-1} \\
\onslide<3->{&= 11} \\[10pt]
\onslide<4->{f(-2) &= 3(-2)^2-1} \\
\onslide<5->{&= 11} \\[10pt]
\onslide<6->{f(0) &= 3(0)^2-1} \\
\onslide<7->{&= -1}
\end{align*}
\end{frame}

\begin{frame}{Example 3}
(c) \newline\\
\begin{minipage}{0.6\textwidth}
\begin{tikzpicture}[scale=0.8, domain = -2.15:2.15]
	\draw [step = 0.5cm, color = gray!110, dotted] (-3.5,-3.5) grid (3.5,3.5);
	\draw[<->, > = latex] (-3.5,0) -- (3.5,0);
	\node at (3.5,0) [anchor = west] {\tiny $x$};
	\draw[<->, > = latex] (0,-3.5) -- (0,3.5);
	\node at (0,3.5) [anchor = south west] {\tiny $y$};
	\foreach \x in {-3,-2,-1,1,2,3}
	\draw[shift = {(\x,0)}] (0pt,2pt) -- (0pt, -2pt) node[below] {\footnotesize $\x$};
	\foreach \y in {-3,-2, -1, 1, 2, 3}
	\draw[shift = {(0,\y)}] (2pt,0pt) -- (-2pt,0pt) node[left] {\footnotesize $\y$};
	\draw [<->, > = latex, color = blue, very thick]	plot (\x, {\x*\x - 1});
\end{tikzpicture}
\end{minipage}
\begin{minipage}{0.25\textwidth}
\begin{align*}
\onslide<2->{f(2) &= 3} \\[12pt]
\onslide<3->{f(-2) &= 3} \\[12pt]
\onslide<4->{f(0) &= -1}
\end{align*}
\end{minipage}
\end{frame}

\end{document}


\documentclass[jou]{apa6}

\usepackage[american]{babel}

\usepackage{csquotes}
\usepackage[style=apa,sortcites=true,sorting=nyt,backend=biber]{biblatex}
\DeclareLanguageMapping{american}{american-apa}
\addbibresource{bibliography.bib}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Discrete Structures
%% The start of RBS stuff
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Working internal and external links in PDF
\usepackage{hyperref}
% Extra math symbols in LaTeX
\usepackage{amsmath}
\usepackage{gensymb}
\usepackage{amssymb}
% Enumerations with (a), (b), etc.
\usepackage{enumerate}

\let\OLDitemize\itemize
\renewcommand\itemize{\OLDitemize\addtolength{\itemsep}{-6pt}}

\usepackage{etoolbox}
\makeatletter
\preto{\@verbatim}{\topsep=3pt \partopsep=3pt }
\makeatother

% These sizes redefine APA for A4 paper size
\oddsidemargin 0.0in
\evensidemargin 0.0in
\textwidth 6.27in
\headheight 1.0in
\topmargin -24pt
\headheight 12pt
\headsep 12pt
\textheight 9.19in

\title{Sample Quiz 8}
\author{Discrete Structures, Spring 2020}
\affiliation{RBS}

\leftheader{Discrete Sample Quiz 8}

\abstract{%
}

%\keywords{}

\setlength\parindent{0pt}

\begin{document}

%\thispagestyle{empty}

\twocolumn
\section{Worksheet 9: Advanced Counting}

\vspace{10pt}
{\bf Question 1 (Rosen7e, Ch.8, Q4).}
Denote by $a_n$ the number of ways to go down an $n$-step staircase if you go down $1$, $2$, or $3$ steps at a time.
(In other words, $a_n$ denotes the number of ways how $n$ can be expressed as a sum of $1,2,3$, if the
order in the sum matters.)\\
Define $a_n$ as a recurrent sequence. Assume that the sequence starts from $a_1$.

\vspace{10pt}
{\bf Question 2 (Rosen7e, Ch.8, Q11).}
A vending machine accepts only \$1 coins, \$1 bills, and \$2 bills. Let $a_n$ denote the
number of ways of depositing $n$ dollars in the vending machine, where the order in which the coins and bills
are deposited matters.\\
{\bf (A)} Find a recurrence relation for $a_n$ and give the necessary initial condition(s).\\
{\bf (B)} Find an explicit formula for $a_n$ by solving the recurrence relation in part {\bf (A)}.

\vspace{10pt}
{\bf Question 3 (Rosen7e, Ch.8, Q16-Q20).}
For each item solve the recurrence relation (characteristic equation
or simply guess the pattern for the terms):\\
{\bf (A)} $a_n = a_{n-2}$, $a_0 = 2$, $a_1 = -1$.\\
{\bf (B)} $a_n = 2a_{n-1} + 2a_{n-2}$, $a_0 = 0$, $a_1 = 1$.\\
{\bf (C)} $a_n = 3na_{n-1}$, $a_0 = 2$.\\
{\bf (D)} $a_n = a_{n-1} + 3n$, $a_0 = 5$.\\
{\bf (E)} $a_n = 2a_{n-1} + 5$, $a_0 = 3$.

\vspace{10pt}
{\bf Question 4 (Rosen7e, Ch.8, Q24)}
Assume that the characteristic equation for a homogeneous
linear recurrence relation with constant coefficients
is $(r + 2)(r + 4)^2 = 0$.\\
{\bf (A)} Describe the form for the general solution to the recurrence relation.\\
{\bf (B)} Define a recurrent sequence that leads to this characteristic equation.

\vspace{10pt}
{\bf Question 5 (Rosen7e, Ch.8, Q27)}
The Catalan numbers $C_n$ count the number of strings of $n$ pluses ($+$) and $n$ minuses ($-$)
with the following property: as each
string is read from left to right, the number of pluses encountered is always
at least as large as the number of minuses.\\
{\bf (A)} Verify this by listing these strings of lengths $2$,
$4$, and $6$ and showing that there are $C_1$, $C_2$, and $C_3$ of
these, respectively.\\
{\bf (B)} Explain how counting these strings is the same as counting
the number of ways to correctly parenthesize strings of variables.

{\em Note.} Catalan numbers
$$C_0=1,\; C_1=1,\; C_2=2,\; C_3=5,\; C_4=14,\ldots$$
are defined as ${\displaystyle C_n = \frac{1}{n+1}{2n \choose n}}$.

\vspace{10pt}
{\bf Question 6  (Rosen7e, Ch.8, Q28, Q29)}\\
{\bf (A)} What form does a particular solution of the linear nonhomogeneous
recurrence relation $a_n = 4a_{n-1} - 4a_{n-2} + F(n)$ have when $F(n) = 2^n$?\\
{\bf (B)} What form does a particular solution of the linear nonhomogeneous
recurrence relation $a_n = 4a_{n-1} - 4a_{n-2} + F(n)$ have when $F(n) = n2^n$?

\vspace{10pt}
{\bf Question 7 (Rosen7e, Ch.8, Q32).}
Consider the recurrence relation $a_n = 2a_{n-1} + 3n$.\\
{\bf (A)} Write the associated homogeneous recurrence relation.\\
{\bf (B)} Find the general solution to the associated homogeneous recurrence relation.\\
{\bf (C)} Find a particular solution to the given recurrence relation.\\
{\bf (D)} Write the general solution to the given recurrence relation.\\
{\bf (E)} Find the particular solution to the given recurrence relation when $a_0 = 1$.

\vspace{10pt}
{\bf Question 8 (Rosen7e, Ch.8, Q37).}
Suppose $f(n) = f(n/3) + 2n$, $f(1) = 1$. Find $f(27)$.

\vspace{10pt}
{\bf Question 9  (Rosen7e, Ch.8, Q53-Q63)}
Find the coefficient of $x^8$ in the power series of each of the function:\\
{\bf (A)} $(1 + x^2 + x^4)^3$.\\
{\bf (B)} $(1 + x^2 + x^4 + x^6)^3$.\\
{\bf (C)} $(1 + x^2 + x^4 + x^6 + x^8)^3$.\\
{\bf (D)} $(1 + x^2 + x^4 + x^6 + x^8 + x^{10})^3$.\\
{\bf (E)} $(1 + x^3)^{12}$.\\
{\bf (F)} $(1 + x)(1 + x^2)(1 + x^3)(1 + x^4)(1 + x^5)$.\\
{\bf (G)} $1/(1 - 2x)$.\\
{\bf (H)} $x^3/(1 - 3x)$.\\
{\bf (I)} $1/(1 - x)^2$.\\
{\bf (J)} $x^2/(1 + 2x)^{2}$.\\
{\bf (K)} $1/(1 - 3x^2)$.

\newpage

\subsection{Answers}

\vspace{10pt}
{\bf Question 1.} For staircases with $n = 1,2,3$ steps we
can enumerate all possible sums and count them.\\
$1$-step staircases: $1$.\\
$2$-step staircases: $1+1$, $2$.\\
$3$-step staircases: $1+1+1$, $1+2$, $2+1$, $3$.

$$\left\{ \begin{array}{l}
a_1 = 1, \\
a_2 = 2, \\
a_3 = 4, \\
a_{n} = a_{n-1} + a_{n-2} + a_{n-3},\;n \geq 4.\\
\end{array} \right.$$

Why $a{n}$ can be expressed as sum of $a_{n-1}$, $a_{n-2}$, $a_{n-3}$?\\
Let us consider the first movement downstairs. It can be either $1$ step down (then the
remaining steps can be made in $a_{n-1}$ different ways), or $2$ steps down
(then the remaining steps can be made in $a_{n-2}$ different ways),
or $3$ steps down (then the remaining steps can be made in $a_{n-3}$ different ways).

All these options do not intersect (since any moving downstairs starts with either $1$, $2$ or $3$),
therefore we simply add all these ways together: $a_{n-1} + a_{n-2} + a_{n-3}$.

\vspace{10pt}
{\bf Question 2.}\\
{\bf (A)} Let us find the initial conditions (values $a_1$ and $a_2$) and
the recurrence relation (how to express $a_n$ via the the two previous values).
$$\left\{ \begin{array}{l}
a_1 = 2, \\
a_2 = 5, \\
a_{n} = 2a_{n-1} + a_{n-2},\;n \geq 3.\\
\end{array} \right.$$
The initial conditions can be verified by noticing that 1 dollar can be paid in two ways
(a coin or a bill); 2 dollars can be paid in five ways
(either one \$2 bill or any of the following: coin+coin, coin+bill, bill+coin, bill+bill).\\
If $n \geq 3$, then $a_n$ either starts by a \$1 coin (plus paying the remaining sum in $a_{n-1}$ different ways)
or by a \$1 bill (plus paying the remaining sum in $a_{n-1}$ different ways), or by
a \$2 bill (plus paying the remaining sum in $a_{n-2}$ different ways).
By adding together these mutually exclusive options we get
$$a_{n} = a_{n-1} + a_{n-1} + a_{n-2} = 2a_{n-1} + a_{n-2}.$$

\vspace{10pt}
{\bf Question 3.} \\
{\bf (A)} Characteristic equation: $r^2 - 1 = 0$.
The roots are $r_1 = -1$ and $r_2 = 1$.
So, the general form of this sequence is
$$a_n = A \cdot (-1)^n + B \cdot 1^n.$$
To get $a_0 = 2$ and $a_1 = -1$, we need $A = \frac{3}{2}$
and $B = \frac{1}{2}$. Therefore,
$$a_n = \frac{3}{2} \cdot (-1)^n + \frac{1}{2}.$$
This expression would generate our alternating
sequence
$$2,-1,2,-1,2,-1,\ldots$$

{\bf (B)} The characteristic equation is
$$r^2 - 2r - 2 = 0.$$
And the roots of this square equation are
$r_{1,2} = 1 \pm \sqrt{3}$.

We need to find constants $A$ and $B$ such that the formula
$$a_n = A\cdot r_1^n + B\cdot r_2^n$$
is correct and we get $a_0 = 0$ and $a_1 = 1$.
By substituting $n=0$ and $n=1$ we get this system:
$$\left\{ \begin{array}{l}
A + B = 0\\
A(1 + \sqrt{3}) + B(1 - \sqrt{3}) = 1\\
\end{array} \right.$$
We get that $B = -A$ and $2A\sqrt{3} = 1$.
Therefore $A = \frac{1}{2\sqrt{3}}$ and
$B = -\frac{1}{2\sqrt{3}}$.
The formula to compute $a_n$ is therefore this:
$$a_n = \frac{1}{2\sqrt{3}}\left(1 + \sqrt{3} \right)^n
-\frac{1}{2\sqrt{3}}\left(1 - \sqrt{3} \right)^n.$$

By plugging into this formula numbers from $[0;10]$,
we can compute the first members of the sequence:
$$0,1,2,6,16,44,120,328,896,2448,6688,\ldots$$

{\bf (C)} $a_n$ can be obtained from $a_0 = 2$,
if we multiply $n$ times by $3$ (i.e.\ by $3^n$),
and also by $1 \cdot 2 \cdot \ldots \cdot n = n!$.
Therefore, we get
$$a_n = 2 \cdot 3^n \cdot n!.$$

{\bf (D)} $a_n$ can be obtained from $a_0 = 5$ by
adding some members of an arithmetic progression:
$$a_n = a_0 + (3 + 6 + \ldots + (3n)) = 5 + \frac{3n(n+1)}{2}.$$

\vspace{10pt}
{\bf Question 4}\\
{\bf (A)} $a_n = A\cdot(-2)^n + B\cdot(-4)^n + C\cdot n \cdot (-4)^4$.\\
{\bf (B)} $(r + 2)(r + 4)^2 = (r + 2)(r^2 + 8r + 16) = r^3 + 8r^2 + 16r + 2r^2 + 16r + 32 = r^3 + 10r^2 + 32r + 32$.
From that cubic equation we can get this recurrent relationship:
$$a_{n} = -10a_{n-1} - 32a_{n-2} - 32a_{n-3}.$$
Initial conditions (values for the first three members, say, $a_1$, $a_2$, $a_3$ can be selected in any way).

\end{document}