\section{Introduction}
\label{sec:This}

One of the advantages often attributed to purely functional programming languages is that they are great for \gls{verification} and \emph{equational reasoning} \cite{Wadler87}.
When I started learning to program in a functional language, I didn't know precisely what reasoning about program is and how it can be used to verify certain \emph{properties} of a program. I also didn't know what kind of properties of a program can be verified with formal methods. In order to understand equational reasoning I wrote this article.

\subsection{Why use equational reasoning?}

Equational reasoning can be used as a formal method for software verification. It's a way to prove the correctness of a program. Correctness means that specified properties of the program hold for an arbitrary input.

A property of a program can be specified in the form of an equation. For example, the following equation states a property of the function \verb|fmap|  \footnote{The function \verb|fmap| takes a function \verb|a -> b| as its first parameter and a container that contains elements of type \verb|a|. It returns a container with elements of type \verb|b|. This is called \emph{mapping}. The type of \verb|fmap| is \verb|(a -> b) -> f a -> f b|.}.
\begin{equation}
  \label{eq:firstfunctorlaw}
\text{fmap } \text{id}  =  \text{id}
\end{equation}
 The function \verb|id| is the identity function. The property represented by equation \ref{eq:firstfunctorlaw} says that applying the function \verb|fmap| with the arguments \verb|id| and an arbitrary container is the same as applying the function \verb|id| on a container. Simply put, mapping the identity function over every item in a container has no effect. Given a function definition of \verb|fmap|, we can check if this property holds, using equational reasoning.

The goal of equational reasoning is to  provide certainty that a program has particular properties. Proof of required properties contributes to the reliability and robustness of a software.
For example, the first formally verified micro kernel, seL4, was verified using Haskell and formal verification to ensure proper security functionality and security assurance \cite{Klein09}.
Another example is the streaming library "pipes". The author, Gabriel Gonzales, used equational reasoning to prove the correctness of the library \cite{gonzales13}.

\subsection{Why are functional programming languages great for equational reasoning?}

An equation is a statement that signifies the equality of two expressions (e.g. $1 + 3 = 4$). Reasoning is the process of thinking about something in a logical way in order to form a conclusion. The technique of equational reasoning implies that we show correctness by showing that two expressions of a program are equal.

The reason we can use equational reasoning in functional programming is \gls{referential_transparency}. A piece of code can be replaced by its value; this is true in purely functional languages. The absence of side effects allows us to replace the left-hand side of a definition with the right-hand side and vice versa. For example if you have the following Haskell definition:
\begin{verbatim}
x = 23
\end{verbatim}
That means that any \verb|x| can be substituted by \verb|23|.

This is not possible in programs written in an imperative programming language because there the value of an expression depends on the context of the execution environment. In an imperative language, \verb|x = 23| is an assignment, not an equality. The variable \verb|x| is mutable and the value of the expression \verb|x| can change at runtime. Hence, we must not substitute \verb|x| with \verb|23| in your program because arbitrary values could be assigned to the variable \verb|x| in other statements.

Haskell is a purely functional language, which means that functions do not have side effects. Hence, we can verify programs written in Haskell using equational reasoning. For this reason, all examples in this article are written in Haskell.

\subsection{What properties will we discuss?}

If we want to verify a program, we have to define one or more properties. These properties are part of the program specification. We could define arbitrary properties, but this article mainly deals with properties derived from the so called \emph{type class laws}. In Haskell, type class laws are exhibited by \glspl{typeclass}. For example, the property formed by equation \ref{eq:firstfunctorlaw} is a type class law of the type class \verb|Functor| (cf. appendix \ref{sec:functor}).

Type classes are a concept to describe the behavior of a type.
They are very similar to interfaces. If we want to make a type an instance of a type class, we have to implement the functions given by the \glspl{typesignature} of the type class. Figure \ref{fig:typeclassrelation} illustrates the relation between values, types and type classes in Haskell. A type can be a member of several type classes. A type class can contain several types. A value has exactly one type but several values can be of the same type.

An instance of a type class must obey the corresponding type class laws. These laws allow us to make assumptions about the behavior of the program and to reason about the code. Properties of existing code allow us to rely on expected behavior and deduce further properties for new code. For example, if the property formed by equation \ref{eq:firstfunctorlaw} holds, we can rest assured that \verb|fmap| only applies the function to the items in the container and has no additional effects. If an implementation of \verb|fmap| would change the structure of the container, it would break the property of equation \ref{eq:firstfunctorlaw}.

The Haskell compiler does not enforce type class laws. We need to verify them ourselfes when implementing an instance of a type class. We can use equational reasoning to prove that an implementation of a type class obeys the corresponding laws.

\tikzset{multi  attribute/.style={attribute ,double  distance=1.5pt}}
\tikzset{derived  attribute/.style={attribute ,dashed}}
\tikzset{total/.style={double  distance=1.5pt}}
\tikzset{every  entity/.style={draw=orange , fill=orange!20}}
\tikzset{every  attribute/.style={draw=MediumPurple1, fill=MediumPurple1!20}}
\tikzset{every  relationship/.style={draw=Chartreuse2, fill=Chartreuse2!20}}

\begin{figure}
\centering
\begin{tikzpicture}[node distance=10em]
  \node[entity](type){type};
  \node[entity](typeclass)[right of=type]{type class};
  \node[entity](value)[left of=type]{value};
  \draw (type) -- (typeclass) node [very near end, above=1pt] {*} node [very near start, above=1pt] {*};
\draw (value) -- (type) node [very near end, above=1pt] {1} node [very near start, above=1pt] {*};

\end{tikzpicture}
\caption{Relation between value, type and type class}
\label{fig:typeclassrelation}
\end{figure}
\subsection{Overview}

This article will illustrate in detail how equational reasoning works in practice using the type class laws of the type class \verb|Monoid| as the running example. The article doesn't require prior knowledge of type classes or \glspl{monoid}. In order to understand every step of the process, the next two sections will explain the fundamental concepts.
Section \ref{sec:typeclasses} will give a short introduction to type classes and section \ref{sec:monoid} describes monoids. In section \ref{sec:equationalreasoning} we will give an introduction to equational reasoning in general. In section \ref{sec:example} we will prove that an implementation of the \verb|Monoid| type class obeys the first monoid law.

This article give answers to the following questions:
\begin{itemize}
\item What are desired properties of a program? Where do they come from? Why are they useful?
\item What is equational reasoning? What is the difference between testing and property proving? How do we use equational reasoning to verify properties of a program?
\end{itemize}


\documentclass[12pt]{article}

\include{preamble}

\title{Math 341 / 650 Spring 2019 \\ Midterm Examination One}
\author{Professor Adam Kapelner}

\date{Tuesday, March 5, 2019}

\begin{document}
\maketitle

\noindent Full Name \line(1,0){410}

\thispagestyle{empty}

\section*{Code of Academic Integrity}

\footnotesize
Since the college is an academic community, its fundamental purpose is the pursuit of knowledge. Essential to the success of this educational mission is a commitment to the principles of academic integrity. Every member of the college community is responsible for upholding the highest standards of honesty at all times. Students, as members of the community, are also responsible for adhering to the principles and spirit of the following Code of Academic Integrity.

Activities that have the effect or intention of interfering with education, pursuit of knowledge, or fair evaluation of a student's performance are prohibited. Examples of such activities include but are not limited to the following definitions:

\paragraph{Cheating} Using or attempting to use unauthorized assistance, material, or study aids in examinations or other academic work or preventing, or attempting to prevent, another from using authorized assistance, material, or study aids. Example: using an unauthorized cheat sheet in a quiz or exam, altering a graded exam and resubmitting it for a better grade, etc.
\\

\noindent I acknowledge and agree to uphold this Code of Academic Integrity. \\

\begin{center}
\line(1,0){250} ~~~ \line(1,0){100}\\
~~~~~~~~~~~~~~~~~~~~~signature~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ date
\end{center}

\normalsize

\section*{Instructions}

This exam is seventy five minutes and closed-book. You are allowed \textbf{one} page (front and back) of a \qu{cheat sheet.} You may use a graphing calculator of your choice. Please read the questions carefully. If the question reads \qu{compute,} this means the solution will be a number otherwise you can leave the answer in \textit{any} widely accepted mathematical notation which could be resolved to an exact or approximate number with the use of a computer. I advise you to skip problems marked \qu{[Extra Credit]} until you have finished the other questions on the exam, then loop back and plug in all the holes. I also advise you to use pencil. The exam is 100 points total plus extra credit. Partial credit will be granted for incomplete answers on most of the questions. \fbox{Box} in your final answers. Good luck!

\pagebreak

\begin{table}[htp]
\centering
\small
\begin{tabular}{l | llll}
Distribution                  & Quantile  & PMF / PDF  &CDF       & Sampling  \\
of r.v. &  Function & function         & function &  Function \\ \hline
beta & \texttt{qbeta}($p$, $\alpha$, $\beta$)
& \texttt{d-}($x$, $\alpha$, $\beta$)
& \texttt{p-}($x$, $\alpha$, $\beta$)
& \texttt{r-}($\alpha$, $\beta$) \\
betabinomial & \texttt{qbetabinom}($p$, $n$, $\alpha$, $\beta$)
& \texttt{d-}($x$, $n$, $\alpha$, $\beta$)
& \texttt{p-}($x$, $n$, $\alpha$, $\beta$)
& \texttt{r-}($n$, $\alpha$, $\beta$) \\

betanegativebinomial & \texttt{qbeta\_nbinom}($p$, $r$, $\alpha$, $\beta$)
& \texttt{d-}($x$, $r$, $\alpha$, $\beta$)
& \texttt{p-}($x$, $r$, $\alpha$, $\beta$)
& \texttt{r-}($r$, $\alpha$, $\beta$) \\

binomial & \texttt{qbinom}($p$, $n$, $\theta$)
& \texttt{d-}($x$, $n$, $\theta$)
& \texttt{p-}($x$, $n$, $\theta$)
& \texttt{r-}($n$, $\theta$) \\

exponential & \texttt{qexp}($p$, $\theta$)
& \texttt{d-}($x$, $\theta$)
& \texttt{p-}($x$, $\theta$)
& \texttt{r-}($\theta$) \\

gamma & \texttt{qgamma}($p$, $\alpha$, $\beta$)
& \texttt{d-}($x$, $\alpha$, $\beta$)
& \texttt{p-}($x$, $\alpha$, $\beta$)
& \texttt{r-}($\alpha$, $\beta$) \\

geometric & \texttt{qgeom}($p$, $\theta$)
& \texttt{d-}($x$, $\theta$)
& \texttt{p-}($x$, $\theta$)
& \texttt{r-}($\theta$) \\

inversegamma & \texttt{qinvgamma}($p$, $\alpha$, $\beta$)
& \texttt{d-}($x$, $\alpha$, $\beta$)
& \texttt{p-}($x$, $\alpha$, $\beta$)
& \texttt{r-}($\alpha$, $\beta$) \\

negative-binomial & \texttt{qnbinom}($p$, $r$, $\theta$)
& \texttt{d-}($x$, $r$, $\theta$)
& \texttt{p-}($x$, $r$, $\theta$)
& \texttt{r-}($r$, $\theta$) \\

normal (univariate) & \texttt{qnorm}($p$, $\theta$, $\sigma$)
& \texttt{d-}($x$, $\theta$, $\sigma$)
& \texttt{p-}($x$, $\theta$, $\sigma$)
& \texttt{r-}($\theta$, $\sigma$) \\

%normal (multivariate) &
%& \multicolumn{2}{l}{\texttt{dmvnorm}($\x$, $\muvec$, $\bSigma$)}
%& \texttt{r-}($\muvec$, $\bSigma$) \\

poisson & \texttt{qpois}($p$, $\theta$)
& \texttt{d-}($x$, $\theta$)
& \texttt{p-}($x$, $\theta$)
& \texttt{r-}($\theta$) \\

T (standard) & \texttt{qt}($p$, $\nu$)
& \texttt{d-}($x$, $\nu$)
& \texttt{p-}($x$, $\nu$)
& \texttt{r-}($\nu$) \\

%T (nonstandard) & \texttt{qt.scaled}($p$, $\nu$, $\mu$, $\sigma$)
%& \texttt{d-}($x$, $\nu$, $\mu$, $\sigma$)
%& \texttt{p-}($x$, $\nu$, $\mu$, $\sigma$)
%& \texttt{r-}($\nu$, $\mu$, $\sigma$) \\

uniform & \texttt{qunif}($p$, $a$, $b$)
& \texttt{d-}($x$, $a$, $b$)
& \texttt{p-}($x$, $a$, $b$)
& \texttt{r-}($a$, $b$) \\

\end{tabular}
\caption{Functions from $\texttt{R}$ (in alphabetical order) that can be used on this exam with their arguments. The hyphen in colums 3, 4 and 5 is shorthand notation for the full text of the r.v. which can be found in column 2.
}
\label{tab:eqs}
\end{table}

\problem Let $\theta$ represent the true probability a specific couple (i.e. a mother and father) will have a male baby. Assume the model $\mathcal{F} = \binomial{n}{\theta}$ where $n$ is the number of children this mother-father couple have.

\begin{figure}[htp]
\centering
\includegraphics[width=1.8in]{family.jpg}
\end{figure}

\benum

\subquestionwithpoints{6} Describe what the assumption $\mathcal{F}$ means in this real-world context for the couple John and Susan. Indicate what is known/unknown.\spc{6}

\subquestionwithpoints{5} John and Susan do not have any kids yet. Using the principle of indifference, what is a distribution that describes the uncertainty in their $\theta$?\spc{1}

\subquestionwithpoints{6} John and Susan have four kids which are all girls. Assuming the prior in (b), provide the explicit PMF or PDF of the distribution that describes the uncertainty in their $\theta$ after you see the gender of their first four kids. Simplify. \spc{2}

\subquestionwithpoints{7} Plot this PMF or PDF (as best as you can) in the space below. It does not have to be drawn to scale but label the axes and important points on the axes. \spc{7}

\subquestionwithpoints{6} Assuming the prior in (b) and the data in (c), find all three Bayesian point estimates for John and Susan's $\theta$ and notate them appropriately. Mark them on the plot in (d). \spc{3}

\subquestionwithpoints{4} Assuming the prior in (b) and the data in (c), what is the distribution of $X^*$, the r.v. modeling the gender of John and Susan's fifth child? \spc{1}

\subquestionwithpoints{4} Assuming the prior in (b) and the data in (c), what is your best guess (the one that minimizes mean squared error) of the gender of John and Susan's fifth child? \spc{1}

\subquestionwithpoints{10} Assuming the prior in (b) and the data in (c), find a credible region for $\theta$. Notate your answer appropriately. Then estimate where this region would be on your plot in (d) \emph{as best as you can}. \spc{4}

\subquestionwithpoints{10} Assuming the prior in (b) and the data in (c), test the theory that their preference for girl or boy births is uneven \emph{as best as you can}. State the hypotheses clearly using mathematical notation. Find the Bayesian $p_{val}$ if possible.\spc{7}

\subquestionwithpoints{5} Although the prior in (b) is reasonable and the data in (c) is very possible, scientists believe the point estimates you found in (e) are too low. They can say this because they examined tons of data from couples before John and Susan. In this case, we can incorporate that prior observation into a new prior:  $\prob{\theta} = \betanot{34}{32}$. Assuming this prior and the data in (c), find all three Bayesian point estimates for John and Susan's $\theta$ and notate them appropriately. Round to two decimal places. \spc{4}

\subquestionwithpoints{5} One of the three Bayesian point estimates in (j) is a \qu{shrinkage estimator}. Which one? Calculate the degree of shrinkage. Round to two decimal places. \spc{1}

\subquestionwithpoints{7} Is the prior in (j) \qu{informative} or \qu{uninformative} in context of the data in (c)? Provide mathematical evidence for your answer. \spc{3}

\eenum

\problem Consider $\Xoneton \iid \betanot{\theta}{1}$.

\benum

\subquestionwithpoints{6} Find $\mathcal{L}\parens{\theta ; \Xoneton}$. Simplify so that your answer does not include the $B(\cdot,\cdot)$ function or the $\Gamma(\cdot)$ function.\spc{4}

\subquestionwithpoints{4} Find $\loglik{\theta ; \Xoneton}$. Simplify as much as possible.\spc{6}

\subquestionwithpoints{3} Find $\thetahatmle$. \spc{3}

\eenum

\problem Below are some theoretical problems that are independent from each other.

\benum

\subquestionwithpoints{6} In the frequentist perspective where no randomness is allowed in $\theta$, prove that $\prob{X} = \cprob{X}{\theta = \theta^*}$ where $\theta^*$ is the true value of $\theta$.\spc{4}

\subquestionwithpoints{6} If $\mathcal{F} = \iid \bernoulli{\theta}$, prove that $\cprob{X_2}{\theta,X_1} = \cprob{X_2}{\theta}$.\spc{6}

\subquestionwithpoints{3} [Extra credit] Demonstrate for $H_0 : \theta \leq \theta_0$ or $H_0 : \theta \geq \theta_0$ that $\cprob{X}{H_0} = \oneover{\prob{H_0}} \int_{H_0} \cprob{X}{\theta} \prob{\theta} d\theta$.

\eenum

\end{document}

\problem Consider the $\iid$ Bernoulli model if $\theta$ known, $n=2$, $\Theta_0 = \braces{0.5, 0.9}$ and the prior of indifference. Note that $\X$ refers the vector of the $n=2$ random variables.

\benum

\subquestionwithpoints{10} Illustrate the space $\mathcal{\X} \times \Theta$ to scale and properly denoted as we did in class. \spc{16}

\eenum

\problem Imagine you are a celebrity who has a instagram page where you post photos. This is important to you so you own a business account. Here is typically what you see:

\begin{figure}[htp]
\centering
\includegraphics[width=2.25in]{instagram.png}
\end{figure}

\noindent In case you can't read the text in the image, the \qu{reach} is how many unique people viewed your photo post and the \qu{engagement} is the number of people who liked it or commented on it.

The proportion of engagement : reach is a critical number --- it demonstrates how hooked your audience is. We'll call this $\theta$ going forward and we'll try to estimate it. As of now, we do not have any reason to exclude any values from $\Theta$.

We'll assume each unique person who visits the post of the photo as an independent Bernoulli trial where 1 represents they engaged with the post and 0 represents that they did not engage.

\benum

\subquestionwithpoints{4} What would the uninformative prior be for $\theta$? \spc{1}

\subquestionwithpoints{4} Assume the uninformative prior. Your post just got 10 views but no engagement. What is your best guess of $\theta$ right now using the Bayesian formulation? \spc{0.5}

\subquestionwithpoints{4} You just got another 10 views but no engagement. What is your best guess of $\theta$ right now (at $n=20$) using the Bayesian formulation? \spc{1}

\subquestionwithpoints{4} What is your best guess of $\theta$ right now (at $n=20$) using the Frequentist formulation? \spc{1}

\subquestionwithpoints{4} Why is the Frequentist formulation not realistic? \spc{3}

\subquestionwithpoints{4} After these 20 observations, what is the posterior distribution? Notate it and its distribution correctly below. \spc{1}

\subquestionwithpoints{6} Plot the PDF of the posterior distribution as best as you can. Include important tick marks on both the $x$ and $y$ axes as well as labels for both the $x$ and $y$ axes. \spc{6}

\subquestionwithpoints{6} Create a 98\% credible region for $\theta$. Use notation from Table~\ref{tab:eqs}. \spc{3}

\subquestionwithpoints{2} Is the interval you just created also the 98\% HDR? Yes/No. \spc{-0.5}

\subquestionwithpoints{6} What is the unconditional probability of those 20 observations? Answer exactly. \spc{1}

\subquestionwithpoints{4} What is the probability that the engagement proportion is greater than 0.5?  You do not need to evaluate the integral. \spc{2}

\subquestionwithpoints{5} What is the probability that the next viewer will not engage? Use the posterior predictive distribution to answer.\spc{2}

We now use previous posts to build a better prior. If we look at previous $\theta$ estimates, we can fit a $\alpha=17$ and $\beta=245$ beta distribution. Use this as the prior from now on.

\subquestionwithpoints{4} This prior is equivalent to how many pseudo-engagements on the post? \spc{2}

\subquestionwithpoints{4} What is the prior expectation? \spc{2}

\subquestionwithpoints{6} Find $\thetahatmmse$, $\thetahatmae$ and $\thetahatmap$ considering the $n=20$ non-engagements from part (d). \spc{2}

\subquestionwithpoints{4} If we are using $\thetahatmmse$ to estimate $\theta$, calculate $\rho$, the shrinkage proportion. Round to the nearest percent. \spc{2}

\subquestionwithpoints{4} Does the shrinkage indicate a \emph{strong} prior? Yes / no. \spc{-0.5}

\subquestionwithpoints{6} Create a 95\% credible region for the probability of \emph{not engaging}. \spc{5}

\eenum

\problem These are purely theoretical exercises.

\benum

\subquestionwithpoints{5} Prove $B(1,1) = 1$ from first principles. \spc{6}

\subquestionwithpoints{4} If $\Gamma(\half) = \sqrt{\pi}$, compute $\Gamma(\frac{7}{2})$ to the nearest 2 digits. \spc{4}
\eenum

\end{document}

\problem This question continues our discussion about extrasensory perception. A famous series of experiments called the Ganzfeld experiments work as follows. A human subject called the \qu{receiver} sits in a room in darkness with their eyes covered and with headphones to filter out any noise. An example subject is pictured below:

\begin{figure}[htp]
\centering
\includegraphics[width=2in]{Ganzfeld.jpg}
\end{figure}

\noindent Another human subject called the transmitter sits in another room. In the beginning of the experiment, the transmitter is shown an image. The transmitter than tries to \qu{telephathically transmit} the image to the receiver. At the conclusion of the experiment, the receiver is shown four images --- three decoys and the true image the transmitter was given --- and is asked to choose one. If the receiver chooses the true image, this is termed a \qu{hit}.

\benum

\subquestionwithpoints{4} Under the null hypothesis of \qu{no ESP}, the receiver chooses one image randomly from four images where one image make the \qu{hit}. Create a r.v. $X$ for the \qu{hit} under the null. Indicate the type of r.v. and the value of $\theta$. \spc{2}

\subquestionwithpoints{5} A psychic claims that his ability to identify the target image is 50\%. Create a prior on both the null $\theta$ and the psychic's claimed $\theta$. Use the principle of indifference.\spc{1}

\subquestionwithpoints{7} We run three independent experimental trials of which the psychic gets two of the three correct. Calculate $\prob{X}$.   \spc{5}

\subquestionwithpoints{6} Find the probability that the psychic's assessment of his abilities is correct given this data. \spc{5}

\subquestionwithpoints{7} Calculate the probability the next experiment for this psychic (i.e. the fourth, unobserved experiment) will be a \qu{hit}. \spc{5}

\subquestionwithpoints{6} We now put a uniform prior on $\theta$ across all values in the support. Find the probability that this psychic has \qu{better than normal} abilities given the data. You are free to leave your answer in notation from Table~\ref{tab:eqs}. \spc{2}

\subquestionwithpoints{5} Instead of a uniform prior, you want to factor in what you've seen previously. You figure your prior experience is worth 18 trials and you haven't seen any evidence of ESP in those trials. Using the conjugate prior we discussed in class, what is the prior for $\theta$ now? \spc{1}

\subquestionwithpoints{7} Illustrate this prior the best you can. No need for perfect scale. Make sure to label axes appropriately. \spc{10}

\subquestionwithpoints{4} Compute the prior mean. \spc{2}

\subquestionwithpoints{5} Given this prior and the data, calculate the shrinkage proportion towards $\expe{\theta}$ if you were to use $\thetahatmmse$ for your point estimation strategy. \spc{1}

\subquestionwithpoints{5} If there was a lot of data, what would be the influence of the prior on the point estimate of $\theta$? Try to use only one word. \spc{0}

\subquestionwithpoints{6} Given this prior and the data, fill in the following boxes with only the following symbols: $<, ~\leq, ~>, ~\geq, ~=$ to indicate the numerical relationships.\vspace{-0.4cm}

\beqn
\thetahatmap~~\framebox(20,20){}~~ \thetahatmmse, \quad \thetahatmae~~\framebox(20,20){}~~ \thetahatmmse, \quad \thetahatmap~~\framebox(20,20){}~~ \thetahatmae
\eeqn

\subquestionwithpoints{10} In 2010, Lance Storm, Patrizio Tressoldi, and Lorenzo Di Risio analyzed 29 ganzfeld studies from 1997 to 2008 amassing data on 1,498 trials with different receiver and transmitter subjects. Of the 1,498 trials, 483 were hits. Test the existence of ESP in the Ganzfeld experiments using the Frequentist (i.e. the \textit{non-Bayesian}) two-sided test. You must (1) write your hypotheses clearly, (2) choose your own significance level and (3) state clearly the conclusion of the test.\spc{7}

\subquestionwithpoints{4} Using the Storm et al. (2010) Ganzfeld data and a uniform prior, create a 95\% credible region for $\theta$. You are free to leave your answer in notation from Table~\ref{tab:eqs}. \spc{2}

\subquestionwithpoints{4} Describe in one sentence how you would use the answer from the previous question to test the existence of ESP in the Storm et al. (2010) Ganzfeld data using one of the Bayesian two-sided hypothesis test strategies discussed in class.\spc{2}

\subquestionwithpoints{10} Assess the evidence for the ESP phenomenon in the Ganzfeld experiments using Bayes Factors. You will not be able to solve for $B$ numerically but you need to get as far as you can. 5 points extra credit if you can solve numerically and interpret your result. You can use $\natlog{n!} \approx n\natlog{n} - n + 1$. \spc{8}

\eenum

\problem This is a theoretical question.

\benum
\subquestionwithpoints{5} Consider the case of assessing the evidence of $\theta_a$ versus $\theta_0$. Show that $B$ is the odds ratio of the posterior odds to the prior odds.

\eenum

\end{document}
%-------------------------------------------------------

\problem This question is ironically about theory of testing.

\begin{figure}[htp]
\centering
\includegraphics[width=4in]{examination.jpg}
\end{figure}

\noindent Every hitter's \emph{sample} batting average (BA) is defined as:

\beqn
BA := \frac{\text{sample \# of hits}}{\text{sample \# of at bats}}
\eeqn

In this problem we care about estimating a hitter's \emph{true} batting average which we call $\theta$. Each player has a different $\theta$ but we focus in this problem on one specific player. In order to estimate the player's true batting average, we use the sample batting average as defined above.

\benum
\subquestionwithpoints{2} For the remainder of the problem, we assume that each at bat (for any player) are \emph{conditionally} $\iid$ based on the players' true batting average, $\theta$. So if a player has $n$ at bats, then each successful hit in each at bat can be modeled via

\beqn
X_1~|~\theta, ~X_2~|~\theta, \ldots, ~X_n~|~\theta \iid \bernoulli{\theta}.
\eeqn

Under this model above, if the player had $n=4$ at bats, would the $\prob{X_3,X_2,X_4,X_1}$ be equal to the $\prob{X_1,X_2,X_3,X_4}$? Yes / no. \spc{1}

\subquestionwithpoints{3} If the player had $n=4$ at bats and $\sum_{i=0}^n x_i=0$ hits, compute $\thetahatmle$.\spc{2}

\subquestionwithpoints{4} Compute a frequentist confidence interval for $\theta$ given the data in (b).\spc{2}

\subquestionwithpoints{3} Describe in English the main problem with the interval in (c).\spc{2}

\subquestionwithpoints{2} Set the following prior: $\theta \sim \stduniform$. Is this an informative prior for the true batting average? Yes/no \spc{1}

\subquestionwithpoints{5} Given the prior in (e) and the data in (b), find the posterior distribution of this player's true batting average.\spc{3}

\subquestionwithpoints{3} Based on your posterior distribution in (f), give your best estimate to the value of $\theta$ which minimizes squared error loss.\spc{3}

\subquestionwithpoints{2} Based on your posterior distribution in (f), describe using an integral or \texttt{R}-language expression your best estimate to the value of $\theta$ which minimizes absolute error loss but do not compute.\spc{4}

\subquestionwithpoints{4} Based on your posterior distribution in (f), give your best estimate to the value of $\theta$ using the posterior mode.\spc{2}

\subquestionwithpoints{5} Find an integral expression for the probability this hitter bats above a 300 batting average (which means the true batting average is 0.3 or greater). Do not compute. \spc{2}

\subquestionwithpoints{5} Assuming you have access to \texttt{R} and its function \texttt{qbeta}, give the 95\% credible region for $\theta$. The three arguments for \texttt{qbeta} are (1) quantile (2) alpha and (3) beta. Then, provide an interpretation for this interval. \spc{2}

\subquestionwithpoints{3} What would the Jeffrey's prior be in our model situation described in (a)? \spc{1}

\subquestionwithpoints{2} Would the posterior under the Haldane prior be proper given the data in (b)? Yes / No. \spc{0.5}

\subquestionwithpoints{5} The batting average is \textit{only} measured as the batting average and never logged or transformed. Would there be any value in using the Jeffrey's prior instead of the prior in (e)? Discuss. \spc{5}

\subquestionwithpoints{5} Looking at the entire dataset for 6,061 batters who had 100 or more at bats, I fit a beta function to the sample batting averages and estimated $\alpha = 42.3$ and $\beta = 127.7$ (which we called \qu{empirical Bayes} estimates in class). Consider building a prior from this estimate as

\beqn
\theta \sim \betanot{42.3}{127.7}.
\eeqn

Would a prior based on these hyperparameter estimates be \qu{objective}? Yes / No. Why? \spc{3}

\subquestionwithpoints{2} Is the prior from (o) considered a \qu{conjugate prior}? Yes / No.\spc{0.5}

\subquestionwithpoints{3} Using the prior from (o), find the $\thetahatmmse$ without considering the data whatsoever. Round to 3 digits. \spc{3}

\subquestionwithpoints{4} Using the prior from (o) and the data from (b), find the posterior $\thetahatmmse$. Round to 3 digits. \spc{2}

\subquestionwithpoints{5} The posterior estimate from (q) is different from the frequentist estimate in (b) due to shrinkage. What is the proportion of shrinkage for the posterior estimate in (q)? We denoted this as $\rho$ in class. Round to 3 digits. \spc{3}

\subquestionwithpoints{4} [Extra Credit] Using the Bayesian CLT, compute a 95\% credible region for $\theta$ for the data in (b) and the prior in (o). Round to 3 digits. \spc{3}

\subquestionwithpoints{3} Based on the data in (b) and the prior in (o), what is the probability this batter gets a hit on his next at bat? \spc{3}

\subquestionwithpoints{5} Based on the data in (b) and the prior in (o), write an exact expression for the batter getting 14 or more hits on the next 20 at bats. You can leave your answer in terms of the beta function. Do not compute explicitly. \spc{5}

\subquestionwithpoints{6} Based on the data in (b) and the prior in (o), find the kernel of the distribution for the number of hits this batter gets in the next $m$ at bats. Partial credit is given. \spc{3}

\subquestionwithpoints{2} Based on the data in (b) and the prior in (o), the joint posterior predictive distribution for $n=4$ looks like as follows.

%library(VGAM)
%barplot(dbetabinom.ab(0 : 4, 4, shape1 = 42.3 + 0, shape2 = 127.7 + 4), names = 0 : 4, xlab = "X* (# hits)", ylab = "prob(X*|X)")
\begin{figure}[htp]
\centering
\includegraphics[width=3.0in]{post_pred.pdf}
\end{figure}

Does the data from (b) look abnormal for this model? Yes / no. \spc{0.5}

\subquestionwithpoints{7} Test the following hypotheses by finding an integral or \texttt{R}-language expression for the Bayesian $p$-val for the data in (b) and prior in (o):

\beqn
&& H_0: \theta \geq \theta_0 \\
&& H_a: \theta < \theta_0
\eeqn

where $\theta_0 = \expe{\theta}$. That is, we're testing if this batter is truly \qu{below-average} as compared to the 6,061 career major league baseball players from the official dataset.\spc{8}

\subquestionwithpoints{7} Write an integral or \texttt{R}-language expression for $K$, the Bayes Factor in favor of $H_a$.\spc{9}

\subquestionwithpoints{3} For the model in (a), specify $\mathcal{F}$ (the likelihood model) of a hit at a single at bat. \spc{3}

\subquestionwithpoints{3} [Extra credit] For the the data in (b) and prior in (o), compute $\expesub{X}{\cexpesub{\theta}{\theta}{X}}$ using the Law of Iterated Expectation. \spc{2}

\subquestionwithpoints{3} [Extra credit] For the data in (b), what is the frequentist predictive distribution?

\eenum

\end{document}


\documentclass{beamer}

\mode<presentation>
{
  \usetheme{Hawke}
  \setbeamercovered{transparent}
}

\usepackage[english]{babel}
\usepackage[latin1]{inputenc}
\usepackage{times}
\usepackage[T1]{fontenc}
\usepackage{multimedia}
\usepackage{mathtools}

%%%%%%
% My Commands
%%%%%%

\newcommand{\ml}{{\sc matlab}}
\newcommand{\bb}{{\boldsymbol{b}}}
\newcommand{\bx}{{\boldsymbol{x}}}
\newcommand{\bfm}[1]{{\boldsymbol{#1}}}

%%%%

\title[Lecture 2]{Lecture 2 - Linear Systems 1}
\author[I. Hawke]{I.~Hawke}
\institute[University of Southampton]
{
  School of Mathematics, \\
  University of Southampton, UK
}

\date[Semester 1] % (optional, should be abbreviation of conference name)
{MATH3018/6141, Semester 1}

\subject{Numerical methods}

\pgfdeclareimage[height=0.5cm]{university-logo}{mathematics_7469}
\logo{\pgfuseimage{university-logo}}

\AtBeginSection[]
{
  \begin{frame}<beamer>
    \frametitle{Outline}
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\begin{frame}
  \titlepage
\end{frame}

\section{Linear systems}

\subsection{Reminders}

\begin{frame}
  \frametitle{Revisions and reminders}

  The system of linear equations
  \begin{equation*}
    \left\{
      \begin{aligned}
          x +   y & = 1 \\
        2 x + 2 y & = 3
      \end{aligned}
    \right.
  \end{equation*}
  has no solutions; the system
  \begin{equation*}
    \left\{
      \begin{aligned}
        0.9999999  x +   y & = 0.9999999 \\
        2          x + 2 y & = 2.9999999
      \end{aligned}
    \right.
  \end{equation*}
  does. \pause

  In general form
  \begin{equation*}
    A \bx = \bb
  \end{equation*}
  has a unique solution iff $\det (A) \ne 0$.

\note{
 Written in the general form
  \begin{equation*}
    A \bx = \bb
  \end{equation*}
  with $A$ an $n \times n$ matrix, we know that the system has a
  unique solution iff $\det (A) \ne 0$.
}

\end{frame}

\subsection{Aim}

\begin{frame}
  \frametitle{Aim of condition number}

  The two systems
  \begin{equation*}
    \left\{
      \begin{aligned}
          x +   y & = 1 \\
        2 x + 2 y & = 3
      \end{aligned}
    \right., \quad
    \left\{
      \begin{aligned}
        0.9999999  x +   y & = 0.9999999 \\
        2          x + 2 y & = 2.9999999
      \end{aligned}
    \right.
  \end{equation*}
  are indistinguishable at 6-digit precision; such a ``computer''
  cannot tell if \emph{either} system has a well-defined solution!
  \pause

  \vspace{1ex}

  Systems are \emph{ill-conditioned}: a ``small'' input change leads
  to an enormous change in the solution. \pause The \emph{condition
    number} should predict ill-conditioned systems without finding
  solutions first. \pause

  \vspace{1ex}

  Note: as the very notion of ``small'' depends on the precision, so
  must the interpretation of the condition number.

\end{frame}

\subsection{Options}

\begin{frame}
  \frametitle{Determinants}

  Think about the determinant $\det(A)$. If it vanishes there is no
  solution. \pause

  \vspace{1ex}

  But the matrix
  \begin{equation*}
    A = 10^{-1}
    \begin{pmatrix}
      1 & 0 & \dots & 0 \\
      0 & 1 & \ddots & \vdots \\
      \vdots & \ddots & \ddots & 0 \\
      0 & \dots & 0 & 1
    \end{pmatrix}
  \end{equation*}
  has determinant $10^{-n}$: arbitrarily small, whilst the matrix is
  trivial to invert.

  \vspace{1ex}

  Determinant $\ne$ condition number.

\end{frame}

\begin{frame}
  \frametitle{Perturbations}

  Model the effect of errors by using a parameter $\epsilon$:
  $\epsilon = 0$ means the exact solution. We write
  \begin{equation*}
    \left( A + \epsilon F \right) \bx (\epsilon) = \bb.
  \end{equation*}
  %
  \begin{center}
    \includegraphics<1|handout:0>[width=0.7\textwidth]{figures/Perturbations}
  \end{center}
  %
  \pause
  %
  Differentiate wrt $\epsilon$ giving $A \dot{\bx} = - F \bx + {\cal
    O}(\epsilon)$ and Taylor expand to find
  \begin{equation*}
    \bx(\epsilon) - \underbrace{\bx(0)}_{\smash{\text{exact}}} =
    \epsilon \dot{\bx}(0) + {\cal O}\left( \epsilon^2 \right) = -
    \epsilon A^{-1}  F \bx + {\cal O}\left( \epsilon^2 \right).
  \end{equation*} \pause
  %
  To get a \emph{relative} error we need a magnitude or absolute
  value, giving
  \begin{equation*}
    \frac{\| \bx(\epsilon) - \bx \|}{\| \bx \|} \sim \| A^{-1} \| \| A
    \| \left( \epsilon \frac{\|F\|}{\|A\|} \right).
  \end{equation*} \pause
  The part we can measure is the \emph{condition number}
  $K(A) = \| A \| \| A^{-1} \|$.

\end{frame}

\subsection{Vector norms}

 \begin{frame}
   \frametitle{Vector norms}

   Norms are ``mathematical distance functions''. Focus on real
   vectors, e.g.\ to find the error $\| \bx_{\text{num}} -
   \bx_{\text{exact}} \|$. Interesting norms are
   \begin{equation*}
     \| \bx \|_1 \equiv \sum_{j=1}^n | x_j |, \quad
     \| \bx \|_2 \equiv \left[ \sum_{j=1}^n ( x_j )^2 \right]^{1/2}, \quad
     \| \bx \|_{\infty} \equiv \max_j | x_j |.
   \end{equation*} \pause

   {\bf Example}: If $\bx = (-1, 2, 1)$ then
   \begin{align*}
     \| \bx \|_1 & = | -1 | + | 2 | + | 1| &&= 4, \\
     \| \bx \|_2 & = \sqrt{(-1)^2 + (2)^2 + (1)^2} &&= \sqrt{6}, \\
     \| \bx \|_{\infty} & = \max (| -1 |, | 2 |, | 1 |) && = 2.
   \end{align*}

   \note{
   Norms are ``mathematical distance functions''; they impose a
   structure on a mathematical space describing the separation of
   points in that space. The norms above are examples of $p$-norms on
   a simple space. In general norms obey simple properties like
   linearity, positivity, and the triangle inequality.
   }

 \end{frame}

 \begin{frame}
   \frametitle{Vector norms - geometrically}

   \begin{center}
     \includegraphics[width=0.9\textwidth]{figures/norms}
   \end{center}
   By plotting the region of $\mathbb{R}^2$ defined by $\| \bx \| \le
   1$ we see the different geometric meanings of the norms.

 \end{frame}

 \subsection{Matrix norms}

 \begin{frame}
   \frametitle{Matrix norms}

   Our suggested condition number $K(A) = \| A \| \| A^{-1} \|$ needs
   a \emph{matrix}, not vector norm.

   \vspace{1ex}

   Induce a norm with respect to a vector: If $\bfm{y}$ is a vector
   then $A\bfm{y}$ is also a vector, so we can use a vector
   norm. \pause

   \vspace{1ex}

   To get a \emph{representative, finite} answer we use
   \begin{equation*}
     \| A \| = \max_{\| \bfm{y} \| = 1} \| A \bfm{y} \|.
   \end{equation*}

   \vspace{1ex}

   {\bf Remark}: All the norms used in the above \emph{must} be the
   same.

   \note{
   The vector norms above are a good way of quantifying the difference
   between known solutions, such as the computed solution and the true
   solution. However, we want a prediction \emph{without} computation
   of how good we expect the solution to be. To do this we need to
   extend to matrix norms.
   }

 \end{frame}

 \begin{frame}
   \frametitle{Examples of matrix norms - 1 norm}

   The matrix 1-norm reduces to the maximum of the 1-norms of the
   column vectors of $A$. \pause

   Let
   \begin{equation*}
     A =
     \begin{pmatrix}
       1 & 2 & 3 \\
       4 & 5 & 4 \\
       -1 & 0 & 2
     \end{pmatrix}.
   \end{equation*} \pause
   The column vectors are
   \begin{equation*}
     \bfm{c}_1 =
     \begin{pmatrix}
       1 \\ 4 \\ -1
     \end{pmatrix}, \quad
     \bfm{c}_2 =
     \begin{pmatrix}
       2 \\ 5 \\ 0
     \end{pmatrix}, \quad
     \bfm{c}_3 =
     \begin{pmatrix}
       3 \\ 4 \\ 2
     \end{pmatrix}.
   \end{equation*} \pause
   The resulting 1-norms are
   \begin{equation*}
     \| \bfm{c}_1 \|_1 = 6, \quad
     \| \bfm{c}_2 \|_1 = 7, \quad
     \| \bfm{c}_3 \|_1 = 9.
   \end{equation*} \pause
   Therefore the 1-norm of $A$ is
   \begin{equation*}
     \| A \|_1 = \max ( \| \bfm{c}_1 \|_1, \| \bfm{c}_2 \|_1, \| \bfm{c}_3
     \|_1 ) = 9.
   \end{equation*}
 \end{frame}

 \begin{frame}
   \frametitle{Examples of matrix norms - $\infty$ norm}

   The matrix infinity-norm reduces to the maximum of the 1-norms of
   the row vectors of $A$. \pause

   Let
   \begin{equation*}
     A =
     \begin{pmatrix}
       1 & 2 & 3 \\
       4 & 5 & 4 \\
       -1 & 0 & 2
     \end{pmatrix}
   \end{equation*}
   as before. \pause The  vectors are
   \begin{equation*}
     \bfm{r}_1 =
     \begin{pmatrix}
       1 & 2 & 3
     \end{pmatrix}, \quad
     \bfm{r}_2 =
     \begin{pmatrix}
       4 & 5 & 4
     \end{pmatrix}, \quad
     \bfm{r}_3 =
     \begin{pmatrix}
       -1 & 0 & 2
     \end{pmatrix}.
   \end{equation*} \pause
   The resulting 1-norms are
   \begin{equation*}
     \| \bfm{r}_1 \|_1 = 6, \quad
     \| \bfm{r}_2 \|_1 = 13, \quad
     \| \bfm{r}_3 \|_1 = 3.
   \end{equation*} \pause
   Therefore the $\infty$-norm of $A$ is
   \begin{equation*}
     \| A \|_{\infty} = \max ( \| \bfm{r}_1 \|_1, \| \bfm{r}_2 \|_1, \| \bfm{r}_3
     \|_1 ) = 13.
   \end{equation*}

 \end{frame}

 \subsection{The condition number}

 \begin{frame}
   \frametitle{The condition number}

   Define the condition number as
   \begin{equation*}
     K(A) = \| A \| \| A^{-1} \|.
   \end{equation*}
   Any norm can be used, but for simplicity the 1- or $\infty$-norms
   are usual. \pause

   \begin{overlayarea}{\textwidth}{0.5\textheight}
     \only<2>
     {
       Loose interpretation: the condition number is
       \emph{the amount that inverting the matrix will increase any
         intrinsic error in the coefficients}.
     }
     \only<3->
     {
       More strictly, define the \emph{weighted residual} as
       \begin{equation*}
         {\cal E} = \frac{\| A \bx_{\text{num}} - \bb \|}{\| \bb \|}.
       \end{equation*}
       This can be minimized numerically. \pause It can be shown that:
       \begin{equation*}
         \frac{1}{K(A)} {\cal E} \le \frac{ \| \bx_{\text{num}} -
           \bx_{\text{exact}} \|}{\| \bx_{\text{exact}} \|} \le  K(A)
         {\cal E} .
       \end{equation*}
       Controlling ${\cal E}$ does not control relative error if $K(A)
       \gg 1$!
     }
   \end{overlayarea}

 \end{frame}

 \begin{frame}
   \frametitle{Condition number example}

   Consider the matrix
   \begin{equation*}
     A =
     \begin{pmatrix}
       1 & 2 \\
       \tfrac{499}{1000} & \tfrac{1001}{1000}
     \end{pmatrix}.
   \end{equation*} \pause
   The inverse is
   \begin{equation*}
     A^{-1} =
     \begin{pmatrix}
       \tfrac{1001}{3} & -\tfrac{2000}{3}  \\
       -\tfrac{499}{3} & \tfrac{1000}{3}
     \end{pmatrix}.
   \end{equation*} \pause
   The matrix norms are
   \begin{align*}
     \| A \|_1 & = 3.001, & \| A^{-1} \|_1 & = 1000 \\
     \| A \|_{\infty} & = 3, & \| A^{-1} \|_{\infty} & = 1000
     \tfrac{1}{3}.
   \end{align*} \pause
   Hence the condition number is $K(A) = 3001$. We should expect that
   if the coefficients of the matrix are uncertain in the third
   significant figure then the result of solving a linear system with
   this coefficient matrix will be uncertain to ${\cal O}(1)$ (see
   course notes).

 \end{frame}

 \section{Summary}

 \subsection{Summary}

 \begin{frame}
   \frametitle{Summary}

   \begin{itemize}
   \item Linear systems where a ``small'' change in the coefficients
     means a large change in the solution are called
     \emph{ill-conditioned}.
   \item The \emph{condition number} $K(A) = \| A \| \| A^{-1} \|$
     gives a measure for any given matrix without having to compute
     the solutions.
   \item Whether a condition number is ``too large'' or not depends on
     the precision to which you are working and the accuracy with
     which the coefficients are known.
   \item The condition number depends on matrix norms; for simplicity
     we only consider simple matrix norms induced by standard vector
     norms.
   \item The condition number can be approximated rapidly
     \emph{without} inverting the matrix; this makes it useful as an
     pre-check.
   \end{itemize}

 \end{frame}

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


\documentclass[12pt]{article}

\include{preamble}

\title{Math 341 / 650 Spring 2020 \\ Midterm Examination One}
\author{Professor Adam Kapelner}

\date{Thursday, February 27, 2020}

\begin{document}
\maketitle

\noindent Full Name \line(1,0){410}

\thispagestyle{empty}

\section*{Code of Academic Integrity}

\footnotesize
Since the college is an academic community, its fundamental purpose is the pursuit of knowledge. Essential to the success of this educational mission is a commitment to the principles of academic integrity. Every member of the college community is responsible for upholding the highest standards of honesty at all times. Students, as members of the community, are also responsible for adhering to the principles and spirit of the following Code of Academic Integrity.

Activities that have the effect or intention of interfering with education, pursuit of knowledge, or fair evaluation of a student's performance are prohibited. Examples of such activities include but are not limited to the following definitions:

\paragraph{Cheating} Using or attempting to use unauthorized assistance, material, or study aids in examinations or other academic work or preventing, or attempting to prevent, another from using authorized assistance, material, or study aids. Example: using an unauthorized cheat sheet in a quiz or exam, altering a graded exam and resubmitting it for a better grade, etc.
\\

\noindent I acknowledge and agree to uphold this Code of Academic Integrity. \\

\begin{center}
\line(1,0){250} ~~~ \line(1,0){100}\\
~~~~~~~~~~~~~~~~~~~~~signature~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ date
\end{center}

\normalsize

\section*{Instructions}

This exam is seventy five minutes and closed-book. You are allowed \textbf{one} page (front and back) of a \qu{cheat sheet.} You may use a graphing calculator of your choice. Please read the questions carefully. If the question reads \qu{compute,} this means the solution will be a number otherwise you can leave the answer in \textit{any} widely accepted mathematical notation which could be resolved to an exact or approximate number with the use of a computer. I advise you to skip problems marked \qu{[Extra Credit]} until you have finished the other questions on the exam, then loop back. I also advise you to use pencil. The exam is 100 points total plus extra credit. Partial credit will be granted for incomplete answers on most of the questions. \fbox{Box} in your final answers. NO FOOD but drinks okay. Good luck!

\pagebreak

\begin{table}[htp]
\centering
\small
\begin{tabular}{l | llll}
Distribution                  & Quantile  & PMF / PDF  &CDF       & Sampling  \\
of r.v. &  Function & function         & function &  Function \\ \hline
beta & \texttt{qbeta}($p$, $\alpha$, $\beta$)
& \texttt{d-}($x$, $\alpha$, $\beta$)
& \texttt{p-}($x$, $\alpha$, $\beta$)
& \texttt{r-}($\alpha$, $\beta$) \\
betabinomial & \texttt{qbetabinom}($p$, $n$, $\alpha$, $\beta$)
& \texttt{d-}($x$, $n$, $\alpha$, $\beta$)
& \texttt{p-}($x$, $n$, $\alpha$, $\beta$)
& \texttt{r-}($n$, $\alpha$, $\beta$) \\

betanegativebinomial & \texttt{qbeta\_nbinom}($p$, $r$, $\alpha$, $\beta$)
& \texttt{d-}($x$, $r$, $\alpha$, $\beta$)
& \texttt{p-}($x$, $r$, $\alpha$, $\beta$)
& \texttt{r-}($r$, $\alpha$, $\beta$) \\

binomial & \texttt{qbinom}($p$, $n$, $\theta$)
& \texttt{d-}($x$, $n$, $\theta$)
& \texttt{p-}($x$, $n$, $\theta$)
& \texttt{r-}($n$, $\theta$) \\

exponential & \texttt{qexp}($p$, $\theta$)
& \texttt{d-}($x$, $\theta$)
& \texttt{p-}($x$, $\theta$)
& \texttt{r-}($\theta$) \\

gamma & \texttt{qgamma}($p$, $\alpha$, $\beta$)
& \texttt{d-}($x$, $\alpha$, $\beta$)
& \texttt{p-}($x$, $\alpha$, $\beta$)
& \texttt{r-}($\alpha$, $\beta$) \\

geometric & \texttt{qgeom}($p$, $\theta$)
& \texttt{d-}($x$, $\theta$)
& \texttt{p-}($x$, $\theta$)
& \texttt{r-}($\theta$) \\

inversegamma & \texttt{qinvgamma}($p$, $\alpha$, $\beta$)
& \texttt{d-}($x$, $\alpha$, $\beta$)
& \texttt{p-}($x$, $\alpha$, $\beta$)
& \texttt{r-}($\alpha$, $\beta$) \\

negative-binomial & \texttt{qnbinom}($p$, $r$, $\theta$)
& \texttt{d-}($x$, $r$, $\theta$)
& \texttt{p-}($x$, $r$, $\theta$)
& \texttt{r-}($r$, $\theta$) \\

normal (univariate) & \texttt{qnorm}($p$, $\theta$, $\sigma$)
& \texttt{d-}($x$, $\theta$, $\sigma$)
& \texttt{p-}($x$, $\theta$, $\sigma$)
& \texttt{r-}($\theta$, $\sigma$) \\

%normal (multivariate) &
%& \multicolumn{2}{l}{\texttt{dmvnorm}($\x$, $\muvec$, $\bSigma$)}
%& \texttt{r-}($\muvec$, $\bSigma$) \\

poisson & \texttt{qpois}($p$, $\theta$)
& \texttt{d-}($x$, $\theta$)
& \texttt{p-}($x$, $\theta$)
& \texttt{r-}($\theta$) \\

T (standard) & \texttt{qt}($p$, $\nu$)
& \texttt{d-}($x$, $\nu$)
& \texttt{p-}($x$, $\nu$)
& \texttt{r-}($\nu$) \\

%T (nonstandard) & \texttt{qt.scaled}($p$, $\nu$, $\mu$, $\sigma$)
%& \texttt{d-}($x$, $\nu$, $\mu$, $\sigma$)
%& \texttt{p-}($x$, $\nu$, $\mu$, $\sigma$)
%& \texttt{r-}($\nu$, $\mu$, $\sigma$) \\

uniform & \texttt{qunif}($p$, $a$, $b$)
& \texttt{d-}($x$, $a$, $b$)
& \texttt{p-}($x$, $a$, $b$)
& \texttt{r-}($a$, $b$) \\

\end{tabular}
\caption{Functions from $\texttt{R}$ (in alphabetical order) that can be used on this exam with their arguments. The hyphen in colums 3, 4 and 5 is shorthand notation for the full text of the r.v. which can be found in column 2.
}
\label{tab:eqs}
\end{table}

\problem Let $\mathcal{F}$ be binomial with known sample size $n = 3$. The data is all \qu{successes} i.e. $x = 3$. For all questions that have numerical answers, use three significant digits e.g. 0.123 and 1.23$\times 10^{-5}$ or fractions.

\benum

\subquestionwithpoints{2} Find the maximum likelihood estimate for $\theta$. \spc{0}

\subquestionwithpoints{4} What is the main problem with your estimate in (a)? \spc{2}

\subquestionwithpoints{3} Find the $CI_{\theta, 99\%}$. \spc{0.5}

\subquestionwithpoints{4} Does the interval in (c) fulfill the second goal of statistical inference? Yes / no and explain your answer. \spc{6}

\subquestionwithpoints{2} We will now conduct Bayesian inference. Consider the reduced parameter space $\Theta_0 = \braces{0.50, 0.99} \subset \Theta = (0, 1)$. We believe strongly in $\theta = 0.5$ but we want to give some credence to the alternate theory. Thus we establish a prior of

\beqn
\prob{\theta} = \begin{cases}
0.50 \withprob 0.9 \\
0.99 \withprob 0.1 \\
\end{cases}
\eeqn

Is this the \qu{prior of indifference} for the reduced parameter space? Yes / no and explain.\spc{2}

\subquestionwithpoints{5} Find $\thetahatmap$. \spc{3}

\subquestionwithpoints{5} Find $\prob{X = x}$. \spc{4}

\subquestionwithpoints{5} Find the posterior predictive probability $\cprob{X_* = 1}{X = x}$ where $X_*$ denotes the next observation. \spc{5}

\subquestionwithpoints{3} We will now consider the entire parameter space for the binomial model i.e. $\Theta = (0, 1)$. We will use the prior $\theta \sim \betanot{\half}{\half}$. We will see later in class that this is called the \qu{Jeffrey's Prior}. Is this an uninformative prior? Yes / no and explain.\spc{2}

\subquestionwithpoints{2} Is this the \qu{prior of indifference}? Yes / no and explain. \spc{1}

\subquestionwithpoints{2} How many pseudosuccesses and pseudofailures is within this prior?\spc{1}

\subquestionwithpoints{3} What is $\expe{\theta}$?\spc{0}

\subquestionwithpoints{5} Find $\cprob{\theta}{X = x}$. \spc{1}

\subquestionwithpoints{6} Draw $\cprob{\theta}{X = x}$ to the best of your ability. Label all axes and critical points. \spc{6}

\subquestionwithpoints{2} Does $\thetahatmap$ exist? Yes / No. \spc{-0.5}

\subquestionwithpoints{4} Find $\thetahatmmse$ and denote it in the illustration in (n). \spc{1}

\subquestionwithpoints{4} What is the proportion of shrinkage towards the prior expectation if you employ the posterior expectation as your point estimate? \spc{1}

\subquestionwithpoints{5} Find the $CR_{\theta, 99\%}$. \spc{1}
\subquestionwithpoints{5} Find the $HDR_{\theta, 99\%}$ and denote it in the illustration in (n). \spc{1}

\subquestionwithpoints{10} Test if $\theta > 0.5$. Write out the hypotheses and declare the $\alpha$ level you are comfortable with. Estimate the Bayesian $p$-value from the illustration of the posterior distribution in (n) and provide the conclusion of the test. \spc{8}

\subquestionwithpoints{4} Find the posterior predictive probability $\cprob{X_* = 1}{X = x}$ where $X_*$ denotes the next observation. \spc{1}

\subquestionwithpoints{3} What is your best guess of $X_*$? \spc{2}
\eenum

\problem Consider $\Xoneton \iid \betanot{1}{\theta}$.

\benum

\subquestionwithpoints{6} Find $\mathcal{L}\parens{\theta ; \Xoneton}$. Simplify so that your answer does not include the $B(\cdot,\cdot)$ function or the $\Gamma(\cdot)$ function.\spc{4}

\subquestionwithpoints{3} Find $\loglik{\theta ; \Xoneton}$. Simplify as much as possible.\spc{2}

\subquestionwithpoints{3} Find $\thetahatmle$. \spc{3}

\subquestionwithpoints{8} [Extra Credit] Consider $\Xoneton \iid \betanot{\theta_1}{\theta_2}$. Find the MLE for $\theta_1$ and the MLE for $\theta_2$. Partial credit is given.\spc{3}
\eenum

\end{document}


\documentclass{article}
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}

%%%%%%%%%% Start TeXmacs macros
\newcommand{\tmaffiliation}[1]{\\ #1}
\newcommand{\tmop}[1]{\ensuremath{\operatorname{#1}}}
\newcommand{\tmtextbf}[1]{{\bfseries{#1}}}
%%%%%%%%%% End TeXmacs macros

\begin{document}

\title{Chapter 4 Confidence Interval Estimation}

\author{
  Yuejina Mo
  \tmaffiliation{May 21, 2018}
}

\maketitle

\section*{Part II}

\tmtextbf{Q4.8} A process produces bags of refined sugar. The weights of the
contents of these bags are normally distributed with standard deviation 1.2
ounces. The contents of a random sample of 25 bags had a mean weight of 19.8
ounces. Find the upper and lower confidence limits of a 99\% confidence
interval for the true mean weight for all bags of sugar produced by the
process.

Ans:

For a 99\% confidence interval the reliability factor is $Z_{0.005} = 2.58$
and with a sample mean of 19.8, n=25, and a standard deviation of 1.2, the
confidence limits are as follows:
\[ \tmop{Upper} \tmop{confidence} \tmop{limit} = \overset{-}{x} + Z_{\alpha /
   2} \frac{\sigma}{\sqrt{n}} = 19.8 + 2.58 \frac{1.2}{\sqrt{25}} = 20.42 \]
\[ \tmop{Lower} \tmop{confidence} \tmop{limit} = \overset{-}{x} - Z_{\alpha /
   2} \frac{\sigma}{\sqrt{n}} = 19.8 - 2.58 \frac{1.2}{\sqrt{25}} = 19.18 \]
Source: 國立交通大學, 唐麗英老師上課講義,
http://ocw.nctu.edu.tw/course/stat021/CH7.pdf

\end{document}

Assignments and Suggested Solutions/Chapter4-11510511-YuejianMo.tex


\documentclass{beamer}
%
% Choose how your presentation looks.
%
% For more themes, color themes and font themes, see:
% http://deic.uab.es/~iblanes/beamer_gallery/index_by_theme.html
%

% Theme License: https://creativecommons.org/licenses/by/4.0/

\mode<presentation>
{
  \usetheme{metropolis}   % https://github.com/matze/mtheme
  \usecolortheme{default} % or try albatross, beaver, crane, ...
  \usefonttheme{default}  % or try serif, structurebold, ...
  \setbeamertemplate{navigation symbols}{}
  %% \setbeamertemplate{caption}[numbered]
  \setbeamertemplate{caption}{\raggedright\insertcaption\par}
}

\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage{xspace}
\PassOptionsToPackage{hyphens}{url}\usepackage{hyperref}

\title{An Introduction to \latex}
\author{Jake Vossen}
\institute{Colorado School of Mines - ACM-W}
\date{2020-02-26}
\newcommand{\latex}{\LaTeX\xspace}

\begin{document}
\begin{frame}{Something to consider...}
    While you are waiting - create an account on \url{overleaf.com}
    \begin{figure}
    \includegraphics[scale=.13]{invisible_formatting_2x.png}
    \caption{https://xkcd.com/2109/}
  \end{figure}
\end{frame}
\begin{frame}
  \titlepage
\end{frame}

% Uncomment these lines for an automatically generated outline.
%\begin{frame}{Outline}
%  \tableofcontents
%\end{frame}

\begin{frame}{A quick note on pronunciation...}
  The ``TeX'' in \latex are the capital letters tau, epsilon, and chi witch derive words like ``technology'' in English today. This makes the logical ways to pronounce it \textit{lah-teck} and \textit{lay-teck}.
\end{frame}

\section{What is \latex?}

\begin{frame}[fragile]
  \frametitle{What is \latex}

  It is \textbf{not} a ``what you see is what you get'' editor (WYSIWYG). \latex converts plain text files (.tex) into (typically) PDFs

  %% text
\begin{verbatim}
  \begin{equation}
        x = \frac{ - b \pm \sqrt{b^2 - 4 a c}}{2a}
  \end{equation}
\end{verbatim}
\par
  \hrulefill\par
%% \hline
  \begin{equation}
        x = \frac{ - b \pm \sqrt{b^2 - 4 a c}}{2a}
  \end{equation}

\end{frame}

\begin{frame}[fragile]
\frametitle{More \latex}
\begin{verbatim}
\begin{equation}
  \int_0^a x^n dx = \frac{1}{n+1}\
  a^{n+1} \qquad n \geq 0
\end{equation}
\end{verbatim}\par
  \hrulefill\par

\begin{equation}
  \int_0^a x^n dx = \frac{1}{n+1}\
  a^{n+1} \qquad n \geq 0
\end{equation}

\end{frame}

\begin{frame}[fragile]
  \frametitle{More \latex}
  \begin{verbatim}
    \begin{itemize}
      \item{Point 1}
      \item {Point 2}
    \end{itemize}
  \end{verbatim}\par
    \hrulefill\par

    \begin{itemize}
      \item{Point 1}
      \item {Point 2}
    \end{itemize}

  \end{frame}

\begin{frame}[fragile]
  \frametitle{More \latex}
  \begin{verbatim}
    \begin{enumerate}
      \item{Point 1}
      \item {Point 2}
    \end{enumerate}
  \end{verbatim}\par
    \hrulefill\par

    \begin{enumerate}
      \item{Point 1}
      \item {Point 2}
    \end{enumerate}

  \end{frame}

\section{The Why}

\begin{frame}[fragile]
  \frametitle{Widespread use in Academia}

  \begin{enumerate}
    \item {Almost all journals, especially in Math, CS, and Physics
      either prefer or mandate \latex as the document submission
      format}
    \item {Discrete Math (CSCI 358) uses \latex for all of your homework}
    \item {It can make your project / homework submissions look much more professional}
   \end{enumerate}

\vskip 1cm

\end{frame}

\begin{frame}[fragile]
  \frametitle{Better looking equations (arguably)}

  \begin{itemize}

  \item{Word}

  \includegraphics[scale=.75]{word-equation.png}

  \item{\latex}

  \begin{equation}
  \int_0^a x^n dx = \frac{1}{n+1}\
  a^{n+1} \qquad n \geq 0
  \end{equation}

  \end{itemize}

\end{frame}

\begin{frame}{Citation management}
  \frametitle{Easy citation management!}
  \begin{itemize}
    \item<1-> {Packages provide all the different citation styles}
    \item<2-> {Auto update order, style, and formatting}
    \item<3-> {Sometimes, journals (+ arXiv) will provide \latex citation entries so you don't have to type anything!}
    \item<4-> {(Dagny shows example)}
  \end{itemize}
\end{frame}

\begin{frame}{Citation management}
  \frametitle{Inserting code}
  Syntax highlighting in the code you add to your reports!
\end{frame}

\begin{frame}{Nerd Quality of Life}
  \begin{itemize}
    \item<1-> {Use your favorite text editor to write those long papers}
    \item<2-> {Use git to version control and collaborate on writeups}
    \item<3-> {More control over how your document is formatted without having drag everything to how you like it}
    \item<4-> {A better ``default'' look instead of just bolding headings, huge upgrade from Google Docs}
    \item<5-> {1000x better citation management}
  \end{itemize}
\end{frame}

\section{Breaking up a \latex document}

\begin{frame}[fragile]

  \frametitle{``Hello World'' document}
  \begingroup
  \fontsize{8pt}{10pt}\selectfont
  \begin{verbatim}
    \documentclass{article}
    \usepackage[margin=1in]{geometry}
    \begin{document}

    \title{Title Here}
    \author{Jacob Vossen}
    \maketitle
    \begin{abstract}
    The abstract text goes here.
    \end{abstract}
    \section{Introduction}
    Here is the text of your introduction.
    \subsection{Subsection Heading Here}
    Write your subsection text here.
    \section{Conclusion}
    Write your conclusion here.

    \end{document}
  \end{verbatim}
\endgroup
\end{frame}

\begin{frame}
  \begin{itemize}
    \item This slide exists to remind Jake to show you the example Hello World document
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{The Header}
  \begin{verbatim}
  \documentclass{article}
  \usepackage[margin=1in]{geometry}
  \begin{document}
  \end{verbatim}

  \begin{itemize}
    \item{Document class - typically \texttt{article}, but you would change this if you where working on a book or thesis (or slides like these)}
    \item {Packages - easiest way to extend \latex to include more functionality (in this case, setting margins away from the default 2 inches)}
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{The Title}
  \begin{verbatim}
   \title{Title Here}
   \author{Jacob Vossen}
   \maketitle
  \end{verbatim}

  \begin{itemize}
    \item {Title is automatically created and formatted with \texttt{\\maketitle}}
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{The Body}

  \begin{verbatim}
    \section{Introduction}
    Here is the text of your introduction.
    \subsection{Subsection Heading Here}
    Write your subsection text here.

  \end{verbatim}
% TODO Add verbatim to the subsection texts
  This is the meat of writing \latex documents, using section, subsection, and subsubsection etc to structure your document, adding in equations when you need them.

\end{frame}

\begin{frame}{Examples}
  \begin{itemize}
  \item<1-> {If you didn't guess already, these slides are in \latex ! You can find them at \url{https://github.com/jakevossen5/acm-w-latex-talk}}
  \item<2-> {Some other examples...}
  \end{itemize}
\end{frame}

\begin{frame}{How to get Started with \latex}
  In approvement order of most tech-intensive to least tech-intensive
  \begin{enumerate}
    \item{Overleaf - The Google Docs equivelent of \latex - everything is in browser}
    \item { $\langle \text{Texpad, Texstudio, Texmaker} \rangle$ are all desktop apps designed for out of the box \latex usage}
    \item {Using \texttt{pdflatex} (on macOS: \texttt{brew cask install mactex}) + the text editor of your choice} %TODO verb
  \end{enumerate}
\end{frame}

\begin{frame}{Things That Suck in \latex}

  \begin{itemize}
    \item<1-> {The error messages are not the best, usually point to something quite a ways after what is actually wrong}
    \item<2-> {You have to rely almost entirely on third party documentation (Overleaf)}
    \item<3-> {Making tables can be a pain (use \url{https://www.tablesgenerator.com})}
    \item<4-> {Adding diagrams exactly where you want them can be difficult as well}
    \item<5-> {It can be frustrating when your English homework doesn't compile}
  \end{itemize}

  % Git, text editors, etc
\end{frame}

\begin{frame}{Getting Help on \latex}
  Official documentation is not great, try the following places for good documentation / tutorials

  \begin{enumerate}
    \item<1->{\url{overleaf.com/learn} provides a \textbf{ton} of information on getting started with \latex}
    \item<2-> {\url{tex.stackexchange.com}}
    \item<3-> {Wikipedia equations are all written in \latex, if you have a specific equation or aren't sure how to do something}
    \item<4-> {Google - you are a comp sci after all}
  \end{enumerate}
\end{frame}

\begin{frame}{Personal Suggestions}
  \begin{enumerate}
    \item<1-> {VSCode + \latex workshop for good side by side editing}
    \item<2-> {Keep a note file somewhere with all the header items you want to have in all of your documents. Copy paste this to start your new documents}
    \item<3-> {Be willing to put some upfront cost in for long term benefit - it can be a bit of a frustrating experience to start but a rewarding one if you commit to it.}
    \item<4-> {Use git!}
  \end{enumerate}
\end{frame}

\section{Questions?}

\end{document}


\chapter{To verify truth table for NOT gate}
%\ref{sec:background}.

%\section{Aim}
%\label{sec:objectives}
%	To verify the truth table for AND Gate.

\section{Apparatus}
%\label{sec:objectives}
	\begin{itemize}
		\tightlist
		\item Kit for realization of gates
		\item Connecting Leads
	\end{itemize}

\section{Theory}
	The NOT gate is an electronic circuit that produces an inverted version of the input at its output. It is also known as an inverter. If the input variable is A, the inverted output is known as NOT A. This is also shown as A' or A with a bar over the top, as shown at the outputs.
	\begin{figure}[h]
		\centering
		\includegraphics{img/exp3/1}
		\caption{Symbol for NOT gate}
		\label{fig:3:1}
	\end{figure}
	\begin{figure}[h]
		\centering
		\includegraphics{img/exp3/2}
		\caption{Truth Table for NOT gate}
		\label{fig:3:2}
	\end{figure}
	NOT gate can be realized through transistor.The input is connected through resistor R2 to the transistor’s base. When no voltage is present on the input, the transistor turns off. When the transistor is off, no current flows through the collector-emitter path. Thus, current from the supply voltage (Vcc) flows through resistor R1 to the output. In this way, the circuit’s output is high when its input is low.

	When voltage is present at the input, the transistor turns on, allowing current to flow through the collector-emitter circuit directly to ground. This ground path creates a shortcut that bypasses the output, which causes the output to go low.

	In this way, the output is high when the input is low and low when the input is high.

	\begin{figure}[h]
		\centering
		\includegraphics[width=0.3\linewidth]{img/exp3/3}
		\caption{Circut for making NOT gate}
		\label{fig:3:3}
	\end{figure}

\section{Procedure}
	\subsubsection{Simulator 1}
	\begin{itemize}
		\tightlist
		\item Connect the supply(+5V) to the circuit.
		\item Press the switches for inputs "A".
		\item Press the switch 2 for input "A".
		\item Repeat step-2 and step-3 for all state of inputs.
	\end{itemize}

	\subsubsection{Simulator 2}
	\begin{itemize}
		\tightlist
		\item Enter the Boolean input "A".
		\item Enter the Boolean output for your corresponding inputs.
		\item Click on "Check" Button to verify your output.
		\item Click "Print" if you want to get print out of Truth Table.
	\end{itemize}

\section{Observations}
	\begin{figure}[h]
		\centering
		\includegraphics[width=0.9\linewidth]{img/exp3/4}
		\caption{}
		\label{fig:3:4}
	\end{figure}
		\begin{figure}[h]
		\centering
		\includegraphics[width=0.9\linewidth]{img/exp3/5}
		\caption{}
		\label{fig:3:5}
	\end{figure}
		\begin{figure}[h]
		\centering
		\includegraphics[width=0.9\linewidth]{img/exp3/6}
		\caption{}
		\label{fig:3:6}
	\end{figure}

\section{Conclusion}
As seen above , NOT gate negates the value of input . If the input is low , the output is higher and vice versa.

\section{Precautions}
	\begin{enumerate}
		\tightlist
		\item Make the connections when power supply is OFF.
		\item Ensure that the connections are tight.
		\item Change the status of inputs only when power supply is OFF.
	\end{enumerate}