```python
from IPython.display import Image
Image('../../../python_for_probability_statistics_and_machine_learning.jpg')
```

[Python for Probability, Statistics, and Machine Learning](https://www.springer.com/fr/book/9783319307152)

```python
from __future__ import division
%pylab inline
```

    Populating the interactive namespace from numpy and matplotlib

It is sometimes very difficult to unequivocally attribute outcomes to causal
factors. For example, did your  experiment generate the outcome you were hoping
for or not?  Maybe something did happen, but the effect is not pronounced
enough to separate it from inescapable measurement errors or other
factors in the ambient environment?  Hypothesis testing is a powerful
statistical method to address these questions.  Let's begin by again
considering our coin-tossing experiment with unknown parameter $p$.  Recall
that the individual coin-flips are Bernoulli distributed.  The first step is
to establish separate hypotheses.  First, $H_0$ is the so-called null
hypothesis. In our case this can be

$$
H_0 \colon \theta <  \frac{1}{2}
$$

 and the alternative hypothesis is then

$$
H_1 \colon \theta \geq  \frac{1}{2}
$$

 With this set up, the question now boils down to figuring out which
hypothesis the data is most consistent with.  To choose between these, we need
a statistical test that is a function, $G$, of the sample set
$\mathbf{X}_n=\left\{ X_i \right\}_n $ into the real line, where $X_i$ is the
heads or tails outcome ($X_i \in \lbrace 0,1 \rbrace$). In other words, we
compute $G(\mathbf{X}_n)$ and check if it exceeds a threshold $c$. If not, then
we declare $H_0$ (otherwise, declare $H_1$). Notationally, this is the
following:

$$
\begin{align*}
 G(\mathbf{X}_n) < c & \Rightarrow H_0   \\\
 G(\mathbf{X}_n) \geq c & \Rightarrow H_1
\end{align*}
$$

 In summary, we have the observed data $\mathbf{X}_n$ and a function
$G$ that maps that data onto the real line. Then, using the
constant $c$ as a threshold, the inequality effectively divides the real line
into two parts, one corresponding to each of the hypotheses.

Whatever this test $G$ is, it will make mistakes of two types --- false
negatives and false positives. The false positives arise from the case where we
declare $H_0$ when the test says we should declare $H_1$.  This is
summarized in the Table ref{tbl:decision}.

$$
\begin{table}
\footnotesize
\centering
\begin{tabular}{l|p{1.3in}|p{1.3in}}
\multicolumn{1}{c}{ } & \multicolumn{1}{c}{Declare $H_0$ } & \multicolumn{1}{c}{ Declare $H_1$ } \\
\hline
$H_0\:$ True & Correct                        & False positive (Type I error) \\
$H_1\:$ True & False negative (Type II error) & Correct (true-detect)      \\
\hline
\end{tabular}
\caption{Truth table for hypotheses testing.}
\label{tbl:decision} \tag{1}
\end{table}
$$

 For this example, here are the false positives (aka false alarms):

$$
P_{FA} = \mathbb{P}\left( G(\mathbf{X}_n) > c \mid \theta \leq \frac{1}{2} \right)
$$

 Or, equivalently,

$$
P_{FA} = \mathbb{P}\left( G(\mathbf{X}_n) > c \mid H_0 \right)
$$

 Likewise, the other error is a false negative, which we can write
analogously as

$$
P_{FN} = \mathbb{P}\left( G(\mathbf{X}_n) < c \vert H_1\right)
$$

 By choosing some acceptable values for either of these errors,
we can solve for the other one.  The practice is usually to pick a value of
$P_{FA}$ and then find the corresponding value of $P_{FN}$.  Note that it is
traditional in engineering to speak about *detection probability*, which is
defined as

$$
P_{D} = 1- P_{FN} = \mathbb{P}\left( G(\mathbf{X}_n) > c \mid H_1\right)
$$

 In other words, this is the probability of declaring $H_1$ when the
test exceeds the threshold. This is otherwise known as the *probability of a
true detection* or *true-detect*.

## Back to the Coin Flipping Example

In our previous maximum likelihood discussion, we wanted to derive an
estimator for the *value* of the probability of heads for the coin
flipping experiment. For hypthesis testing, we want to ask a softer
question: is the probability of heads greater or less than $\nicefrac{1}{2}$? As we
just established, this leads to the two hypotheses:

$$
H_0 \colon \theta < \frac{1}{2}
$$

 versus,

$$
H_1 \colon \theta > \frac{1}{2}
$$

 Let's assume we have five observations.  Now we need the $G$ function
and a threshold $c$ to help pick between the two hypotheses. Let's count the
number of heads observed in five observations as our
criterion. Thus, we have

$$
G(\mathbf{X}_5) := \sum_{i=1}^5 X_i
$$

 and, suppose further that we pick $H_1$  only if exactly five out of
five observations are heads. We'll call this the *all-heads* test.

Now, because all of the $X_i$ are random variables, so is $G$ and we must
find the corresponding probability mass function for $G$.  Assuming the
individual coin tosses are independent, the probability of five heads is $\theta^5$.
This means that the probability of rejecting the $H_0$ hypothesis (and choosing
$H_1$, because there are only two choices here) based on the unknown underlying
probability is $\theta^5$. In the parlance, this is known and the *power function*
as in denoted by $\beta$ as in

$$
\beta(\theta) = \theta^5
$$

 Let's get a quick plot this in [Figure](#fig:Hypothesis_testing_001).

```python
%matplotlib inline

from matplotlib.pylab import subplots
import numpy as np
fig,ax=subplots()
fig.set_size_inches((6,3))
xi = np.linspace(0,1,50)
_=ax.plot(xi, (xi)**5,'-k',label='all heads')
_=ax.set_xlabel(r'$\theta$',fontsize=22)
_=ax.plot(0.5,(0.5)**5,'ko')
fig.tight_layout()
#fig.savefig('fig-statistics/Hypothesis_Testing_001.png')
```

Power function for the all-heads test. The dark circle indicates the value of the function indicating $\alpha$.

 Now, we have the following false alarm probability,

$$
P_{FA} = \mathbb{P}( G(\mathbf{X}_n)= 5 \vert H_0) =\mathbb{P}( \theta^5 \vert H_0)
$$

 Notice that this is a function of $\theta$, which means there are
many false alarm probability values that correspond to this test. To be on the
conservative side, we'll pick the supremum (i.e., maximum) of this function,
which is known as the *size* of the test, traditionally denoted by $\alpha$,

$$
\alpha = \sup_{\theta \in \Theta_0} \beta(\theta)
$$

 with domain $\Theta_0 = \lbrace \theta < 1/2 \rbrace$ which in our case is

$$
\alpha = \sup_{\theta < \frac{1}{2}} \theta^5 = \left(\frac{1}{2}\right)^5 = 0.03125
$$

 Likewise, for the detection probability,

$$
\mathbb{P}_{D}(\theta) = \mathbb{P}( \theta^5 \vert H_1)
$$

 which is again a function of the parameter $\theta$. The problem with
this test is that the $P_{D}$ is pretty low for most of the domain of
$\theta$. For instance, values in the nineties for $P_{D}$
only happen when $\theta > 0.98$. In other words, if the coin produces
heads 98 times out of 100, then we can detect $H_1$ reliably. Ideally, we want
a test that is zero for the domain corresponding to $H_0$ (i.e., $\Theta_0$) and
equal to one otherwise. Unfortunately, even if we increase the length of the
observed sequence, we cannot escape this effect with this test. You can try
plotting $\theta^n$ for larger and larger values of $n$ to see this.

### Majority Vote Test

Due to the problems with the detection probability in the all-heads test, maybe
we can think of another test that will have the performance we want? Suppose we
reject $H_0$ if the majority of the observations are heads. Then, using the
same reasoning as above, we have

$$
\beta(\theta) = \sum_{k=3}^5 \binom{5}{k} \theta^k(1-\theta)^{5-k}
$$

[Figure](#fig:Hypothesis_testing_002) shows the power function
for both the majority vote and the all-heads tests.

```python
fig,ax=subplots()
fig.set_size_inches((6,3))
from sympy.abc import theta,k # get some variable symbols
import sympy as S
xi = np.linspace(0,1,50)
expr=S.Sum(S.binomial(5,k)*theta**(k)*(1-theta)**(5-k),(k,3,5)).doit()
_=ax.plot(xi, (xi)**5,'-k',label='all heads')
_=ax.plot(xi, S.lambdify(theta,expr)(xi),'--k',label='majority vote')
_=ax.plot(0.5, (0.5)**5,'ko')
_=ax.plot(0.5, S.lambdify(theta,expr)(0.5),'ko')
_=ax.set_xlabel(r'$\theta$',fontsize=22)
_=ax.legend(loc=0)
fig.tight_layout()
#fig.savefig('fig-statistics/Hypothesis_Testing_002.png')
```

Compares the power function for the all-heads test with that of the majority-vote test.

 In this case, the new test has *size*

$$
\alpha = \sup_{\theta < \frac{1}{2}} \theta^{5} + 5 \theta^{4} \left(- \theta + 1\right) + 10 \theta^{3} \left(- \theta + 1\right)^{2} = \frac{1}{2}
$$

 As before we only get to upwards of 90% for detection
probability only when the underlying parameter $\theta > 0.75$.
Let's see what happens when we consider more than five samples. For
example, let's suppose that we have $n=100$ samples and we want to
vary the threshold for the majority vote test. For example, let's have
a new test where we declare $H_1$ when $k=60$ out of the 100 trials
turns out to be heads. What is the $\beta$ function in this case?

$$
\beta(\theta) = \sum_{k=60}^{100} \binom{100}{k} \theta^k(1-\theta)^{100-k}
$$

 This is too complicated to write by hand, but the statistics module
in Sympy has all the tools we need to compute this.

```python
>>> from sympy.stats import P, Binomial
>>> theta = S.symbols('theta',real=True)
>>> X = Binomial('x',100,theta)
>>> beta_function = P(X>60)
>>> print beta_function.subs(theta,0.5) # alpha
0.0176001001088524
>>> print beta_function.subs(theta,0.70)
0.979011423996075
```

    0.0176001001088524
    0.979011423996075

    0.979011423996075

 These results are much better than before because the $\beta$
function is much steeper. If we declare $H_1$ when we observe 60 out of 100
trials are heads, then we wrongly declare heads approximately 1.8% of the
time.  Otherwise, if it happens that the true value for $p>0.7$, we will
conclude correctly approximately 97% of the time. A quick simulation can sanity
check these results as shown below:

```python
from scipy import stats
rv=stats.bernoulli(0.5) # true p = 0.5
# number of false alarms ~ 0.018
print sum(rv.rvs((1000,100)).sum(axis=1)>60)/1000.
```

    0.016

 The above code is pretty dense so let's unpack it. In the first line, we use the `scipy.stats` module to define the
Bernoulli random variable for the coin flip. Then, we use the `rvs` method of
the variable to generate 1000 trials of the experiment where each trial
consists of 100 coin flips. This generates a $1000 \times 100$ matrix where the
rows are the individual trials and the columns are the outcomes of each
respective set of 100 coin flips. The `sum(axis=1)` part computes the sum across the
columns. Because the values of the embedded matrix are only `1` or `0` this
gives us the count of flips that are heads per row. The next `>60` part
computes the boolean 1000-long vector of values that are bigger than 60. The
final `sum` adds these up. Again, because the entries in the array are `True`
or `False` the `sum` computes the count of times the number of heads has
exceeded 60 per 100 coin flips in each of 1000 trials. Then, dividing this
number by 1000 gives a quick approximation of false alarm probability we
computed above for this case where the true value of $p=0.5$.

## Receiver Operating Characteristic

Because the majority vote test is a binary test, we can compute the *Receiver
Operating Characteristic* (ROC) which is the graph of the $(P_{FA},
P_D)$. The term comes from radar systems but is a very general method for
consolidating all of these issues into a single graph. Let's consider a typical
signal processing example with two hypotheses. In $H_0$, there is noise but no
signal present at the receiver,

$$
H_0 \colon X =  \epsilon
$$

 where $\epsilon \sim \mathcal{N}(0,\sigma^2)$ represents additive
noise. In the alternative hypothesis, there is a  deterministic signal at the receiver,

$$
H_1 \colon X = \mu + \epsilon
$$

 Again, the problem is to choose between these two hypotheses. For
$H_0$, we have $X \sim \mathcal{N}(0,\sigma^2)$ and for $H_1$, we have $ X \sim
\mathcal{N}(\mu,\sigma^2)$.  Recall that we only observe values for $x$ and
must pick either $H_0$ or $H_1$ from these observations. Thus, we need a
threshold, $c$, to compare $x$ against in order to distinguish the two
hypotheses. [Figure](#fig:Hypothesis_testing_003) shows the probability density
functions under each of the hypotheses. The dark vertical line is the threshold
$c$. The gray shaded area is the probability of detection, $P_D$ and the shaded
area is the probability of false alarm, $P_{FA}$. The test evaluates every
observation of $x$ and concludes $H_0$ if $x<c$ and $H_1$ otherwise.

The two density functions for the $H_0$ and $H_1$ hypotheses. The shaded gray area is the detection probability and the shaded blue area is the probability of false alarm. The vertical line is the decision threshold.

**Programming Tip.**

The shading shown in [Figure](#fig:Hypothesis_testing_003) comes from
Matplotlib's `fill_between` function. This function has a `where` keyword
argument to specify which part of the plot to apply shading with specified
`color` keyword argument. Note there is also a `fill_betweenx` function that
fills horizontally.  The `text` function can place formatted
text anywhere in the plot and can utilize basic \LaTeX{} formatting.
See the IPython notebook corresponding to this section for the source code.

As we slide the threshold left and right along the horizontal axis, we naturally change the corresponding areas under
each of the curves shown in [Figure](#fig:Hypothesis_testing_003) and thereby
change the values of $P_D$ and $P_{FA}$. The contour that emerges from sweeping
the threshold this way is the ROC as shown in [Figure](#fig:Hypothesis_testing_004). This figure also shows the diagonal line which
corresponds to making decisions based on the flip of a fair coin. Any
meaningful test must do better than coin flipping so the more the ROC bows up
to the top left corner of the graph, the better. Sometimes ROCs are quantified
into a single number called the *area under the curve* (AUC), which varies from
0.5 to 1.0 as shown. In our example, what separates the two probability density
functions is the value of $\mu$. In a real situation, this would be determined
by signal processing methods that include many complicated trade-offs. The key
idea is that whatever those trade-offs are, the test itself boils down to the
separation between these two density functions --- good tests separate the two
density functions and bad tests do not. Indeed, when there is no separation, we
arrive at the diagonal-line coin-flipping situation we just discussed.

What values for $P_D$ and $P_{FA}$ are considered *acceptable* depends on the
application. For example, suppose you are testing for a fatal disease. It could
be that you are willing to except a relatively high $P_{FA}$ value if that
corresponds to a good $P_D$ because the test is relatively cheap to administer
compared to the alternative of missing a detection. On the other hand,
may be a false alarm triggers an expensive response, so that minimizing
these alarms is more important than potentially missing a detection. These
trade-offs can only be determined by the application and design factors.

The Receiver Operating Characteristic (ROC) corresponding to [Figure](#fig:Hypothesis_testing_003).

## P-Values

There are a lot of moving parts in hypothesis testing. What we need
is a way to consolidate the findings. The idea is that we want to find
the minimum level at which the test rejects $H_0$. Thus, the p-value
is the probability, under $H_0$, that the test-statistic is at least
as extreme as what was actually observed.  Informally, this means
that smaller values imply that $H_0$ should be rejected, although
this doesn't mean that large values imply that $H_0$ should be
retained. This is because a large p-value can arise from either $H_0$
being true or the test having low statistical power.

If $H_0$ is true, the p-value is uniformly distributed in the interval $(0,1)$.
If $H_1$ is true, the distribution of the p-value will concentrate closer to
zero. For continuous distributions, this can be proven rigorously and implies
that if we reject $H_0$ when the corresponding p-value is less than $\alpha$,
then the probability of a false alarm is $\alpha$. Perhaps it helps to
formalize this a bit before computing it. Suppose $\tau(X)$ is a test
statistic that rejects $H_0$ as it gets bigger. Then, for each sample $x$,
corresponding to the data we actually have on-hand, we define

$$
p(x) = \sup_{\theta \in \Theta_0} \mathbb{P}_{\theta}(\tau(X) > \tau(x))
$$

 This equation states that the supremum (i.e., maximum)
probability that the test statistic, $\tau(X)$, exceeds the value for
the test statistic on this particular data ($\tau(x)$) over the
domain $\Theta_0$ is defined as the p-value. Thus, this embodies a
worst-case scenario over all values of $\theta$.

Here's one way to think about this. Suppose you rejected $H_0$, and someone
says that you just got *lucky* and somehow just drew data that happened to
correspond to a rejection of $H_0$.   What p-values provide is a way to address
this by capturing the odds of just a favorable data-draw.  Thus, suppose that
your p-value is 0.05. Then, what you are showing is that the odds of just
drawing that data sample, given $H_0$ is in force, is just 5%. This means that
there's a 5% chance that you somehow lucked out and got a favorable draw of
data.

Let's make this concrete with an example. Given, the majority-vote rule above,
suppose we actually do observe three of five heads. Given the $H_0$, the
probability of observing this event is the following:

$$
p(x) =\sup_{\theta \in \Theta_0} \sum_{k=3}^5\binom{5}{k} \theta^k(1-\theta)^{5-k} = \frac{1}{2}
$$

 For the all-heads test, the corresponding computation is the following:

$$
p(x) =\sup_{\theta \in \Theta_0} \theta^5 = \frac{1}{2^5} = 0.03125
$$

From just looking at these p-values, you might get the feeling that the second
test is better, but we still have the same detection probability issues we
discussed above; so, p-values help in summarizing some aspects of our
hypothesis testing, but they do *not* summarize all the salient aspects of the
*entire* situation.

## Test Statistics

As we have seen, it is difficult to derive good test statistics for hypothesis
testing without a systematic process.  The Neyman-Pearson Test is derived from
fixing a false-alarm value ($\alpha$) and then maximizing  the detection
probability. This results in the Neyman-Pearson Test,

$$
L(\mathbf{x}) = \frac{f_{X|H_1}(\mathbf{x})}{f_{X|H_0}(\mathbf{x})} \stackrel[H_0]{H_1}{\gtrless} \gamma
$$

 where $L$ is the likelihood ratio and where the threshold
$\gamma$ is chosen such that

$$
\int_{x:L(\mathbf{x})>\gamma} f_{X|H_0}(\mathbf{x}) d\mathbf{x}=\alpha
$$

 The Neyman-Pearson Test is one of a family of tests that use
the likelihood ratio.

**Example.** Suppose we have a receiver and we want to distinguish
whether just noise ($H_0$) or signal pluse noise ($H_1$) is received.
For the noise-only case, we have  $x\sim \mathcal{N}(0,1)$ and for the
signal pluse noise case we have $x\sim \mathcal{N}(1,1)$.  In other
words, the mean of the distribution shifts in the presence of the
signal. This is a very common problem in signal processing and
communications. The Neyman-Pearson Test then boils down to the
following,

$$
L(x)= e^{-\frac{1}{2}+x}\stackrel[H_0]{H_1}{\gtrless}\gamma
$$

 Now we have to find the threshold $\gamma$ that solves the
maximization problem that characterizes the Neyman-Pearson Test. Taking
the natural  logarithm and re-arranging gives,

$$
x\stackrel[H_0]{H_1}{\gtrless} \frac{1}{2}+\log\gamma
$$

 The next step is find $\gamma$ corresponding to the desired
$\alpha$ by computing it from the following,

$$
\int_{1/2+\log\gamma}^{\infty} f_{X|H_0}(x)dx = \alpha
$$

 For example, taking $\alpha=1/100$, gives
$\gamma\approx 6.21$. To summarize the test in this case, we have,

$$
x\stackrel[H_0]{H_1}{\gtrless} 2.32
$$

 Thus, if we measure $X$ and see that its value
exceeds the threshold above, we declare $H_1$ and otherwise
declare $H_0$. The following code shows how to
solve this example using Sympy and Scipy. First, we
set up the likelihood ratio,

```python
import sympy as S
from sympy import stats
s = stats.Normal('s',1,1) # signal+noise
n = stats.Normal('n',0,1) # noise
x = S.symbols('x',real=True)
L = stats.density(s)(x)/stats.density(n)(x)
```

 Next, to find the $\gamma$ value,

```python
g = S.symbols('g',positive=True) # define gamma
v=S.integrate(stats.density(n)(x),
             (x,S.Rational(1,2)+S.log(g),S.oo))
```

**Programming Tip.**

Providing additional information regarding the Sympy variable by using the
keyword argument `positive=True` helps the internal simplification algorithms
work faster and better. This is especially useful when dealing with complicated
integrals that involve special functions. Furthermore, note that we used the
`Rational` function to define the `1/2` fraction, which is another way of
providing hints to Sympy. Otherwise, it's possible that the floating-point
representation of the fraction could  disguise the simple fraction and
thereby miss internal simplification opportunities.

 We want to solve for `g` in the above expression. Sympy has some
built-in numerical solvers as in the following,

```python
print S.nsolve(v-0.01,3.0) # approx 6.21
```

    6.21116124253284

 Note that in this situation it is better to use the numerical
solvers because Sympy `solve` may grind along for a long time to
resolve this.

### Generalized Likelihood Ratio Test

The likelihood ratio test can be generalized using the following statistic,

$$
\Lambda(\mathbf{x})= \frac{\sup_{\theta\in\Theta_0} L(\theta)}{\sup_{\theta\in\Theta} L(\theta)}=\frac{L(\hat{\theta}_0)}{L(\hat{\theta})}
$$

 where $\hat{\theta}_0$ maximizes $L(\theta)$ subject to
$\theta\in\Theta_0$ and $\hat{\theta}$ is the maximum likelihood estimator.
The intuition behind this generalization of the Likelihood Ratio Test is that
the denomimator is the usual maximum likelihood estimator and the numerator is
the maximum likelihood estimator, but over a restricted domain ($\Theta_0$).
This means that the ratio is always less than unity because the maximum
likelihood estimator over the entire space will always be at least as maximal
as that over the more restricted space. When this $\Lambda$ ratio gets small
enough, it means that the maximum likelihood estimator over the entire domain
($\Theta$) is larger which means that it is safe to reject the null hypothesis
$H_0$.  The tricky part is that the statistical distribution of $\Lambda$ is
usually eye-wateringly difficult.  Fortunately, Wilks Theorem says that with
sufficiently large $n$, the distribution of $-2\log\Lambda$ is approximately
chi-square with $r-r_0$ degrees of freedom, where $r$ is the number of free
parameters for $\Theta$ and $r_0$ is the number of free parameters in
$\Theta_0$.  With this result, if we want an approximate test at level
$\alpha$, we can reject $H_0$ when $-2\log\Lambda \ge \chi^2_{r-r_0}(\alpha)$
where $\chi^2_{r-r_0}(\alpha)$ denotes the $1-\alpha$ quantile of the
$\chi^2_{r-r_0}$ chi-square distribution.  However, the problem with this
result is that there is no definite way of knowing how big $n$ should be. The
advantage of this generalized likelihood ratio test is that it
can test multiple hypotheses simultaneously, as illustrated
in the following example.

**Example.** Let's return to our coin-flipping example, except now we have
three different coins. The likelihood function is then,

$$
L(p_1,p_2,p_3) = \texttt{binom}(k_1;n_1,p_1)\texttt{binom}(k_2;n_2,p_2)\texttt{binom}(k_3;n_3,p_3)
$$

 where $\texttt{binom}$ is the binomial distribution with
the given parameters. For example,

$$
\texttt{binom}(k;n,p) =\sum_{k=0}^n \binom{n}{k} p^k(1-p)^{n-k}
$$

  The null hypothesis is that all three coins have the
same probability of heads, $H_0:p=p_1=p_2=p_3$. The alternative hypothesis is
that at least one of these probabilites is different. Let's consider the
numerator of the $\Lambda$ first, which will give us the maximum likelihood
estimator of $p$. Because the null hypothesis is that all the $p$ values are
equal, we can just treat this as one big binomial distribution with
$n=n_1+n_2+n_3$ and $k=k_1+k_2+k_3$ is the total number of heads observed for
any coin.  Thus, under the null hypothesis, the distribution of $k$ is binomial
with parameters $n$ and $p$. Now, what is the maximum likelihood estimator for
this distribution? We have worked this problem before and have the following,

$$
\hat{p}_0= \frac{k}{n}
$$

 In other words, the maximum likelihood estimator under the null
hypothesis is the proportion of ones observed in the sequence of $n$ trials
total. Now, we have to substitute this in for the likelihood under the null
hypothesis to finish the numerator of $\Lambda$,

$$
L(\hat{p}_0,\hat{p}_0,\hat{p}_0) = \texttt{binom}(k_1;n_1,\hat{p}_0)\texttt{binom}(k_2;n_2,\hat{p}_0)\texttt{binom}(k_3;n_3,\hat{p}_0)
$$

For the denomimator of $\Lambda$, which represents the case of maximizing over
the entire space, the maximum likelihood estimator for each separate binomial
distribution is likewise,

$$
\hat{p}_i= \frac{k_i}{n_i}
$$

 which makes the likelihood in the denominator the following,

$$
L(\hat{p}_1,\hat{p}_2,\hat{p}_3) = \texttt{binom}(k_1;n_1,\hat{p}_1)\texttt{binom}(k_2;n_2,\hat{p}_2)\texttt{binom}(k_3;n_3,\hat{p}_3)
$$

 for each of the $i\in \lbrace 1,2,3 \rbrace$ binomial distributions. Then, the
$\Lambda$ statistic is then the following,

$$
\Lambda(k_1,k_2,k_3) = \frac{L(\hat{p}_0,\hat{p}_0,\hat{p}_0)}{L(\hat{p}_1,\hat{p}_2,\hat{p}_3)}
$$

 Wilks theorems states that $-2\log\Lambda$  is chi-square
distributed. We can compute this example with the statistics tools in  Sympy and
Scipy.

```python
from scipy.stats import binom, chi2
import numpy as np
# some sample parameters
p0,p1,p2 = 0.3,0.4,0.5
n0,n1,n2 = 50,180,200
brvs= [ binom(i,j) for i,j in zip((n0,n1,n2),(p0,p1,p2))]
def gen_sample(n=1):
    'generate samples from separate binomial distributions'
    if n==1:
        return [i.rvs() for i in brvs]
    else:
        return [gen_sample() for k in range(n)]
```

**Programming Tip.**

Note the recursion in the definition of the `gen_sample` function where a
conditional clause of the function calls itself. This is a quick way to reusing
code and generating vectorized output. Using `np.vectorize` is another way, but
the code is simple enough in this case to use the conditional clause. In
Python, it is generally bad for performance to have code with nested recursion
because of how the stack frames are managed.  However,  here we are only
recursing once so this is not an issue.

 Next, we compute the logarithm of the numerator of the $\Lambda$
statistic,

```python
from __future__ import division
np.random.seed(1234)
```

```python
k0,k1,k2 = gen_sample()
print k0,k1,k2
pH0 = sum((k0,k1,k2))/sum((n0,n1,n2))
numer = np.sum([np.log(binom(ni,pH0).pmf(ki))
                  for ni,ki in
                      zip((n0,n1,n2),(k0,k1,k2))])
print numer
```

    12 68 103
    -15.5458638366

 Note that we used the null hypothesis estimate for the $\hat{p}_0$.
Likewise, for the logarithm of the denominator we have the following,

```python
denom = np.sum([np.log(binom(ni,pi).pmf(ki))
                  for ni,ki,pi in
                     zip((n0,n1,n2),(k0,k1,k2),(p0,p1,p2))])
print denom
```

    -8.42410648079

 Now, we can compute the logarithm of the $\Lambda$ statistic as
follows and see what the corresponding value is according to Wilks theorem,

```python
chsq=chi2(2)
logLambda =-2*(numer-denom)
print logLambda
print 1- chsq.cdf(logLambda)
```

    14.2435147116
    0.000807346708329

 Because the value reported above is less than the 5% significance
level, we reject the null  hypothesis that all the coins have the same
probability of heads. Note that there are two degrees of freedom because the
difference in the number of parameters between the null hypothesis ($p$) and
the alternative ($p_1,p_2,p_3$) is two. We can build a quick Monte
Carlo simulation to check the probability of detection for this example using
the following code, which is just a combination of the last few code blocks,

```python
c= chsq.isf(.05) # 5% significance level
out = []
for k0,k1,k2 in gen_sample(100):
   pH0 = sum((k0,k1,k2))/sum((n0,n1,n2))
   numer = np.sum([np.log(binom(ni,pH0).pmf(ki))
                     for ni,ki in
                         zip((n0,n1,n2),(k0,k1,k2))])
   denom = np.sum([np.log(binom(ni,pi).pmf(ki))
                     for ni,ki,pi in
                        zip((n0,n1,n2),(k0,k1,k2),(p0,p1,p2))])
   out.append(-2*(numer-denom)>c)

print np.mean(out) # estimated probability of detection
```

    0.59

 The above simulation shows the estimated probability of
detection, for this set of example parameters. This relative low
probability of detection means that while the test is unlikely (i.e.,
at the 5% significance level) to mistakenly pick the null hypothesis,
it is likewise missing many of the $H_1$ cases (i.e., low probability
of detection). The trade-off between which is more important is up to
the particular context of the problem. In some situations, we may
prefer additional false alarms in exchange for missing fewer $H_1$
cases.

### Permutation Test

The Permutation Test is good way to test whether or not
samples  samples come from the same distribution. For example, suppose that

$$
X_1, X_2, \ldots, X_m \sim F
$$

 and also,

$$
Y_1, Y_2, \ldots, Y_n \sim G
$$

 That is, $Y_i$ and $X_i$ come from different distributions. Suppose
we have some test statistic, for example

$$
T(X_1,\ldots,X_m,Y_1,\ldots,Y_n)  = \vert\overline{X}-\overline{Y}\vert
$$

 Under the null hypothesis for which $F=G$, any of the
$(n+m)!$ permutations are equally likely. Thus, suppose for
each of the $(n+m)!$ permutations, we have the computed
statistic,

$$
\lbrace T_1,T_2,\ldots,T_{(n+m)!} \rbrace
$$

 Then, under the null hypothesis, each of these values is equally
likely. The distribution of $T$ under the null hypothesis is the *permutation
distribution* that puts weight $1/(n+m)!$ on each $T$-value. Suppose $t_o$ is
the observed value of the test statistic and assume that large $T$ rejects the
null hypothesis, then the p-value for the permutation test is the following,

$$
P(T>t_o)= \frac{1}{(n+m)!} \sum_{j=1}^{(n+m)!} I(T_j>t_o)
$$

 where $I()$ is the indicator function. For large $(n+m)!$, we can
sample randomly from the set of all permutations to estimate this p-value.

**Example.** Let's return to our coin-flipping example from last time, but
now we have only two coins. The hypothesis is that both coins
have the same probability of heads. We can use the built-in
function in Numpy to compute the random permutations.

```python
x=binom(10,0.3).rvs(5) # p=0.3
y=binom(10,0.5).rvs(3) # p=0.5
z = np.hstack([x,y]) # combine into one array
t_o = abs(x.mean()-y.mean())
out = [] # output container
for k in range(1000):
   perm = np.random.permutation(z)
   T=abs(perm[:len(x)].mean()-perm[len(x):].mean())
   out.append((T>t_o))

print 'p-value = ', np.mean(out)
```

    p-value =  0.0

 Note that the size of total permutation space is
$8!=40320$ so we are taking relatively few (i.e., 100) random
permutations from this space.

### Wald Test

The Wald Test is an asympotic test. Suppose we have $H_0:\theta=\theta_0$ and
otherwise $H_1:\theta\ne\theta_0$, the corresponding statistic is defined as
the following,

$$
W=\frac{\hat{\theta}_n-\theta_0}{\texttt{se}}
$$

 where $\hat{\theta}$ is the maximum likelihood estimator and
$\texttt{se}$ is the standard error,

$$
\texttt{se} = \sqrt{\mathbb{V}(\hat{\theta}_n)}
$$

 Under general conditions, $W\overset{d}{\to} \mathcal{N}(0,1)$.
Thus, an asympotic test at level $\alpha$ rejects when $\vert W\vert>
z_{\alpha/2}$ where $z_{\alpha/2}$ corresponds to $\mathbb{P}(\vert
Z\vert>z_{\alpha/2})=\alpha$ with $Z \sim \mathcal{N}(0,1)$.  For our favorite
coin-flipping example, if $H_0:\theta=\theta_0$, then

$$
W = \frac{\hat{\theta}-\theta_0}{\sqrt{\hat{\theta}(1-\hat{\theta})/n}}
$$

 We can simulate this using the following code at the usual
5% significance level,

```python
from scipy import stats
theta0 = 0.5 # H0
k=np.random.binomial(1000,0.3)
theta_hat = k/1000. # MLE
W = (theta_hat-theta0)/np.sqrt(theta_hat*(1-theta_hat)/1000)
c = stats.norm().isf(0.05/2) # z_{alpha/2}
print abs(W)>c # if true, reject H0
```

    True

 This rejects $H_0$ because the true $\theta=0.3$ and the null hypothesis
is that $\theta=0.5$.  Note that $n=1000$ in this case which puts us well inside the
asympotic range of the result. We can re-do this example to estimate
the detection probability for this example as in the following code,

```python
theta0 = 0.5 # H0
c = stats.norm().isf(0.05/2.) # z_{alpha/2}
out = []
for i in range(100):
   k=np.random.binomial(1000,0.3)
   theta_hat = k/1000. # MLE
   W = (theta_hat-theta0)/np.sqrt(theta_hat*(1-theta_hat)/1000.)
   out.append(abs(W)>c) # if true, reject H0

print np.mean(out) # detection probability
```

    1.0

## Testing Multiple Hypotheses

Thus far, we have focused primarily on two competing hypotheses. Now, we
consider multiple comparisons. The general situation is the following.  We test
the null hypothesis against  a sequence of $n$ competing hypotheses $H_k$.  We
obtain p-values for each hypothesis so now we have multiple p-values to
consider $\lbrace p_k \rbrace$. To boil this sequence down to a single
criterion, we can make the following argument. Given $n$ independent hypotheses
that are all untrue, the probability of getting at least one false alarm is the
following,

$$
P_{FA} = 1-(1-p_0)^n
$$

 where $p_0$ is the individual p-value threshold (say, 0.05). The
problem here is that $P_{FA}\rightarrow 1$ as $n\rightarrow\infty$.  If we want
to make many comparisons at once and control the overall false alarm rate the
overall p-value should be computed under the assumption that none of the
competing hypotheses is valid. The most common way to address this is with the
Bonferroni correction which says that the individual significance level should
be reduced to $p/n$. Obviously, this  makes it much harder to declare
significance for any particular hypothesis. The natural consequence of this
conservative restriction is to reduce the statistical power of the experiment,
thus making it more likely the true effects will be missed.

In 1995, Benjamini and Hochberg devised a simple method that tells which
p-values are statistically significant. The procedure is to sort the list of
p-values in ascending order, choose a false-discovery rate (say, $q$), and then
find the largest p-value in the sorted list such that $p_k \le k q/n$, where
$k$ is the p-value's position in the sorted list. Finally, declare that $p_k$
value and all the others less than it statistically significant. This procedure
guarantees that the proportion of false-positives is less than $q$  (on
average). The Benjamini-Hochberg procedure (and its derivatives) is fast and
effective and is widely used for testing hundreds of primarily false hypotheses
when studying genetics or diseases. Additionally, this
procedure provides better statistical power than the Bonferroni correction.

In this section, we discussed the structure of statistical hypothesis testing
and defined the various  terms that are commonly used for this process, along
with the illustrations of what they mean in our running coin-flipping example.
From an engineering standpoint, hypothesis testing is not as common as
confidence-intervals and point estimates. On the other hand, hypothesis testing
is very common in social and medical science, where one must deal with
practical constraints that may limit the sample size or other aspects of the
hypothesis testing rubric. In engineering, we can usually have much more
control over the samples and models we employ because they are typically
inanimate objects that can be measured repeatedly and consistently. This is
obviously not so with human studies, which generally have other ethical and
legal considerations.


Probabilistic Programming
=====
and Bayesian Methods for Hackers
========

##### Version 0.1

`Original content created by Cam Davidson-Pilon`

`Ported to Python 3 and PyMC3 by Max Margenot (@clean_utensils) and Thomas Wiecki (@twiecki) at Quantopian (@quantopian)`
___

Welcome to *Bayesian Methods for Hackers*. The full Github repository is available at [github/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers](https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers). The other chapters can be found on the project's [homepage](https://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/). We hope you enjoy the book, and we encourage any contributions!

Chapter 1
======
***

The Philosophy of Bayesian Inference
------

> You are a skilled programmer, but bugs still slip into your code. After a particularly difficult implementation of an algorithm, you decide to test your code on a trivial example. It passes. You test the code on a harder problem. It passes once again. And it passes the next, *even more difficult*, test too! You are starting to believe that there may be no bugs in this code...

If you think this way, then congratulations, you already are thinking Bayesian! Bayesian inference is simply updating your beliefs after considering new evidence. A Bayesian can rarely be certain about a result, but he or she can be very confident. Just like in the example above, we can never be 100% sure that our code is bug-free unless we test it on every possible problem; something rarely possible in practice. Instead, we can test it on a large number of problems, and if it succeeds we can feel more *confident* about our code, but still not certain.  Bayesian inference works identically: we update our beliefs about an outcome; rarely can we be absolutely sure unless we rule out all other alternatives.

### The Bayesian state of mind

Bayesian inference differs from more traditional statistical inference by preserving *uncertainty*. At first, this sounds like a bad statistical technique. Isn't statistics all about deriving *certainty* from randomness? To reconcile this, we need to start thinking like Bayesians.

The Bayesian world-view interprets probability as measure of *believability in an event*, that is, how confident we are in an event occurring. In fact, we will see in a moment that this is the natural interpretation of probability.

For this to be clearer, we consider an alternative interpretation of probability: *Frequentist*, known as the more *classical* version of statistics, assume that probability is the long-run frequency of events (hence the bestowed title). For example, the *probability of plane accidents* under a frequentist philosophy is interpreted as the *long-term frequency of plane accidents*. This makes logical sense for many probabilities of events, but becomes more difficult to understand when events have no long-term frequency of occurrences. Consider: we often assign probabilities to outcomes of presidential elections, but the election itself only happens once! Frequentists get around this by invoking alternative realities and saying across all these realities, the frequency of occurrences defines the probability.

Bayesians, on the other hand, have a more intuitive approach. Bayesians interpret a probability as measure of *belief*, or confidence, of an event occurring. Simply, a probability is a summary of an opinion. An individual who assigns a belief of 0 to an event has no confidence that the event will occur; conversely, assigning a belief of 1 implies that the individual is absolutely certain of an event occurring. Beliefs between 0 and 1 allow for weightings of other outcomes. This definition agrees with the probability of a plane accident example, for having observed the frequency of plane accidents, an individual's belief should be equal to that frequency, excluding any outside information. Similarly, under this definition of probability being equal to beliefs, it is meaningful to speak about probabilities (beliefs) of presidential election outcomes: how confident are you candidate *A* will win?

Notice in the paragraph above, I assigned the belief (probability) measure to an *individual*, not to Nature. This is very interesting, as this definition leaves room for conflicting beliefs between individuals. Again, this is appropriate for what naturally occurs: different individuals have different beliefs of events occurring, because they possess different *information* about the world. The existence of different beliefs does not imply that anyone is wrong. Consider the following examples demonstrating the relationship between individual beliefs and probabilities:

- I flip a coin, and we both guess the result. We would both agree, assuming the coin is fair, that the probability of Heads is 1/2. Assume, then, that I peek at the coin. Now I know for certain what the result is: I assign probability 1.0 to either Heads or Tails (whichever it is). Now what is *your* belief that the coin is Heads? My knowledge of the outcome has not changed the coin's results. Thus we assign different probabilities to the result.

-  Your code either has a bug in it or not, but we do not know for certain which is true, though we have a belief about the presence or absence of a bug.

-  A medical patient is exhibiting symptoms $x$, $y$ and $z$. There are a number of diseases that could be causing all of them, but only a single disease is present. A doctor has beliefs about which disease, but a second doctor may have slightly different beliefs.

This philosophy of treating beliefs as probability is natural to humans. We employ it constantly as we interact with the world and only see partial truths, but gather evidence to form beliefs. Alternatively, you have to be *trained* to think like a frequentist.

To align ourselves with traditional probability notation, we denote our belief about event $A$ as $P(A)$. We call this quantity the *prior probability*.

John Maynard Keynes, a great economist and thinker, said "When the facts change, I change my mind. What do you do, sir?" This quote reflects the way a Bayesian updates his or her beliefs after seeing evidence. Even — especially — if the evidence is counter to what was initially believed, the evidence cannot be ignored. We denote our updated belief as $P(A |X )$, interpreted as the probability of $A$ given the evidence $X$. We call the updated belief the *posterior probability* so as to contrast it with the prior probability. For example, consider the posterior probabilities (read: posterior beliefs) of the above examples, after observing some evidence $X$:

1\. $P(A): \;\;$ the coin has a 50 percent chance of being Heads. $P(A | X):\;\;$ You look at the coin, observe a Heads has landed, denote this information $X$, and trivially assign probability 1.0 to Heads and 0.0 to Tails.

2\.   $P(A): \;\;$  This big, complex code likely has a bug in it. $P(A | X): \;\;$ The code passed all $X$ tests; there still might be a bug, but its presence is less likely now.

3\.  $P(A):\;\;$ The patient could have any number of diseases. $P(A | X):\;\;$ Performing a blood test generated evidence $X$, ruling out some of the possible diseases from consideration.

It's clear that in each example we did not completely discard the prior belief after seeing new evidence $X$, but we *re-weighted the prior* to incorporate the new evidence (i.e. we put more weight, or confidence, on some beliefs versus others).

By introducing prior uncertainty about events, we are already admitting that any guess we make is potentially very wrong. After observing data, evidence, or other information, we update our beliefs, and our guess becomes *less wrong*. This is the alternative side of the prediction coin, where typically we try to be *more right*.

### Bayesian Inference in Practice

 If frequentist and Bayesian inference were programming functions, with inputs being statistical problems, then the two would be different in what they return to the user. The frequentist inference function would return a number, representing an estimate (typically a summary statistic like the sample average etc.), whereas the Bayesian function would return *probabilities*.

For example, in our debugging problem above, calling the frequentist function with the argument "My code passed all $X$ tests; is my code bug-free?" would return a *YES*. On the other hand, asking our Bayesian function "Often my code has bugs. My code passed all $X$ tests; is my code bug-free?" would return something very different: probabilities of *YES* and *NO*. The function might return:

>    *YES*, with probability 0.8; *NO*, with probability 0.2

This is very different from the answer the frequentist function returned. Notice that the Bayesian function accepted an additional argument:  *"Often my code has bugs"*. This parameter is the *prior*. By including the prior parameter, we are telling the Bayesian function to include our belief about the situation. Technically this parameter in the Bayesian function is optional, but we will see excluding it has its own consequences.

#### Incorporating evidence

As we acquire more and more instances of evidence, our prior belief is *washed out* by the new evidence. This is to be expected. For example, if your prior belief is something ridiculous, like "I expect the sun to explode today", and each day you are proved wrong, you would hope that any inference would correct you, or at least align your beliefs better. Bayesian inference will correct this belief.

Denote $N$ as the number of instances of evidence we possess. As we gather an *infinite* amount of evidence, say as $N \rightarrow \infty$, our Bayesian results (often) align with frequentist results. Hence for large $N$, statistical inference is more or less objective. On the other hand, for small $N$, inference is much more *unstable*: frequentist estimates have more variance and larger confidence intervals. This is where Bayesian analysis excels. By introducing a prior, and returning probabilities (instead of a scalar estimate), we *preserve the uncertainty* that reflects the instability of statistical inference of a small $N$ dataset.

One may think that for large $N$, one can be indifferent between the two techniques since they offer similar inference, and might lean towards the computationally-simpler, frequentist methods. An individual in this position should consider the following quote by Andrew Gelman (2005)[1], before making such a decision:

> Sample sizes are never large. If $N$ is too small to get a sufficiently-precise estimate, you need to get more data (or make more assumptions). But once $N$ is "large enough," you can start subdividing the data to learn more (for example, in a public opinion poll, once you have a good estimate for the entire country, you can estimate among men and women, northerners and southerners, different age groups, etc.). $N$ is never enough because if it were "enough" you'd already be on to the next problem for which you need more data.

### Are frequentist methods incorrect then?

**No.**

Frequentist methods are still useful or state-of-the-art in many areas. Tools such as least squares linear regression, LASSO regression, and expectation-maximization algorithms are all powerful and fast. Bayesian methods complement these techniques by solving problems that these approaches cannot, or by illuminating the underlying system with more flexible modeling.

#### A note on *Big Data*
Paradoxically, big data's predictive analytic problems are actually solved by relatively simple algorithms [2][4]. Thus we can argue that big data's prediction difficulty does not lie in the algorithm used, but instead on the computational difficulties of storage and execution on big data. (One should also consider Gelman's quote from above and ask "Do I really have big data?")

The much more difficult analytic problems involve *medium data* and, especially troublesome, *really small data*. Using a similar argument as  Gelman's above, if big data problems are *big enough* to be readily solved, then we should be more interested in the *not-quite-big enough* datasets.

### Our Bayesian framework

We are interested in beliefs, which can be interpreted as probabilities by thinking Bayesian. We have a *prior* belief in event $A$, beliefs formed by previous information, e.g., our prior belief about bugs being in our code before performing tests.

Secondly, we observe our evidence. To continue our buggy-code example: if our code passes $X$ tests, we want to update our belief to incorporate this. We call this new belief the *posterior* probability. Updating our belief is done via the following equation, known as Bayes' Theorem, after its discoverer Thomas Bayes:

\begin{align}
 P( A | X ) = & \frac{ P(X | A) P(A) } {P(X) } \\\\[5pt]
& \propto P(X | A) P(A)\;\; (\propto \text{is proportional to })
\end{align}

The above formula is not unique to Bayesian inference: it is a mathematical fact with uses outside Bayesian inference. Bayesian inference merely uses it to connect prior probabilities $P(A)$ with an updated posterior probabilities $P(A | X )$.

##### Example: Mandatory coin-flip example

Every statistics text must contain a coin-flipping example, I'll use it here to get it out of the way. Suppose, naively, that you are unsure about the probability of heads in a coin flip (spoiler alert: it's 50%). You believe there is some true underlying ratio, call it $p$, but have no prior opinion on what $p$ might be.

We begin to flip a coin, and record the observations: either $H$ or $T$. This is our observed data. An interesting question to ask is how our inference changes as we observe more and more data? More specifically, what do our posterior probabilities look like when we have little data, versus when we have lots of data.

Below we plot a sequence of updating posterior probabilities as we observe increasing amounts of data (coin flips).

```python
"""
The book uses a custom matplotlibrc file, which provides the unique styles for
matplotlib plots. If executing this book, and you wish to use the book's
styling, provided are two options:
    1. Overwrite your own matplotlibrc file with the rc-file provided in the
       book's styles/ dir. See http://matplotlib.org/users/customizing.html
    2. Also in the styles is  bmh_matplotlibrc.json file. This can be used to
       update the styles in only this notebook. Try running the following code:

        import json
        s = json.load(open("../styles/bmh_matplotlibrc.json"))
        matplotlib.rcParams.update(s)

"""

# The code below can be passed over, as it is currently not important, plus it
# uses advanced topics we have not covered yet. LOOK AT PICTURE, MICHAEL!
%matplotlib inline
from IPython.core.pylabtools import figsize
import numpy as np
from matplotlib import pyplot as plt
figsize(11, 9)

import scipy.stats as stats

dist = stats.beta
n_trials = [0, 1, 2, 3, 4, 5, 8, 15, 50, 500]
data = stats.bernoulli.rvs(0.5, size=n_trials[-1])
x = np.linspace(0, 1, 100)

# For the already prepared, I'm using Binomial's conj. prior.
for k, N in enumerate(n_trials):
    sx = plt.subplot(len(n_trials)/2, 2, k+1)
    plt.xlabel("$p$, probability of heads") \
        if k in [0, len(n_trials)-1] else None
    plt.setp(sx.get_yticklabels(), visible=False)
    heads = data[:N].sum()
    y = dist.pdf(x, 1 + heads, 1 + N - heads)
    plt.plot(x, y, label="observe %d tosses,\n %d heads" % (N, heads))
    plt.fill_between(x, 0, y, color="#348ABD", alpha=0.4)
    plt.vlines(0.5, 0, 4, color="k", linestyles="--", lw=1)

    leg = plt.legend()
    leg.get_frame().set_alpha(0.4)
    plt.autoscale(tight=True)

plt.suptitle("Bayesian updating of posterior probabilities",
             y=1.02,
             fontsize=14)

plt.tight_layout()
```

The posterior probabilities are represented by the curves, and our uncertainty is proportional to the width of the curve. As the plot above shows, as we start to observe data our posterior probabilities start to shift and move around. Eventually, as we observe more and more data (coin-flips), our probabilities will tighten closer and closer around the true value of $p=0.5$ (marked by a dashed line).

Notice that the plots are not always *peaked* at 0.5. There is no reason it should be: recall we assumed we did not have a prior opinion of what $p$ is. In fact, if we observe quite extreme data, say 8 flips and only 1 observed heads, our distribution would look very biased *away* from lumping around 0.5 (with no prior opinion, how confident would you feel betting on a fair coin after observing 8 tails and 1 head?). As more data accumulates, we would see more and more probability being assigned at $p=0.5$, though never all of it.

The next example is a simple demonstration of the mathematics of Bayesian inference.

##### Example: Bug, or just sweet, unintended feature?

Let $A$ denote the event that our code has **no bugs** in it. Let $X$ denote the event that the code passes all debugging tests. For now, we will leave the prior probability of no bugs as a variable, i.e. $P(A) = p$.

We are interested in $P(A|X)$, i.e. the probability of no bugs, given our debugging tests $X$. To use the formula above, we need to compute some quantities.

What is $P(X | A)$, i.e., the probability that the code passes $X$ tests *given* there are no bugs? Well, it is equal to 1, for a code with no bugs will pass all tests.

$P(X)$ is a little bit trickier: The event $X$ can be divided into two possibilities, event $X$ occurring even though our code *indeed has* bugs (denoted $\sim A\;$, spoken *not $A$*), or event $X$ without bugs ($A$). $P(X)$ can be represented as:

\begin{align}
P(X ) & = P(X \text{ and } A) + P(X \text{ and } \sim A) \\\\[5pt]
 & = P(X|A)P(A) + P(X | \sim A)P(\sim A)\\\\[5pt]
& = P(X|A)p + P(X | \sim A)(1-p)
\end{align}

We have already computed $P(X|A)$ above. On the other hand, $P(X | \sim A)$ is subjective: our code can pass tests but still have a bug in it, though the probability there is a bug present is reduced. Note this is dependent on the number of tests performed, the degree of complication in the tests, etc. Let's be conservative and assign $P(X|\sim A) = 0.5$. Then

\begin{align}
P(A | X) & = \frac{1\cdot p}{ 1\cdot p +0.5 (1-p) } \\\\
& = \frac{ 2 p}{1+p}
\end{align}
This is the posterior probability. What does it look like as a function of our prior, $p \in [0,1]$?

```python
figsize(12.5, 4)
p = np.linspace(0, 1, 50)
plt.plot(p, 2*p/(1+p), color="#348ABD", lw=3)
#plt.fill_between(p, 2*p/(1+p), alpha=.5, facecolor=["#A60628"])
plt.scatter(0.2, 2*(0.2)/1.2, s=140, c="#348ABD")
plt.xlim(0, 1)
plt.ylim(0, 1)
plt.xlabel("Prior, $P(A) = p$")
plt.ylabel("Posterior, $P(A|X)$, with $P(A) = p$")
plt.title("Are there bugs in my code?");
```

We can see the biggest gains if we observe the $X$ tests passed when the prior probability, $p$, is low. Let's settle on a specific value for the prior. I'm a strong programmer (I think), so I'm going to give myself a realistic prior of 0.20, that is, there is a 20% chance that I write code bug-free. To be more realistic, this prior should be a function of how complicated and large the code is, but let's pin it at 0.20. Then my updated belief that my code is bug-free is 0.33.

Recall that the prior is a probability: $p$ is the prior probability that there *are no bugs*, so $1-p$ is the prior probability that there *are bugs*.

Similarly, our posterior is also a probability, with $P(A | X)$ the probability there is no bug *given we saw all tests pass*, hence $1-P(A|X)$ is the probability there is a bug *given all tests passed*. What does our posterior probability look like? Below is a chart of both the prior and the posterior probabilities.

```python
figsize(12.5, 4)
colours = ["#348ABD", "#A60628"]

prior = [0.20, 0.80]
posterior = [1./3, 2./3]
plt.bar([0, .7], prior, alpha=0.70, width=0.25,
        color=colours[0], label="prior distribution",
        lw="3", edgecolor=colours[0])

plt.bar([0+0.25, .7+0.25], posterior, alpha=0.7,
        width=0.25, color=colours[1],
        label="posterior distribution",
        lw="3", edgecolor=colours[1])

plt.xticks([0.20, .95], ["Bugs Absent", "Bugs Present"])
plt.title("Prior and Posterior probability of bugs present")
plt.ylabel("Probability")
plt.legend(loc="upper left");
```

Notice that after we observed $X$ occur, the probability of bugs being absent increased. By increasing the number of tests, we can approach confidence (probability 1) that there are no bugs present.

This was a very simple example of Bayesian inference and Bayes rule. Unfortunately, the mathematics necessary to perform more complicated Bayesian inference only becomes more difficult, except for artificially constructed cases. We will later see that this type of mathematical analysis is actually unnecessary. First we must broaden our modeling tools. The next section deals with *probability distributions*. If you are already familiar, feel free to skip (or at least skim), but for the less familiar the next section is essential.

_______

## Probability Distributions

**Let's quickly recall what a probability distribution is:** Let $Z$ be some random variable. Then associated with $Z$ is a *probability distribution function* that assigns probabilities to the different outcomes $Z$ can take. Graphically, a probability distribution is a curve where the probability of an outcome is proportional to the height of the curve. You can see examples in the first figure of this chapter.

We can divide random variables into three classifications:

-   **$Z$ is discrete**: Discrete random variables may only assume values on a specified list. Things like populations, movie ratings, and number of votes are all discrete random variables. Discrete random variables become more clear when we contrast them with...

-   **$Z$ is continuous**: Continuous random variable can take on arbitrarily exact values. For example, temperature, speed, time, color are all modeled as continuous variables because you can progressively make the values more and more precise.

- **$Z$ is mixed**: Mixed random variables assign probabilities to both discrete and continuous random variables, i.e. it is a combination of the above two categories.

### Discrete Case
If $Z$ is discrete, then its distribution is called a *probability mass function*, which measures the probability $Z$ takes on the value $k$, denoted $P(Z=k)$. Note that the probability mass function completely describes the random variable $Z$, that is, if we know the mass function, we know how $Z$ should behave. There are popular probability mass functions that consistently appear: we will introduce them as needed, but let's introduce the first very useful probability mass function. We say $Z$ is *Poisson*-distributed if:

$$P(Z = k) =\frac{ \lambda^k e^{-\lambda} }{k!}, \; \; k=0,1,2, \dots $$

$\lambda$ is called a parameter of the distribution, and it controls the distribution's shape. For the Poisson distribution, $\lambda$ can be any positive number. By increasing $\lambda$, we add more probability to larger values, and conversely by decreasing $\lambda$ we add more probability to smaller values. One can describe $\lambda$ as the *intensity* of the Poisson distribution.

Unlike $\lambda$, which can be any positive number, the value $k$ in the above formula must be a non-negative integer, i.e., $k$ must take on values 0,1,2, and so on. This is very important, because if you wanted to model a population you could not make sense of populations with 4.25 or 5.612 members.

If a random variable $Z$ has a Poisson mass distribution, we denote this by writing

$$Z \sim \text{Poi}(\lambda) $$

One useful property of the Poisson distribution is that its expected value is equal to its parameter, i.e.:

$$E\large[ \;Z\; | \; \lambda \;\large] = \lambda $$

We will use this property often, so it's useful to remember. Below, we plot the probability mass distribution for different $\lambda$ values. The first thing to notice is that by increasing $\lambda$, we add more probability of larger values occurring. Second, notice that although the graph ends at 15, the distributions do not. They assign positive probability to every non-negative integer.

```python
figsize(12.5, 4)

import scipy.stats as stats
a = np.arange(16)
poi = stats.poisson
lambda_ = [1.5, 4.25]
colours = ["#348ABD", "#A60628"]

plt.bar(a, poi.pmf(a, lambda_[0]), color=colours[0],
        label="$\lambda = %.1f$" % lambda_[0], alpha=0.60,
        edgecolor=colours[0], lw="3")

plt.bar(a, poi.pmf(a, lambda_[1]), color=colours[1],
        label="$\lambda = %.1f$" % lambda_[1], alpha=0.60,
        edgecolor=colours[1], lw="3")

plt.xticks(a + 0.4, a)
plt.legend()
plt.ylabel("probability of $k$")
plt.xlabel("$k$")
plt.title("Probability mass function of a Poisson random variable; differing \
$\lambda$ values");
```

### Continuous Case
Instead of a probability mass function, a continuous random variable has a *probability density function*. This might seem like unnecessary nomenclature, but the density function and the mass function are very different creatures. An example of continuous random variable is a random variable with *exponential density*. The density function for an exponential random variable looks like this:

$$f_Z(z | \lambda) = \lambda e^{-\lambda z }, \;\; z\ge 0$$

Like a Poisson random variable, an exponential random variable can take on only non-negative values. But unlike a Poisson variable, the exponential can take on *any* non-negative values, including non-integral values such as 4.25 or 5.612401. This property makes it a poor choice for count data, which must be an integer, but a great choice for time data, temperature data (measured in Kelvins, of course), or any other precise *and positive* variable. The graph below shows two probability density functions with different $\lambda$ values.

When a random variable $Z$ has an exponential distribution with parameter $\lambda$, we say *$Z$ is exponential* and write

$$Z \sim \text{Exp}(\lambda)$$

Given a specific $\lambda$, the expected value of an exponential random variable is equal to the inverse of $\lambda$, that is:

$$E[\; Z \;|\; \lambda \;] = \frac{1}{\lambda}$$

```python
a = np.linspace(0, 4, 100)
expo = stats.expon
lambda_ = [0.5, 1]

for l, c in zip(lambda_, colours):
    plt.plot(a, expo.pdf(a, scale=1./l), lw=3,
             color=c, label="$\lambda = %.1f$" % l)
    plt.fill_between(a, expo.pdf(a, scale=1./l), color=c, alpha=.33)

plt.legend()
plt.ylabel("PDF at $z$")
plt.xlabel("$z$")
plt.ylim(0,1.2)
plt.title("Probability density function of an Exponential random variable;\
 differing $\lambda$");
```

### But what is $\lambda \;$?

**This question is what motivates statistics**. In the real world, $\lambda$ is hidden from us. We see only $Z$, and must go backwards to try and determine $\lambda$. The problem is difficult because there is no one-to-one mapping from $Z$ to $\lambda$. Many different methods have been created to solve the problem of estimating $\lambda$, but since $\lambda$ is never actually observed, no one can say for certain which method is best!

Bayesian inference is concerned with *beliefs* about what $\lambda$ might be. Rather than try to guess $\lambda$ exactly, we can only talk about what $\lambda$ is likely to be by assigning a probability distribution to $\lambda$.

This might seem odd at first. After all, $\lambda$ is fixed; it is not (necessarily) random! How can we assign probabilities to values of a non-random variable? Ah, we have fallen for our old, frequentist way of thinking. Recall that under Bayesian philosophy, we *can* assign probabilities if we interpret them as beliefs. And it is entirely acceptable to have *beliefs* about the parameter $\lambda$.

##### Example: Inferring behaviour from text-message data

Let's try to model a more interesting example, one that concerns the rate at which a user sends and receives text messages:

>  You are given a series of daily text-message counts from a user of your system. The data, plotted over time, appears in the chart below. You are curious to know if the user's text-messaging habits have changed over time, either gradually or suddenly. How can you model this? (This is in fact my own text-message data. Judge my popularity as you wish.)

```python
figsize(12.5, 3.5)
count_data = np.loadtxt("data/txtdata.csv")
n_count_data = len(count_data)
plt.bar(np.arange(n_count_data), count_data, color="#348ABD")
plt.xlabel("Time (days)")
plt.ylabel("count of text-msgs received")
plt.title("Did the user's texting habits change over time?")
plt.xlim(0, n_count_data);
```

Before we start modeling, see what you can figure out just by looking at the chart above. Would you say there was a change in behaviour during this time period?

How can we start to model this? Well, as we have conveniently already seen, a Poisson random variable is a very appropriate model for this type of *count* data. Denoting day $i$'s text-message count by $C_i$,

$$ C_i \sim \text{Poisson}(\lambda)  $$

We are not sure what the value of the $\lambda$ parameter really is, however. Looking at the chart above, it appears that the rate might become higher late in the observation period, which is equivalent to saying that $\lambda$ increases at some point during the observations. (Recall that a higher value of $\lambda$ assigns more probability to larger outcomes. That is, there is a higher probability of many text messages having been sent on a given day.)

How can we represent this observation mathematically? Let's assume that on some day during the observation period (call it $\tau$), the parameter $\lambda$ suddenly jumps to a higher value. So we really have two $\lambda$ parameters: one for the period before $\tau$, and one for the rest of the observation period. In the literature, a sudden transition like this would be called a *switchpoint*:

$$
\lambda =
\begin{cases}
\lambda_1  & \text{if } t \lt \tau \cr
\lambda_2 & \text{if } t \ge \tau
\end{cases}
$$

If, in reality, no sudden change occurred and indeed $\lambda_1 = \lambda_2$, then the $\lambda$s posterior distributions should look about equal.

We are interested in inferring the unknown $\lambda$s. To use Bayesian inference, we need to assign prior probabilities to the different possible values of $\lambda$. What would be good prior probability distributions for $\lambda_1$ and $\lambda_2$? Recall that $\lambda$ can be any positive number. As we saw earlier, the *exponential* distribution provides a continuous density function for positive numbers, so it might be a good choice for modeling $\lambda_i$. But recall that the exponential distribution takes a parameter of its own, so we'll need to include that parameter in our model. Let's call that parameter $\alpha$.

\begin{align}
&\lambda_1 \sim \text{Exp}( \alpha ) \\\
&\lambda_2 \sim \text{Exp}( \alpha )
\end{align}

$\alpha$ is called a *hyper-parameter* or *parent variable*. In literal terms, it is a parameter that influences other parameters. Our initial guess at $\alpha$ does not influence the model too strongly, so we have some flexibility in our choice.  A good rule of thumb is to set the exponential parameter equal to the inverse of the average of the count data. Since we're modeling $\lambda$ using an exponential distribution, we can use the expected value identity shown earlier to get:

$$\frac{1}{N}\sum_{i=0}^N \;C_i \approx E[\; \lambda \; |\; \alpha ] = \frac{1}{\alpha}$$

An alternative, and something I encourage the reader to try, would be to have two priors: one for each $\lambda_i$. Creating two exponential distributions with different $\alpha$ values reflects our prior belief that the rate changed at some point during the observations.

What about $\tau$? Because of the noisiness of the data, it's difficult to pick out a priori when $\tau$ might have occurred. Instead, we can assign a *uniform prior belief* to every possible day. This is equivalent to saying

\begin{align}
& \tau \sim \text{DiscreteUniform(1,70) }\\\\
& \Rightarrow P( \tau = k ) = \frac{1}{70}
\end{align}

So after all this, what does our overall prior distribution for the unknown variables look like? Frankly, *it doesn't matter*. What we should understand is that it's an ugly, complicated mess involving symbols only a mathematician could love. And things will only get uglier the more complicated our models become. Regardless, all we really care about is the posterior distribution.

We next turn to PyMC3, a Python library for performing Bayesian analysis that is undaunted by the mathematical monster we have created.

Introducing our first hammer: PyMC3
-----

PyMC3 is a Python library for programming Bayesian analysis [3]. It is a fast, well-maintained library. The only unfortunate part is that its documentation is lacking in certain areas, especially those that bridge the gap between beginner and hacker. One of this book's main goals is to solve that problem, and also to demonstrate why PyMC3 is so cool.

We will model the problem above using PyMC3. This type of programming is called *probabilistic programming*, an unfortunate misnomer that invokes ideas of randomly-generated code and has likely confused and frightened users away from this field. The code is not random; it is probabilistic in the sense that we create probability models using programming variables as the model's components. Model components are first-class primitives within the PyMC3 framework.

B. Cronin [5] has a very motivating description of probabilistic programming:

>   Another way of thinking about this: unlike a traditional program, which only runs in the forward directions, a probabilistic program is run in both the forward and backward direction. It runs forward to compute the consequences of the assumptions it contains about the world (i.e., the model space it represents), but it also runs backward from the data to constrain the possible explanations. In practice, many probabilistic programming systems will cleverly interleave these forward and backward operations to efficiently home in on the best explanations.

Because of the confusion engendered by the term *probabilistic programming*, I'll refrain from using it. Instead, I'll simply say *programming*, since that's what it really is.

PyMC3 code is easy to read. The only novel thing should be the syntax. Simply remember that we are representing the model's components ($\tau, \lambda_1, \lambda_2$ ) as variables.

```python
import pymc3 as pm
import theano.tensor as tt

with pm.Model() as model:
    alpha = 1.0/count_data.mean()  # Recall count_data is the
                                   # variable that holds our txt counts
    lambda_1 = pm.Exponential("lambda_1", alpha)
    lambda_2 = pm.Exponential("lambda_2", alpha)

    tau = pm.DiscreteUniform("tau", lower=0, upper=n_count_data - 1)
```

In the code above, we create the PyMC3 variables corresponding to $\lambda_1$ and $\lambda_2$. We assign them to PyMC3's *stochastic variables*, so-called because they are treated by the back end as random number generators.

```python
with model:
    idx = np.arange(n_count_data) # Index
    lambda_ = pm.math.switch(tau > idx, lambda_1, lambda_2)
```

This code creates a new function `lambda_`, but really we can think of it as a random variable: the random variable $\lambda$ from above. The `switch()` function assigns `lambda_1` or `lambda_2` as the value of `lambda_`, depending on what side of `tau` we are on. The values of `lambda_` up until `tau` are `lambda_1` and the values afterwards are `lambda_2`.

Note that because `lambda_1`, `lambda_2` and `tau` are random, `lambda_` will be random. We are **not** fixing any variables yet.

```python
with model:
    observation = pm.Poisson("obs", lambda_, observed=count_data)
```

The variable `observation` combines our data, `count_data`, with our proposed data-generation scheme, given by the variable `lambda_`, through the `observed` keyword.

The code below will be explained in Chapter 3, but I show it here so you can see where our results come from. One can think of it as a *learning* step. The machinery being employed is called *Markov Chain Monte Carlo* (MCMC), which I also delay explaining until Chapter 3. This technique returns thousands of random variables from the posterior distributions of $\lambda_1, \lambda_2$ and $\tau$. We can plot a histogram of the random variables to see what the posterior distributions look like. Below, we collect the samples (called *traces* in the MCMC literature) into histograms.

```python
### Mysterious code to be explained in Chapter 3.
with model:
    step = pm.Metropolis()
    trace = pm.sample(10000, tune=5000,step=step)
```

    100%|██████████| 10000/10000 [00:02<00:00, 4511.50it/s]

```python
lambda_1_samples = trace['lambda_1']
lambda_2_samples = trace['lambda_2']
tau_samples = trace['tau']
```

```python
figsize(12.5, 10)
#histogram of the samples:

ax = plt.subplot(311)
ax.set_autoscaley_on(False)

plt.hist(lambda_1_samples, histtype='stepfilled', bins=30, alpha=0.85,
         label="posterior of $\lambda_1$", color="#A60628", normed=True)
plt.legend(loc="upper left")
plt.title(r"""Posterior distributions of the variables
    $\lambda_1,\;\lambda_2,\;\tau$""")
plt.xlim([15, 30])
plt.xlabel("$\lambda_1$ value")

ax = plt.subplot(312)
ax.set_autoscaley_on(False)
plt.hist(lambda_2_samples, histtype='stepfilled', bins=30, alpha=0.85,
         label="posterior of $\lambda_2$", color="#7A68A6", normed=True)
plt.legend(loc="upper left")
plt.xlim([15, 30])
plt.xlabel("$\lambda_2$ value")

plt.subplot(313)
w = 1.0 / tau_samples.shape[0] * np.ones_like(tau_samples)
plt.hist(tau_samples, bins=n_count_data, alpha=1,
         label=r"posterior of $\tau$",
         color="#467821", weights=w, rwidth=2.)
plt.xticks(np.arange(n_count_data))

plt.legend(loc="upper left")
plt.ylim([0, .75])
plt.xlim([35, len(count_data)-20])
plt.xlabel(r"$\tau$ (in days)")
plt.ylabel("probability");
```

### Interpretation

Recall that Bayesian methodology returns a *distribution*. Hence we now have distributions to describe the unknown $\lambda$s and $\tau$. What have we gained? Immediately, we can see the uncertainty in our estimates: the wider the distribution, the less certain our posterior belief should be. We can also see what the plausible values for the parameters are: $\lambda_1$ is around 18 and $\lambda_2$ is around 23. The posterior distributions of the two $\lambda$s are clearly distinct, indicating that it is indeed likely that there was a change in the user's text-message behaviour.

What other observations can you make? If you look at the original data again, do these results seem reasonable?

Notice also that the posterior distributions for the $\lambda$s do not look like exponential distributions, even though our priors for these variables were exponential. In fact, the posterior distributions are not really of any form that we recognize from the original model. But that's OK! This is one of the benefits of taking a computational point of view. If we had instead done this analysis using mathematical approaches, we would have been stuck with an analytically intractable (and messy) distribution. Our use of a computational approach makes us indifferent to mathematical tractability.

Our analysis also returned a distribution for $\tau$. Its posterior distribution looks a little different from the other two because it is a discrete random variable, so it doesn't assign probabilities to intervals. We can see that near day 45, there was a 50% chance that the user's behaviour changed. Had no change occurred, or had the change been gradual over time, the posterior distribution of $\tau$ would have been more spread out, reflecting that many days were plausible candidates for $\tau$. By contrast, in the actual results we see that only three or four days make any sense as potential transition points.

### Why would I want samples from the posterior, anyways?

We will deal with this question for the remainder of the book, and it is an understatement to say that it will lead us to some amazing results. For now, let's end this chapter with one more example.

We'll use the posterior samples to answer the following question: what is the expected number of texts at day $t, \; 0 \le t \le 70$ ? Recall that the expected value of a Poisson variable is equal to its parameter $\lambda$. Therefore, the question is equivalent to *what is the expected value of $\lambda$ at time $t$*?

In the code below, let $i$ index samples from the posterior distributions. Given a day $t$, we average over all possible $\lambda_i$ for that day $t$, using $\lambda_i = \lambda_{1,i}$ if $t \lt \tau_i$ (that is, if the behaviour change has not yet occurred), else we use $\lambda_i = \lambda_{2,i}$.

```python
figsize(12.5, 5)
# tau_samples, lambda_1_samples, lambda_2_samples contain
# N samples from the corresponding posterior distribution
N = tau_samples.shape[0]
expected_texts_per_day = np.zeros(n_count_data)
for day in range(0, n_count_data):
    # ix is a bool index of all tau samples corresponding to
    # the switchpoint occurring prior to value of 'day'
    ix = day < tau_samples
    # Each posterior sample corresponds to a value for tau.
    # for each day, that value of tau indicates whether we're "before"
    # (in the lambda1 "regime") or
    #  "after" (in the lambda2 "regime") the switchpoint.
    # by taking the posterior sample of lambda1/2 accordingly, we can average
    # over all samples to get an expected value for lambda on that day.
    # As explained, the "message count" random variable is Poisson distributed,
    # and therefore lambda (the poisson parameter) is the expected value of
    # "message count".
    expected_texts_per_day[day] = (lambda_1_samples[ix].sum()
                                   + lambda_2_samples[~ix].sum()) / N

plt.plot(range(n_count_data), expected_texts_per_day, lw=4, color="#E24A33",
         label="expected number of text-messages received")
plt.xlim(0, n_count_data)
plt.xlabel("Day")
plt.ylabel("Expected # text-messages")
plt.title("Expected number of text-messages received")
plt.ylim(0, 60)
plt.bar(np.arange(len(count_data)), count_data, color="#348ABD", alpha=0.65,
        label="observed texts per day")

plt.legend(loc="upper left");
```

Our analysis shows strong support for believing the user's behavior did change ($\lambda_1$ would have been close in value to $\lambda_2$ had this not been true), and that the change was sudden rather than gradual (as demonstrated by $\tau$'s strongly peaked posterior distribution). We can speculate what might have caused this: a cheaper text-message rate, a recent weather-to-text subscription, or perhaps a new relationship. (In fact, the 45th day corresponds to Christmas, and I moved away to Toronto the next month, leaving a girlfriend behind.)

##### Exercises

1\.  Using `lambda_1_samples` and `lambda_2_samples`, what is the mean of the posterior distributions of $\lambda_1$ and $\lambda_2$?

```python
#type your code here.
```

2\.  What is the expected percentage increase in text-message rates? `hint:` compute the mean of `lambda_1_samples/lambda_2_samples`. Note that this quantity is very different from `lambda_1_samples.mean()/lambda_2_samples.mean()`.

```python
#type your code here.
```

3\. What is the mean of $\lambda_1$ **given** that we know $\tau$ is less than 45.  That is, suppose we have been given new information that the change in behaviour occurred prior to day 45. What is the expected value of $\lambda_1$ now? (You do not need to redo the PyMC3 part. Just consider all instances where `tau_samples < 45`.)

```python
#type your code here.
```

### References

-  [1] Gelman, Andrew. N.p.. Web. 22 Jan 2013. [N is never large enough](http://andrewgelman.com/2005/07/31/n_is_never_larg).
-  [2] Norvig, Peter. 2009. [The Unreasonable Effectiveness of Data](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/35179.pdf).
- [3] Salvatier, J, Wiecki TV, and Fonnesbeck C. (2016) Probabilistic programming in Python using PyMC3. *PeerJ Computer Science* 2:e55 <https://doi.org/10.7717/peerj-cs.55>
- [4] Jimmy Lin and Alek Kolcz. Large-Scale Machine Learning at Twitter. Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data (SIGMOD 2012), pages 793-804, May 2012, Scottsdale, Arizona.
- [5] Cronin, Beau. "Why Probabilistic Programming Matters." 24 Mar 2013. Google, Online Posting to Google . Web. 24 Mar. 2013. <https://plus.google.com/u/0/107971134877020469960/posts/KpeRdJKR6Z1>.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunss.otf');
    }
    div.cell{
        width:800px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: Helvetica, serif;
    }
    h4{
        margin-top:12px;
        margin-bottom: 3px;
       }
    div.text_cell_render{
        font-family: Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 145%;
        font-size: 130%;
        width:800px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro", source-code-pro,Consolas, monospace;
    }
    .prompt{
        display: None;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 22pt;
        color: #4057A1;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>

```python

```