# Bayesian Statistics 

## Chapter Zero Review

Omission

## Chapter One Introduction 

### Conditional Probability 

As we have mentioned in `Using Python in Advanced Statistics`, **probability** is a number within in 0 to 1(included 0 and 1),and the meaning of probability is that the credibility(可信度) of a event or a prediction. According to this idea, **COnditional Probability** is the probability under some kinds of conditions. For example, if there is a rain tomorrow in Zhuhai. I check the data, found that in tomorrow's date, there are about 30% that will rain. **This is the probability**. If I want to know the possibility of rain tomorrow if it is already raining today. **This is the conditional probability**, which is a probability of an event happen based on a happened fact. 

The usual notation for conditional probability is $p(A\mid B)$, which means under B event's condition, the probability of A.

*PS: if the probability of an event is 0, we says it is **false**; in the opposite, if the probability is 1, it is **true**.*

### Conjoint Probability 

After known what is **conditional probability**, we can easily know what is **conjoint probability**, which is another way to say the probability that two things are true. This time, the notation is: $p(A B)$ or $P(A\ and \ B)$.


```python
import matplotlib.pyplot as plt
import numpy as np 
import sympy
from matplotlib_venn import venn2, venn3_circles

plt.figure(figsize=(4,4))
v = venn2(subsets=(2,2,1), set_labels = ('A', 'B'))



plt.title("Sample Venn diagram - plot one")
plt.annotate('P(AB)', xy=v.get_label_by_id('11').get_position() - np.array([0, 0.05]), xytext=(-70,-70),
             ha='center', textcoords='offset points', bbox=dict(boxstyle='round,pad=0.5', fc='gray', alpha=0.1),
             arrowprops=dict(arrowstyle='->', connectionstyle='arc3,rad=0.5',color='gray'))
plt.show()
```

The conjointed part can represent the conjoint probability that $A$ and $B$ are both true. For instance, if $A$ and $B$ are independent, we know that the $P(A\ and \ B) = P(A)P(B) \Leftrightarrow P(A \mid B) = p(A)$, for whether A is true or not will not affect B; but if $A$ and $B$ are not independent, just like the weather, if we know that it raind today, it is more likely that it will rain tomorrow, so $p(A|B) > p(A)$. 

In general, the probability of a conjunction is : $\mathbf  p(A\ and \ B) = p(A)p(B \mid A)$

### Bayes's theorem 

Known what is conditional probability and cojoint probability, we can move on to the Headlight: **Bayes's theorem**. First, we should make sure we are in the same channel: 
   1. conjoint probability: $p(A\ and \ B) = p(B\ and \ A)$ 
   2. probability of a conjunction: $p(A\ and \ B) = p(A)p(B \mid A)$
   3. interchange $A$ and $B$: $p(B\ and \ A) = p(B)p(A \mid B)$
   
In the light of point 2 and point 3, we can combine the two equations to one euqation: $\Rightarrow p(A) \ p(B \mid A) = p(B) \ p(A \mid B)$, divided by $p(B)$ in the meantime and we can get the **Bayes's theorem !** : $\Rightarrow \frac{p(A) \ p(B \mid A)}{p(B)} =  p(A \mid B)$

We can have a example, Allen B.Downey called it **The Cookie Problem**: Suppose there are two bowls of cookies. Bowl-1 contains $30$ vanilla cookies and $10$ chocolate cookies. Bowl-2 contains $20$ of each.
Now suppose you choose one of the bowls at random and, without looking, select a cookie at random. The cookie is vanilla. What is the probability that it came from Bowl-1?
This is a conditional probability; we want $p(Bowl-1|vanilla)$, but it is not obvious how to compute it. If I asked a different question — the probability of a vanilla cookie given Bowl-1 — it would be easy: 

$$p(vanilla\mid Bowl-1) = \frac{3}{4}$$

Sadly, $p(A|B)$ is not the same as $p(B|A)$, but there is a way to get from one
to the other: Bayes’s theorem.

Using **Bayes's Theorem**, we can easily solve the cookie problem: <br> 
First, assume that $B_1$ represent the probility that cookie from Bowl-1 and $p(V)$ is the probability of taking out a Vanilla. 

$$P(B_1\mid V) = \frac{p(B_1)p(V\mid B_1)}{p(V)}$$

There is another way to think of Bayes's theorem: it gives us a way to update the probability of a hypothesis, $H$, in light of some body of data,$D$. 

This way of thinking about Bayes's theorem is called the **diachronic interpretation**. "Diachronic" means that something is happening over time; in this case the probability of the hypotheses changes, over time, as we see new data. 

Rewriting Bayes's theorem with $H$ and $D$ yields:

$$P(H|D) = \frac{p(H)p(D|H)}{p(D)}$$

In this interpretation, each term has a name:

* $p(H)$ is the probability of the hypothesis before we see the data, called the prior probability, or just **prior**;
* $p(D)$ is the probability of the data under any hypothesis, called the normalizing constant(total probability);
* $P(H|D)$ is waht we want to compute, the probability of the hypothesis after we see the data, called the **posterior**;
* $P(D|H)$ is the probability of the data under the hypothesis, called the likelihood

In the classical Statistics theory, the data is the random variable. The the Cookie Problem for example, if we want to know the probability of taking out vanilla cookies from Bowl-1, the random variable is $1,2,3,\dots$, the data or the number. However, if we want to know the probability of which Bowls does the vanilla cookie come from, as we got a vanilla cookies already, this time, the random varaible is the probability, not the data. This is the biggest difference between Bayesian and Frequentist. 

## Chapter Two Computational Statistics 

In statistics, a **distribution** is a set of values and thier correponding probabilites. 

You can get more information about Probabilities and Statistics <a href="https://terenceliu98.github.io/Jupytersite//Html/Use_PY_in_Advanced_Statstics.html">here</a>

Here is a example of rolling a dice, as we all know that the probability of each side is $\frac{1}{6}$, then, this is a Discrete Uniform Distribution. 


```python
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt
import matplotlib

fig, ax = plt.subplots(1, 1)
a = [1,2,3,4,5,6]
sample = np.random.randint(len(a), size = 10000)

dict = {}
for key in sample:
    dict[key] = dict.get(key, 0) + 1
len(dict)

x = [dict[i] for i in dict] # change dict to list 
pmf = [dict[i]/len(sample) for i in dict] #calculate pmf of each element in list 

ax.plot(a,pmf,'bo')
ax.vlines(a, 0, pmf, colors='b', lw=5, alpha=0.5)

line1 = [(0, 0.1666667), (10, 0.1666667)]
(line1_xs, line1_ys) = zip(*line1)
ax.add_line(matplotlib.lines.Line2D(line1_xs, line1_ys, linewidth=1, color='red') )        
plt.show()
```


```python

```

7d8233876fe406b71f2c3ab008a25959e9569209
ipynb
Jupyter Notebook
jupyterfiles/Using-Python-in-Bayes.ipynb
ecwu/Using-Python-Series
e6408f92516a7146d572b1d95403ec0c1c90097d
MIT
jupyterfiles/Using-Python-in-Bayes.ipynb
ecwu/Using-Python-Series
e6408f92516a7146d572b1d95403ec0c1c90097d
MIT
jupyterfiles/Using-Python-in-Bayes.ipynb
ecwu/Using-Python-Series
e6408f92516a7146d572b1d95403ec0c1c90097d
MIT
Qwen/Qwen-72B
1. YES
2. YES