```python
%matplotlib inline
import numpy as np
import pandas as pd
import matplotlib.pylab as plt
import seaborn as sns
sns.set_context('notebook')

RANDOM_SEED = 20090425
```

---

# Comparing Two Groups with a Continuous or Binary Outcome

Statistical inference is a process of learning from incomplete or imperfect (error-contaminated) data. Can account for this "imperfection" using either a sampling model or a measurement error model.

### Statistical hypothesis testing

The *de facto* standard for statistical inference is statistical hypothesis testing. The goal of hypothesis testing is to evaluate a **null hypothesis**. There are two possible outcomes:

- reject the null hypothesis
- fail to reject the null hypothesis

Rejection occurs when a chosen test statistic is higher than some pre-specified threshold valuel; non-rejection occurs otherwise.

Notice that neither outcome says anything about the quantity of interest, the **research hypothesis**.

Setting up a statistical test involves several subjective choices by the user that are rarely justified based on the problem or decision at hand:

- statistical test to use
- null hypothesis to test
- significance level

Choices are often based on arbitrary criteria, including "statistical tradition" (Johnson 1999). The resulting evidence is indirect, incomplete, and typically overstates the evidence against the null hypothesis (Goodman 1999).

Most importantly to applied users, the results of statistical hypothesis tests are very easy to misinterpret.

### Estimation

Instead of testing, a more informative and effective approach for inference is based on **estimation** (be it frequentist or Bayesian). That is, rather than testing whether two groups are different, we instead pursue an estimate of *how different* they are, which is fundamentally more informative.

Additionally, we include an estimate of **uncertainty** associated with that difference which includes uncertainty due to our lack of knowledge of the model parameters (*epistemic uncertainty*) and uncertainty due to the inherent stochasticity of the system (*aleatory uncertainty*).

# An Introduction to Bayesian Statistical Analysis

Though many of you will have taken a statistics course or two during your undergraduate (or graduate education, most of those who have will likely not have had a course in *Bayesian* statistics. Most introductory courses, particularly for non-statisticians, still do not cover Bayesian methods at all. Even today, Bayesian courses (similarly to statistical computing courses!) are typically tacked onto the curriculum, rather than being integrated into the program.

In fact, Bayesian statistics is not just a particular method, or even a class of methods; it is an entirely **different paradigm** for doing statistical analysis.

> Practical methods for making inferences from data using probability models for quantities we observe and about which we wish to learn.
*-- Gelman et al. 2013*

A Bayesian model is described by parameters, uncertainty in those parameters is described using probability distributions.

All conclusions from Bayesian statistical procedures are stated in terms of **probability statements**

This confers several benefits to the analyst, including:

- ease of interpretation, summarization of uncertainty
- can incorporate uncertainty in parent parameters
- easy to calculate summary statistics

### Bayesian vs Frequentist Statistics: *What's the difference?*

Any statistical inferece paradigm, Bayesian or otherwise, involves at least the following:

1. Some **unknown quantities** about which we are interested in learning or testing. We call these *parameters*.
2. Some **data** which have been observed, and hopefully contain information about.
3. One or more **models** that relate the data to the parameters, and is the instrument that is used to learn.

### The Frequentist World View

- The **data** that have been observed are considered **random**, because they are realizations of random processes, and hence will vary each time one goes to observe the system.
- Model **parameters** are considered **fixed**. A parameter's true value is uknown and fixed, and so we *condition* on them.

In mathematical notation, this implies a (very) general model of the following form:

\\[f(y | \theta)\\]

Here, the model \\(f\\) accepts data values \\(y\\) as an argument, conditional on particular values of \\(\theta\\).

Frequentist inference typically involves deriving **estimators** for the unknown parameters. Estimators are formulae that return estimates for particular estimands, as a function of data. They are selected based on some chosen optimality criterion, such as *unbiasedness*, *variance minimization*, or *efficiency*.

> For example, lets say that we have collected some data on the prevalence of autism spectrum disorder (ASD) in some defined population. Our sample includes \\(n\\) sampled children, \\(y\\) of them having been diagnosed with autism. A frequentist estimator of the prevalence \\(p\\) is:

>

> $$\hat{p} = \frac{y}{n}$$
>

> Why this particular function? Because it can be shown to be unbiased and minimum-variance.

It is important to note that, in a frequentist world, new estimators need to be derived for every estimand that is introduced.

### The Bayesian World View

- Data are considered **fixed**. They used to be random, but once they were written into your lab notebook/spreadsheet/IPython notebook they do not change.
- Model parameters themselves may not be random, but Bayesians use probability distribtutions to describe their uncertainty in parameter values, and are therefore treated as **random**. In some cases, it is useful to consider parameters as having been sampled from probability distributions.

This implies the following form:

\\[p(\theta | y)\\]

This formulation used to be referred to as ***inverse probability***, because it infers from observations to parameters, or from effects to causes.

Bayesians do not seek new estimators for every estimation problem they encounter. There is only one estimator for Bayesian inference: **Bayes' Formula**.

## Bayes' Formula

Now that we have some probability under our belt, we turn to Bayes' formula. While frequentist statistics uses different estimators for different problems, Bayes formula is the **only estimator** that Bayesians need to obtain estimates of unknown quantities that we care about.

The equation expresses how our belief about the value of \\(\theta\\), as expressed by the **prior distribution** \\(P(\theta)\\) is reallocated following the observation of the data \\(y\\).

The innocuous denominator \\(P(y)\\) usuallt cannot be computed directly, and is actually the expression in the numerator, integrated over all \\(\theta\\):

\\[Pr(\theta|y) = \frac{Pr(y|\theta)Pr(\theta)}{\int Pr(y|\theta)Pr(\theta) d\theta}\\]

The intractability of this integral is one of the factors that has contributed to the under-utilization of Bayesian methods by statisticians.

### Priors

Once considered a controversial aspect of Bayesian analysis, the prior distribution characterizes what is known about an unknown quantity before observing the data from the present study. Thus, it represents the information state of that parameter. It can be used to reflect the information obtained in previous studies, to constrain the parameter to plausible values, or to represent the population of possible parameter values, of which the current study's parameter value can be considered a sample.

### Likelihood functions

The likelihood represents the information in the observed data, and is used to update prior distributions to posterior distributions. This updating of belief is justified becuase of the **likelihood principle**, which states:

> Following observation of \\(y\\), the likelihood \\(L(\theta|y)\\) contains all experimental information from \\(y\\) about the unknown \\(\theta\\).

Bayesian analysis satisfies the likelihood principle because the posterior distribution's dependence on the data is **only through the likelihood**. In comparison, most frequentist inference procedures violate the likelihood principle, because inference will depend on the design of the trial or experiment.

Remember from the density estimation section that the likelihood is closely related to the probability density (or mass) function. The difference is that the likelihood varies the parameter while holding the observations constant, rather than *vice versa*.

## Bayesian Inference, in 3 Easy Steps

Gelman et al. (2013) describe the process of conducting Bayesian statistical analysis in 3 steps.

### Step 1: Specify a probability model

As was noted above, Bayesian statistics involves using probability models to solve problems. So, the first task is to *completely specify* the model in terms of probability distributions. This includes everything: unknown parameters, data, covariates, missing data, predictions. All must be assigned some probability density.

This step involves making choices.

- what is the form of the sampling distribution of the data?
- what form best describes our uncertainty in the unknown parameters?

### Discrete Random Variables

$$X = \{0,1\}$$

$$Y = \{\ldots,-2,-1,0,1,2,\ldots\}$$

**Probability Mass Function**:

For discrete $X$,

$$Pr(X=x) = f(x|\theta)$$

***e.g. Poisson distribution***

The Poisson distribution models unbounded counts:

$$Pr(X=x)=\frac{e^{-\lambda}\lambda^x}{x!}$$

* $X=\{0,1,2,\ldots\}$
* $\lambda > 0$

$$E(X) = \text{Var}(X) = \lambda$$

```python
from pymc3 import Poisson

x = Poisson.dist(mu=1)
samples = x.random(size=10000)
```

```python
samples.mean()
```

    1.0073000000000001

```python
plt.hist(samples, bins=len(set(samples)));
```

### Continuous Random Variables

$$X \in [0,1]$$

$$Y \in (-\infty, \infty)$$

**Probability Density Function**:

For continuous $X$,

$$Pr(x \le X \le x + dx) = f(x|\theta)dx \, \text{ as } \, dx \rightarrow 0$$

***e.g. normal distribution***

$$f(x) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left[-\frac{(x-\mu)^2}{2\sigma^2}\right]$$

* $X \in \mathbf{R}$
* $\mu \in \mathbf{R}$
* $\sigma>0$

$$\begin{align}E(X) &= \mu \cr
\text{Var}(X) &= \sigma^2 \end{align}$$

```python
from pymc3 import Normal

y = Normal.dist(mu=-2, sd=4)
samples = y.random(size=10000)
```

```python
samples.mean()
```

    -1.9495826503791522

```python
samples.std()
```

    3.9997450711227618

```python
plt.hist(samples);
```

### Step 2: Calculate a posterior distribution

The mathematical form \\(p(\theta | y)\\) that we associated with the Bayesian approach is referred to as a **posterior distribution**.

> posterior /pos·ter·i·or/ (pos-tēr´e-er) later in time; subsequent.

Why posterior? Because it tells us what we know about the unknown \\(\theta\\) *after* having observed \\(y\\).

This posterior distribution is formulated as a function of the probability model that was specified in Step 1. Usually, we can write it down but we cannot calculate it analytically. In fact, the difficulty inherent in calculating the posterior distribution for most models of interest is perhaps the major contributing factor for the lack of widespread adoption of Bayesian methods for data analysis. Various strategies for doing so comprise this tutorial.

**But**, once the posterior distribution is calculated, you get a lot for free:

- point estimates
- credible intervals
- quantiles
- predictions

### Step 3: Check your model

Though frequently ignored in practice, it is critical that the model and its outputs be assessed before using the outputs for inference. Models are specified based on assumptions that are largely unverifiable, so the least we can do is examine the output in detail, relative to the specified model and the data that were used to fit the model.

Specifically, we must ask:

- does the model fit data?
- are the conclusions reasonable?
- are the outputs sensitive to changes in model structure?

## Estimation for one group

Before we compare two groups using Bayesian analysis, let's start with an even simpler scenario: statistical inference for one group.

For this we will use Gelman et al.'s (2007) radon dataset. In this dataset the amount of the radioactive gas radon has been measured among different households in all counties of several states. Radon gas is known to be the highest cause of lung cancer in non-smokers. It is believed to be more strongly present in households containing a basement and to differ in amount present among types of soil.

>  the US EPA has set an action level of 4 pCi/L. At or above this level of radon, the EPA recommends you take corrective measures to reduce your exposure to radon gas.

Let's import the dataset:

```python
radon = pd.read_csv('../data/radon.csv', index_col=0)
radon.head()
```

      idnum
      state
      state2
      stfips
      zip
      region
      typebldg
      floor
      room
      basement
      ...
      pcterr
      adjwt
      dupflag
      zipflag
      cntyfips
      county
      fips
      Uppm
      county_code
      log_radon

      0
      5081.0
      MN
      MN
      27.0
      55735
      5.0
      1.0
      1.0
      3.0
      N
      ...
      9.7
      1146.499190
      1.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      0.832909

      1
      5082.0
      MN
      MN
      27.0
      55748
      5.0
      1.0
      0.0
      4.0
      Y
      ...
      14.5
      471.366223
      0.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      0.832909

      2
      5083.0
      MN
      MN
      27.0
      55748
      5.0
      1.0
      0.0
      4.0
      Y
      ...
      9.6
      433.316718
      0.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      1.098612

      3
      5084.0
      MN
      MN
      27.0
      56469
      5.0
      1.0
      0.0
      4.0
      Y
      ...
      24.3
      461.623670
      0.0
      0.0
      1.0
      AITKIN
      27001.0
      0.502054
      0
      0.095310

      4
      5085.0
      MN
      MN
      27.0
      55011
      3.0
      1.0
      0.0
      4.0
      Y
      ...
      13.8
      433.316718
      0.0
      0.0
      3.0
      ANOKA
      27003.0
      0.428565
      1
      1.163151

5 rows × 29 columns

Let's focus on the (log) radon levels measured in a single county (Hennepin).

Suppose we are interested in:

- whether the mean log-radon value is greater than 4 pCi/L in Hennepin county
- the probability that any randomly-chosen household in Hennepin county has a reading of greater than 4

```python
hennepin_radon = radon.query('county=="HENNEPIN"').log_radon
sns.distplot(hennepin_radon)
```

```python
hennepin_radon.shape
```

    (105,)

### The model

Recall that the first step in Bayesian inference is specifying a **full probability model** for the problem.

This consists of:

- a likelihood function(s) for the observations
- priors for all unknown quantities

The measurements look approximately normal, so let's start by assuming a normal distribution as the sampling distribution (likelihood) for the data.

$$y_i \sim N(\mu, \sigma^2)$$

(don't worry, we can evaluate this assumption)

This implies that we have 2 unknowns in the model; the mean and standard deviation of the distribution.

#### Prior choice

How do we choose distributions to use as priors for these parameters?

There are several considerations:

- discrete vs continuous values
- the support of the variable
- the available prior information

While there may likely be prior information about the distribution of radon values, we will assume no prior knowledge, and specify a **diffuse** prior for each parameter.

Since the mean can take any real value (since it is on the log scale), we will use another normal distribution here, and specify a large variance to allow the possibility of very large or very small values:

$$\mu \sim N(0, 10^2)$$

For the standard deviation, we know that the true value must be positive (no negative variances!). I will choose a uniform prior bounded from below at zero and from above at a value that is sure to be higher than any plausible value the true standard deviation (on the log scale) could take.

$$\sigma \sim U(0, 10)$$

We can encode these in a Python model, using the PyMC3 package, as follows:

```python
from pymc3 import Model, Uniform

with Model() as radon_model:

    μ = Normal('μ', mu=0, sd=10)
    σ = Uniform('σ', 0, 10)
```

> ## Software
> Today there is an array of software choices for Bayesians, including both open source software (*e.g.*, Stan, PyMC, JAGS, emcee) and commercial (*e.g.*, SAS, Stata). These examples can be replicated in any of these environments.

All that remains is to add the likelihood, which takes $\mu$ and $\sigma$ as parameters, and the log-radon values as the set of observations:

```python
with radon_model:

    y = Normal('y', mu=μ, sd=σ, observed=hennepin_radon)
```

Now, we will fit the model using a numerical approach called **variational inference**. This will estimate the posterior distribution using an optimized approximation, and then draw samples from it.

```python
from pymc3 import fit

with radon_model:

    samples = fit(random_seed=RANDOM_SEED).sample(1000)
```

    Average Loss = 117.86: 100%|██████████| 10000/10000 [00:00<00:00, 14385.86it/s]
    Finished [100%]: Average Loss = 117.86

```python
from pymc3 import plot_posterior

plot_posterior(samples, varnames=['μ'], ref_val=np.log(4), color='LightSeaGreen');
```

The plot shows the posterior distribution of $\mu$, along with an estimate of the 95% posterior **credible interval**.

The output

    83.1% < 1.38629 < 16.9%%

informs us that the probability of $\mu$ being less than $\log(4)$ is 83.1%% and the corresponding probability of being greater than $\log(4)$ is 16.9%.

> The posterior probability that the mean level of household radon in Henneprin County is greater than 4 pCi/L is 0.17.

### Prediction

What is the probability that a given household has a log-radon measurement larger than one? To answer this, we make use of the **posterior predictive distribution**.

$$p(z |y) = \int_{\theta} p(z |\theta) p(\theta | y) d\theta$$

where here $z$ is the predicted value and y is the data used to fit the model.

We can estimate this from the posterior samples of the parameters in the model.

```python
mus = samples['μ']
sigmas = samples['σ']
```

```python
radon_samples = Normal.dist(mus, sigmas).random()
```

```python
(radon_samples > np.log(4)).mean()
```

    0.46999999999999997

> The posterior probability that a randomly-selected household in Henneprin County contains radon levels in excess of 4 pCi/L is 0.48.

### Model checking

But, ***how do we know this model is any good?***

Its important to check the fit of the model, to see if its assumptions are reasonable. One way to do this is to perform **posterior predictive checks**. This involves generating simulated data using the model that you built, and comparing that data to the observed data.

One can choose a particular statistic to compare, such as tail probabilities or quartiles, but here it is useful to compare them graphically.

We already have these simulations from the previous exercise!

```python
sns.distplot(radon_samples, label='simulated')
sns.distplot(hennepin_radon, label='observed')
plt.legend()
```

### Prior sensitivity

Its also important to check the sensitivity of your choice of priors to the resulting inference.

Here is the same model, but with drastically different (though still uninformative) priors specified:

```python
from pymc3 import Flat, HalfCauchy

with Model() as prior_sensitivity:

    μ = Flat('μ')
    σ = HalfCauchy('σ', 5)

    dist = Normal('dist', mu=μ, sd=σ, observed=hennepin_radon)

    sensitivity_samples = fit(random_seed=RANDOM_SEED).sample(1000)
```

    Average Loss = 114.32: 100%|██████████| 10000/10000 [00:00<00:00, 14780.74it/s]
    Finished [100%]: Average Loss = 114.32

```python
plot_posterior(sensitivity_samples, varnames=['μ'], ref_val=np.log(4), color='LightSeaGreen');
```

Here is the original model for comparison:

```python
plot_posterior(samples, varnames=['μ'], ref_val=np.log(4), color='LightSeaGreen');
```

## Two Groups with Continiuous Outcome

To illustrate how this Bayesian estimation approach works in practice, we will use a fictitious example from Kruschke (2012) concerning the evaluation of a clinical trial for drug evaluation. The trial aims to evaluate the efficacy of a "smart drug" that is supposed to increase intelligence by comparing IQ scores of individuals in a treatment arm (those receiving the drug) to those in a control arm (those recieving a placebo). There are 47 individuals and 42 individuals in the treatment and control arms, respectively.

```python
drug = pd.DataFrame(dict(iq=(101,100,102,104,102,97,105,105,98,101,100,123,105,103,100,95,102,106,
        109,102,82,102,100,102,102,101,102,102,103,103,97,97,103,101,97,104,
        96,103,124,101,101,100,101,101,104,100,101),
                         group='drug'))
placebo = pd.DataFrame(dict(iq=(99,101,100,101,102,100,97,101,104,101,102,102,100,105,88,101,100,
           104,100,100,100,101,102,103,97,101,101,100,101,99,101,100,100,
           101,100,99,101,100,102,99,100,99),
                            group='placebo'))

trial_data = pd.concat([drug, placebo], ignore_index=True)
trial_data.hist('iq', by='group');
```

Since there appear to be extreme ("outlier") values in the data, we will choose a Student-t distribution to describe the distributions of the scores in each group. This sampling distribution adds **robustness** to the analysis, as a T distribution is less sensitive to outlier observations, relative to a normal distribution.

The three-parameter Student-t distribution allows for the specification of a mean $\mu$, a precision (inverse-variance) $\lambda$ and a degrees-of-freedom parameter $\nu$:

$$f(x|\mu,\lambda,\nu) = \frac{\Gamma(\frac{\nu + 1}{2})}{\Gamma(\frac{\nu}{2})} \left(\frac{\lambda}{\pi\nu}\right)^{\frac{1}{2}} \left[1+\frac{\lambda(x-\mu)^2}{\nu}\right]^{-\frac{\nu+1}{2}}$$

the degrees-of-freedom parameter essentially specifies the "normality" of the data, since larger values of $\nu$ make the distribution converge to a normal distribution, while small values (close to zero) result in heavier tails.

Thus, the likelihood functions of our model are specified as follows:

$$\begin{align}
y^{(drug)}_i &\sim T(\nu, \mu_1, \sigma_1) \\
y^{(placebo)}_i &\sim T(\nu, \mu_2, \sigma_2)
\end{align}$$

As a simplifying assumption, we will assume that the degree of normality $\nu$ is the same for both groups.

### Exercise

Draw 10000 samples from a Student-T distribution (`StudentT` in PyMC3) with parameter `nu=3` and compare the distribution of these values to a similar number of draws from a Normal distribution with parameters `mu=0` and `sd=1`.

```python
from pymc3 import StudentT

t = StudentT.dist(nu=3).random(size=10000)
n = Normal.dist(0, 1).random(size=10000)
```

```python
sns.distplot(t, label='Student-T')
sns.distplot(n, label='Normal')
plt.legend()
plt.xlim(-10,10);
```

### Prior choice

Since the means are real-valued, we will apply normal priors. Since we know something about the population distribution of IQ values, we will center the priors at 100, and use a standard deviation that is more than wide enough to account for plausible deviations from this population mean:

$$\mu_k \sim N(100, 10^2)$$

```python
with Model() as drug_model:

    μ_0 = Normal('μ_0', 100, sd=10)
    μ_1 = Normal('μ_1', 100, sd=10)
```

Similarly, we will use a uniform prior for the standard deviations, with an upper bound of 20.

```python
with drug_model:
    σ_0 = Uniform('σ_0', lower=0, upper=20)
    σ_1 = Uniform('σ_1', lower=0, upper=20)
```

For the degrees-of-freedom parameter $\nu$, we will use an **exponential** distribution with a mean of 30; this allocates high prior probability over the regions of the parameter that describe the range from normal to heavy-tailed data under the Student-T distribution.

```python
from pymc3 import Exponential

with drug_model:
    ν = Exponential('ν_minus_one', 1/29.) + 1

```

```python
sns.distplot(Exponential.dist(1/29).random(size=10000), kde=False);
```

```python
from pymc3 import StudentT

with drug_model:

    drug_like = StudentT('drug_like', nu=ν, mu=μ_1, lam=σ_1**-2, observed=drug.iq)
    placebo_like = StudentT('placebo_like', nu=ν, mu=μ_0, lam=σ_0**-2, observed=placebo.iq)
```

Now that the model is fully specified, we can turn our attention to tracking the posterior quantities of interest. Namely, we can calculate the difference in means between the drug and placebo groups.

As a joint measure of the groups, we will also estimate the "effect size", which is the difference in means scaled by the pooled estimates of standard deviation. This quantity can be harder to interpret, since it is no longer in the same units as our data, but it is a function of all four estimated parameters.

```python
from pymc3 import Deterministic

with drug_model:

    diff_of_means = Deterministic('difference of means', μ_1 - μ_0)

    effect_size = Deterministic('effect size',
                        diff_of_means / np.sqrt((σ_1**2 + σ_0**2) / 2))

```

```python
with drug_model:

    drug_trace = fit(random_seed=RANDOM_SEED).sample(1000)
```

    Average Loss = 231.2: 100%|██████████| 10000/10000 [00:01<00:00, 7510.30it/s]
    Finished [100%]: Average Loss = 231.2

```python
plot_posterior(drug_trace[100:],
                varnames=['μ_0', 'μ_1', 'σ_0', 'σ_1', 'ν_minus_one'],
                color='#87ceeb');
```

```python
plot_posterior(drug_trace[100:],
          varnames=['difference of means', 'effect size'],
          ref_val=0,
          color='#87ceeb');
```

> The posterior probability that the mean IQ of subjects in the treatment group is greater than that of the control group is 0.99.

### Exercise

Load the `nashville_precip.txt` dataset. Build a model to compare rainfall in January and July.

- What's the probability that the expected rainfall in January is larger than in July?
- What's the probability that January rainfall exceeds July rainfall in a given year?

```python
nash_precip = pd.read_table('../data/nashville_precip.txt',
                            delimiter='\s+', na_values='NA', index_col=0)
nash_precip.head()
```

      Jan
      Feb
      Mar
      Apr
      May
      Jun
      Jul
      Aug
      Sep
      Oct
      Nov
      Dec

      Year

      1871
      2.76
      4.58
      5.01
      4.13
      3.30
      2.98
      1.58
      2.36
      0.95
      1.31
      2.13
      1.65

      1872
      2.32
      2.11
      3.14
      5.91
      3.09
      5.17
      6.10
      1.65
      4.50
      1.58
      2.25
      2.38

      1873
      2.96
      7.14
      4.11
      3.59
      6.31
      4.20
      4.63
      2.36
      1.81
      4.28
      4.36
      5.94

      1874
      5.22
      9.23
      5.36
      11.84
      1.49
      2.87
      2.65
      3.52
      3.12
      2.63
      6.12
      4.19

      1875
      6.15
      3.06
      8.14
      4.22
      1.73
      5.63
      8.12
      1.60
      3.79
      1.25
      5.46
      4.30

```python
# %load ../exercises/rainfall.py
```

## Two Groups with Binary Outcome

Now that we have seen how to generalize normally-distributed data to another distribution, we are equipped to analyze other data types. Binary outcomes are common in clinical research:

- survival/death
- true/false
- presence/absence
- positive/negative

> *Never, ever dichotomize continuous or ordinal variables prior to statistical analysis*

In practice, binary outcomes are encoded as ones (for event occurrences) and zeros (for non-occurrence). A single binary variable is distributed as a **Bernoulli** random variable:

$$f(x \mid p) = p^{x} (1-p)^{1-x}$$

Such events are sometimes reported as sums of individual events, such as the number of individuals in a group who test positive for a condition of interest. Sums of Bernoulli events are distributed as **binomial** random variables.

$$f(x \mid n, p) = \binom{n}{x} p^x (1-p)^{n-x}$$

The parameter in both models is $p$, the probability of the occurrence of an event. In terms of inference, we are typically interested in whether $p$ is larger or smaller in one group relative to another.

To demonstrate the comparison of two groups with binary outcomes using Bayesian inference, we will use a sample pediatric dataset. Data on 671 infants with very low (<1600 grams) birth weight from 1981-87 were collected at Duke University Medical Center. Of interest is the relationship between the outcome intra-ventricular hemorrhage (IVH) and predictor such as birth weight, gestational age, presence of pneumothorax and mode of delivery.

```python
vlbw = pd.read_csv('../data/vlbw.csv', index_col=0).dropna(axis=0, subset=['ivh', 'pneumo'])
vlbw.head()
```

      birth
      exit
      hospstay
      lowph
      pltct
      race
      bwt
      gest
      inout
      twn
      ...
      vent
      pneumo
      pda
      cld
      pvh
      ivh
      ipe
      year
      sex
      dead

      5
      81.593002
      81.598999
      2.0
      6.969997
      54.0
      black
      925.0
      28.0
      born at Duke
      0.0
      ...
      1.0
      1.0
      0.0
      0.0
      definite
      definite
      NaN
      81.594055
      female
      1

      6
      81.601997
      81.771004
      62.0
      7.189999
      NaN
      white
      940.0
      28.0
      born at Duke
      0.0
      ...
      1.0
      0.0
      0.0
      0.0
      absent
      absent
      absent
      81.602295
      female
      0

      13
      81.683998
      81.853996
      62.0
      7.179996
      182.0
      black
      1110.0
      28.0
      born at Duke
      0.0
      ...
      0.0
      1.0
      0.0
      1.0
      absent
      absent
      absent
      81.684448
      male
      0

      14
      81.689003
      81.877998
      69.0
      7.419998
      361.0
      white
      1180.0
      28.0
      born at Duke
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      absent
      absent
      absent
      81.689880
      male
      0

      16
      81.696999
      81.952004
      93.0
      7.239998
      255.0
      black
      770.0
      26.0
      born at Duke
      0.0
      ...
      1.0
      0.0
      0.0
      1.0
      absent
      absent
      absent
      81.698120
      male
      0

5 rows × 26 columns

To demonstrate binary data analysis, we will try to estimate the difference between the probability of an intra-ventricular hemorrhage for infants with a pneumothorax.

```python
pd.crosstab(vlbw.ivh, vlbw.pneumo)
```

      pneumo
      0.0
      1.0

      ivh

      absent
      359
      73

      definite
      45
      30

      possible
      6
      4

We will create a binary outcome by combining `definite` and `possible` into a single outcome.

```python
ivh = vlbw.ivh.isin(['definite', 'possible']).astype(int).values
x = vlbw.pneumo.astype(int).values
```

### Prior choice

What should we choose as a prior distribution for $p$?

We could stick with a normal distribution, but note that the value of $p$ is **constrained** by the laws of probability. Namely, we cannot have values smaller than zero nor larger than one. So, choosing a normal distribution will result in ascribing positive probability to unsupported values of the parameter. In many cases, this will still work in practice, but will be inefficient for calculating the posterior and will not accurately represent the prior information about the parameter.

A common choice in this context is the **beta distribution**, a continuous distribution with 2 parameters and whose support is on the unit interval:

$$ f(x \mid \alpha, \beta) = \frac{x^{\alpha - 1} (1 - x)^{\beta - 1}}{B(\alpha, \beta)}$$

- Support: $x \in (0, 1)$
- Mean: $\dfrac{\alpha}{\alpha + \beta}$
- Variance: $\dfrac{\alpha \beta}{(\alpha+\beta)^2(\alpha+\beta+1)}$

```python
from pymc3 import Beta

params = (5, 1), (1, 3), (5, 5), (0.5, 0.5), (1, 1)

fig, axes = plt.subplots(1, len(params), figsize=(14, 4), sharey=True)
for ax, (alpha, beta) in zip(axes, params):
    sns.distplot(Beta.dist(alpha, beta).random(size=10000), ax=ax, kde=False)
    ax.set_xlim(0, 1)
    ax.set_title(r'$\alpha={0}, \beta={1}$'.format(alpha, beta));
```

So let's use a beta distribution to model our prior knowledge of the probabilities for both groups. Setting $\alpha = \beta = 1$ will result in a uniform distribution of prior mass:

```python
with Model() as ivh_model:

    p = Beta('p', 1, 1, shape=2)
```

We can now use `p` as the parameter of our Bernoulli likelihood. Here, `x` is a vector of zeros an ones, which will extract the approproate group probability for each subject:

```python
from pymc3 import Bernoulli

with ivh_model:

    bb_like = Bernoulli('bb_like', p=p[x], observed=ivh)
```

Finally, since we are interested in the difference between the probabilities, we will keep track of this difference:

```python
with ivh_model:

    p_diff = Deterministic('p_diff', p[1] - p[0])
```

```python
with ivh_model:
    ivh_trace = fit(random_seed=RANDOM_SEED).sample(1000)
```

    Average Loss = 226.28: 100%|██████████| 10000/10000 [00:00<00:00, 13352.85it/s]
    Finished [100%]: Average Loss = 226.28

```python
plot_posterior(ivh_trace[100:], varnames=['p'], color='#87ceeb');
```

We can see that the probability that `p` is larger for the pneumothorax with probability one.

```python
plot_posterior(ivh_trace[100:], varnames=['p_diff'], ref_val=0, color='#87ceeb');
```

## References and Resources

- Goodman, S. N. (1999). Toward evidence-based medical statistics. 1: The P value fallacy. Annals of Internal Medicine, 130(12), 995–1004. http://doi.org/10.7326/0003-4819-130-12-199906150-00008
- Johnson, D. (1999). The insignificance of statistical significance testing. Journal of Wildlife Management, 63(3), 763–772.
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian Data Analysis, Third Edition. CRC Press.
- Kruschke, J.K. *Doing Bayesian Data Analysis, Second Edition: A Tutorial with R, JAGS, and Stan.* 2015. Academic Press / Elsevier.
- O'Shea M, Savitz D.A., Hage M.L., Feinstein K.A.: *Prenatal events and the risk of subependymal / intraventricular haemorrhage in very low birth weight neonates*. **Paediatric and Perinatal Epdiemiology** 1992;6:352-362

notebooks/2. Basic Bayesian Inference.ipynb


# Introducción a Python

## Introducción

[Python](https://www.python.org/) es un lenguaje de propósito general sencillo y expresivo. Permite acceder cómodamente a una amplia colección de bibliotecas útiles en todos los campos de la informática. Su uso en ciencia y tecnología es cada vez mayor.

Es un lenguaje interpretado, con tipos de datos dinámicos y manejo automático de memoria, que puede utilizarse tanto para escribir programas de forma tradicional como para experimentar en un entorno interactivo. Incluye las construcciones más importantes de programación funcional y admite programación orientada a objetos.

La sintaxis es simple e intuitiva pero hay que tener en cuenta algunas características:

- Los bloques de instrucciones de las construcciones condicionales, bucles y funciones se delimitan mediante la "indentación del código": no se utiliza "end" ni `{` `}`.

- Los índices para acceder a los arrays o listas comienzan en 0 y acaban en tamaño-1. Las secuencias (*range*) usadas en bucles o *list comprehensions* no incluyen el límite superior.

- Algunas funciones tienen la sintaxis tradicional `f(x)`, `g(x,a)`, mientras que otras se expresan como `x.f()`, `x.g(a)`, etc., indicando que el "objeto" `x` se modifica de alguna forma.

- Los arrays y las listas son "mutables": su asignación a otra variable **no** crea una copia del objeto original sino una "referencia" a través de la cual se puede modificar la estructura original.

- Las funciones pueden leer directamente el valor de variables globales, pero para modificarlas hay que declararlas como `global`. La asignación de variables dentro de una función crea variables locales.

- La única forma de crear un ámbito de variables es definir una función. Los índices de los bucles son visibles a la salida.

### Instalación

[Anaconda](https://www.anaconda.com/distribution)

Si partimos de la instalación mínima [miniconda](https://conda.io/miniconda.html) necesitamos los siguiente paquetes:

    > conda install jupyter numpy scipy sympy matplotlib

Arrancamos el "servidor" de notebooks con

    > jupyter notebook

Las celdas de código se evalúan pulsando Mayúsculas-Entrar.

## Tipos simples

Cadenas de caracteres:

```python
s = 'Hola'
```

```python
s
```

    'Hola'

```python
print(s)
```

    Hola

```python
type(s)
```

    str

Se admiten diferentes tipos de delimitadores y cadenas multilínea.

```python
"Hola" + ''' amigos!'''
```

    'Hola amigos!'

Variables lógicas:

```python
c = 3 < 4
```

```python
type(c)
```

    bool

```python
c and (2==1+1) or not (3 != 5)
```

    True

Números reales aproximados con coma flotante de doble precisión:

```python
x = 3.5
```

```python
type(x)
```

    float

Los enteros tienen tamaño ilimitado:

```python
x = 20
```

```python
type(x)
```

    int

```python
x**x
```

    104857600000000000000000000

Variable compleja:

```python
(1+1j)*(1-1j)
```

    (2+0j)

```python
import cmath

cmath.sqrt(-1)
```

    1j

## Control

Condiciones:

```python
k = 7

if k%2 == 0:
    print(k," es par")
else:
    print(k," es impar")
    print("me gustan los impares")
```

    7  es impar
    me gustan los impares

Bucles:

```python
for k in [1,2,3]:
    print(k)
```

    1
    2
    3

```python
for k in range(5):
    print(k)
```

    0
    1
    2
    3
    4

```python
k = 1
p = 1
while k < 5:
    p = p*k
    k = k+1
p
```

    24

## Contenedores

### Tuplas

```python
t = (2,'rojo')
```

```python
t
```

    (2, 'rojo')

```python
t[0]
```

    2

Son inmutables.

### Listas

```python
l = [1,-2,67,0,8,1,3]
```

```python
type(l)
```

    list

También admite elementos de diferentes tipos, incluyendo otras listas, tuplas, o cualquier otro tipo de datos, aunque lo normal es trabajar con listas homogéneas (con elementos del mismo tipo) cuyos elementos pueden procesarse todos de la misma manera usando un bucle.

La extracción de elementos ("indexado"), la longitud de la lista y la suma de sus elementos se consiguen exactamente igual que en las tuplas:

```python
l[2], len(l), sum(l)
```

    (67, 7, 78)

Sin embargo, las listas se diferencian en una característica fundamental. Son **mutables**: podemos añadir o quitar elementos de ellas.

```python
l.append(28)

l
```

    [1, -2, 67, 0, 8, 1, 3, 28]

```python
l += [-2,4]

l
```

    [1, -2, 67, 0, 8, 1, 3, 28, -2, 4]

```python
l.remove(0)

l
```

    [1, -2, 67, 8, 1, 3, 28, -2, 4]

```python
l[2] = 7

l
```

    [1, -2, 7, 8, 1, 3, 28, -2, 4]

```python
l.pop()
```

    4

```python
l
```

    [1, -2, 7, 8, 1, 3, 28, -2]

```python
del l[2]
```

```python
l
```

    [1, -2, 8, 1, 3, 28, -2]

```python
l.insert(3,100)

l
```

    [1, -2, 8, 100, 1, 3, 28, -2]

### Conjuntos

El tipo `set` trata de reproducir el concepto matemático de conjunto. Se construye con llaves y los elementos duplicados se eliminan automáticamente.

```python
C = {1,2,7,1,8,2,1}
C
```

    {1, 2, 7, 8}

Las operaciones de conjuntos están disponibles con símbolos o con "métodos" (funciones en forma de sufijo) . Los detalles pueden encontrarse en la [documentación](https://docs.python.org/3.6/library/stdtypes.html?highlight=set#set).

```python
C.union({0,8})
```

    {0, 1, 2, 7, 8}

```python
C | {0,8}
```

    {0, 1, 2, 7, 8}

```python
C & {5,2}
```

    {2}

```python
C - {2,8,0,5}
```

    {1, 7}

```python
5 in C
```

    False

```python
{1,2} < {5,2,1}
```

    True

### Diccionarios

Es un array asociativo (el índice puede ser cualquier tipo (inmutable)). Es una estructura muy utilizada en Python.

```python
d = {'lunes': 8, 'martes' : [1,2,3], 3: 5}
```

```python
d['martes']
```

    [1, 2, 3]

```python
d.keys()
```

    dict_keys(['lunes', 'martes', 3])

```python
d.values()
```

    dict_values([8, [1, 2, 3], 5])

### Iteración en contenedores

Si queremos procesar todos los elementos de un contenedor podemos hacer un bucle y acceder a cada uno de ellos con la operación de indexado.

```python
lista = [1,2,3,4,5]

for k in range(len(lista)):
    print(lista[k])
```

    1
    2
    3
    4
    5

Esta construcción es tan común que en Python podemos escribirla de forma mucho más natural:

```python
for x in lista:
    print(x)
```

    1
    2
    3
    4
    5

Esto funciona incluso en contenedores como `set` que no admiten el indexado. Los tipos contenedores se pueden "recorrer" directamente mediante un bucle `for`, visitando todos sus elementos.

### Conversión

El nombre de un contenedor es a la vez una función para construir un contenedor de ese tipo a partir de otro contenedor cualquiera.

```python
l = [4,2,2,3,3,3,3,1]

tuple(l)
```

    (4, 2, 2, 3, 3, 3, 3, 1)

```python
set(l)
```

    {1, 2, 3, 4}

```python
list({5,4,3})
```

    [3, 4, 5]

```python
list(range(10))
```

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Esta característica funciona con cualquier otro tipo, no solo con contenedores:

```python
float(5)
```

    5.0

```python
int('54')
```

    54

Si la conversión no es posible se producirá un error.

### Subsecuencias

```python
l = list(range(20))

l
```

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]

```python
l[:5]
```

    [0, 1, 2, 3, 4]

```python
l[4:]
```

    [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]

```python
l[-3:]
```

    [17, 18, 19]

```python
l[5:10:2]
```

    [5, 7, 9]

```python
l[::-1]
```

    [19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

```python
l[10:14] = [0,0,0]

l
```

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 0, 0, 14, 15, 16, 17, 18, 19]

### *List comprehensions*

Cuando utilizamos un bucle para recorrer una lista o realizar un gran número de cálculos los resultados intermedios se pueden imprimir si se desea, pero en cualquier caso al final se pierden.

Muchas veces surge la necesidad de construir una lista (o cualquier otro tipo de contenedor) a partir de los elementos de otra. Una forma de programarlo es empezar con una lista vacía e iterar mediante un bucle añadiendo elementos.

Supongamos que queremos construir una lista con los 100 primeros números cuadrados $1,4,9,16,\ldots,10000$. En principio parece razonable hacer lo siguiente:

```python
r = []
for k in range(1,101):
    r.append(k**2)

print(r)
```

    [1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121, 144, 169, 196, 225, 256, 289, 324, 361, 400, 441, 484, 529, 576, 625, 676, 729, 784, 841, 900, 961, 1024, 1089, 1156, 1225, 1296, 1369, 1444, 1521, 1600, 1681, 1764, 1849, 1936, 2025, 2116, 2209, 2304, 2401, 2500, 2601, 2704, 2809, 2916, 3025, 3136, 3249, 3364, 3481, 3600, 3721, 3844, 3969, 4096, 4225, 4356, 4489, 4624, 4761, 4900, 5041, 5184, 5329, 5476, 5625, 5776, 5929, 6084, 6241, 6400, 6561, 6724, 6889, 7056, 7225, 7396, 7569, 7744, 7921, 8100, 8281, 8464, 8649, 8836, 9025, 9216, 9409, 9604, 9801, 10000]

No está mal, pero los lenguajes modernos proporcionan una herramienta mucho más elegante para expresar este tipo de cálculos. Se conoce como [list comprehension](https://en.wikipedia.org/wiki/List_comprehension) y trata de imitar la notación matemática para definir conjuntos:

$$ r = \{ k^2 \; : \; \forall k \in \mathbb{N},  \;1 \leq k \leq 100 \} $$

```python
r = [ k**2 for k in range(1,101) ]

print(r)
```

    [1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121, 144, 169, 196, 225, 256, 289, 324, 361, 400, 441, 484, 529, 576, 625, 676, 729, 784, 841, 900, 961, 1024, 1089, 1156, 1225, 1296, 1369, 1444, 1521, 1600, 1681, 1764, 1849, 1936, 2025, 2116, 2209, 2304, 2401, 2500, 2601, 2704, 2809, 2916, 3025, 3136, 3249, 3364, 3481, 3600, 3721, 3844, 3969, 4096, 4225, 4356, 4489, 4624, 4761, 4900, 5041, 5184, 5329, 5476, 5625, 5776, 5929, 6084, 6241, 6400, 6561, 6724, 6889, 7056, 7225, 7396, 7569, 7744, 7921, 8100, 8281, 8464, 8649, 8836, 9025, 9216, 9409, 9604, 9801, 10000]

```python
[ k for k in range(100) if k%7 == 0 ]
```

    [0, 7, 14, 21, 28, 35, 42, 49, 56, 63, 70, 77, 84, 91, 98]

```python
[(a,b) for a in range(1,7) for b in range(1,7) if a + b >= 10 ]
```

    [(4, 6), (5, 5), (5, 6), (6, 4), (6, 5), (6, 6)]

```python
{ a+b for a in range(1,7) for b in range(1,7) }
```

    {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}

```python
sum([k**2 for k in range(100+1)])
```

    338350

### Desestructuración

En Python es posible asignar nombres a los elementos de una secuencia de forma muy natural.

Supongamos que tenemos una tupla como la siguiente

```python
t = (3,4,5)
```

y queremos operar con sus elementos. Podemos acceder con un índice:

```python
t[1] + t[2]
```

    9

No hay ningún problema pero el acceso con índice se hace pesado si los elementos aparecen varias veces en el código. En estos casos es mejor ponerles nombre. Podemos hacer

```python
b = t[1]
c = t[2]

b+c
```

    9

Sin embargo Python nos permite algo más elegante:

```python
_,b,c = t

b+c
```

    9

(El nombre `_`  se suele usar cuando no necesitamos ese elemento.)

Usando esta característica podemos escribir varias asignaciones de una vez:

```python
x,y = 23,45
```

Un nombre con asterisco captura dentro de una lista todos los elementos restantes:

```python
s = 'Alberto'

x, y, *z, w = s
```

```python
y
```

    'l'

```python
z
```

    ['b', 'e', 'r', 't']

La desestructuración de argumentos es muy práctica en combinación con las *list comprehensions*:

```python
l = [(k,k**2) for k in range(5)]
l
```

    [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)]

```python
[a+b for a,b in l]
```

    [0, 2, 6, 12, 20]

## Funciones

```python
def sp(n):
    r = n**2+n+41
    return r
```

```python
sp(5)
```

    71

Se pueden devolver varios resultados en una tupla:

```python
import math

def ecsec(a,b,c):
    d = math.sqrt(b**2- 4*a*c)
    s1 = (-b+d)/2/a
    s2 = (-b-d)/2/a
    return (s1,s2)
```

```python
ecsec(2,-6,4)
```

    (2.0, 1.0)

Los paréntesis de la tupla son opcionales.

```python
a,b = ecsec(1,-3,2)

b
```

    1.0

Las variables globales son visibles dentro de las funciones y las asignaciones crean variables locales (a menos que el nombre se declare `global`).

```python
a = 5

b = 8

def f(x):
    b = a+1
    return b

print(f(3))
print(b)
```

    6
    8

```python
a = 5

b = 8

def f(x):
    global b
    b = a+1
    return b

print(f(3))
print(b)
```

    6
    6

Argumentos por omisión:

```python
def incre(x,y=1):
    return x + y

print(incre(5))
print(incre(5,3))
```

    6
    8

Argumentos por nombre:

```python
incre(y=3, x=2)
```

    5

Documentación:

```python
# ? sum
help(sum)
```

    Help on built-in function sum in module builtins:

    sum(iterable, start=0, /)
        Return the sum of a 'start' value (default: 0) plus an iterable of numbers

        When the iterable is empty, return the start value.
        This function is intended specifically for use with numeric values and may
        reject non-numeric types.

```python
def fun(n):
    """Una función muy simple que calcula el triple de su argumento."""
    return 3*n
```

```python
help(fun)
```

    Help on function fun in module __main__:

    fun(n)
        Una función muy simple que calcula el triple de su argumento.

### Bibliotecas

Las funciones definidas en un archivo se pueden utilizar directamente haciendo un `import`. Existe una convención para definir una función `main` que se ejecuta cuando el archivo se arranca como programa y suele usarse para ejecutar tests.

### Programación funcional

En Python 3 las construcciones funcionales crean secuencias "bajo demanda".

```python
map(sp,range(5))
```

    <map at 0x7efcb013a1d0>

```python
for k in map(sp,range(5)):
    print(k)
```

    41
    43
    47
    53
    61

```python
list(map(sp,range(5)))
```

    [41, 43, 47, 53, 61]

```python
list(filter(lambda x: x%2 == 1, range(10)))
```

    [1, 3, 5, 7, 9]

Es poco frecuente usar explícitamente map y filter, ya que su efecto se consigue de forma más cómoda con list comprehensions:

```python
[k**2 for k in range(10) if k >5 ]
```

    [36, 49, 64, 81]

```python
def divis(n):
    return [k for k in range(2,n) if n%k==0]
```

```python
divis(12)
```

    [2, 3, 4, 6]

```python
divis(1001)
```

    [7, 11, 13, 77, 91, 143]

```python
def perfect(n):
    return sum(divis(n)) + 1 == n
```

```python
perfect(4)
```

    False

```python
perfect(6)
```

    True

```python
def prime(n):
    return divis(n)==[]
```

```python
[k for k in range(2,21) if prime(k)]
```

    [2, 3, 5, 7, 11, 13, 17, 19]

```python
from functools import reduce
import operator

def product(l):
    return reduce(operator.mul,l,1)
```

```python
product(range(1,10+1))
```

    3628800

Función que construye funciones:

```python
def mkfun(y):
    return lambda x: x+y
```

```python
f = mkfun(1)
g = mkfun(5)

print(f(10))
print(g(10))
```

    11
    15

```python
fs = list(map(mkfun,range(1,6)))

print(fs[0](10))
print(fs[4](10))
```

    11
    15

## Arrays

Gran parte del éxito de Python se debe a [numpy](http://www.numpy.org/).

```python
import numpy as np
```

Construcción a partir de listas (u otros contenedores):

```python
m = np.array([[5,3, 2,10],
              [2,0, 7, 0],
              [1,1,-3, 6]])
```

```python
m[1,2]
```

    7

Inspección de su tipo y estructura:

```python
type(m)
```

    numpy.ndarray

```python
m.dtype
```

    dtype('int64')

```python
m.shape
```

    (3, 4)

```python
m.ndim
```

    2

```python
len(m)
```

    3

```python
m.size
```

    12

Las operaciones elemento a elemento son automáticas:

```python
5*m + 2
```

    array([[ 27,  17,  12,  52],
           [ 12,   2,  37,   2],
           [  7,   7, -13,  32]])

Constructores especiales:

```python
np.zeros([2,3])
```

    array([[0., 0., 0.],
           [0., 0., 0.]])

```python
np.ones([4])
```

    array([1., 1., 1., 1.])

```python
np.linspace(0,5,11)
```

    array([0. , 0.5, 1. , 1.5, 2. , 2.5, 3. , 3.5, 4. , 4.5, 5. ])

```python
np.arange(10)
```

    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

```python
np.arange(1,10,0.5)
```

    array([1. , 1.5, 2. , 2.5, 3. , 3.5, 4. , 4.5, 5. , 5.5, 6. , 6.5, 7. ,
           7.5, 8. , 8.5, 9. , 9.5])

```python
np.eye(7)
```

    array([[1., 0., 0., 0., 0., 0., 0.],
           [0., 1., 0., 0., 0., 0., 0.],
           [0., 0., 1., 0., 0., 0., 0.],
           [0., 0., 0., 1., 0., 0., 0.],
           [0., 0., 0., 0., 1., 0., 0.],
           [0., 0., 0., 0., 0., 1., 0.],
           [0., 0., 0., 0., 0., 0., 1.]])

Iteración, a lo largo de la primera dimensión:

```python
for e in np.arange(4):
    print(e)
```

    0
    1
    2
    3

```python
for e in m:
    print(e)
```

    [ 5  3  2 10]
    [2 0 7 0]
    [ 1  1 -3  6]

```python
sum(m)
```

    array([ 8,  4,  6, 16])

```python
np.sum(m,axis=1)
```

    array([20,  9,  5])

Operaciones matriciales:

```python
m.T
```

    array([[ 5,  2,  1],
           [ 3,  0,  1],
           [ 2,  7, -3],
           [10,  0,  6]])

```python
v = np.array([3,2,-5,8])
```

El producto de matrices, el producto escalar de vectores, y su generalización para arrays multidimensionales se expresa con el símbolo `@` (que representa a la función `dot`).

```python
m @ v
```

    array([ 91, -29,  68])

```python
np.diag([10,0,1]) @ m
```

    array([[ 50,  30,  20, 100],
           [  0,   0,   0,   0],
           [  1,   1,  -3,   6]])

Las funciones matemáticas están optimizadas para operar con arrays elemento a elemento:

```python
x = np.linspace(0,2*np.pi,30)

x
```

    array([0.        , 0.21666156, 0.43332312, 0.64998469, 0.86664625,
           1.08330781, 1.29996937, 1.51663094, 1.7332925 , 1.94995406,
           2.16661562, 2.38327719, 2.59993875, 2.81660031, 3.03326187,
           3.24992343, 3.466585  , 3.68324656, 3.89990812, 4.11656968,
           4.33323125, 4.54989281, 4.76655437, 4.98321593, 5.1998775 ,
           5.41653906, 5.63320062, 5.84986218, 6.06652374, 6.28318531])

```python
y = np.sin(x) + np.cos(2*x)
y
```

    array([ 1.        ,  1.12254586,  1.06727539,  0.87270255,  0.60038006,
            0.32232498,  0.10669282,  0.00439546,  0.03917335,  0.20298123,
            0.45755084,  0.74183837,  0.9839623 ,  1.1153946 ,  1.08473957,
            0.86850154,  0.47679154, -0.04714542, -0.63356055, -1.19782715,
           -1.65497221, -1.93447969, -1.99267137, -1.82040717, -1.44469911,
           -0.92394405, -0.33764588,  0.22749718,  0.69260498,  1.        ])

Reconfiguración de los elementos:

```python
np.arange(12)
```

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])

```python
np.arange(12).reshape(3,2,2)
```

    array([[[ 0,  1],
            [ 2,  3]],

           [[ 4,  5],
            [ 6,  7]],

           [[ 8,  9],
            [10, 11]]])

### Matrices por bloques

```python
np.append(m,[[100,200,300,400],
             [0,  10,  0,  1] ],axis=0)
```

    array([[  5,   3,   2,  10],
           [  2,   0,   7,   0],
           [  1,   1,  -3,   6],
           [100, 200, 300, 400],
           [  0,  10,   0,   1]])

```python
np.hstack([np.zeros([3,3]),np.ones([3,2])])
```

    array([[0., 0., 0., 1., 1.],
           [0., 0., 0., 1., 1.],
           [0., 0., 0., 1., 1.]])

```python
np.vstack([np.eye(3),5*np.ones([2,3])])
```

    array([[1., 0., 0.],
           [0., 1., 0.],
           [0., 0., 1.],
           [5., 5., 5.],
           [5., 5., 5.]])

numpy proporciona un tipo especial `matrix` para los arrays de 2 dimensiones pero [se recomienda no usarlo](https://stackoverflow.com/questions/4151128/what-are-the-differences-between-numpy-arrays-and-matrices-which-one-should-i-u) o usarlo con cuidado.

### Automatic broadcasting

Las operaciones elemento a elemento argumentos con las mismas dimensiones. Pero si alguna dimensión es igual a uno, se sobreentiende que los elementos se replican en esa dimensión para coincidir con el otro array.

```python
m = np.array([[1, 2, 3, 4]
             ,[5, 6, 7, 8]
             ,[9,10,11,12]])
```

```python
m + [[10],
     [20],
     [30]]
```

    array([[11, 12, 13, 14],
           [25, 26, 27, 28],
           [39, 40, 41, 42]])

```python
m + [100,200,300,400]
```

    array([[101, 202, 303, 404],
           [105, 206, 307, 408],
           [109, 210, 311, 412]])

```python
np.array([[1,2,3,4]])   + np.array([[100],
                                    [200],
                                    [300]])
```

    array([[101, 102, 103, 104],
           [201, 202, 203, 204],
           [301, 302, 303, 304]])

### Slices

Extracción de elementos y "submatrices" o "subarrays", seleccionando intervalos de filas, columnas, etc.:

```python
m = np.arange(42).reshape(6,7)
m
```

    array([[ 0,  1,  2,  3,  4,  5,  6],
           [ 7,  8,  9, 10, 11, 12, 13],
           [14, 15, 16, 17, 18, 19, 20],
           [21, 22, 23, 24, 25, 26, 27],
           [28, 29, 30, 31, 32, 33, 34],
           [35, 36, 37, 38, 39, 40, 41]])

```python
m[1,2]
```

    9

```python
m[2:5,1:4]
```

    array([[15, 16, 17],
           [22, 23, 24],
           [29, 30, 31]])

```python
m[:3, 4:]
```

    array([[ 4,  5,  6],
           [11, 12, 13],
           [18, 19, 20]])

```python
m[[1,0,0,2,1],:]
```

    array([[ 7,  8,  9, 10, 11, 12, 13],
           [ 0,  1,  2,  3,  4,  5,  6],
           [ 0,  1,  2,  3,  4,  5,  6],
           [14, 15, 16, 17, 18, 19, 20],
           [ 7,  8,  9, 10, 11, 12, 13]])

Los índices negativos indican que se empieza a contar desde el final.

```python
# las dos últimas columnas y todas las filas menos las tres últimas.
m[:-3,-2:]
```

    array([[ 5,  6],
           [12, 13],
           [19, 20]])

```python
# la penúltima columna
m[:,-2]
```

    array([ 5, 12, 19, 26, 33, 40])

```python
# la penúltima columna pero como array 2D (matriz), para que se vea como un vector columna
m[:,[-2]]
```

    array([[ 5],
           [12],
           [19],
           [26],
           [33],
           [40]])

### Masks

Extracción de elementos que cumplen una condición:

```python
n = np.arange(10)

n
```

    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

```python
n < 5
```

    array([ True,  True,  True,  True,  True, False, False, False, False,
           False])

```python
n[n<5]
```

    array([0, 1, 2, 3, 4])

```python
k = np.arange(1,101)

(k ** 2)[(k>10) & (k**3 < 2000)]
```

    array([121, 144])

### I/O

La función `np.loadtxt` permite cargar los datos de los arrays a partir de ficheros de texto. También es posible guardar y recuperar arrays en formato binario.

## Gráficas

Uno de los paquetes gráficos más conocidos es `matplotlib`, que puede utilizarse con un interfaz muy parecido al de Matlab/Octave.

```python
import numpy as np
import matplotlib.pyplot as plt

# para insertar los gráficos en el notebook
%matplotlib inline

# para generar ventanas independientes
# %matplotlib qt
# %matplotib  tk
```

```python
x=np.linspace(0,2*np.pi,200)
```

```python
plt.plot(np.sin(x))
```

```python
plt.plot(np.cos(x),np.sin(x)); plt.axis('equal');
```

```python
plt.plot(x,np.sin(x), x,np.cos(x));
```

```python
plt.plot(x,np.sin(x),color='red')
plt.plot(x,np.sin(2*x),color='black')
plt.plot([1,2.5],[-0.5,0],'.',markersize=15);
plt.legend(['hola','fun','puntos']);
plt.xlabel('x'); plt.ylabel('y'); plt.title('bonito plot'); plt.axis('tight');
```

El gráfico se puede exportar en el formato deseado:

```python
# plt.savefig('result.pdf')  # o .svg, .png, .jpg, etc.
```

```python
plt.plot(x,np.exp(x)); plt.axis([0,3,-1,5]);
```

```python
for k in [1,2,3]:
    plt.plot(x,np.sin(k*x))
plt.grid()
```

```python
def espiral(n):
    t = np.linspace(0,n*2*np.pi,1000)
    r = 3 * t
    x = r * np.cos(t)
    y = r * np.sin(t)
    plt.plot(x,y)
    plt.axis('equal')
    plt.axis('off')

espiral(4)
```

```python
import numpy.random as rnd

def randwalk(n,s):
    p = s*rnd.randn(n,2)
    r = np.cumsum(p,axis=0)
    x = r[:,0]
    y = r[:,1]
    plt.plot(x,y)
    plt.axis('equal');
```

```python
plt.figure(figsize=(4,4))
randwalk(1000,1)
```

```python
plt.figure(figsize=(8,8))
x = np.linspace(0,6*np.pi,100);

plt.subplot(2,2,1)
plt.plot(x,np.sin(x),'r')

plt.subplot(2,2,2)
plt.plot(x,np.cos(x))

plt.subplot(2,2,3)
plt.plot(x,np.sin(2*x))

plt.subplot(2,2,4)
plt.plot(x,np.cos(2*x),'g');
```

```python
x,y = np.mgrid[-3:3:0.2,-3:3:0.2]

z = x**2-y**2-1
```

```python
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm

fig = plt.figure(figsize=(8,6))
ax = fig.add_subplot(111, projection='3d')

ax.plot_surface(x,y,x**2-y**2, cmap=cm.coolwarm, linewidth=0.5, rstride=2, cstride=2);
```

```python
plt.figure(figsize=(6,6))
plt.contour(x,y, z , colors=['k']);
plt.axis('equal');
```

### Animaciones

```python
# si se produce un error:
# conda install -c menpo ffmpeg

from matplotlib import animation, rc
from IPython.display import HTML
rc('animation', html='html5')
```

```python
x = np.linspace(0,2,100)

def wave(lam,freq,x,t):
    return 1*np.sin(2*np.pi*(x/lam - t*freq))
```

```python
fig, ax = plt.subplots()
plt.grid()
plt.title('onda viajera')
plt.xlabel('x');
plt.close();
ax.set_xlim(( 0, 2))
ax.set_ylim((-1.1, 1.1))

line1, = ax.plot([], [], '-')
#line2, = ax.plot([], [], '.', markerSize=20)

lam  = 0.8
freq = 1/4

def animate(i):
    t = i/25
    line1.set_data(x,wave(lam,freq,x,t))
    #line2.set_data(1,f(lam,freq,1,t))
    return ()

animation.FuncAnimation(fig, animate, frames=100, interval=1000/25, blit=True)
```

### Data frames

El módulo `pandas` proporciona el tipo "dataframe", muy utilizado en análisis de datos. Permite leer conjuntos de datos almacenados en archivos que pueden estar incluso en una máquina remota.

```python
import pandas as pd

df = pd.read_table('https://robot.inf.um.es/material/data/ConstanteHubbleDatos-1.txt', sep='\s+', comment='#')
df
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>V(km/s)</th>
      <th>Redshift</th>
      <th>Magnitud</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>18287</td>
      <td>0.060998</td>
      <td>17.62</td>
    </tr>
    <tr>
      <th>1</th>
      <td>5691</td>
      <td>0.018983</td>
      <td>15.00</td>
    </tr>
    <tr>
      <th>2</th>
      <td>26382</td>
      <td>0.088000</td>
      <td>18.59</td>
    </tr>
    <tr>
      <th>3</th>
      <td>5996</td>
      <td>0.020000</td>
      <td>15.54</td>
    </tr>
    <tr>
      <th>4</th>
      <td>19202</td>
      <td>0.064051</td>
      <td>15.30</td>
    </tr>
    <tr>
      <th>5</th>
      <td>23684</td>
      <td>0.079000</td>
      <td>16.56</td>
    </tr>
    <tr>
      <th>6</th>
      <td>11702</td>
      <td>0.039034</td>
      <td>17.14</td>
    </tr>
    <tr>
      <th>7</th>
      <td>17284</td>
      <td>0.057653</td>
      <td>13.50</td>
    </tr>
    <tr>
      <th>8</th>
      <td>13491</td>
      <td>0.045000</td>
      <td>17.80</td>
    </tr>
    <tr>
      <th>9</th>
      <td>10566</td>
      <td>0.035244</td>
      <td>15.25</td>
    </tr>
    <tr>
      <th>10</th>
      <td>14718</td>
      <td>0.049094</td>
      <td>15.60</td>
    </tr>
    <tr>
      <th>11</th>
      <td>13491</td>
      <td>0.045000</td>
      <td>14.52</td>
    </tr>
    <tr>
      <th>12</th>
      <td>16325</td>
      <td>0.054453</td>
      <td>15.30</td>
    </tr>
    <tr>
      <th>13</th>
      <td>20686</td>
      <td>0.069000</td>
      <td>16.80</td>
    </tr>
    <tr>
      <th>14</th>
      <td>1808</td>
      <td>0.006031</td>
      <td>11.16</td>
    </tr>
    <tr>
      <th>15</th>
      <td>7603</td>
      <td>0.025361</td>
      <td>15.18</td>
    </tr>
    <tr>
      <th>16</th>
      <td>1018</td>
      <td>0.003395</td>
      <td>12.24</td>
    </tr>
    <tr>
      <th>17</th>
      <td>321</td>
      <td>0.001071</td>
      <td>13.00</td>
    </tr>
    <tr>
      <th>18</th>
      <td>3106</td>
      <td>0.010360</td>
      <td>12.49</td>
    </tr>
    <tr>
      <th>19</th>
      <td>9426</td>
      <td>0.031442</td>
      <td>14.53</td>
    </tr>
    <tr>
      <th>20</th>
      <td>7464</td>
      <td>0.024897</td>
      <td>15.21</td>
    </tr>
    <tr>
      <th>21</th>
      <td>15143</td>
      <td>0.050512</td>
      <td>17.40</td>
    </tr>
    <tr>
      <th>22</th>
      <td>407</td>
      <td>0.001358</td>
      <td>10.87</td>
    </tr>
    <tr>
      <th>23</th>
      <td>7257</td>
      <td>0.024207</td>
      <td>14.60</td>
    </tr>
    <tr>
      <th>24</th>
      <td>9193</td>
      <td>0.030664</td>
      <td>15.10</td>
    </tr>
    <tr>
      <th>25</th>
      <td>12137</td>
      <td>0.040485</td>
      <td>14.75</td>
    </tr>
    <tr>
      <th>26</th>
      <td>4264</td>
      <td>0.014224</td>
      <td>14.98</td>
    </tr>
    <tr>
      <th>27</th>
      <td>4381</td>
      <td>0.014615</td>
      <td>14.15</td>
    </tr>
    <tr>
      <th>28</th>
      <td>22484</td>
      <td>0.075000</td>
      <td>17.43</td>
    </tr>
    <tr>
      <th>29</th>
      <td>15162</td>
      <td>0.050575</td>
      <td>16.50</td>
    </tr>
    <tr>
      <th>30</th>
      <td>30000</td>
      <td>0.101000</td>
      <td>18.90</td>
    </tr>
    <tr>
      <th>31</th>
      <td>12981</td>
      <td>0.043300</td>
      <td>15.23</td>
    </tr>
    <tr>
      <th>32</th>
      <td>8803</td>
      <td>0.029364</td>
      <td>14.90</td>
    </tr>
  </tbody>
</table>
</div>

Puede convertirse en un array normal:

```python
A = np.array(df)
A
```

    array([[1.8287e+04, 6.0998e-02, 1.7620e+01],
           [5.6910e+03, 1.8983e-02, 1.5000e+01],
           [2.6382e+04, 8.8000e-02, 1.8590e+01],
           [5.9960e+03, 2.0000e-02, 1.5540e+01],
           [1.9202e+04, 6.4051e-02, 1.5300e+01],
           [2.3684e+04, 7.9000e-02, 1.6560e+01],
           [1.1702e+04, 3.9034e-02, 1.7140e+01],
           [1.7284e+04, 5.7653e-02, 1.3500e+01],
           [1.3491e+04, 4.5000e-02, 1.7800e+01],
           [1.0566e+04, 3.5244e-02, 1.5250e+01],
           [1.4718e+04, 4.9094e-02, 1.5600e+01],
           [1.3491e+04, 4.5000e-02, 1.4520e+01],
           [1.6325e+04, 5.4453e-02, 1.5300e+01],
           [2.0686e+04, 6.9000e-02, 1.6800e+01],
           [1.8080e+03, 6.0310e-03, 1.1160e+01],
           [7.6030e+03, 2.5361e-02, 1.5180e+01],
           [1.0180e+03, 3.3950e-03, 1.2240e+01],
           [3.2100e+02, 1.0710e-03, 1.3000e+01],
           [3.1060e+03, 1.0360e-02, 1.2490e+01],
           [9.4260e+03, 3.1442e-02, 1.4530e+01],
           [7.4640e+03, 2.4897e-02, 1.5210e+01],
           [1.5143e+04, 5.0512e-02, 1.7400e+01],
           [4.0700e+02, 1.3580e-03, 1.0870e+01],
           [7.2570e+03, 2.4207e-02, 1.4600e+01],
           [9.1930e+03, 3.0664e-02, 1.5100e+01],
           [1.2137e+04, 4.0485e-02, 1.4750e+01],
           [4.2640e+03, 1.4224e-02, 1.4980e+01],
           [4.3810e+03, 1.4615e-02, 1.4150e+01],
           [2.2484e+04, 7.5000e-02, 1.7430e+01],
           [1.5162e+04, 5.0575e-02, 1.6500e+01],
           [3.0000e+04, 1.0100e-01, 1.8900e+01],
           [1.2981e+04, 4.3300e-02, 1.5230e+01],
           [8.8030e+03, 2.9364e-02, 1.4900e+01]])

```python
x = A[:,0]
y = A[:,2]

# x,_,y = A.T

plt.plot(x,y,'.');
```

## Cálculo científico

### Números pseudoaleatorios y estadística elemental

`numpy` permite generar arrays de números pseudoaleatorios con diferentes tipos de distribuciones (uniforme, normal, etc.).

Tiene también funciones de estadística descriptiva para calcular características de conjuntos de datos tales como la media, mediana, desviación típica, máximo y mínimo, etc.

Como ejemplo, podemos estudiar la distribución de puntuaciones al lanzar 3 dados.

```python
dados = np.random.randint(1,6+1,(100,3))
dados[:10]
```

    array([[3, 6, 1],
           [6, 1, 1],
           [4, 4, 2],
           [4, 5, 2],
           [5, 5, 4],
           [2, 3, 2],
           [1, 6, 6],
           [4, 1, 2],
           [2, 3, 3],
           [2, 1, 1]])

```python
s = np.sum(dados,axis=1)
s
```

    array([10,  8, 10, 11, 14,  7, 13,  7,  8,  4, 14, 16, 10, 11, 14, 17,  8,
           11,  9, 14, 17,  3, 14,  6, 14, 10,  7,  8, 12,  9,  9, 14, 11, 11,
            8, 13, 12,  7, 12,  9,  9,  9,  7,  9, 12,  4, 13, 14, 17,  7,  9,
            9, 11,  6, 13, 15, 11,  8, 14, 14, 12,  5,  9, 13, 13,  4,  8, 16,
           13, 12, 11, 14,  7,  7,  7, 10, 10, 13, 11, 11, 10,  8, 11, 10,  7,
           14, 13, 13, 10, 16,  6, 15,  9, 18, 11, 16,  7,  4,  7, 16])

```python
plt.hist(s,bins=np.arange(2,19)+0.5);
```

```python
s.mean()
```

    10.6

```python
s.std()
```

    3.3763886032268267

### Implementación eficiente

Las operaciones de `numpy` están "optimizadas" (escritas internamente en código C eficiente).

```python
x = np.random.rand(10**8)
```

```python
x
```

    array([0.39520383, 0.36193568, 0.16302059, ..., 0.66497153, 0.07501657,
           0.8121129 ])

```python
%%time

np.mean(x)
```

    CPU times: user 129 ms, sys: 465 µs, total: 130 ms
    Wall time: 127 ms

    0.500018403633832

```python
%%timeit

np.mean(x)
```

    85.5 ms ± 8.76 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

```python
%%timeit

x @ x
```

    87.1 ms ± 6.98 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Si la misma operación se realiza "manualmente" con instrucciones normales de Python requiere mucho más tiempo:

```python
%%time

s = 0
for e in x:
    s += e
print(s/len(x))
```

    0.5000184036336627
    CPU times: user 17.2 s, sys: 0 ns, total: 17.2 s
    Wall time: 17.2 s

Por tanto, si usamos los módulos apropiados los programas en Python no tienen por qué ser más lentos que los de otros lenguajes de programación. Python es "glue code", un pegamento para combinar bibliotecas de funciones, escritas en cualquier lenguaje, que resuelven eficientemente problemas específicos.

### Álgebra lineal

El submódulo `linalg` ofrece las operaciones usuales de álgebra lineal.

```python
import scipy.linalg as la
```

Por ejemplo, podemos calcular fácilmente el módulo de un vector:

```python
la.norm([1,2,3,4,5])
```

    7.416198487095663

o el determinante de una matriz:

```python
la.det([[1,2],
        [3,4]])
```

    -2.0

Observa que muchas de las funciones que trabajan con arrays admiten también otros contenedores como listas o tuplas, que son transformadas automáticamente en arrays.

Un problema muy importante es la resolución de sistemas de ecuaciones lineales. Si tenemos que resolver un sistema como

$$
\begin{align*}
x + 2y &= 3\\
3x+4y  &= 5
\end{align*}
$$

Lo expresamos en forma matricial $AX=B$ y podemos resolverlo con la inversa de $A$, o directamente con `solve`.

```python
m = np.array([[1,2],
              [3,4]])
```

```python
m
```

    array([[1, 2],
           [3, 4]])

```python
la.inv(m)
```

    array([[-2. ,  1. ],
           [ 1.5, -0.5]])

```python
la.inv(m) @ np.array([3,5])
```

    array([-1.,  2.])

Es mejor (más eficiente y numéricamente estable) usar la función `solve`:

```python
la.solve(m,[3,5])
```

    array([-1.,  2.])

La solución se debería mostrar como una columna, pero en Python los arrays de una dimensión se imprimen como una fila porque no siempre representan vectores matemáticos. Si lo preferimos podemos usar matrices de una sola columna.

```python
x = la.solve(m,[[3],
                [5]])

x
```

    array([[-1.],
           [ 2.]])

```python
m @ x
```

    array([[3.],
           [5.]])

Si el lado derecho de la ecuación matricial $A X = B$ es una matriz, la solución $X$ también lo será.

### Computación matricial

Python proporciona una [amplia colección](https://docs.scipy.org/doc/scipy/reference/linalg.html) de funciones de álgebra lineal numérica.

```python
la.eigh([[1,2],
         [2,3]])
```

    (array([-0.23606798,  4.23606798]), array([[-0.85065081,  0.52573111],
            [ 0.52573111,  0.85065081]]))

### Mínimos cuadrados

Como ejemplo de uso de las herramientas de álgebra lineal realizaremos el ajuste de un modelo polinomial a unas observaciones ficticias. Encontraremos la solución de mínimo error cuadrático a un sistema de ecuaciones sobredeterminado.

En primer lugar generamos unos datos de prueba artificiales que simulan observaciones contaminadas con ruido de una función no lineal.

```python
import matplotlib.pyplot as plt
%matplotlib inline

x = np.linspace(0,2,30)

y = np.sin(x) + 0.05*np.random.randn(x.size)

plt.plot(x,y,'.');
```

Vamos a ajustar un modelo del tipo $y = ax^2 + bx + c$. Los coeficientes desconocidos $a$, $b$ y $c$ se pueden obtener resolviendo un sistema de ecuaciones lineales.

La matriz de coeficientes tiene potencias de $x$ hasta el grado que nos interesa.

```python
A = np.vstack([x**2, x, np.ones(x.size)]).T

A
```

    array([[0.        , 0.        , 1.        ],
           [0.00475624, 0.06896552, 1.        ],
           [0.01902497, 0.13793103, 1.        ],
           [0.04280618, 0.20689655, 1.        ],
           [0.07609988, 0.27586207, 1.        ],
           [0.11890606, 0.34482759, 1.        ],
           [0.17122473, 0.4137931 , 1.        ],
           [0.23305589, 0.48275862, 1.        ],
           [0.30439952, 0.55172414, 1.        ],
           [0.38525565, 0.62068966, 1.        ],
           [0.47562426, 0.68965517, 1.        ],
           [0.57550535, 0.75862069, 1.        ],
           [0.68489893, 0.82758621, 1.        ],
           [0.80380499, 0.89655172, 1.        ],
           [0.93222354, 0.96551724, 1.        ],
           [1.07015458, 1.03448276, 1.        ],
           [1.2175981 , 1.10344828, 1.        ],
           [1.3745541 , 1.17241379, 1.        ],
           [1.54102259, 1.24137931, 1.        ],
           [1.71700357, 1.31034483, 1.        ],
           [1.90249703, 1.37931034, 1.        ],
           [2.09750297, 1.44827586, 1.        ],
           [2.3020214 , 1.51724138, 1.        ],
           [2.51605232, 1.5862069 , 1.        ],
           [2.73959572, 1.65517241, 1.        ],
           [2.97265161, 1.72413793, 1.        ],
           [3.21521998, 1.79310345, 1.        ],
           [3.46730083, 1.86206897, 1.        ],
           [3.72889417, 1.93103448, 1.        ],
           [4.        , 2.        , 1.        ]])

El lado derecho del sistema es directamente el vector con los valores de $y$, la variable independiente del modelo.

```python
B = np.array(y)

B
```

    array([-0.03376601,  0.1262018 ,  0.10308713,  0.22008185,  0.22293447,
            0.36197156,  0.33667892,  0.34638041,  0.51909713,  0.65396166,
            0.60524298,  0.6565335 ,  0.80283792,  0.80984674,  0.90612391,
            0.86432539,  0.82420153,  0.8831923 ,  0.96355321,  0.95857022,
            1.00002704,  1.00481373,  0.99556629,  0.97193459,  0.9837597 ,
            1.01496467,  0.95858499,  1.00338407,  0.92323694,  0.91704651])

El sistema que hay que resolver está sobredeterminado: tiene solo tres incógnitas y tantas ecuaciones como observaciones de la función.

$$A \begin{bmatrix}a\\b\\c\end{bmatrix}= B$$

La solución de [mínimo error cuadrático](https://en.wikipedia.org/wiki/Least_squares) para los coeficientes del modelo se obtiene de manera directa:

```python
sol = la.lstsq(A,B)[0]

sol
```

    array([-0.39434454,  1.28360329, -0.05193268])

```python
ye = A @ sol

plt.plot(x,y,'.',x,ye,'r');
```

Se puede experimentar con polinomios de mayor o menor grado.

### Solución numérica de ecuaciones no lineales

Resuelve

$$x^4=16$$

```python
import scipy as sci

sci.roots([1,0,0,0,-16])
```

    array([-2.00000000e+00+0.j,  1.66533454e-16+2.j,  1.66533454e-16-2.j,
            2.00000000e+00+0.j])

Resuelve

$$sin(x)+cos(2x)=0$$

```python
import scipy.optimize as opt

opt.fsolve(lambda x: sci.sin(x) + sci.cos(2*x), 0)
```

    array([-0.52359878])

Resuelve

$$
\begin{align*}
x^2 - 3y &= 10\\
sin(x)+y &= 5
\end{align*}
$$

```python
def fun(z):
    x,y = z
    return [ x**2 - 3*y - 10
           , sci.sin(x) + y - 5]

opt.fsolve(fun,[0.1,-0.1])
```

    array([5.2511881 , 5.85832548])

### Minimización

Encuentra $(x,y)$ que minimiza $(x-1)^2 + (y-2)^2-x+3y$

```python
def fun(z):
    x,y = z
    return (x-1)**2 + (y-2)**2 - x + 3*y

opt.minimize(fun,[0.1,-0.1])
```

          fun: 2.500000000000014
     hess_inv: array([[ 0.57758622, -0.18103452],
           [-0.18103452,  0.92241375]])
          jac: array([ 0.00000000e+00, -2.38418579e-07])
      message: 'Optimization terminated successfully.'
         nfev: 12
          nit: 2
         njev: 3
       status: 0
      success: True
            x: array([1.49999999, 0.49999988])

### Derivación numérica

Calcula una aproximación numérica para $f'(2)$ cuando $f(x) = \sin(2x)*\exp(\cos(x))$

```python
from scipy.misc import derivative

derivative(lambda x: sci.sin(2*x)*sci.exp(sci.cos(x)),2,1E-6)
```

    -0.40836700757052036

```python
(lambda x: (-np.sin(x)*np.sin(2*x) + 2*np.cos(2*x))*np.exp(np.cos(x)))(2)
```

    -0.40836700756782335

### Integración numérica

Calcula una aproximación numérica a la integral definida

$$\int_0^1 \frac{4}{1+x^2}dx$$

```python
from scipy.integrate import quad

quad(lambda x: 4/(1+x**2),0,1)
```

    (3.1415926535897936, 3.4878684980086326e-14)

### ecuaciones diferenciales

Resuelve

$$\ddot{x}+0.95x+0.1\dot{x}=0$$

para $x(0)=10$, $\dot{x}(0)=0, t\in[0,20]$

```python
from scipy.integrate import odeint

def xdot(z,t):
    x,v = z
    return [v,-0.95*x-0.1*v]

t = np.linspace(0,20,1000)
r = odeint(xdot,[10,0],t)
# plt.plot(r);
plt.plot(t,r[:,0],t,r[:,1]);
```

```python
plt.plot(r[:,0],r[:,1]);
```

### Cálculo simbólico

[sympy](http://www.sympy.org/en/index.html)

```python
import sympy

x = sympy.Symbol('x')
```

```python
sympy.diff( sympy.sin(2*x**3) , x)
```

    6*x**2*cos(2*x**3)

```python
sympy.integrate(1/(1+x))
```

    log(x + 1)

## Varios

### Vídeos

```python
from IPython.display import YouTubeVideo
```

```python
YouTubeVideo('p7bzE1E5PMY')
```

### xkcd

```python
plt.xkcd()
plt.plot(np.sin(np.linspace(0, 10)))
plt.title('Whoo Hoo!!!');
```

### Estilo del notebook

```python
# podemos "tunearlo"
#from IPython.display import HTML
#HTML(open('../css/nb1.css').read())
```