```python
%matplotlib inline
import numpy as np
import pandas as pd
import scipy.stats
import matplotlib.pyplot as plt

from matplotlib import animation
from matplotlib import rcParams
rcParams['figure.dpi'] = 120
from IPython.display import HTML
from IPython.display import YouTubeVideo
from functools import partial
YouTubeVideo_formato = partial(YouTubeVideo, modestbranding=1, disablekb=0,
                               width=640, height=360, autoplay=0, rel=0, showinfo=0)
```

# Estadística inferencial

La inferencia busca

> Extraer **conclusiones** a partir de **hechos u observaciones** a través de un **método o premisa**

En el caso particular de la **inferencia estadística** podemos realizar las siguientes asociaciones

- Hechos: Datos
- Premisa: Modelo probabilístico
- Conclusión: Una cantidad no observada que es interesante

Y lo que buscamos es

> Cuantificar la incerteza de la conclusión dado los datos y el modelo

La inferencia estadística puede dividirse en los siguientes tres niveles

1. Ajustar un modelo a nuestros datos
1. Verificar que el modelo sea confiable
1. Responder una pregunta usando el modelo

En esta lección estudiaremos las herramientas más utilizadas asociadas a cada uno de estos niveles

1. **Estimador de máxima verosimilitud**
1. **Bondad de ajuste** e **Intervalos de confianza**
1. **Test de hipótesis**

## Ajuste de modelos: Estimación de máxima verosimilitud

En este nivel de inferencia se busca **ajustar** un modelo teórico sobre nuestros datos. En esta lección nos enfocaremos en **modelos de tipo parámetrico**. Un modelo parámetrico es aquel donde **se explicita una distribución de probabilidad**.

Recordemos que una distribución tiene **parámetros**. Por ejemplo la distribución Gaussiana (univariada) se describe por su media $\mu$ y su varianza $\sigma^2$. Luego ajustar una distribución Gaussiana corresponde a encontrar el valor de $\mu$ y $\sigma$ que hace que el modelo se parezca lo más posible a la distribución empírica de los datos.

A continuación veremos los pasos necesarios para ajustar una distribución a nuestros datos

### ¿Qué distribución ajustar?

Antes de ajustar debemos realizar un supuesto sobre la distribución para nuestro modelo. En general podemos ajustar cualquier distribución pero un mal supuesto podría invalidar nuestra inferencia

Podemos usar las herramientas de **estadística descriptiva** para estudiar nuestros datos y tomar esta decisión de manera informada

En el siguiente ejemplo, un histograma de los datos revela que un modelo gaussiano no es una buena decisión

¿Por qué? La distribución empírica es claramente asimétrica, su cola derecha es más pesada que su cola izquierda. La distribución Gaussiana es simétrica por lo tanto no es apropiada en este caso ¿Qué distribución podría ser más apropiada?

### ¿Cómo ajustar mi modelo? Estimación de máxima verosimilitud

A continuación describiremos un procedimiento para ajustar modelos paramétricos llamado *maximum likelihood estimation* (MLE)

Sea un conjunto de datos $\{x_1, x_2, \ldots, x_N\}$

**Supuesto 1** Los datos siguen el modelo $f(x;\theta)$ donde $f(\cdot)$ es una distribución y $\theta$ son sus parámetros

$$
f(x_1, x_2, \ldots, x_N |\theta)
$$

**Supuesto 2** Las observaciones son independientes e idénticamente distribuidas (iid)

- Si dos variables son independientes se cumple que $P(x, y) = P(x)P(y)$
- Si son además idénticamente distribuidas entonces tienen **la misma distribución y parámetros**

Usando esto podemos escribir

$$
\begin{align}
f(x_1, x_2, \ldots, x_N |\theta) &= f(x_1|\theta) f(x_2|\theta) \ldots f(x_N|\theta) \nonumber \\
& = \prod_{i=1}^N f(x_i|\theta) \nonumber \\
& = \mathcal{L}(\theta)
\end{align}
$$

donde $\mathcal{L}(\theta)$ se conoce como la verosimilitud o probabilidad inversa de $\theta$

Si consideramos que los datos son fijos podemos buscar el valor de $\theta$ de máxima verosimilitud

$$
\begin{align}
\hat \theta &= \text{arg} \max_\theta \mathcal{L}(\theta) \nonumber \\
&= \text{arg} \max_\theta \log \mathcal{L}(\theta) \nonumber \\
&= \text{arg} \max_\theta \sum_{i=1}^N \log f(x_i|\theta)
\end{align}
$$

El segundo paso es valido por que el máximo de $g(x)$ y $\log(g(x))$ es el mismo. El logaritmo es monoticamente creciente. Además aplicar el logaritmo es muy conveniente ya que convierte la multiplicatoria en una sumatoria.

Ahora sólo falta encontrar el máximo. Podemos hacerlo

- Analíticamente, derivando con respecto a $\theta$ e igualando a cero
- Usando técnicas de optimización iterativas como gradiente descedente

**Ejemplo:** La pesa defectuosa

Su profesor quiere medir su peso pero sospecha que su pesa está defectuosa. Para comprobarlo mide su peso $N$ veces obteniendo un conjunto de observaciones $\{x_i\}$. ¿Es posible obtener un estimador del peso real $\hat x$ a partir de estas observaciones?

Modelaremos las observaciones como

$$
x_i = \hat x + \varepsilon_i
$$

donde $\varepsilon_i$ corresponde al ruido o error del instrumento y asumiremos que $\varepsilon_i \sim \mathcal{N}(0, \sigma_\varepsilon^2)$, es decir que el ruido es **independiente** y **Gaussiano** con media cero y **varianza** $\sigma_\varepsilon^2$ **conocida**

Entonces la distribución de $x_i$ es

$$
f(x_i|\hat x) = \mathcal{N}(\hat x, \sigma_\varepsilon^2)
$$

Para encontrar $\hat x$, primero escribimos el logaritmo de la **verosimilitud**

$$
\begin{align}
\log \mathcal{L}(\hat x) &=  \sum_{i=1}^N \log f(x_i|\hat x) \nonumber \\
&= \sum_{i=1}^N  \log \frac{1}{\sqrt{2\pi\sigma_\varepsilon^2}}  \exp \left ( - \frac{1}{2\sigma_\varepsilon^2} (x_i - \hat x)^2 \right)  \nonumber \\
&= -\frac{N}{2}\log(2\pi\sigma_\varepsilon^2)  - \frac{1}{2\sigma_\varepsilon^2}  \sum_{i=1}^N  (x_i - \hat x)^2  \nonumber
\end{align}
$$

Luego debemos resolver

$$
\begin{align}
\hat \theta &= \text{arg} \max_\theta \log \mathcal{L}(\theta) \nonumber \\
&= \text{arg} \max_\theta - \frac{1}{2\sigma_\varepsilon^2}  \sum_{i=1}^N  (x_i - \hat x)^2
\end{align}
$$

donde podemos ignorar el primer término de la verosimilitud ya que no depende de $\theta$. Para encontrar el máximo derivamos la expresión anterior e igualamos a cero

$$
-\frac{1}{2\sigma_\varepsilon^2} \sum_{i=1}^N 2(x_i - \hat x ) = 0.
$$

Finalmente si despejamos llegamos a que

$$
\hat x = \frac{1}{N} \sum_{i=1}^N x_i,
$$

que se conoce como el estimador de máxima verosimilitud **para la media de una Gaussiana**

Recordemos que podemos comprobar que es un máximo utilizando la segunda derivada

### Estimación MLE con `scipy`

Como vimos en la lección anterior el módulo [`scipy.stats`](https://docs.scipy.org/doc/scipy/reference/stats.html) provee de un gran número de distribuciones teóricas organizadas como

- continuas de una variable
- discretas de una variable
- multivariadas

Las distribuciones comparten muchos de sus métodos, a continuación revisaremos los más importantes. A modo de ejemplo consideremos la distribución Gaussiana (Normal)

```python
from scipy.stats import norm
dist = norm() # Esto crea una Gaussiana con media 0 y desviación estándar (std) 1
dist = norm(loc=2, scale=2) # Esto crea una Gaussiana con media 2 y std 2
```

**Crear una muestra aleatoria con `rvs`**

Luego de crear un objeto distribución podemos obtener una muestra aleatoria usando el método el atributo `rvs`

```python
dist = norm(loc=2, scale=2)
dist.rvs(size=10, # Cantidad de números aleatorios generados
         random_state=None #Semilla aleatoria
        )
```

Esto retorna un arreglo de 10 números generados aleatoriamente a partir de `dist`

**Evaluar la función de densidad de probabilidad**

La función de densidad de la Gaussiana es

$$
f(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi \sigma^2}} \exp \left( -\frac{1}{2\sigma^2} (x-\mu)^2 \right)
$$

La densidad de un objeto distribución continuo puede obtenerse con el método `pdf` el cual es función de `x`

```python
dist = norm(loc=2, scale=2)
p = dist.pdf(x # Un ndrray que representa x en la ecuación superior
            )
plt.plot(x, p) # Luego podemos graficar la fdp
```

De forma equivalente, si deseamos la función de densidad acumulada usamos el método `cdf`

Para objetos distribución discretos debemos usar el atributo `pmf`

**Ajustar los parámetros con MLE**

Para hacer el ajuste se usa el método `fit`

```python
params = norm.fit(data # Un ndarray con los datos
                 )
```

En el caso de la Gaussiana el vector `params` tiene dos componentes `loc` y `scale`. La cantidad de parámetros depende de la distribución que estemos ajustando. También es importante notar que para ajustar se usa `norm` (clase abstracta) y no `norm()` (instancia)

Una vez que tenemos los parámetros ajustados podemos usarlos con

```python
dist = norm(loc=params[0], scale=params[1])
```

Para distribuciones que tienen más de dos parámetros podemos usar

```python
dist = norm(*params[:-2], loc=params[-2], scale=params[-1])
```

### Ejercicio

Observe la siguiente distribución y reflexione ¿Qué características resaltan de la misma? ¿Qué distribución sería apropiado ajustar en este caso?

```python
df = pd.read_csv('../data/cancer.csv', index_col=0)
df = df[["diagnosis", "radius1", "texture1"]]
x = df["radius1"].values
fig, ax = plt.subplots(figsize=(5, 3), tight_layout=True)
ax.hist(x, bins=20, density=True)
ax.set_xlabel('Radio del nucleo');
```

- Seleccione una distribución de `scipy.stats`  ajustela a los datos
- Grafique la pdf teórica sobre el histograma

```python

```

## Verificación de modelos: Tests de bondad de ajuste

Una vez que hemos ajustado un modelo es buena práctica verificar que tan confiable es este ajuste. Las herramientas más típicas para medir que tan bien se ajusta nuestra distribución teórica son

- el [test de Akaike](https://en.wikipedia.org/wiki/Akaike_information_criterion)
- los [gráficos cuantil-cuantil](https://es.wikipedia.org/wiki/Gr%C3%A1fico_Q-Q) (QQ plot)
- el test no-paramétrico de Kolmogorov-Smirnov (KS)

A continuación revisaremos el test de KS para bondad de ajuste

**El test de Kolmogorov-Smirnov**

Es un test no-paramétrico que compara una muestra de datos estandarizados (distribución empírica) con una distribución de densidad acumulada (CDF) teórica. Este test busca refutar la siguiente hipótesis

> **Hipótesis nula:** Las distribuciones son idénticas

Para aplicar el test primero debemos **estandarizar** los datos. Estandarizar se refiere a la transformación

$$
z = \frac{x - \mu_x}{\sigma_x}
$$

es decir los datos estándarizados tienen media cero y desviación estándar uno

Esto puede hacerse fácilmente con NumPy usando

```python
z = (x - np.mean(x))/np.std(x)
```

### Test de KS con `scipy`

Podemos realizar el test de KS con la función [`scipy.stats.kstest`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.kstest.html) donde

```python
scipy.stats.kstest(rvs, # Una muestra de observaciones estandarizadas
                   cdf, # Una distribución acumulada teórica, por ejemplo scipy.stats.norm.cdf
                   ...
                  )
```

Esta función retorna el valor del estadístico de KS y su *p-value* asociado. Mientras más cerca de cero sea el estadístico de KS mejor es el ajuste.

Más adelante haremos un repaso de tests de hipótesis en detalle. De momento recordemos que si el *p-value* es menor que una confianza $\alpha=0.05$ entonces rechazamos la hipótesis nula con confianza $1-\alpha = 0.95$ o $95\%$

### Ejercicio

Considere la muestra de datos anterior
- Seleccione un conjunto de distribuciones teóricas
- Encuentra la que tiene mejor ajuste usando `kstest`

```python

```

## Responder preguntas con nuestro modelo: Test de hipótesis

Se aplica un tratamiento nuevo a una muestra de la población

- ¿Es el tratamiento efectivo?
- ¿Existe una diferencia entre los que tomaron el tratamiento y los que no?

El test de hipótesis es un procedimiento estadístico para comprobar si el resultado de un experimento es significativo en la población

Para esto formulamos dos escenarios cada uno con una hipótesis asociada

- Hipótesis nula ($H_0$): Por ejemplo
    - "El experimento no produjo diferencia"
    - "El experimento no tuvo efecto"
    - "Las observaciones son producto del azar"
- Hipótesis alternativa ($H_A$): Usualmente el complemento de $H_0$

> El test de hipótesis se diseña para medir que tan fuerte es la evidencia **en contra** de la hipótesis nula

### Algoritmo general de un test de hipótesis

El siguiente es el algoritmo general de un test de hipótesis paramétrico

1. Definimos $H_0$ y $H_A$
1. Definimos un estadístico $T$
1. Asumimos una distribución para $T$ dado que $H_0$ es cierto
1. Seleccionamos un nivel de significancia $\alpha$
1. Calculamos el $T$ para nuestros datos $T_{data}$
1. Calculamos el **p-value**
    - Si nuestro test es de una cola:
        - Superior: $p = P(T>T_{data})$
        - Inferior: $p = P(T<T_{data})$
    - Si nuestro test es dos colas: $p = P(T>T_{data}) + P(T<T_{data})$

Finalmente:

`Si`  $p < \alpha$

> Rechazamos la hipótesis nula con confianza (1-$\alpha$)

`De lo contrario`

> No hay suficiente evidencia para rechazar la hipótesis nula

El valor de $\alpha$ nos permite controlar el **[Error tipo I](https://es.wikipedia.org/wiki/Errores_de_tipo_I_y_de_tipo_II)**, es decir el error que cometemos si rechazamos $H_0$ cuando en realidad era cierta (falso positivo)

Tipicamente se usa $\alpha=0.05$ o $\alpha=0.01$

**Errores de interpretación comunes**

Muchas veces se asume que el p-value es la probabilidad de que $H_0$ sea cierta dado nuestras observaciones

$$
p = P(H_0 | T> T_{data})
$$

Esto es un **grave error**. Formálmente el **p-value** es la probabilidad de observar un valor de $T$ más extremo que el observado, es decir

$$
p = P(T> T_{data} | H_0)
$$

Otro error común es creer que no ser capaz de rechazar $H_0$ es lo mismo que aceptar $H_0$

No tener suficiente evidencia para rechazar no es lo mismo que aceptar

### Un primer test de hipótesis: El t-test de una muestra

Sea un conjunto de $N$ observaciones iid $X = {x_1, x_2, \ldots, x_N}$ con media muestral $\bar x = \sum_{i=1}^N x_i$

El t-test de una muestra es un test de hipótesis que busca verificar si $\bar x$ es significativamente distinta de la **media poblacional** $\mu$, en el caso de que **no conocemos la varianza poblacional** $\sigma^2$

Las hipótesis son

- $H_0:$ $\bar x = \mu$
- $H_A:$ $\bar x \neq \mu$ (dos colas)

El estadístico de prueba es

$$
t = \frac{\bar x - \mu}{\hat \sigma /\sqrt{N-1}}
$$

donde $\hat \sigma = \sqrt{ \frac{1}{N} \sum_{i=1}^N (x_i - \bar x)^2}$ es la desviación estándar muestral (sesgada)

Si asumimos que $\bar x$ se distribuye $\mathcal{N}(\mu, \frac{\sigma^2}{N})$ entonces
$t$ se distribuye [t-student](https://en.wikipedia.org/wiki/Student%27s_t-distribution) con $N-1$ grados de libertad

- Para muestras iid y $N$ grande el supuesto se cumple por teorema central del límite
- Si $N$ es pequeño debemos verificar la normalidad de los datos

### Aplicación de t-test para probar que la regresión es significativa

En un modelo de regresión lineal donde tenemos $N$ ejemplos

$$
y_i = x_i \theta_1 + \theta_0, ~ i=1, 2, \ldots, N
$$

Podemos probar que la correlación entre $x$ es $y$ es significativa con un test sobre $\theta_1$

Por ejemplo podemos plantear las siguientes hipótesis

- $H_0:$ La pendiente es nula $\theta_1= 0$
- $H_A:$ La pendiente no es nula: $\theta_1\neq 0$ (dos colas)

Y asumiremos que $\theta_1$ es normal pero que desconocemos su varianza. Bajo este supuesto se puede formular el siguiente estadístico de prueba

$$
t = \frac{(\theta_1-\theta^*) }{\text{SE}_{\theta_1}/\sqrt{N-2}} = \frac{ r\sqrt{N-2}}{\sqrt{1-r^2}},
$$

donde $r$ es el coeficiente de correlación de Pearson (detalles más adelante) y la última expresión se obtiene reemplazando  $\theta^*=0$ y $\text{SE}_{\theta_1} = \sqrt{ \frac{\frac{1}{N} \sum_i (y_i - \hat y_i)^2}{\text{Var}(x)}}$.

El estadístico tiene distribución t-student con dos grados de libertad (modelo de dos parámetros)

## Ejercicio formativo: Regresión lineal

En lecciones anteriores estudiamos el modelo de regresión lineal el cual nos permite estudiar si existe correlación entre variables continuas. También vimos como ajustar los parámetros del modelo usando el método de mínimos cuadrados. En este ejercicio formativo veremos como verificar si el modelo de regresión ajustado es correcto

Luego de revisar este ejercicio usted habrá aprendido

- La interpretación probabilística de la regresión lineal y la relación entre mínimos cuadrados ordinarios y la estimación por máxima verosimilitud
- El estadístico $r$ para medir la fuerza de la correlación entre dos variables
- Un test de hipótesis para verificar que la correlación encontrada es estadística significativa

Usaremos el siguiente dataset de consumo de helados. Referencia: [A handbook of small datasets](https://www.routledge.com/A-Handbook-of-Small-Data-Sets/Hand-Daly-McConway-Lunn-Ostrowski/p/book/9780367449667), estudio realizado en los años 50

```python
df = pd.read_csv('../data/helados.csv', header=0, index_col=0)
df.columns = ['consumo', 'ingreso', 'precio', 'temperatura']
display(df.head())
```

El dataset tiene la temperatura promedio del día (grados Fahrenheit), el precio promedio de los helados comprados (dolares), el ingreso promedio familiar semanal de las personas que compraron helado (dolares) y el consumo ([pintas](https://en.wikipedia.org/wiki/Pint) per capita).

A continuación se muestra un gráfico de dispersión del consumo en función de las demás variables. ¿Cree usted que existe correlación en este caso?

```python
fig, ax = plt.subplots(1, 3, figsize=(8, 3), tight_layout=True, sharey=True)
for i, col in enumerate(df.columns[1:]):
    ax[i].scatter(df[col], df["consumo"], s=10)
    ax[i].set_xlabel(col)
ax[0].set_ylabel(df.columns[0]);
```

### Interpretación probabilística y MLE de la regresión lineal

Sea $y$ el consumo y $x$ la temperatura.

Asumiremos errores gaussianos iid

$$
y_i = \hat y_i + \epsilon_i, \epsilon_i \sim \mathcal{N}(0, \sigma^2),
$$

y un modelo lineal de **dos parámetros** (linea recta)

$$
\hat y_i = \theta_0 + \theta_1 x_i
$$

Bajo estos supuestos el estimador de máxima verosimilitud es

$$
\begin{align}
\hat \theta &= \text{arg}\max_\theta \log \mathcal{L}(\theta) \nonumber \\
&=\text{arg}\max_\theta  - \frac{1}{2\sigma^2} \sum_{i=1}^N (y_i - \theta_0 - \theta_1 x_i)^2 \nonumber
\end{align}
$$

Es decir que el estimador de máxima verosimilitud es equivalente al de mínimos cuadrados ordanrios $\hat \theta= (X^T X)^{-1} X^T y$ que vimos anteriormente

**Importante:** Cuando utilizamos la solución de mínimos cuadrados estamos asumiendo implicitamente que las observaciones son iid y que la verosimilitud es Gaussiana

Derivando con respecto a los parámetros e igualado a cero tenemos que

$$
\begin{align}
\sum_i y_i  - N\theta_0 - \theta_1  \sum_i x_i &= 0 \nonumber \\
\sum_i y_i x_i - \theta_0 \sum_i x_i - \theta_1 \sum_i x_i^2 &= 0 \nonumber
\end{align}
$$

Finalmente podemos despejar

$$
\begin{align}
\theta_0 &= \bar y - \theta_1 \bar x \nonumber \\
\theta_1 &= \frac{\sum_i x_i y_i - N \bar x \bar y}{\sum_i x_i^2 - M \bar x^2}  \nonumber \\
&= \frac{ \sum_i (y_i - \bar y)(x_i - \bar x)}{\sum_i (x_i - \bar x)^2}  = \frac{\text{COV}(x, y)}{\text{Var}(x)}
\end{align}
$$

de donde reconocemos las expresiones para la covarianza entre $x$ e $y$ y la varianza de $x$

### Coeficiente de correlación de Pearson

La fuerza de la correlación se suele medir usando

$$
r^2 = 1 - \frac{\sum_i ( y_i - \hat y_i)^2}{\sum_i ( y_i - \bar y)^2} = 1 - \frac{\frac{1}{M} \sum_i (y_i - \hat y_i)^2}{\text{Var}(y)} = \frac{\text{COV}^2(x, y)}{\text{Var}(x) \text{Var}(y)}
$$

donde $r = \frac{\text{COV}(x, y)}{\sqrt{\text{Var}(x) \text{Var}(y)}} \in [-1, 1]$ se conoce como [coeficiente de correlación de Pearson](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)

donde

- si $r=1$ existe una correlación lineal perfecta
- si $r=-1$ existe una anticorrelación lineal perfecta
- si $r=0$ no hay correlación lineal entre las variables

En general un $r>0.5$ se considera una correlación importante

**Calculando $r$ con y los parámetros de la regresión lineal**

Podemos usar el atributo de dataframe

```python
df.corr()
```

Que retorna la matriz de correlaciones lineales

```python
df.corr()
```

Si queremos también el valor de los parámetros podemos usar la función de scipy

```python
scipy.stats.linregress(x, # Variable independiente unidimensional
                       y # Variable dependiente unidimensional
                      )
```

Esta función retorna una tupla con

- Valor de la pendiente: $\theta_1$
- Valor de la intercepta: $\theta_0$
- Coeficiente de correlación $r$
- p-value
- Error estándar del ajuste

```python
fig, ax = plt.subplots(1, 3, figsize=(8, 3), tight_layout=True, sharey=True)
ax[0].set_ylabel(df.columns[0]);

for i, col in enumerate(df.columns[1:]):
    res = scipy.stats.linregress(df[col], df["consumo"])
    x_plot = np.linspace(np.amin(df[col]), np. amax(df[col]), num=100)
    ax[i].scatter(df[col], df["consumo"], label='datos', s=10)
    ax[i].plot(x_plot, res.slope*x_plot + res.intercept, lw=2, c='r', label='modelo');
    ax[i].set_xlabel(col)
    ax[i].set_title(f"$r$: {res.rvalue:0.5f}")
    ax[i].legend()
```

Es decir que visualmente parece existir

- una correlación positiva alta entre consumo y temperatura
- una correlación negativa moderada entre consumo y precio
- una correlación cercana a cero entre consumo e ingreso

### Test de hipótesis y conclusiones

La función `linregress` implementa el t-test sobre $\theta_1$ que vimos anteriormente. Usemos estos resultados para verificar si las correlaciones son estadísticamente significativas

```python
alpha = 0.05

for i, col in enumerate(df.columns[1:]):
    res = scipy.stats.linregress(df[col], df["consumo"])
    print(f"{col}: \t p-value:{res.pvalue:0.4f} \t ¿Menor que {alpha}?: {res.pvalue < alpha}")
```

Como complemento visualizemos

- las distribuciones bajo la hipótesis nula: linea azul
- los límites dados por $\alpha$: linea punteada negra
- El valor del observado para cada una de las variables: linea roja

```python
fig, ax = plt.subplots(1, 3, figsize=(8, 2), tight_layout=True, sharey=True)
ax[0].set_ylabel(df.columns[0]);

N = df.shape[0]
t = np.linspace(-7, 7, num=1000)
dist = scipy.stats.t(loc=0, scale=1, df=N-2) # dos grados de libertad

for i, col in enumerate(df.columns[1:]):
    res = scipy.stats.linregress(df[col], df["consumo"])
    t_data = res.rvalue*np.sqrt(N-2)/np.sqrt(1.-res.rvalue**2)
    ax[i].plot(t, dist.pdf(t))
    ax[i].plot([dist.ppf(alpha/2)]*2, [0, np.amax(dist.pdf(t))], 'k--')
    ax[i].plot([dist.ppf(1-alpha/2)]*2, [0, np.amax(dist.pdf(t))], 'k--')
    ax[i].plot([t_data]*2, [0, np.amax(dist.pdf(t))], 'r-')
    ax[i].set_xlabel(col)
```

**Conclusión**

Basado en los p-values y considerando $\alpha=0.05$

¿Qué podemos decir de las correlaciones con el consumo de helados?

> Rechazamos la hipótesis nula de que no existe correlación entre temperatura y consumo con un 95% de confianza

Para las variables ingreso y precio no existe suficiente evidencia para rechazar $H_0$

### Reflexión final

En el ejercicio anterior usamos t-test para una regresión lineal entre dos variables ¿Qué prueba puedo usar si quiero hacer regresión lineal multivariada?

> Se puede usar [ANOVA](https://pythonfordatascience.org/anova-python/)

¿Qué pasa si...

- mis datos tienen una relación que no es lineal?
- $\theta_1$ no es Gaussiano/normal?
- si el ruido no es Gaussiano?
- si el ruido es Gaussiano pero su varianza cambia en el tiempo?

> En estos casos no se cumplen los supuestos del modelo o del test, por ende el resultado no es confiable

Si mis supuestos no se cumplen con ninguna prueba parámetrica, la opión es utilizar pruebas no-paramétricas

## Prueba no-paramétrica: *Bootstrap*

Podemos estimar la incerteza de un estimador de forma no-paramétrica usando **muestreo tipo *bootstrap***

Esto consiste en tomar nuestro conjunto de datos de tamaño $N$ y crear $T$ nuevos conjuntos que "se le parezcan". Luego se calcula el valor del estimador que estamos buscando en los $T$ conjuntos. Con esto obtenemos una distribución para el estimador como muestra el siguiente diagrama

Para crear los subconjuntos podríamos suponer independencia y utilizar **muestreo con reemplazo**. Esto consiste en tomar $N$ muestras al azar permitiendo repeticiones, como muestra el siguiente diagrama

Si no es posible suponer indepdencia se puede realizar bootstrap basado en residuos y bootstrap dependiente. Puedes consultar más detalles sobre [*bootstrap*](https://www.stat.cmu.edu/~cshalizi/402/lectures/08-bootstrap/lecture-08.pdf) [aquí](http://homepage.divms.uiowa.edu/~rdecook/stat3200/notes/bootstrap_4pp.pdf) y [acá](https://www.sagepub.com/sites/default/files/upm-binaries/21122_Chapter_21.pdf). A continuación nos enfocaremos en el clásico muestreo con reemplazo y como implementarlo en Python

### Implementación con Numpy y Scipy

La función `numpy.random.choice` permite remuestrear un conjunto de datos

Por ejemplo para la regresión lineal debemos remuestrar las parejas/tuplas $(x_i, y_i)$

Luego calculamos y guardamos los parámetros del modelo para cada remuestreo. En este ejemplo haremos $1000$ repeticiones del conjunto de datos

```python
df = pd.read_csv('../data/helados.csv', header=0, index_col=0)
df.columns = ['consumo', 'ingreso', 'precio', 'temperatura']

x, y = df["temperatura"].values, df["consumo"].values
params = scipy.stats.linregress(x, y)

def muestreo_con_reemplazo(x, y):
    N = len(x)
    idx = np.random.choice(N, size=N, replace=True)
    return x[idx], y[idx]

def boostrap_linregress(x, y, T=100):
    # Parámetros: t0, t1 y r
    params = np.zeros(shape=(T, 3))
    for t in range(T):
        res = scipy.stats.linregress(*muestreo_con_reemplazo(x, y))
        params[t, :] = [res.intercept, res.slope, res.rvalue]
    return params

boostrap_params = boostrap_linregress(x, y, T=1000)
```

### Intervalos de confianza empíricos

Veamos la distribución empírica de $r$ obtenida usando bootstrap

En la figura de abajo tenemos

- Histograma azul: Distribución bootstrap de $r$
- Linea roja: $r$ de los datos
- Lineas punteadas negras: Intervalo de confianza empírico al 95%

```python
r_bootstrap = boostrap_params[:, 2]

fig, ax = plt.subplots(figsize=(4, 3), tight_layout=True)
hist_val, hist_lim, _ = ax.hist(r_bootstrap, bins=20, density=True)

ax.plot([params.rvalue]*2, [0, np.max(hist_val)], 'r-', lw=2)
IC = np.percentile(r_bootstrap, [2.5, 97.5])
ax.plot([IC[0]]*2, [0, np.max(hist_val)], 'k--', lw=2)
ax.plot([IC[1]]*2, [0, np.max(hist_val)], 'k--', lw=2)

print(f"Intervalo de confianza al 95% de r: {IC}")
```

De la figura podemos notar que el 95% de la distribución empírica esta sobre $r=0.5$

También podemos notar que la distribución empírica de $r$ no es simétrica, por lo que aplicar un t-test parámetrico sobre $r$ no hubiera sido correcto

### Visualizando la incerteza del modelo

Usando la distribución empírica de los parámetros $\theta_0$ y $\theta_1$ podemos visualizar la incerteza de nuestro modelo de regresión lineal

En la figura de abajo tenemos
- Puntos azules: Datos
- Linea roja: Modelo de regresión lineal en los datos
- Sombra rojo claro: $\pm 2$ desviaciones estándar del modelo en base a la distribución empírica

```python
fig, ax = plt.subplots(figsize=(4, 3), tight_layout=True)
ax.set_ylabel('Consumo')
ax.set_xlabel('Temperatura')
ax.scatter(x, y, zorder=100, s=10, label='datos')

def model(theta0, theta1, x):
    return x*theta1 + theta0

ax.plot(x_plot, model(params.intercept, params.slope, x_plot),
        c='r', lw=2, label='mejor ajuste')

dist_lines = model(boostrap_params[:, 0], boostrap_params[:, 1], x_plot.reshape(-1, 1)).T
mean_lines, std_lines = np.mean(dist_lines, axis=0), np.std(dist_lines, axis=0)
ax.fill_between(x_plot,
                mean_lines - 2*std_lines,
                mean_lines + 2*std_lines,
                color='r', alpha=0.25, label='incerteza')
plt.legend();
```

```python

```


```python
# Mount Google Drive
from google.colab import drive # import drive from google colab

ROOT = "/content/drive"     # default location for the drive
print(ROOT)                 # print content of ROOT (Optional)

drive.mount(ROOT,force_remount=True)
```

    /content/drive
    Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly

    Enter your authorization code:
    ··········
    Mounted at /content/drive

#  Neuromatch Academy: Week 1, Day 1, Tutorial 2
# Model Types: "How" models
__Content creators:__ Matt Laporte, Byron Galbraith, Konrad Kording

__Content reviewers:__ Dalin Guo, Aishwarya Balwani, Madineh Sarvestani, Maryam Vaziri-Pashkam, Michael Waskom

___
# Tutorial Objectives
This is tutorial 2 of a 3-part series on different flavors of models used to understand neural data. In this tutorial we will explore models that can potentially explain *how* the spiking data we have observed is produced

To understand the mechanisms that give rise to the neural data we save in Tutorial 1,  we will build simple neuronal models and compare their spiking response to real data. We will:
- Write code to simulate a  simple "leaky integrate-and-fire" neuron model
- Make the model more complicated — but also more realistic — by adding more physiologically-inspired details

```python
#@title Video 1: "How" models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='PpnagITsb3E', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=PpnagITsb3E

# Setup

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```

```python
#@title Figure Settings
import ipywidgets as widgets #interactive display

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")

```

```python
#@title Helper Functions
def histogram(counts, bins, vlines=(), ax=None, ax_args=None, **kwargs):
  """Plot a step histogram given counts over bins."""
  if ax is None:
    _, ax = plt.subplots()

  # duplicate the first element of `counts` to match bin edges
  counts = np.insert(counts, 0, counts[0])

  ax.fill_between(bins, counts, step="pre", alpha=0.4, **kwargs)  # area shading
  ax.plot(bins, counts, drawstyle="steps", **kwargs)  # lines

  for x in vlines:
    ax.axvline(x, color='r', linestyle='dotted')  # vertical line

  if ax_args is None:
    ax_args = {}

  # heuristically set max y to leave a bit of room
  ymin, ymax = ax_args.get('ylim', [None, None])
  if ymax is None:
    ymax = np.max(counts)
    if ax_args.get('yscale', 'linear') == 'log':
      ymax *= 1.5
    else:
      ymax *= 1.1
      if ymin is None:
        ymin = 0

  if ymax == ymin:
    ymax = None

  ax_args['ylim'] = [ymin, ymax]

  ax.set(**ax_args)
  ax.autoscale(enable=False, axis='x', tight=True)

def plot_neuron_stats(v, spike_times):
  fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(12, 5))

  # membrane voltage trace
  ax1.plot(v[0:100])
  ax1.set(xlabel='Time', ylabel='Voltage')
  # plot spike events
  for x in spike_times:
    if x >= 100:
      break
    ax1.axvline(x, color='red')

  # ISI distribution
  isi = np.diff(spike_times)
  n_bins = np.arange(isi.min(), isi.max() + 2) - .5
  counts, bins = np.histogram(isi, n_bins)
  vlines = []
  if len(isi) > 0:
    vlines = [np.mean(isi)]
  xmax = max(20, int(bins[-1])+5)
  histogram(counts, bins, vlines=vlines, ax=ax2, ax_args={
    'xlabel': 'Inter-spike interval',
    'ylabel': 'Number of intervals',
    'xlim': [0, xmax]
  })
  plt.show()
```

# Section 1: The Linear Integrate-and-Fire Neuron

How does a neuron spike?

A neuron charges and discharges an electric field across its cell membrane. The state of this electric field can be described by the _membrane potential_. The membrane potential rises due to excitation of the neuron, and when it reaches a threshold a spike occurs. The potential resets, and must rise to a threshold again before the next spike occurs.

One of the simplest models of spiking neuron behavior is the linear integrate-and-fire model neuron. In this model, the neuron increases its membrane potential $V_m$ over time in response to excitatory input currents $I$ scaled by some factor $\alpha$:

\begin{align}
  dV_m = {\alpha}I
\end{align}

Once $V_m$ reaches a threshold value a spike is produced, $V_m$ is reset to a starting value, and the process continues.

Here, we will take the starting and threshold potentials as $0$ and $1$, respectively. So, for example, if $\alpha I=0.1$ is constant---that is, the input current is constant---then $dV_m=0.1$, and at each timestep the membrane potential $V_m$ increases by $0.1$ until after $(1-0)/0.1 = 10$ timesteps it reaches the threshold and resets to $V_m=0$, and so on.

Note that we define the membrane potential $V_m$ as a scalar: a single real (or floating point) number. However, a biological neuron's membrane potential will not be exactly constant at all points on its cell membrane at a given time. We could capture this variation with a more complex model (e.g. with more numbers). Do we need to?

The proposed model is a 1D simplification. There are many details we could add to it, to preserve different parts of the complex structure and dynamics of a real neuron. If we were interested in small or local changes in the membrane potential, our 1D simplification could be a problem. However, we'll assume an idealized "point" neuron model for our current purpose.

#### Spiking Inputs

Given our simplified model for the neuron dynamics, we still need to consider what form the input $I$ will take. How should we specify the firing behavior of the presynaptic neuron(s) providing the inputs to our model neuron?

Unlike in the simple example above, where $\alpha I=0.1$, the input current is generally not constant. Physical inputs tend to vary with time. We can describe this variation with a distribution.

We'll assume the input current $I$ over a timestep is due to equal contributions from a non-negative ($\ge 0$) integer number of input spikes arriving in that timestep. Our model neuron might integrate currents from 3 input spikes in one timestep, and 7 spikes in the next timestep. We should see similar behavior when sampling from our distribution.

Given no other information about the input neurons, we will also assume that the distribution has a mean (i.e. mean rate, or number of spikes received per timestep), and that the spiking events of the input neuron(s) are independent in time. Are these reasonable assumptions in the context of real neurons?

A suitable distribution given these assumptions is the Poisson distribution, which we'll use to model $I$:

\begin{align}
  I \sim \mathrm{Poisson}(\lambda)
\end{align}

where $\lambda$ is the mean of the distribution: the average rate of spikes received per timestep.

### Exercise 1: Compute $dV_m$

For your first exercise, you will write the code to compute the change in voltage $dV_m$ (per timestep) of the linear integrate-and-fire model neuron. The rest of the code to handle numerical integration is provided for you, so you just need to fill in a definition for `dv` in the `lif_neuron` function below. The value of $\lambda$ for the Poisson random variable is given by the function argument `rate`.

The [`scipy.stats`](https://docs.scipy.org/doc/scipy/reference/stats.html) package is a great resource for working with and sampling from various probability distributions. We will use the `scipy.stats.poisson` class and its method `rvs` to produce Poisson-distributed random samples. In this tutorial, we have imported this package with the alias `stats`, so you should refer to it in your code as `stats.poisson`.

```python
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  """ Simulate a linear integrate-and-fire neuron.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    rate (int): The mean rate of incoming spikes

  """
  # precompute Poisson samples for speed
  exc = stats.poisson(rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []

  ################################################################################
  # Students: compute dv, then comment out or remove the next line
  #raise NotImplementedError("Excercise: compute the change in membrane potential")
  ################################################################################

  for i in range(1, n_steps):

    dv = alpha*exc[i]

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines after completing the lif_neuron function
v, spike_times = lif_neuron()
plot_neuron_stats(v, spike_times)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D1_ModelTypes/solutions/W1D1_Tutorial2_Solution_f8960ca1.py)

*Example output:*

## Interactive Demo: Linear-IF neuron
Like last time, you can now explore how various parametes of the LIF model influence the ISI distribution.

```python
#@title

#@markdown You don't need to worry about how the code works – but you do need to **run the cell** to enable the sliders.

def _lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  exc = stats.poisson(rate).rvs(n_steps)
  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = alpha * exc[i]
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0
  return v, spike_times

@widgets.interact(
  n_steps=widgets.FloatLogSlider(1000.0, min=2, max=4),
  alpha=widgets.FloatLogSlider(0.01, min=-2, max=-1),
  rate=widgets.IntSlider(10, min=5, max=20)
)
def plot_lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  v, spike_times = _lif_neuron(int(n_steps), alpha, rate)
  plot_neuron_stats(v, spike_times)
```

    interactive(children=(FloatLogSlider(value=1000.0, description='n_steps', min=2.0), FloatLogSlider(value=0.01,…

```python
#@title Video 2: Linear-IF models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='QBD7kulhg4U', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=QBD7kulhg4U

# Section 2: Inhibitory signals

Our linear integrate-and-fire neuron from the previous section was indeed able to produce spikes. However, our ISI histogram doesn't look much like empirical ISI histograms seen in Tutorial 1, which had an exponential-like shape. What is our model neuron missing, given that it doesn't behave like a real neuron?

In the previous model we only considered excitatory behavior -- the only way the membrane potential could decrease was upon a spike event. We know, however, that there are other factors that can drive $V_m$ down. First is the natural tendency of the neuron to return to some steady state or resting potential. We can update our previous model as follows:

\begin{align}
  dV_m = -{\beta}V_m + {\alpha}I
\end{align}

where $V_m$ is the current membrane potential and $\beta$ is some leakage factor. This is a basic form of the popular Leaky Integrate-and-Fire model neuron (for a more detailed discussion of the LIF Neuron, see the Appendix).

We also know that in addition to excitatory presynaptic neurons, we can have inhibitory presynaptic neurons as well. We can model these inhibitory neurons with another Poisson random variable:

\begin{align}
I = I_{exc} - I_{inh} \\
I_{exc} \sim \mathrm{Poisson}(\lambda_{exc}) \\
I_{inh} \sim \mathrm{Poisson}(\lambda_{inh})
\end{align}

where $\lambda_{exc}$ and $\lambda_{inh}$ are the average spike rates (per timestep) of the excitatory and inhibitory presynaptic neurons, respectively.

### Exercise 2: Compute $dV_m$ with inhibitory signals

For your second exercise, you will again write the code to compute the change in voltage $dV_m$, though now of the LIF model neuron described above. Like last time, the rest of the code needed to handle the neuron dynamics are provided for you, so you just need to fill in a definition for `dv` below.

```python
def lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """

  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []

  ###############################################################################
  # Students: compute dv, then comment out or remove the next line
  #raise NotImplementedError("Excercise: compute the change in membrane potential")
  ################################################################################

  for i in range(1, n_steps):

    dv = (exc[i]-inh[i])*alpha -beta*v[i-1]
    #use v[i-1] because we're calculating v[i] in this step, so it's based on v[i-1]

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines do make the plot once you've completed the function
v, spike_times = lif_neuron_inh()
plot_neuron_stats(v, spike_times)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W1D1_ModelTypes/solutions/W1D1_Tutorial2_Solution_4d9a2677.py)

*Example output:*

## Interactive Demo: LIF + inhibition neuron

```python
#@title
#@markdown **Run the cell** to enable the sliders.
def _lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = -beta * v[i-1] + alpha * (exc[i] - inh[i])
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

@widgets.interact(n_steps=widgets.FloatLogSlider(1000.0, min=2.5, max=4),
                  alpha=widgets.FloatLogSlider(0.5, min=-1, max=1),
                  beta=widgets.FloatLogSlider(0.1, min=-1, max=0),
                  exc_rate=widgets.IntSlider(12, min=10, max=20),
                  inh_rate=widgets.IntSlider(12, min=10, max=20))
def plot_lif_neuron(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  v, spike_times = _lif_neuron_inh(int(n_steps), alpha, beta, exc_rate, inh_rate)
  plot_neuron_stats(v, spike_times)
```

```python
#@title Video 3: LIF + inhibition
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='Aq7JrxRkn2w', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=Aq7JrxRkn2w

#Summary

In this tutorial we gained some intuition for the mechanisms that produce the observed behavior in our real neural data. First, we built a simple neuron model with excitatory input and saw that it's behavior, measured using the ISI distribution, did not match our real neurons. We then improved our model by adding leakiness and inhibitory input. The behavior of this balanced model was much closer to the real neural data.

# Bonus

### Why do neurons spike?

A neuron stores energy in an electric field across its cell membrane, by controlling the distribution of charges (ions) on either side of the membrane. This energy is rapidly discharged to generate a spike when the field potential (or membrane potential) crosses a threshold. The membrane potential may be driven toward or away from this threshold, depending on inputs from other neurons: excitatory or inhibitory, respectively. The membrane potential tends to revert to a resting potential, for example due to the leakage of ions across the membrane, so that reaching the spiking threshold depends not only on the amount of input ever received following the last spike, but also the timing of the inputs.

The storage of energy by maintaining a field potential across an insulating membrane can be modeled by a capacitor. The leakage of charge across the membrane can be modeled by a resistor. This is the basis for the leaky integrate-and-fire neuron model.

### The LIF Model Neuron

The full equation for the LIF neuron is

\begin{align}
C_{m}\frac{dV_m}{dt} = -(V_m - V_{rest})/R_{m} + I
\end{align}

where $C_m$ is the membrane capacitance, $R_M$ is the membrane resistance, $V_{rest}$ is the resting potential, and $I$ is some input current (from other neurons, an electrode, ...).

In our above examples we set many of these parameters to convenient values ($C_m = R_m = dt = 1$, $V_{rest} = 0$) to focus more on the general behavior of the model. However, these too can be manipulated to achieve different dynamics, or to ensure the dimensions of the problem are preserved between simulation units and experimental units (e.g. with $V_m$ given in millivolts, $R_m$ in megaohms, $t$ in milliseconds).


# Quality control

**ENGSCI233: Computational Techniques and Computer Systems**

*Department of Engineering Science, University of Auckland*

It is not realistic to write code that is completely free from **bugs**. However, we should strive to eliminate as many as possible from our work. Although this is not a software design course, there are a number of good practices that we can borrow from that field. With practice, you will develop a set of useful habits - **unit testing, version control, and writing specifications** - that will help to minimise bugs, and make it easy for other people (and your future self) to understand, use and modify your code.

You need to know:
- How to write a unit test, a function that tests that a specific part of your code has been correctly implemented.
- The key elements of a specification, a brief description that informs a user how your code works: inputs, outputs, purpose of a function, preconditions and post conditions, and writing a Python docstring.
- Practical version control. A repository as a cloud-based copy of your code - you can clone a copy, make changes and map them back up to the cloud.

```python
# loading support files - only execute this block if running via Google Colab; if so, execute it before anything else.
# download notebook files
!git clone https://github.com/bryan-ruddy/ENGSCI233_2021.git
%cd ENGSCI233_2021/quality_control
```

```python
# imports and environment: this cell must be executed before any other in the notebook
%matplotlib notebook
from quality_control233 import*
```

Content in this module has been drawn from this MIT OpenCourseWare course:

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-005-software-construction-spring-2016/readings/

## 1 Unit testing

***Checking our code is bug-free.***

Large software projects might comprise thousands of lines of code. These must be written sufficiently well that the "whole program" achieves its objective. Code will generally be organised in a modular fashion, as a collection of functions and subroutines. Many of these will perform a small, specific task. These in turn will be called by other functions, achieving perhaps some task of intermediate complexity, and so on and so forth.

A bug or error in an elementary function can propagate its effects to other parts of the code, compromising the software. **Unit testing** is the practice of checking for and catching these errors. We do this by actively **trying** to make the code fail. You should put aside any feelings of pride and accomplishment in your work. Instead, approach the task with the methodical, sociopathic brutality of a university lecturer writing a final exam0.

0  it's a joke settle down

Let's look at an example:

Consider the function below that computes the negative square of an input number, i.e., $-x^2$ for input $x$.

```python
# run this cell to make the function available
def neg_square(x):
    return (-x)*(-x)
```

You should immediately see that `neg_square` has been improperly implemented. However, **it will still return a result**, i.e., no error is raised. This means that, if we're not paying attention, this bug has the potential to cause mischief elsewhere in our code.

A **unit test** is another function that we write whose express purpose is to test that one of our functions is **working correctly**. But what does it mean to be "working correctly"?

- The function should return the correct result.
- The function should return the correct result **for every possible value** $x$.
- The function should return the correct result **for every possible value** $x$, and **anticipate the stupidity** of the user, e.g., `neg_square('an apple')`.

The right test depends on how rigourous you need to be, and the tolerance and implications of failure by your software. Let's take a look at one example.

```python
# make sure to run the cell above defining `neg_square` before running this cell
def test_neg_square():
    assert neg_square(2) == -4

test_neg_square()
```

The test above raises an `AssertionError` on the line with the `assert` command. Indeed, this is the express purpose of `assert` - to raise an error in the program when a condition evaluates to `False`. So the unit test is doing its job, signalling loud and clear that there is a bug in your code.

***Fix the implementation of ***`neg_square`*** above so that the it passes the unit test.***

### 1.1 Subdomains and edge cases

You should now have a unit test verifying that `neg_square` works for the specific case of $x=2$. Common sense would tell us it should also work for $x=3$ or $x=1003$, but these are basically the same inputs: **positive integers larger than 1**. And it is not really practical to run a test for all integers larger than 1 - there are a lot of them.

How about other input types? Negative integers? Floats? The special case of zero? An integer so large that squaring it will cause overflow? Strange and sometimes unexpected things can happen when you pass extreme or idiosyncratic values into your functions.

When designing a unit test, you'll usually want to try an input from each of the different sub-domains - all positive integers, all negative integers - and edges between sub-domains - zero, negative infinity.

***Copy-paste the unit test above and add `assert` statements to check proper behaviour for positive and negative integers and floats, zero and infinities.***

```python
# use np.inf to represent "infinity"
import numpy as np

def test_neg_square():
    # your code here
    pass

test_neg_square()
```

If your function contains `if/else` branches, then a single unit test may miss a buggy line of code if it is in the wrong branch. **Statement coverage** is the idea that you should write multiple unit tests to invoke code on the different branches, running as may lines of code (statements) as possible. One hundred percent statement coverage may not be practical.

### 1.2 Testing suites

Even modestly complex programs can run to thousands of lines codes and tens or hundreds of functions. Writing and running unit tests for all of these can be exhausting **but is good practice**. When you discover bugs in your code, you should immediately write a unit test for it.

Another coding philosophy is **test-driven development** or test-first programming: first, write a unit test, then, write a function that passes it. You won't need to do test-driven development in this course, but you may encounter it in industry (when doing internships, etc).

In instrumentation development, some of your code may be written to interact directly with custom hardware. This may mean you need to test your code with the hardware, or with a physical prototype of it. This is called **hardware-in-the-loop** testing. Again, you won't need to do this in this course, but may encounter it in industry, especially if you work on safety-critical or failure critical systems. (Think ventilators, or [rockets](https://www.youtube.com/watch?v=xahiWQQKw7Y&t=101s).)

Each time you sit down to write some code, you should run all your unit tests - the test suite - before starting (especially if you're working on someone else's code) and again once you have finished (especially if other people are working with yours!)

With lots of tests, this can be a painful process. Thankfully, there are several automated testing programs to streamline the process. We will use one in the lab called `py.test`.

## 2 Specifications

***Communicating the purpose of our code.***

**You write a function for someone else to use**1.

Let's define some terminology and use it to unpack that statement.

- The **implementor** (you). The person who **writes** the function.
- The **client** (someone else). The person who **uses** the function.
- The **contract**. The unspoken division of labour. You (the implementor) are writing the function and someone else (the client) is using it.
- The **firewall**. The unspoken division of knowledge. You (the implementor) don't need to know **the context** in which the function is being used. Someone else (the client) doesn't need to know **the algorithmic implementation** of the function.

Makes sense? Of course it does. But let's just think through some of the implications anyway...

- The implementor can change the inner workings of a function, say, for efficiency, without consulting the client and **without breaking the client's code that uses the function**.
- The client doesn't have to be an expert in efficient, robust or obscure algorithms.

So far, this is all just philosophy. The **specification** is where we turn it into reality.

1  Sometimes, the "someone else" is ourselves. But, because this person is in the future, we shall consider them a separate individual. If this is confusing, I recommend watching the movie Looper (2012).

### 2.1 Writing a specification

The specification provides both the implementor and the client with an unambiguous, agreed upon description of the function. It should state:

- Inputs/arguments/parameters to the functions.
- Any preconditions on these inputs, e.g., input `a` must be a `True/False` boolean; input list `xs` must be sorted.
- Outputs/returns of the function.
- Any postconditions on the outputs, e.g., output `ix` is the **first** appearance of input `x` in input list `xs`, which potentially contains repetitions.

In Python, we shall present the specification as a docstring, a concise commented description immediately below the function header. Let's look at an example:

```python
''' Find the position of a number in an array.

    Parameters:
    -----------
    x : float
        item to locate
    xs : array-like
        a list of values
    first : boolean (optional)
        if True, returns the index of the first appearance of x (default False)
    last : boolean (optional)
        if True, returns the index of the last appearance of x (default False)

    Returns:
    --------
    ix : array-like
        index location of x in xs

    Notes:
    ------
    xs should be sorted from smallest to largest
'''
```

***- - - - QUESTIONS TO CONTEMPLATE - - - -***

```python
# PART ONE
# --------
# What are the inputs for this specification?

# What are the outputs for this specification?

```

```python
# PART TWO
# --------
# What are the preconditions for this specification?

# What are the postconditions for this specification?

```

```python
# OPTIONAL CHALLENGE
# ------------------
# Can you think of any other preconditions that should be given?

# Often, there will be a heading "Raises:", which describes what should happen when an error occurs.
# Suggest an error that could occur for an implementation of this specification.

```

An essential feature of the specification above is that it provides sufficient information to BOTH the implementor and the client to do their job. If I asked you **to implement** this specification, you could. If I gave you the name of a function that corresponded to this specification, you could **make use of it**.

In addition, the specification provides **all the information you need** to write a unit-test. Details of the how the implementation works are not required.

Finally, the specification is **language-agnostic** (notwithstanding, I have written it as the classic Python docstring). In practice, you should be able write a function in Python, MATLAB, C, etc., that conforms to the specification above.

***Complete doc-strings for the functions below.***

```python
def neg_square2(x):
    ''' **your docstring here**
    '''
    return -x**2

def find_absolute_min(x, first=False, last = False):
    ''' **your docstring here**
    '''

    assert len(x)>0
    assert not (first and last)

    ax = abs(x)

    axmin = np.min(ax)

    ixmin = np.where(ax == axmin)[0]

    if first:
        ixmin = ixmin[0]
    if last:
        ixmin = ixmin[-1]

    return ixmin

```

### 2.2 Errors and asserts

Specifications as we have described them leave little room for **incompetence**. For instance, the implementor **assumes** that the client will satisfy the appropriate preconditions. Equally, the client **assumes**2 that the implementor has created a bug-free function. At least for the latter, the implementor could point to a **unit-test** as providing some guarantee of quality.

But how should the implementor **guard against** incompetence on the part of the client? Here are two ways:

- Explicitly check that preconditions are satisfied within the implementation. We do this using **assert** statements.
- Monitor for anomalous or unexpected outcomes and `raise` an **error**.

2Remember that, when you *assume*, you make an "ass" out of "u" and "me"...

The cell below calls a function that computes the **harmonic** mean of `xs`:

\begin{equation}
\tilde{x} = \left(\frac1n\sum\limits_{i=0}^{n}\frac{1}{x_i}\right)^{-1}
\end{equation}

It is not defined for any **zero values** of `xs`.

```python
# harmonic mean calculation
xs = [1, 2, 3]
xharm = harmonic_mean(xs)
print(xharm)
```

```python
# Run the cell above with xs = [1, 2, 3]. What happens?

# Try inserting a 0 value into xs and rerunning the cell. What happens?

# Try calling harmonic_mean with an empty list (xs = []). What happens?

# Try calling harmonic_mean with a non-numeric value (xs = [1, 'an apple', 2]). What happens?

# Which of these are 'checked' preconditions, and which are not?
```

While it is sometimes a kindness on the part of the implementor to check preconditions, it may **not always be practical**. For example, the computational expense required to check the precondition *'input array `xs` must be sorted smallest to largest'* may be large compared to *'find the index position of the value `x`'*. Indeed, often the purpose of a precondition is to **save** the implementor some computational expense by guaranteeing desirable qualities of the inputs.

The cell below calls a function that computes the **geometric** mean of `xs`:

\begin{equation}
\hat{x} = \sqrt[^n]{\prod\limits_{i=1}^n x_i}
\end{equation}

It is not defined if `xs` contains BOTH $0$ and $\infty$ ([what is zero times infinity?](https://img.huffingtonpost.com/asset/5b9282ac190000930a503a0f.jpeg?ops=1910_1000))

```python
# geometric mean calculation
xs = [1, 2, 3]
xgeom = geometric_mean(xs)
print(xgeom)
```

```python
# Run the cell above with xs = [1, 2, 3]. What happens?

# Try inserting a 0 value into xs and rerunning the cell. What happens?

# Try calling harmonic_mean with an empty list (xs = []). What happens?

# Try inserting an np.inf value into xs and rerunning the cell. What happens?

# Try inserting a 0 AND an np.inf value into xs and rerunning the cell. What happens?

# Which of these are 'checked' preconditions, which are `errors raised` due to anomalous behaviour,
# and which are normal outcomes?

# OPTIONAL
# Check the implementation of geometric_mean - what fancy trick are we using to compute it? Does the client
# need to know about these fancy tricks?
```

### 2.3 Raising and catching errors

The implementor will sometimes **raise** an error when they want to signal to the client that things are not going well in the code. However, sometimes the client will be prepared to **tolerate and respond** to this misbehaviour. They can do this by **catching** the error with a `try` statement, and redirecting the code to an `except`.

For instance, we have seen how the inputs below to `geometric_mean` raise a `ValueError`. We can catch this error by wrapping the error generating command (`geometric_mean`) inside a `try` block. If an error is raised, the code in the `except` block will be executed.

```python
xs = [1, 2, 3, 0, np.inf]

try:
    xgeom = geometric_mean(xs)
except:
    # default to 0 if mean is not undefined
    xgeom = 0.

print(xgeom)
```

Catching and raising errors, using asserts to check for preconditions, and writing clear specifications, are all steps you can take to minimise the emergence and impact of bugs in your code.

The final topic we need to cover is how to back-up and chronicle changes to your code: **version control**.

## 3 Version control

***Backing up our code.***

Starting a new software project with promises of strict version control is the amateur coder's equivalent to a new year/new gym resolution. Its a good idea. You should do it. You **will** do it. Until one day it's inconvenient and you regress to lazy Leslie from 2018. Nevertheless, let's cover the basics3...

3  For a *better* description of version control than I will give you here, see this [reading](https://ocw.mit.edu/ans7870/6/6.005/s16/classes/05-version-control/).

Your coding project is just a collection of files.

> **If your computer dies tomorrow, wouldn't it be nice to have a backup?**

You make your code better by making changes to those files.

> **Wouldn't it be nice to have a record of all those changes?**

Sometimes you'll make a change that actually makes your code worse.

> **Wouldn't it be nice to roll back to a previous (better) version?**

Sometimes you need to work at a desktop at university and sometimes you'll want work on your laptop at home.

> **Wouldn't it be nice to sync your coding project between two or more machines?**

Sometimes you'll work as part of a team developing different parts of the code.

> **Wouldn't it be nice if there was a way to push out your changes to others, and pull their changes back?**

*The objective of version control is to address these issues.* We will be using a program called [**git**](https://git-scm.com/)4 to help us do that.

4In fact, if you are running this notebook on Google Colab, you are already using git!

### 3.1 Repositories

At the heart of version control is the concept of the **repository**. This is an archive of the current contents of all the files in your code, safely located in the cloud. A repository can be **private**, accessible only to people selected by the **owner**, or **public**, accessible to any who want to look at it. (Typically, only people selected by the owner have the right to modify a public repository; this is why you can access the course files but you can't edit or replace them.)

Anyone who has permission (including, of course, the owner) can `clone` a copy of the repository to their (say, university) computer. All the files will appear in their folders, and you can run them if you wish.

In this, and following sections, I will be following `git` command line terminology. For example, to clone a repository, you would write at the command line

> `git clone *name of repository*`

The repository name typically combines the web address of the hosting entity (e.g., [BitBucket](https://bitbucket.org), [GitHub](https://github.com/)), your username, and a project name. Usually this will be obvious when visiting the repository web interface, e.g., to clone all the notebooks for this entire course

> `git clone https://github.com/bryan-ruddy/ENGSCI233_2021.git`

At the start of this notebook, if you ran it via Google, you will have run this very command so that Google's servers had access to all the support files for the notebook.

### 3.2 Recording your changes

Sometimes, you will make a change, say by fixing a bug in one of your functions, thereby making a change to the file `super_func.py`. So that there is a record of this change, you will `add` (nominate new/modified files) and `commit` (record the change).

Here is the command line terminology for `git`

> `git add .`

> `git commit -m "added a check for preconditions to super_func"`

Other times, you might make a change by adding a unit test, written in a new file `func_i_test.py`. Once again, you will `add` this file to the repository, and then `commit` so there is a record of the change.

> `git add .`

> `git commit -m "added a unit test for super_func"`

Your commits - records of change - are local to your computer. You rejoin them with the online **repository** using a `push`.

> `git push`

### 3.3 Working from  multiple locations

Now, if you go home and want to work on this code, you can `clone` a copy of the repository.

> `git clone *name of repository*`

Make changes to files at home. Then `add`, `commit` and `push` these changes up to the online repository.

> `git add .`

> `git commit -m "fixed a bug in the precondition check"`

> `git push`

Next time you are working at the university, use a `pull` to retrieve the changes made at home, **syncing** your local repository with the online one.

> `git pull`

### 3.4 Managing conflicts

Of course, there is no requirement that the university and home clones are owned by the same person. Suppose that your friend has cloned your repository and you both working on different parts of it **simultaneously**.

Your friend finishes for the night at 8pm4 and `push`es their changes up to the online repository.

Managing your deadlines and workload carefully, you finish at 1am, `commit` and try to `push` up your changes. Unfortunately, the `push` fails because both you and your friend have modified the same line of code, resulting in a conflict.

To resolve the conflict, you must `fetch` the latest version that includes the modifications from your friend, and then manage the conflict locally using a `merge`.

> `git fetch`

> `git merge`

Once the conflict is handled, you can `commit` and `push` a new version, that includes both your changes and the managed conflict.

4lol, casual

### 3.5 Rolling back to previous versions

Often you'll find that some magic-bullet change to your code that was going to make things more efficient just didn't pan out. Things don't even work now and you wish desperately you could roll back to the inefficient, but working, version.

In keeping a record of all your changes - snapshots of your code at different moments in history - version control allows this sort of "saved game" approach to coding. There are two common options:

1. If you want to take a peek at the older version of the code, but later return to the current version, a `checkout` allows you to temporarily roll back.

2. If you want to reset permanently to the older version, you want to do a `revert`.

For more on rolling back to an old version of your code, check out [this StackOverflow thread](https://stackoverflow.com/questions/4114095/how-to-revert-a-git-repository-to-a-previous-commit).

```python
# TERMINOLOGY TEST
# ----------------
# Provide definitions for the following terms as they pertain to version control.
#
# Repository:
# Owner:
# Clone:
# Push/Pull:
# Commit:
# Add:
# Revert:
# Merge:
# Git:
```

### 3.6 Final notes on `git`

Because it is a command line tool, `git` has extraordinary flexibility in its mode of operation through the use of command line flags. For example,

> `git commit -m "fixed a function"`

associates the message `"fixed a function"` to the commit. `-m` is the command line flag that tells `git` "include a message, make it the text that follows". Adjusting this command slightly

> `git commit -m "fixed a function" -q `

will do the same as above, but now the `-q` will request `git` to suppress output information about what code has been changed for this commit.

It is not possible to fully cover `git` functionality in one week of this course. For most users, it will be sufficient to Google what you want to achieve, and then read the explanation and instructions from the first StackOverflow link. For more help with `git` command line, try typing

> `git help`

> `git help add`

#### 3.6.1 Ignoring files

If you're writing and compiling code (even Python compiles itself, creating `*.pyc` files) there will be extra files created that you won't want to track. Placing a `.gitignore` file in your repository tells `git` not to include particular files in the repository.

***Run the cell below to display the contents of `.gitignore`***

```python
%pycat .gitignore
```

Here, we use a bit of special syntax (which you won't be assessed on) to show the file contents in a Jupyter notebook: the % tells the interactive Python interpreter that the line is actually a special command, not a piece of Python code. The `pycat` command shows the contents of a file on the screen, with syntax highlighting that assumes it's a piece of Python code. Similarly, the ! used with the `git` command at the start of the notebook tells the interpreter it is a shell command, not Python code.

##### Example unit test for `neg_square`

```python
def test_neg_square():
    assert neg_square(2) == -4
    assert neg_square(-2) == -4
    assert neg_square(2.5) == -6.25
    assert neg_square(-2.5) == -6.25
    assert neg_square(0) == 0
    assert neg_square(np.inf) == -np.inf
```


# Bevezetés

### Python programozási nyelv

A Python egy open-source (OS), interpretált, általános célú programozási nyelv (vagy script-nyelv).

**Tulajdonságai:**
- Objektum orientált
- Interpretált
    - Nem szükséges fordítani (mint a pl a *C++*-t), elég csak beírni a parancsot, és már futtatható is a kód
    - Alkalmas teszi számítások gyors-prototipizálására
    - Cserébe lassú
- Open-source:
    - Ingyenes
    - Folyamatosan karban tartott
    - Széles körben felhasznált iparban és akadémiában is
    - Nagy "Community", sok segédlettel, fórummal (pl.: [stackoverflow](https://stackoverflow.com/questions/tagged/python))
- Moduláris:
    - Rengetek feladatra létezik "*package*" (pl.: numerikus számításokra *numpy*/*scipy*, szimbolikus számításokra *sympy*, táblázatfájl-kezelésre *CSV*)
    - Csak azt kell behívni, amire szükségünk van
    - Ismerni kell a *package* ekoszisztémát, mik léteznek, mi mire jó, stb...
- Sok IDE (*Integrated Development Environment*) létezik:
    - Alapvetően shell (terminál) alapú
    - Notebook: **_jupyter notebook_**, *jupyter lab*
    - Szövegszerkesztő: *Spyder*, *VS Code* (ingyenes/open source - ezek tartalmaznak *Debugger*-t is)
    - Fizetős szövegszerkeszők (lista nem teljes): *Visual Studio*, *PyCharm*, stb...

### Jupyter notebook működése (+ Python kernel):

Legfontosabb tudnivalók:

- Csak egy *front-end*, ami kommunikál egy *kernel*-lel (ez a kernel menüben választható).
- Két mód létezik:
    - Command mode (cellaműveleteket lehet végezni)
    - Edit mode (szövegbevitel cellába)
- Command mode (`ESC` billentyű lenyomásával érhető el, kék csík a cella kijelölése esetén):
    - Notebook mentése: `s`
    - Cellák hozzáadása: `b` az aktuális cella alá, `a` az aktuális cella fölé
    - Cella törlése: kétszer egymás után a `d` billentyű lenyomása
    - Cella törlésének visszavonása: `z`
    - Cella másolás: `c`, kivágás: `x`, beillesztés az aktuális cella alá: `v`
    - Számozás bekapcsolása a cella soraira: `l` (kis L), vagy `Shift + l` az összes cellára
    - Cellamódok: futtatandó kód: `y`, nyers kód (nem futtatható): `r`, markdown (formázott szöveg): `m`
- Edit mode (`Enter` billenytű lenyomásával érhető el, zöld szín):
    - Sor "kikommentelése"/"visszaállítása": `Ctrl + /`
    - Több kurzor lehelyezése: `Ctrl + Bal egérgomb`
    - Téglalap-szerű kijelölés (rectangular selection): `Alt + Bal egérgomb` "húzása" (dragging)
- Közös
    - Cella futtatása, majd cellaléptetés: `Shift + Enter` (ez létrehoz egy új cellát, ha nincs hova lépnie)
    - Cella futtatása cellaléptetés nélkül: `Ctrl + Enter`

**Jupyter notebook help-jének előhozása**: *Edit mode*-ban `h` lenyomásával
**Python help**: Kurzorral a függvény nevén állva `Shift + Tab` vagy egy cellába `?"fv_név"` beírása és futtatása

### A bevezető felépítése:
- [Alapműveletek](#Alapmuveletek)
- [Összetett függvények](#Osszetettfuggvenyek)
- [Saját függvények](#Sajatfuggvenyek)
- [Osztályok](#Osztalyok)
- [Vezérlési szerkezetek](#Vezerlesiszerkezetek)
- [Külső függvénykönyvtárak](#Kulsofuggvenykonyvtarak)
    - [Szimbolikus matematikai műveletek](#Szimbolikus)
    - [Deriválás/Integrálás](#DerivalIntegral)
    - [Vektor és mátrixszámítások Sympy-ban](#SzimVektorMatrix)
    - [Vektor és mátrixszámítások Scipy-ban](#NumVektorMatrix)
    - [Egyenletek megoldása Scipy-ban](#Egyenletek)
    - [Szimbolikus függvényekből numerikus függvények](#SymToNum)
- [Egyszerű ábrák készítése](#Egyszeruabrak)

# Python bevezető

## Alapműveletek (Shift/Ctrl + Enter-rel futtassuk)

```python
17 + 7 #Összeadás
```

    24

```python
333 - 7 #Kivonás
```

    326

```python
11 * 22 #Szorzás
```

    242

```python
7/9 #Osztás (ez nem egész (int) lesz: külön típus float)
```

    0.7777777777777778

```python
0.3-0.1-0.2 # float: számábrázolási hiba lehet!!
```

    -2.7755575615628914e-17

```python
2**3 # Hatványozás (** és NEM ^!)
```

    8

```python
2**(0.5) # Gyökvönás hatványozás segítségével
```

    1.4142135623730951

```python
5e-3 #normálalak e segítségével (vagy 5E-3)
```

    0.005

Néhány alapművelet működik szövegre is

```python
'str1_' + 'str2_' #Összeadás
```

    'str1_str2_'

```python
2 * 'str2_' #Szorzás
```

    'str2_str2_'

## Összetettebb függvények

```python
sin(2) #szinusz
```

Összetettebb függvények már nincsenek a python alapnyelvben - ilyenkor szükséges behívni külső csomagokat, pl a **math** csomagot

```python
import math
```

```python
sin(2) # ez így továbbra sem létezik
```

```python
math.sin(2)
```

    0.9092974268256817

```python
# Több parancs együttes beírásakor nem látszik, csak az utolsó sor kimenete: print függvény alkalmazása!
print(math.sqrt(2))
print(math.tan(2))
print(math.atan(2))
```

    1.4142135623730951
    -2.185039863261519
    1.1071487177940904

```python
# Kimenet el is rejthető a ; segítségével ("suppress output")
1+1;
```

Amennyiben szükséges, definiálhatunk mi is saját változókat az `=` jellel.
Megjegyzés: a `=` értékadó függvénynek nincs kimenete

```python
a=2
b=3
c=4.0 # automatikus típusadás
```

```python
(a+b*c)**a # a legáltalánosabb típus lesz a kimenet (int < float)
```

    196.0

```python
# Fontos, hogy igyekezzük kerülni védett változó neveket! ILYET NE!
math.sqrt = 1
math.sqrt(2)

# KERNEL RESTART SZÜKSÉGES
```

**Ha véletlenül ilyet teszünk, akkor érdemes újraindítani a *kernel* a fent látható körkörös nyíllal, vagy a *Kernel* $\rightarrow$ *Restart* segítségével**

## Saját függvények

Szerkezet:
```python
def function(*arguments):
    instruction1
    instruction2
    ...
    return result
```

A függvény alá tartozó utasításokat tabulátoros behúzással (indent) kell beírni (nincs `{}` zárójel, vagy `end`). A függvény neve után jönnek az argumentumok majd kettősponttal `:` jelezzük, hogy hol kezdődik a függvény.

```python
def foo(x):
    return 3*x

def bar(x,y):
    a = x+y**2
    return 2*a + 4
```

```python
print(foo(3))
print(foo(3.))
print(foo('szöveg_'))

print(bar(3,4.))
```

    9
    9.0
    szöveg_szöveg_szöveg_
    42.0

Lehetséges úgynevezett anonim függvényeket (*anonymous function* vagy *lambda function*) is létrehozni, amely gyors módja az egyszerű, egysoros függvények létrehozására:

```python
lambda arguments: instruction
```

Ez akár egy változóhoz is hozzárendelhető, mint egy szám vagy string.

```python
double = lambda x : x*2
multiply = lambda x,y : x*y
```

```python
print(double(3))
print(multiply(10,3))
```

    6
    30

## Osztályok

```python
def foo(x):
    return x**2

class MyClass:
    def __init__(self,x,y,z):
        self.square = foo(x)-z
        self.cubic = y**3+foo(y)

    @classmethod
    def createfrom_x(cls,x):
        return MyClass(x,x,x)

    def return_stuff(self):
        return self.square+3*self.cubic
```

```python
mcl=MyClass.createfrom_x(2)
```

```python
mcl.return_stuff()
```

    38

```python
def foo(x):
    return x**2

class MyClass:
    def __init__(self,x,y,z):
        self.square = foo(x)-z
        self.cubic = y**3+foo(y)

    @classmethod
    def createfrom_x(cls,x):
        return MyClass(x,x,x)

    def return_stuff(self):
        return self.square+3*self.cubic
```

```python
mcl=MyClass.createfrom_x(2)
```

```python
mcl.return_stuff()
```

    38

## Vezérlési szerkezetek (Control Flow) - csak a legfontosabbak

### Listák

```python
lista = [1,2,3,4,"valami",[1.0,4]]
```

```python
print(lista[0]) # lista 1. eleme
print(lista[3]) # lista 4. eleme
print(lista[-1]) # negatív számokkal hátulról indexeljük a listát, és (-1)-től indul
print(lista[-2]) # lista utolsó előtti eleme
```

    1
    4
    [1.0, 4]
    valami

```python
print(lista[1:-1]) # egyszerre több elem [inkluzív:exklúzív módon]
print(lista[1:2]) # egyszerre több elem [inkluzív:exklúzív módon]
print(lista[2:]) # lista utolsó elemét is figyelembe vesszük
```

    [2, 3, 4, 'valami']
    [2]
    [3, 4, 'valami', [1.0, 4]]

```python
lista = [2,3,64,89,1,4,9,0,1]

lista.sort()
lista
```

    [0, 1, 1, 2, 3, 4, 9, 64, 89]

### if-then-else

```python
if condition:
    instruction1
elif condition2:
    instruction2
else:
    intsturction3
```

```python
a=4
if a<=3:
    print('"a" nem nagyobb, mint 3')
elif a>=10:
    print('"a" nem kisebb, mint 10')
else:
    print('"a" nagyobb mint 3, de kisebb mint 10')
```

    "a" nagyobb mint 3, de kisebb mint 10

### for ciklus (for loop)
```python
for i in array:
    instruction
```

```python
for i in range(3):
    print(i)

print()

for (i,elem) in enumerate(lista):
    print('lista ',i,'. eleme: ',elem,sep='') # több elem printelése egyszerr, szeparátor = ''

```

    0
    1
    2

    lista 0. eleme: 0
    lista 1. eleme: 1
    lista 2. eleme: 1
    lista 3. eleme: 2
    lista 4. eleme: 3
    lista 5. eleme: 4
    lista 6. eleme: 9
    lista 7. eleme: 64
    lista 8. eleme: 89

## Listák gyors létrehozása (List comprehension)

```python
lista2 = [3*i**2 for i in range(2,5)] # range: 2,3,4
lista2
```

    [12, 27, 48]

```python
lista3 = list(range(10))
lista3
```

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

```python
myfun = lambda x: 3*x**2

lista4 = [myfun(i) for i in range(2,10) if i%3 != 0] # ha i nem osztható 3-al
lista4
```

    [12, 48, 75, 147, 192]

# Külső függvénykönyvtárak:

A mérnöki gyakorlatban előforduló alapszámításokhoz 2 fő csomag importálása ajánlott:
- `sympy`: szimbolikus számításokhoz
- `scipy`/`numpy`: numerikus számításokhoz (pl mátrix algebra)
- `matplotlib.pyplot`: Függvények ábrázolása
A `module`-ok hívása az alábbi módokon történhet
- `import` *`modulename`* : importálja az adott `module`-t. Ekkor a `module`-ban lévő függvények hívása a `module.functionname(arguments)` segítségével történik (Az *1_Alapok.ipynb*-ben láthattunk erre példát a `math` `module` esetében).
- `import` *`modulename`* `as` *`alias`* : hasonló az előzőhöz, de megválasztjuk, hogy milyen *alias*-ként hivatkozunk a `module`-unkra
- `from` *`modulename`* ` import` *`function1, function2, ...`* : csak bizonyos függvények importálása (nem szükséges a `module`-ra hivatkozni a függvények hívása során)
- `from` *`modulename`* ` import *` : a `module` összes függvényének importálása (nem szükséges a `module`-ra hivatkozni a függvények hívása során)

## Szimbolikus matematikai műveletek

```python
import math
import sympy as sp
import scipy as sc
sp.init_printing()
```

```python
F, m, a, b, c, x = sp.symbols("F m a b c x")
```

```python
F=m*a
```

```python
F.subs([(a,7)])
```

```python
F.subs([(a,7),(m,1.1)])
```

```python
((a+b)**3).expand()
```

```python
((a+b)**7 - (b+2*a)**3).expand()
```

```python
(a**2+b**2+2*a*b).factor()
```

```python
sp.factor(a**2+b**2+2*a*b)
```

```python
sp.factor(b**3 + 3*a*b**2 + 3*a**2*b + a**3)
```

```python
a/b+c/b+7/b
```

```python
sp.ratsimp(a/b+c/b+7/b)
```

```python
(a/b+c/b+7/b).ratsimp()
```

```python
(sp.sin(x)**2 + sp.cos(x)**2).simplify()
```

```python
(sp.cos(2*x)).expand()
```

```python
sp.expand_trig(sp.cos(2*x))
```

```python
import scipy.constants
```

```python
sc.constants.golden
```

```python
math.sqrt(-1+0j)
```

```python
sc.sqrt(-1+0j)
```

```python
sp.limit(sp.sin(x)/x,x,0)
```

Taylor-sor megadása. Első paraméter a függvény, második a változó, harmadik az érték ami körül akarjuk a sort kifejteni, negyedik pedig a fokszám:

$$f\left(x\right) \approx \sum\limits_{i=0}^{N} \dfrac{\left(x - x_0\right)^i}{i!} \left.\dfrac{\mathrm{d}^i f}{\mathrm{d} x^i}\right|_{x = x_0}$$

```python
sp.series(sp.sin(x),x,0,20)
```

### Deriválás/Integrálás

```python
a,Δt,x = sp.symbols('a,Δt,x')
```

Deriválás

```python
sp.diff(sp.sin(x**3),x)
```

Többszöri deriválás

```python
sp.diff(sp.sin(x**3),x,3)
```

Integrálás

```python
sp.integrate(1/(1+x),x)
```

Határozott integrál

```python
sp.integrate(1/(1+x),(x,1,2))
```

```python
a = sp.Symbol('a')
```

```python
sp.integrate(1/(x**2 + a),x)
```

Láncszabályt is ismeri a szoftver

```python
y = sp.Symbol('y')
def f(x):
    return x**2
def g(y):
    return sp.sin(y)
```

```python
f(g(y))
```

```python
sp.diff(f(g(y)),y)
```

```python
sp.diff(g(f(x)),x)
```

Sok esetben nem létezik zárt alakú kifejezés a határozatlan integrálhoz. Ebben az esetben használhatjuk a [numerikus integrálást](https://en.wikipedia.org/wiki/Numerical_integration) a határozott integrál számításához:

```python
sp.integrate(sp.sin(sp.cos(x)),x)
```

[numerikus intágráls SymPy-vel](https://docs.sympy.org/latest/modules/integrals/integrals.html#numeric-integrals)

nem triviális

## Vektor- és mátrixszámítások *sympy*-ban
Kisebb méretek esetén, amelyek akár szimbolikus számokat is tartalmaznak

```python
v1= sp.Matrix([2.,3.,4.]) # oszlopvektor
v2= sp.Matrix([[3.,-2.,-7.]]) # sorvektor (még 1 szögletes zárójel)
mx1 = sp.Matrix([[1.,2.,3.],[2.,0.,4.],[3.,4.,1.]])
mx2 = sp.Matrix([[1.,2.,3.],[4.,5.,6.],[7.,8.,9.]])
EM = sp.eye(3) # egységmátrix
```

```python
v1
```

```python
v2
```

```python
v2.multiply(v1)
```

```python
mx2.multiply(v1)
```

```python
v2.multiply(mx2)
```

```python
EM
```

```python
mx1.eigenvals() # sajátértékek és multiplicitásuk (racionális számokkal)
```

```python
mx1.eigenvals(rational=False) # sajátértékek numerikusan
```

```python
mx1.eigenvects() # sajátvektorok numerikusan
```

```python
mx1.det() # mx1 determinánsa
```

```python
Ix,Iy,Ixy = sp.symbols('Ix,Iy,Ixy')
mxSP=sp.Matrix([[Ix,-Ixy],[-Ixy,Iy]])
display(mxSP)

print('\n Sajátértékek, vektorok: \n')
mxSP.eigenvects()
```

```python
mxSP=sp.Matrix([[Ix,0],[0,Iy]])
display(mxSP)

print('\n Sajátértékek, vektorok: \n')
mxSP.eigenvects()
```

## Vektor- és mátrixszámítások *scipy*-ban
Nagy mátrixok és vektorok esetén érdemes ezt használni, vagy ha sok numerikus adattal dolgozik az ember. Továbbá a sajátérték-sajátvektor számítások is jobban megoldott ebben a csomagban

```python
import sympy as sp
import scipy as sc
import scipy.linalg
```

```python
v1= sc.array([2.,3.,4.])
v2= sc.array([3.,-2.,-7.])
mx1 = sc.array([[1.,2.,3.],[2.,0.,4.],[3.,4.,1.]])
mx2 = sc.array([[1.,2.,3.],[4.,5.,6.],[7.,8.,9.]])
```

```python
print( sc.dot(mx2,v1) )  # skalár szorat mx2*v1
print( sc.dot(v1,mx2) )  # skalár szorat transpose(v1)*mx2
print( sc.cross(v1,v2) ) # keresztszorzat v1×v2
```

    [20. 47. 74.]
    [42. 51. 60.]
    [-13.  26. -13.]

```python
(λ,V) = sc.linalg.eig(mx1) # Mátrix sajátértékei, sajátvektorai a λ és a V változókban

# kiíratás 2 értékes tizedes jeggyel és for ciklusra példa
for (i,v) in enumerate(V):
    print(i+1, '. sajátérték és -vektor:',sep='')
    print('λ = ', sp.N(λ[i],3), '; v = ', [sp.N(e,3) for e in v], sep='', end='\n\n')
```

    1. sajátérték és -vektor:
    λ = 6.76; v = [0.529, 0.834, -0.157]

    2. sajátérték és -vektor:
    λ = -1.15; v = [0.543, -0.475, -0.693]

    3. sajátérték és -vektor:
    λ = -3.61; v = [0.653, -0.281, 0.704]

Figyeljük meg, hogy míg a sympy a sajátvektor utolsó koordinátáját 1-nek választja, addig a scipy a sajátvektorokat 1-re normálja!

## Egyenletek megoldása

```python
x, y, z = sp.symbols("x y z")
```

```python
egy1=77*x+6 - 160
```

```python
sp.solve(egy1,x)
```

```python
sp.solve(x**2 - 5, x)
```

```python
x
```

```python
e1 = 3*z + 2*y + 1*x - 7
e2 = 4*z + 0*y + 2*x - 8
e3 = 1*z + 4*y + 3*x - 9
```

```python
sp.solve([e1,e2,e3],[x,y,z])
```

```python
def f(x):
    return x**2 - 4
```

```python
# Numerikus gyökkereséssel
import scipy.optimize

gyok = scipy.optimize.brentq(f,0,10)
print(gyok)
```

    2.0

### Szimbolikus függvényekből numerikus függvények

Ezt a legegyszerűbben [*lambda function*](#Sajatfuggvenyek)-ök segítségével tehetjük meg

```python
a,b,c,x = sp.symbols('a,b,c,x')
adat = [(a, 3.0), (b,-2.), (c,1.0)]

masodfoku_x = a*x**2 + b*x + c

masodfoku_x
```

```python
masodfoku_num = sp.lambdify(x,masodfoku_x.subs(adat))
masodfoku_num(1.)
```

# Egyszerű ábrák készítése

```python
import scipy as sc # Ez egyszerűsíti a sok függvénnyel való munkát
import matplotlib.pyplot as plt # matplotlib csomag python rajzolójának behívása
```

## Függvények ábrázolása egyszerű diagramokon

```python
def f(x):
    return sc.sin(x)*(1-4*sc.exp(0.7*x)/(x**3))

x1 = sc.linspace(2,10,30)
y1 = f(x1)
```

```python
plt.figure(figsize=(20/2.54,12/2.54)) #inch-ben lehet megadni az ábra méretét - nem kötelező de érdemes, ha dokumentációba szánjuk

#plot objektum létrehozása
plt.plot(x1,y1)

# tengelyfeliratok
plt.xlabel('x',fontsize=12)
plt.ylabel('f(x)',fontsize=12)

# rácsozás
plt.grid()

# ábra mentés (még azelőtt kell, mielőtt kirajzolunk)
# plt.savefig("abra1.png", bbox_inches="tight")
# plt.savefig("abra1.pdf", bbox_inches="tight")

# plot kirajzolása
plt.show()

# plot mentése
plt.savefig('abra1.png')  # lehet .pdf, .svg formátumokban is menteni
```

```python
plt.plot(x1,y1,linewidth=3,color='k',linestyle='-.') # vonalvastagság, -szín, és -stílus módosítása
plt.xlabel('x')
plt.ylabel('f(x)')
# megjelenítési határ
plt.ylim(-0.5,0.5) #-0.5...0.5
plt.xlim(None,8) # xmin...8
plt.grid()

plt.show()
```

```python
def g(x):
    return sc.sqrt(x)
def h(x):
    return x**2

x2 = sc.linspace(0.,1.,100)

plt.plot(x2,g(x2))
plt.plot(x2,h(x2))

plt.xlabel('x',fontsize=12)
# felirat
plt.legend((r'$\sqrt{x}$',r'$x^2$'), fontsize=12, loc = 5) # loc: 1:jobbfenn, 2: balfent, 3:ballent,...10
plt.grid()

plt.show()
```

```python
x3 = sc.arange(1,14+1)
y3 = sc.exp(x3)/(1.4e4)
plt.plot(x3, y3, linestyle = '', marker = 'o')
plt.xlabel('szorgalmi hét sorszáma')
plt.ylabel('hallgatók terheltsége [%]-ban')
plt.grid()
plt.show()
```

```python
plt.plot(x3, y3, linestyle = '-', marker = 'o')
plt.yscale('log')
plt.xlabel('szorgalmi hét sorszáma')
plt.ylabel(r'hallgatók terheltsége [%]-ban')
plt.grid()
plt.show()
```

## Parametrikus függvényábrázolás:

```python
t = sc.linspace(0,2*sc.pi,100)
x = sc.sin(t)
y = sc.cos(t)

plt.plot(x, y, linestyle = '--')
plt.grid()
plt.axis('equal')
plt.show()
```

## 3D függvényábrázolás

```python
from mpl_toolkits.mplot3d import Axes3D
```

```python
def f(x, y):
    return sc.sin(sc.exp( - x ** 2 - y ** 2))

x = sc.linspace(-2, 2, 30)
y = sc.linspace(-2, 2, 30)

X, Y = sc.meshgrid(x, y)
Z = f(X, Y)
```

```python
fig = plt.figure(figsize=(16/2.54,10/2.54)) # itt manuálisan létre kell hozni a plot (figure) objektumot
ax = fig.add_subplot(111, projection='3d')
ax.plot_surface(X, Y, Z, cmap='viridis')
plt.show()
```