```julia
HTML("""
<style>
.rendered_html table, .rendered_html th, .rendered_html tr, .rendered_html td {
     font-size: 100%;
}
</style>
""")
```

## 1.1 为什么需要量子计算

### 后摩尔时代

我们对算力的需求在不断增长，但是摩尔定律却不再有效了。

#### 摩尔定律

- 新的思路（三维堆叠，etc）
- 定制芯片（GPU，TPU，FPGA，etc）
- 更换计算模型（量子计算，etc）

### 量子多体物理的维度诅咒

**量子多体物理**是指多个具有量子效应的物理对象所构成的系统。这种系统所构成的状态空间往往随着它的粒子数目的增长而指数增长。而量子多体物理的理论研究，将直接对新型材料，新型药物的研究造成影响。

而由于指数增长的空间大小，经典计算机难以精确模拟复杂的多体系统，而需要进行一些假设，简化等，常见的方法包括但不限于：

- DFT，密度泛函

- DMRG，密度矩阵重整化，张量网络

- QMC，量子蒙特卡洛，变分量子蒙特卡洛

其中的很多方法对于计算资源的消耗是巨大的。例如[PEPS++](https://arxiv.org/abs/1806.03761)这个工作甚至用上了中国最强的超级计算机神威太湖之光整机。

所以有人就想：**能不能用量子系统模拟量子系统呢？**

这个人就是：费恩曼（Feynman）[Simulating Physics with Computers](https://people.eecs.berkeley.edu/~christos/classics/Feynman.pdf)

当然后来在这个想法上有很多工作，用一个可以精确操纵的量子系统模拟另外一个量子系统也称为量子模拟（Quantum Simulation）。这也是被认为一个在近期最有可能成为量子计算机的杀手级应用的方向。

### 作为基础学科的启发性研究

#### 启发经典算法设计

- [Quantum Inspired Recommendation System](https://arxiv.org/abs/1807.04271)
- [Quantum Inspired PCA](https://arxiv.org/abs/1811.00414)
- [Simulated Quantum Annealing Can Be Exponentially Faster than Classical Simulated Annealing](https://arxiv.org/abs/1601.03030)

#### 帮助我们获得精确控制量子系统的能力

- 控制单量子比特系统
- 控制多量子比特系统
- etc...

#### 帮助我们理解这个世界

- 量子机器学习
- 宇宙学：[Quantum Circuit Cosmology: The Expansion of the Universe Since the First Qubit](https://arxiv.org/abs/1702.06959)
- 黑洞：[Quantum Circuit Model of Black Hole Evaporation](https://arxiv.org/abs/1807.07672)
- etc.

### 阅读材料（可能需要VPN）

- [Why we need quantum computing](https://www.research.ibm.com/ibm-q/learn/what-is-quantum-computing/#)
- [Julia语言入门](https://www.bilibili.com/video/av28178443/)

### 课程将用到的材料

- Michael A. Nielsen & Isaac L. Chuang, Quantum Computation & Quantum Information
- Yao.jl - Extensible Efficient Quantum Algorithm Design for Humans

## 1.2 经典的逻辑电路

经典的逻辑电路构成了我们现代计算机的基础，本章我们将着重学习一些经典逻辑电路的基础知识，然后之后我将介绍如何从经典的逻辑电路过度到量子线路。

### 逻辑比特

一般情况下我们往往使用比特，也就是两种不同的物理状态来进行计算，我们将其称为**比特**。而理论上，我们将这两种状态抽象出来，用数学符号表示之，从而简化问题。我们将这样的抽象称为**逻辑比特**，一个理论的逻辑比特，在实际的物理实现上有可能对应多个**物理比特**。

我们可以使用我们的**Yao**来定义一组逻辑比特

```julia
using Yao

ArrayReg(bit"000")
```

    ┌ Info: Recompiling stale cache file /Users/roger/.julia/compiled/v1.1/Yao/TDiQQ.ji for Yao [5872b779-8223-5990-8dd0-5abbb0748c8c]
    └ @ Base loading.jl:1184

    ArrayReg{1, Complex{Float64}, Array...}
        active qubits: 3/3

逻辑比特的向量表示，实际上我们也可以用一个one hot向量表示逻辑比特，例如逻辑比特01，可以表示为

| 向量的元素值 | 对应的逻辑比特 |
| ---------- | ------------ |
|     0      |     00       |
|     1      |     01       |
|     0      |     10       |
|     0      |     11       |

### 逻辑门

逻辑门是操作逻辑比特的一种特殊函数，它的输入是二进制的比特，输出是另外一组二进制的比特。我们常见的操作有逻辑：与，或，非

与：

- 如果两个输入的比特都为0，那么输出0
- 如果两个输入的比特都为1，那么输出1
- 如果两个输入的比特不同，那么也输出0

### 布尔代数和真值表

用文字描述上面的操作非常麻烦，数学上我们有布尔代数和真值表来描述逻辑电路。例如非门：

非：

- 如果输入的比特为 0，那么输出1
- 如果输入的比特为 1，那么输出0

真值表

| $A$ | $\neg A$ |
|-----|----------|
|  0  |    1     |
|  1  |    0     |

如果我们用一个独热编码（onehot）的向量来表示一个逻辑比特，那么我们可以用线性代数来表示逻辑门的运算，例如我们用如下的方式表示单个比特

$$
0 = \begin{pmatrix} 1\\ 0 \end{pmatrix}, \quad 1 = \begin{pmatrix} 0 \\ 1 \end{pmatrix}
$$

我们有

$$
\neg 0 = \begin{pmatrix} 0 & 1\\1 & 0 \end{pmatrix} \begin{pmatrix} 1\\ 0 \end{pmatrix} = \begin{pmatrix} 0\\ 1\end{pmatrix} = 1
$$

同理有

$$
\neg 1 = \begin{pmatrix} 0 & 1\\1 & 0 \end{pmatrix} \begin{pmatrix} 0\\ 1 \end{pmatrix} = \begin{pmatrix} 1\\ 0\end{pmatrix}
$$

我们以后为了简便，将这个矩阵记为 **X**

你可以用Yao来验证上述结论，其中 **X** 即为我们所说的非门

```julia
apply!(ArrayReg(bit"0"), X) == ArrayReg(bit"1")
```

    true

#### 练习

1. 或门（**OR**）是一个两比特门，它的定义是只要两个比特中有一个比特是 1 那么就输出 1，写出它的真值表和矩阵形式
2. Toffli门是一个三比特门，它的输入是 **A,B,C**，定义为 **A, B** 为控制比特，当**A，B** 都为1时，翻转 **C**

#### 答案

#### 1. 或门（**OR**）

真值表

| AB | **OR**(A, B) |
| -- | ------------ |
| 00 | 0            |
| 01 | 1            |
| 10 | 1            |
| 11 | 1            |

矩阵形式

$$
OR = \begin{pmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 1 & 1
\end{pmatrix}
$$

我们有

$$
\begin{pmatrix}1\\0\\0\\0 \end{pmatrix} \cdot OR = \begin{pmatrix}1\\0\end{pmatrix}\quad
\begin{pmatrix}0\\1\\0\\0 \end{pmatrix} \cdot OR = \begin{pmatrix}0\\1\end{pmatrix}\quad
\begin{pmatrix}0\\0\\1\\0 \end{pmatrix} \cdot OR = \begin{pmatrix}0\\1\end{pmatrix}\quad
\begin{pmatrix}0\\0\\0\\1 \end{pmatrix} \cdot OR = \begin{pmatrix}0\\1\end{pmatrix}
$$

#### 2. Toffli门

真值表

|     ABC     | 000 | 001 | 010 | 011 | 100 | 101 | 110 | 111 |
|:-----------:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| Toffli(ABC) | 000 | 001 | 010 | 011 | 100 | 101 | 111 | 110 |

矩阵

```julia
using Latexify

latexarray(Int.(mat(ConstGate.Toffoli)))
```

\begin{equation}
\left[
\begin{array}{cccccccc}
1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\
0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \\
0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 \\
0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 \\
0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\
\end{array}
\right]
\end{equation}

### 可逆计算

可逆计算，顾名思义是说计算过程中不会丢失信息，可以从结果反推出初始状态的计算过程。

```julia
Int.(inv(mat(ConstGate.Toffoli)))
```

    8×8 Array{Int64,2}:
     1  0  0  0  0  0  0  0
     0  1  0  0  0  0  0  0
     0  0  1  0  0  0  0  0
     0  0  0  0  0  0  0  1
     0  0  0  0  1  0  0  0
     0  0  0  0  0  1  0  0
     0  0  0  0  0  0  1  0
     0  0  0  1  0  0  0  0

对一个比特，可逆计算的门有哪些？

两个比特呢？

CNOT（控制翻转）就是一种，因为连续进行两次翻转就会回到原先的状态

```julia
CNOT = control(2, 2, 1=>X)
Int.(mat(CNOT))
```

    4×4 Array{Int64,2}:
     1  0  0  0
     0  1  0  0
     0  0  0  1
     0  0  1  0

三个比特呢？

Toffli门就是三个比特的可逆门，试试看用Toffli连续作用两次会发生什么？

```julia
using Yao

r = rand_state(3)
r1 = copy(r) |> ConstGate.Toffoli |> ConstGate.Toffoli
r1 ≈ r
```

    true

可逆计算要求，存在对应矩阵的逆矩阵，对于一个门的矩阵形式 $A$，存在 $B$ 使得他们的乘积为1

$$
A B = I, \quad \exists B
$$

### 从经典到量子

如果我们不限制这些门矩阵只能是实数0或者1，也不限制状态向量是onehot向量呢？

## 1.3 量子线路模型

上一节我们学习了经典线路模型。而量子线路模型可以看作是经典线路模型的推广

### 量子态

量子线路使用量子态表示当前的计算状态，我们将经典线路中对状态为onehot的要求放松一些：

允许**任意复数，模为1的向量作为当前计算的状态表示。**

而它的物理意义即为量子力学中的量子态。很自然的，他们可以这样叠加

```julia
r = ArrayReg(bit"010") + ArrayReg(bit"110")
```

    ArrayReg{1, Complex{Float64}, Array...}
        active qubits: 3/3

```julia
state(r)
```

    8×1 Array{Complex{Float64},2}:
     0.0 + 0.0im
     0.0 + 0.0im
     1.0 + 0.0im
     0.0 + 0.0im
     0.0 + 0.0im
     0.0 + 0.0im
     1.0 + 0.0im
     0.0 + 0.0im

而量子态可以在物理过程上用Unitary矩阵进行演化。

所以我们要求量子线路中的门也都是unitary，回想上一节的结论：**量子计算的是一种可逆计算**，每个量子门的逆就是它的共轭转置（adjoint，dagger）

恰好，我们上一节介绍的几个可逆的门都是unitary矩阵，例如CNOT和Toffli

```julia
isunitary(CNOT)
```

    true

```julia
isunitary(ConstGate.Toffoli)
```

    true

## Bloch球

对于一个量子比特，它的向量表示为

$$
\begin{pmatrix}
a\\
b
\end{pmatrix}
$$

由于我们要求它的模为1，很自然的，由于 $a^2 + b^2 = 1$ 我们有

$$
a = cos{\theta} e^{i\delta}, \quad b = sin{\theta} e^{i(\phi + \delta)}
$$

忽略全局global phase （为什么？）就可以得到

$$
\Psi = cos{\theta} |0\rangle + sin{\theta} e^{i\phi} |1\rangle
$$

这也就意味着，任何对单比特的操作都可以看成是Bloch球上的旋转。让我们来试几个单比特门

(打开[bloch_sphere.jl](https://github.com/QuantumBFS/SSSS/blob/master/4_quantum/bloch_sphere.jl))

### 1. Pauli门

```julia
mat(X)
```

    2×2 LuxurySparse.PermMatrix{Complex{Float64},Int64,Array{Complex{Float64},1},Array{Int64,1}}:
        0       1.0+0.0im
     1.0+0.0im     0

Pauli门是常数门，Yao会提前分配常数门的矩阵所需的内存，不需要担心使用常数门进行计算会发生额外的内存分配

```julia
@allocated mat(X)
```

    0

而定义一个新的常数门也仅仅只需要一行

```julia
@const_gate MyConstGate = rand(4, 4)
```

```julia
@allocated mat(MyConstGate)
```

    0

```julia
nqubits(MyConstGate)
```

    2

### 2. Phase门

### 3. 旋转门

1. Rx

2. Ry

3. Rz

旋转门的矩阵形式是：

$$
cos{\frac{\theta}{2}} \mathbf{I} - i sin{\frac{\theta}{2}} \mathbf{U}
$$

```julia
mat(Rx(0.1))
```

    2×2 StaticArrays.SArray{Tuple{2,2},Complex{Float64},2,4}:
     0.99875+0.0im            0.0-0.0499792im
         0.0-0.0499792im  0.99875+0.0im

### 4. Hadmard门

## 2.1 制备一个GHZ态

在Yao里我们用block来描述量子线路，最基本的门是primitive block，例如我们这里要用到的X门和H门，primitive block是指没有subblock（子block）的block

```julia
using Yao
subblocks(X)
```

    ()

而我们使用不同的composite block将block组装起来就可以构建出更大的quantum circuit。例如我们可以将H门串起来

```julia
chain(H, H, H, H)
```

    nqubits: 1, datatype: Complex{Float64}
    chain
    ├─ H gate
    ├─ H gate
    ├─ H gate
    └─ H gate

这就构成来一个非常简单的量子线路，它会不断对输入的单比特作用Hadmard门

而有了横向组装门的方法，我们如何在纵向组装呢？你可以用kron，纵向排列的门相当于这些门的张量积。别忘了利用语法糖

```julia
kron(H for _ in 1:4)
```

    nqubits: 4, datatype: Complex{Float64}
    kron
    ├─ 1=>H gate
    ├─ 2=>H gate
    ├─ 3=>H gate
    └─ 4=>H gate

当然你也可以指定位置，但是这个时候就要记得输入比特的数量，因为程序并不能够自己推导出总共的比特数目

```julia
kron(4, 1=>X, 3=>H)
```

    nqubits: 4, datatype: Complex{Float64}
    kron
    ├─ 1=>X gate
    └─ 3=>H gate

但是加入我忘记输入比特数目，或者我暂时还没想好要用多少个比特怎么办？没关系！

```julia
kron(1=>X, 3=>H)
```

    (n -> kron(n, 1 => X gate, 3 => H gate))

Yao在无法推导出比特数目的时候会返回一个以总比特数为输入的匿名函数，你依然可以把它当作正常的block来使用，Yao会在能够推导出比特数目的时候自动把它填进去，例如

```julia
chain(kron(1=>X, 3=>H), kron(H for _ in 1:4))
```

    nqubits: 4, datatype: Complex{Float64}
    chain
    ├─ kron
    │  ├─ 1=>X gate
    │  └─ 3=>H gate
    └─ kron
       ├─ 1=>H gate
       ├─ 2=>H gate
       ├─ 3=>H gate
       └─ 4=>H gate

但是GHZ里还有控制门，这个要怎么写呢？类似的在Yao里我们使用Julia自带的 `Pair` 类型来指定位置，一个1号比特控制2号比特上X门的控制模块可以这么写

```julia
control(4, 1, 2=>X)
```

    nqubits: 4, datatype: Complex{Float64}
    control(1)
    └─ (2,) X gate

注意，如果你不写总比特数，就会返回一个函数

```julia
control(1, 2=>X)
```

    (n -> control(n, 1, 2 => X gate))

那么到这里我们就可以构建上面画的这个GHZ的线路了

```julia
circuit = chain(
    kron(1=>X, (k=>H for k in 2:4)...), # 第一层
    control(2, 1=>X), # 第一个CNOT
    control(4, 3=>X),
    control(3, 1=>X),
    control(4, 3=>X),
    kron(H for _ in 1:4), # 最后一层H
)
```

    nqubits: 4, datatype: Complex{Float64}
    chain
    ├─ kron
    │  ├─ 1=>X gate
    │  ├─ 2=>H gate
    │  ├─ 3=>H gate
    │  └─ 4=>H gate
    ├─ control(2)
    │  └─ (1,) X gate
    ├─ control(4)
    │  └─ (3,) X gate
    ├─ control(3)
    │  └─ (1,) X gate
    ├─ control(4)
    │  └─ (3,) X gate
    └─ kron
       ├─ 1=>H gate
       ├─ 2=>H gate
       ├─ 3=>H gate
       └─ 4=>H gate

注意上面的门的作用顺序和线路图略有不同（线路里两个control是同时作用的），但是是等价的。

接下来让我们来验证一下，这个线路是否能够从000制备出GHZ态

```julia
r = ArrayReg(bit"0000") |> circuit
```

    ArrayReg{1, Complex{Float64}, Array...}
        active qubits: 4/4

```julia
using Plots

results = measure(r; nshots=2000);
histogram(results; nbins=16, legend=nothing, xlabel="bit configuration", xticks=((0:15).+0.5, 0:15))
```

虽然一般的来说，构建线路只需要 `control`, `chain`, `kron` 三种复合模块就可以了，但是对于数值模拟，但是实际使用上我们还提供了很多不同的复合模块来帮助你快速定义模型，比如上面一排Hadmard的例子，你可以直接使用repeat模块来构造

```julia
repeat(4, H)
```

    nqubits: 4, datatype: Complex{Float64}
    repeat on (1, 2, 3, 4)
    └─ H gate

我们不妨来看看repeat的文档，Yao的大部分函数都可以在help模式中查看文档和用例

```julia
?repeat(4, H)
```

```
repeat(n, x::AbstractBlock[, locs]) -> RepeatedBlock{n}
```

Create a [`RepeatedBlock`](@ref) with total number of qubits `n` and the block to repeat on given location or on all the locations.

# Example

This will create a repeat block which puts 4 X gates on each location.

```jldoctest
julia> repeat(4, X)
nqubits: 4, datatype: Complex{Float64}
repeat on (1, 2, 3, 4)
└─ X gate
```

You can also specify the location

```jldoctest
julia> repeat(4, X, (1, 2))
nqubits: 4, datatype: Complex{Float64}
repeat on (1, 2)
└─ X gate
```

But repeat won't copy the gate, thus, if it is a gate with parameter, e.g a `phase(0.1)`, the parameter will change simultaneously.

```jldoctest
julia> g = repeat(4, phase(0.1))
nqubits: 4, datatype: Complex{Float64}
repeat on (1, 2, 3, 4)
└─ phase(0.1)

julia> g.content
phase(0.1)

julia> g.content.theta = 0.2
0.2

julia> g
nqubits: 4, datatype: Complex{Float64}
repeat on (1, 2, 3, 4)
└─ phase(0.2)
```

然后对单个的模块我们还可以直接用put，而不用专门kron

```julia
put(4, 1=>X)
```

    nqubits: 4, datatype: Complex{Float64}
    put on (1)
    └─ X gate

put虽然可以作用任意大小的block，但是它对小的block更高效，对大的block，我们需要稍后会介绍到的concentrator。但是现在我们可以把上面的线路重写为

```julia
circuit2 = chain(
    4,
    put(1=>X),
    repeat(H, 2:4),
    control(2, 1=>X),
    control(4, 3=>X),
    control(3, 1=>X),
    control(4, 3=>X),
    repeat(H, 1:4),
)
```

    nqubits: 4, datatype: Complex{Float64}
    chain
    ├─ put on (1)
    │  └─ X gate
    ├─ repeat on (2, 3, 4)
    │  └─ H gate
    ├─ control(2)
    │  └─ (1,) X gate
    ├─ control(4)
    │  └─ (3,) X gate
    ├─ control(3)
    │  └─ (1,) X gate
    ├─ control(4)
    │  └─ (3,) X gate
    └─ repeat on (1, 2, 3, 4)
       └─ H gate

## 2.2 量子傅立叶变换

量子傅立叶变换可以用下面的线路表示，试试自己用Yao来定义这个线路？

量子傅立叶变换是很多量子算法的基础模块，它是经典的快速傅立叶变换的量子版本，经典的傅立叶变换可以写成

$$
y_k = \sum_{j=0}^{N-1} e^{\frac{2\pi i k j}{N}} x_j
$$

而量子傅立叶变换则定义为

$$
\sum_j \alpha_j |j\rangle \rightarrow \sum_k \hat{\alpha}_k |k\rangle, \quad where \quad \hat{\alpha}_k = \frac{1}{\sqrt{N}}\sum_{j}^{N-1} e^{2\pi ijk/N}\alpha_j
$$

量子傅立叶变换可以用下面的线路表示，试试自己用Yao来定义这个线路？

它会将一个函数变换到频率空间

```julia
using FFTW, Interact, Plots

xs = LinRange(-5, 5, 10000)
l = @layout (1, 2)

f(x) = sin(x^2) * exp(x^2/10)

@manipulate for k in 1:0.1:2
    ys = f.(k * xs)
    plot(xs, [ys, abs.(fft(ys))], layout=l, ylims=(-10, 10), size=(1000, 200))
end
```

    Unable to load WebIO. Please make sure WebIO works for your Jupyter client.

很自然的我们可以用相位偏移来实现这个操作，因为相位偏移（phase shift）门有如下的形式：

$$
\begin{pmatrix}
1 & 0\\
0 & e^{i\theta}
\end{pmatrix}
$$

而我们上面定义的（量子）傅立叶变换可以写成下面的矩阵形式

$$
\begin{pmatrix}
1      & 1          & 1             & \dots           & 1            \\
1      & \omega     & \omega^2      & \dots           & \omega^{N-1} \\
1      & \omega^2   & \omega^4      & \dots           & \omega^{2N-2} \\
\vdots & \vdots     & \vdots        & \vdots          & \vdots \\
1      & \omega^N-1 & \omega^{2N-2} & \vdots          & \omega^{(N-1)(N-1)}
\end{pmatrix} \quad \omega^N = 1, \quad \omega = e^{2\pi i/N}
$$

我们定义下面的记号，对二进制整数有这样两种记号：

$$
\begin{aligned}
(k)_{(10)} &= \text{LSB}[k_1 k_2 \dots k_{n}]_{2} \\
&= k_1 2^{n-1} + k_2 2^{n-2} \dots k_{n} 2^{0}\\
&= \sum_{l=1}^{n} k_l 2^{n - l}
\end{aligned}
$$

或者

$$
\begin{aligned}
(k)_{(10)} &= \text{MSB}[k_1 k_2 \dots k_n]_{2}\\
&= k_1 2^{0} + k_2 2^{1} \dots k_{n} 2^{n-1}\\
&= \sum_{l=1}^{n} k_l 2^{l-1}
\end{aligned}
$$

我们记后者为 MSB (Most Significant Bit numbering) 前者为 LSB (Least Significant Bit numbering)

类似的对二进制小数也有类似的记号：

$$
\begin{aligned}
(0.k)_{(10)} &= \text{LSB}[k_1 k_2 \dots k_n]_{(2)}  /  2^n\\
&= \text{MSB} [0.k_1 k_2 \dots k_n]_{(2)}\\
&= k_1 2^{-1} + k_2 2^{-2} \cdots k_n 2^{-n}\\
&= \sum_{l=1}^{n} k_l 2^{-l} \\
\end{aligned}
$$

$$
\begin{aligned}
(0.k)_{(10)} &= \text{MSB}[k_1 k_2 \cdots k_n]_{(2)}  /  2^n \\
&= \text{LSB} [0.k_1 k_2 \dots k_n]_{(2)}\\
&= k_1 2^{-n} + k_2 2^{-(n-1)} \cdots k_n 2^{-1}\\
&= \sum_{l=1}^{n} k_l 2^{n-l+1}
\end{aligned}
$$

而一般地，我们可以定义

$$
LSB[0.k_{j+1} \cdots k_{j+n}]_{(2)} = \sum_{l=1}^n k_{j+l} 2^{-(j+l)} = LSB[0.k_1 \cdots k_{n}]_{(2)} / 2^j\\
MSB[0.k_{j+1} \cdots k_{j+n}]_{(2)} = \sum_{l=1}^n k_{j+l} 2^{n-(j+l)+1} = MSB[0.k_1 \cdots k_{n}]_{(2)} / 2^j
$$

我们观察到对上面的量子傅立叶变换，不妨假设 $j$ 和 $k$ 都是 $n$ 位二进制数

$$
\begin{aligned}
|x\rangle &= \frac{1}{2^{n/2}} \sum_{k=0}^{2^n-1} e^{2\pi i \cdot \text{LSB}[x_1 x_2\cdots x_n]_{(2)} k/2^n} |k\rangle\\
&= \frac{1}{2^{n/2}} \sum_{k=0}^{2^n-1} e^{2\pi i \cdot \text{LSB}[0.x_1 x_2 \cdots x_n]_{(2)} k} |k\rangle\\
&= \frac{1}{2^{n/2}} \sum_{\{k_1, k_2, \cdots k_n\}\in \{0, 1\}^n} e^{2\pi i \cdot \text{LSB}[0.x_1 x_2 \cdots x_n]_{(2)} \text{MSB}[k_1 k_2 \cdots k_n]_{(2)}} |MSB[k_1 k_2 \cdots k_n]_{(2)} \rangle \\
\end{aligned}
$$

$$
\begin{aligned}
|x\rangle &= \frac{1}{2^{n/2}} \sum_{\{k_1, k_2, \cdots k_n\}\in \{0, 1\}^n} e^{2\pi i \cdot \sum_{l=1}^n k_l 2^{l-1}  \cdot \text{LSB}[0.x_1 x_2 \cdots x_n]_{(2)}} |MSB[k_1 k_2 \cdots k_n]_{(2)} \rangle \\
&= \frac{1}{2^{n/2}} \sum_{\{k_1, k_2, \cdots k_n\}\in \{0, 1\}^n} e^{2\pi i \cdot \sum_{l=1}^n k_{l} \cdot \text{LSB}[0.x_{l} x_{l+1} \cdots x_{n}]_{(2)}} |MSB[k_1 k_2 \cdots k_n]_{(2)}\rangle
\end{aligned}
$$

$$
\begin{aligned}
|x\rangle &= \frac{1}{2^{n/2}} \sum_{\{k_1, k_2, \cdots k_n\}\in \{0, 1\}^n} \bigotimes_{l=1}^n e^{2\pi i k_{l} \cdot \text{LSB}[0.x_{l} x_{l+1} \cdots x_{n}]_{(2)}} |k_l^{\text{MSB}}\rangle\\
&= \frac{1}{2^{n/2}} \bigotimes_{l=1}^n \sum_{k_{l} \in \{0, 1\}} e^{2\pi i k_{l} \cdot \text{LSB}[0.x_{l} x_{l+1} \cdots x_{n}]_{(2)}} |k_l^{\text{MSB}}\rangle\\
&= \frac{1}{2^{n/2}} \bigotimes_{l=1}^n [|0^{\text{MSB}}\rangle + e^{2\pi i \cdot \text{LSB}[0.x_{l} x_{l+1} \cdots x_{n}]_{(2)}} |1^{\text{MSB}}\rangle ] \\
&= \frac{1}{2^{n/2}} \bigotimes_{l=1}^n [|0^{\text{MSB}}\rangle + \omega_{n-l+1}^{\text{LSB}[x_{l} x_{l+1} \cdots x_{n}]_{(2)}} |1^{\text{MSB}}\rangle ] \quad where \quad \omega_k = e^{2\pi i / 2^k}
\end{aligned}
$$

Yao里默认使用和数组角标顺序一致的MSB，下面我们先从 $|\text{MSB}[01]\rangle$ 来看看要怎么做

$$
\begin{aligned}
|\text{LSB}[10] \rangle &= \frac{1}{2} [|0^{MSB}\rangle + \omega_2^2 |1^{MSB}\rangle] \otimes [|0^{MSB}\rangle + |1^{MSB}\rangle] = \frac{1}{2} ( |0\rangle - |1\rangle + |2\rangle - |3\rangle )
\end{aligned}
$$

这个结构可以用这样一个H门+控制shift门+H门来实现

```julia
c = chain(2, put(1=>H), control(2, 1=>shift(2π / 1 << 2)), put(2=>H))
```

    nqubits: 2, datatype: Complex{Float64}
    chain
    ├─ put on (1)
    │  └─ H gate
    ├─ control(2)
    │  └─ (1,) shift(1.5707963267948966)
    └─ put on (2)
       └─ H gate

```julia
ArrayReg(bit"01") |> c |> state
```

    4×1 Array{Complex{Float64},2}:
      0.4999999999999999 + 0.0im
     -0.4999999999999999 + 0.0im
      0.4999999999999999 + 0.0im
     -0.4999999999999999 + 0.0im

对任意的QFT，我们可以用下面这样递归的方式实现

```julia
A(i, j) = control(i, j=>shift(2π/(1<<(i-j+1))))
B(n, i) = chain(n, i==j ? put(i=>H) : A(j, i) for j in i:n)
qft(n) = chain(B(n, i) for i in 1:n)
```

    qft (generic function with 1 method)

```julia
qft(4)
```

    nqubits: 4, datatype: Complex{Float64}
    chain
    ├─ chain
    │  ├─ put on (1)
    │  │  └─ H gate
    │  ├─ control(2)
    │  │  └─ (1,) shift(1.5707963267948966)
    │  ├─ control(3)
    │  │  └─ (1,) shift(0.7853981633974483)
    │  └─ control(4)
    │     └─ (1,) shift(0.39269908169872414)
    ├─ chain
    │  ├─ put on (2)
    │  │  └─ H gate
    │  ├─ control(3)
    │  │  └─ (2,) shift(1.5707963267948966)
    │  └─ control(4)
    │     └─ (2,) shift(0.7853981633974483)
    ├─ chain
    │  ├─ put on (3)
    │  │  └─ H gate
    │  └─ control(4)
    │     └─ (3,) shift(1.5707963267948966)
    └─ chain
       └─ put on (4)
          └─ H gate

### FFT和QFT的关系

通过上面的推导我们发现实际上QFT相当于颠倒了bit numbering的离散FFT（DFT），而我们从上面的线路图里可以很容易数出来，QFT的复杂度是 $O(n^2)$ 量级的。而对于n个比特，离散傅立叶变换需要 $n 2^n$ 的复杂度。但是实际上由于目前并没有真正的量子硬件使用我们在经典模拟的时候可以用DFT来进行模拟。Yao的可扩展性提供了很方便的接口来做到这一点。

我们首先定义一个 `AbstractBlock` 的子类型

```julia
struct QFT{N, T} <: PrimitiveBlock{N, T} end

QFT(::Type{T}, n::Int) where T = QFT{n, T}()
QFT(n::Int) = QFT(ComplexF64, n)
```

    QFT

然后让我们定义它的矩阵形式，而实际上这个时候 `QFT` 作为block已经可以使用了，尽管我们没有为QFT定义特别的计算方式，Yao会使用它的矩阵进行计算。

```julia
qft_circuit(x::QFT{N}) where N = qft(N)
YaoBlocks.mat(x::QFT) = mat(qft_circuit(x))
```

所以接下来我们来定义具体如何计算QFT

```julia
using FFTW, LinearAlgebra

function YaoBlocks.apply!(r::ArrayReg, x::QFT)
    α = sqrt(length(statevec(r)))
    invorder!(r)
    lmul!(α, ifft!(statevec(r)))
    return r
end
```

在Yao里，一个block如何作用在register上的行为是用 `apply!` 函数确定的，也就是说只要为QFT定义了apply函数，我们就可以用FFT来模拟QFT。

现在让我们来检查一下我们定义的QFT和直接用量子线路计算出来的结果是否一致

```julia
r = rand_state(5)
r1 = r |> copy |> QFT(5)
r2 = r |> copy |> qft(5)
r1 ≈ r2
```

    true

到此为止我们就定义好所有必要的方法了，是不是非常简单？

不过也许你是一个追求完美的人，想让你的QFT block的打印信息好看一些？当然没问题，只要定义下面这个方法就可以了

```julia
YaoBlocks.print_block(io::IO, x::QFT{N}) where N = print(io, "QFT($N)")
```

```julia
QFT(5)
```

    QFT(5)

Yao会自己处理和其它block组合的问题

```julia
chain(QFT(5), put(1=>H))
```

    nqubits: 5, datatype: Complex{Float64}
    chain
    ├─ QFT(5)
    └─ put on (1)
       └─ H gate

亦或者是QFT的逆变换（为什么？想想我们之前讲过的）

```julia
QFT(5)'
```

     [†]QFT(5)

## 2.3 相位估计 （Phase Estimation）

我们在上一节封装好了一个QFT block，那么我们接下来就可以用它做一些有用的事情了比如相位估计，相位估计是指下面这个问题：

给定一个作用在m个qubit上的unitary $U$，有 $U |\psi\rangle = e^{2\pi i \theta}|\psi\rangle, \quad 0 \leq \theta < 1$

这个问题可以用这样一个线路来解决，试试自己用Yao定义出来？

相位估计其实需要两个register，我们按照上面的线路来看看它们都做了什么。

首先对register 1，我们作用了一排H门，这个怎么写？

```julia
PE(n) = chain(n, repeat(H))
```

    PE (generic function with 1 method)

作用一下

```julia
ArrayReg(bit"00") |> PE(2) |> state
```

    4×1 Array{Complex{Float64},2}:
     0.4999999999999999 + 0.0im
     0.4999999999999999 + 0.0im
     0.4999999999999999 + 0.0im
     0.4999999999999999 + 0.0im

然后我们来构造一个假的（已知 $\theta$）态和 U 来看看都发生了什么，分这么几步：

```julia
N, M = 3, 5
```

    (3, 5)

1. 随机获得一个unitary，然后用本征值求解器找到所有的本征矢

```julia
P = eigen(rand_unitary(1<<M)).vectors;
```

2. 我们构造一个能够用二进制小数表示的 $\theta$ 因为我们想要凑一个QFT能够变换的形式

```julia
θ = Int(0b110) / 1 << N;
```

3. 然后随便产生一些随机数当其它的phase

```julia
phases = rand(1<<M);
```

4. 我们把其中某个比特的本征值的相位定为 $\theta$

```julia
phases[bit"010"] = θ;
```

5. 然后把本征矢矩阵乘在这个对角矩阵的两边

```julia
U = P * Diagonal(exp.(2π * im * phases)) * P';
```

我们的波函数 $\psi$ 就是本征值矩阵的第 `bit"010"` 个本征矢

```julia
psi = P[:, bit"010"]
norm(U * psi - exp(2π * im * θ) * psi)
```

    9.482712684165183e-15

接下来我们来构造这两个register，也就是把一个 $|000\cdots 00\rangle$ 的register和一个处于 $\psi$ 的register拼接起来

```julia
r = join(ArrayReg(psi), zero_state(N))
```

    ArrayReg{1, Complex{Float64}, Array...}
        active qubits: 8/8

所以在Hadmard之后发生了什么呢？受控制的 U 被作用在了 $\psi$ 上面，首先 $U^{2^k}$ 作用在 $\psi$ 上我们会得到

$$
U^{2^k} |\psi\rangle = e^{2\pi 2^k \theta} |\psi\rangle
$$

类似QFT，也就是说 $\theta$ 也可以写成类似的小数形式，我们用C-U作用在register上也可以获得类似的

$$
\frac{1}{2^{n/2}} \bigotimes_{j=1}^{n} [|0\rangle + e^{2\pi i 2^{n-j}\theta}|1\rangle] |\psi\rangle = \sum_{k=0}^{2^n-1} e^{2\pi i \theta} |k\rangle |\psi\rangle
$$

回忆之前QFT的形式，我们已经凑出来一样的格式了，接下来我们希望能用一个QFT里面的小数和 $\theta$ 相减，而如果我们对前n个比特做iQFT就可以产生一个负号（也就是QFT的adjoint），这样如果某一项为零，其余项为整数，那么对前n个比特我们就可以得到一个某一位概率为1的波函数。那么不妨就对前n个比特做iQFT

$$
\frac{1}{2^{n}}\sum_{x=0}^{2^n-1}\sum_{k=0}^{2^n-1} e^{2\pi i \theta} e^{-2\pi i k x / 2^n} |x\rangle = \frac{1}{2^n}\sum_{x=0}^{2^n-1}\sum_{k=0}^{2^n-1} e^{-\frac{2\pi i k}{2^n} (x - 2^n\theta)} |x\rangle
$$

但是注意 $\theta$ 也许并不能用 n 个比特准确表示（虽然32个比特就足够表示大部分的小数了），我们记 $2^n\theta = a + 2^n \delta$，就有

$$
\frac{1}{2^n}\sum_{x=0}^{2^n-1}\sum_{k=0}^{2^n-1} e^{-\frac{2\pi i k}{2^n} (x - a)} e^{-2\pi i k \delta} |x\rangle
$$

而在这个态上测量到 $|a\rangle$ 的概率为

$$
Pr(a) = |\langle a |\frac{1}{2^n}\sum_{x=0}^{2^n-1}\sum_{k=0}^{2^n-1} e^{-\frac{2\pi i k}{2^n} (x - a)} e^{-2\pi i k \delta} |x\rangle|^2 = \frac{1}{2^n}|\sum_{k=0}^{2^{n-1}} e^{2\pi i k \delta}|^2
$$

回到我们的线路，理解原理以后，构造其实非常简单

但是问题来了，如何只在前n个比特上作用iQFT呢？在Yao里我们定义了 active qubit的概念，所有的block都只作用在当前active的qubit上，我们可以使用 `focus!` 和 `relax!` 函数来调整当前激活的qubit，例如

```julia
r = ArrayReg(bit"10101")
```

    ArrayReg{1, Complex{Float64}, Array...}
        active qubits: 5/5

```julia
focus!(r, 1)
```

    ArrayReg{1, Complex{Float64}, Array...}
        active qubits: 1/5

```julia
r |> X
```

    ArrayReg{1, Complex{Float64}, Array...}
        active qubits: 1/5

```julia
relax!(r, 1)
```

    ArrayReg{1, Complex{Float64}, Array...}
        active qubits: 5/5

```julia
state(r)[bit"10100"]
```

    1.0 + 0.0im

为了方便你定义线路，Yao还提供了 `concentrate` 这个模块，它会在register进入这个模块的时候进行focus，从而作用较小的模块，然后在作用完毕之后再relax回原来的大小和顺序。所以我们只要把iQFT放进concentrate里面就好了，这样我们的PE线路就可以这么定义

```julia
PE(n, m, U) =
    chain(n+m, repeat(H, 1:n), # H
               chain(control(k, n+1:n+m=>matblock(U^(2^(k-1)))) for k in 1:n), # C-U
               concentrate(QFT(n)', 1:n) # iQFT
    )
```

    PE (generic function with 2 methods)

最后让我们来验证一下

```julia
r = join(ArrayReg(psi), zero_state(N))
r |> PE(N, M, U)
results = measure(r, 1:N; nshots=1)
```

    1-element Array{Int64,1}:
     3

别忘了我们获得的比特是反的（因为FFT），所以我们来反转一下

```julia
using BitBasis
estimated_phase = bfloat(results[]; nbits=N)
```

    0.75

### 3 动机 Motivation

- 变分量子线路算法的兴起

- 基于Python的模拟器性能上的劣势

- 易用性和扩展性

### 4 量子体系结构概览

### 5. Yao的设计思路和架构

### 5.1 为算法研究而生

### 5.2 线路的树状表示和IR

### 5.3 利用多重派发进行异构计算

### 5.4 利用多重派发定制线路运行行为

### 5.5 未来和正在进行的工作

### 6. 如何参与Yao的开发


# Finite Elements Lecture 1

```
from IPython.core.display import HTML
css_file = '../ipython_notebook_styles/ngcmstyle.css'
HTML(open(css_file, "r").read())
```

<link href='http://fonts.googleapis.com/css?family=Open+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 1000px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1200px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Open Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:900px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Arvo', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Arvo', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Arvo', serif;
    margin-top:16px;
	font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Arvo', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Arvo', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 90%;
}

</style>

##### Georges Limbert

This will be introduction:

* Fundamental aspects
* Examples of applications
* The direct stiffness matrix method
* The weak form

###### References:

* [The Finite Element Method (Hughes, 2003)](http://www.amazon.co.uk/gp/product/B00CB2MK8K?btkr=1)
* [Belytschko, Liu & Moran](http://www.amazon.co.uk/Nonlinear-Elements-Structures-Mechanical-Engineering/dp/0471987743)
* [Simo and Hughes](http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-0-387-97520-7)
* [Crisfield](http://www.amazon.com/Non-Linear-Finite-Element-Analysis-Structures/dp/047197059X)

## Background

Typical applications for GL:

* Background in applied maths and engineering mechanics
* Nonlinear continuum mechanics of biological tissues/biomaterials
* Multiphysics finite element techniques (theory/implementation/analysis)
* Models used in industry, academia and the military

## Aims

* Introduce the general "philosophy"
* Raise awareness about the applications
* Highlight the concept through simple examples
* Introduce variational formulations of PDEs

## Computational Engineering

* Modelling complex engineering systems
* Assessment of performace and safety before physical prototypes are built and tested
* Explore many design alternatives
* Optimise performance and safety
* Certification requirements

## Finite Element Method

Powerful method to solve PDEs over *complex* geometrical domains.

* Rooted in variational methods
* Principles developed in the 40's, driven by Boeing and large advances in the 60s and 70s.
* Mainly driven by the need to solve elasticitiy and structural analysis problemns encountered in civil and aeronautical engineering.

## Domain of applications

Very wide:

* Biophysics
* Mechanics
* EM
* Aerodynamics
* Structural dynamics
* Heat transfer
* Biology
* Porous media
* Hydrodynamics
* Structural mechanics
* Nanomechanics
* Behavioural sciences
* and on to any domain of natural and physical sciences.

## Example codes

**Commerical**

* ABAQUS
* ANSYS
* COMSOL
* MARC
* NASTRAN
* LS-DYNA

**Open source**

* FEnics
* CalculiX
* FEAPpv
* WARP3D
* Matlab toolboxes

## Example applications

**Simulated knee flexion**

Fibrous models of ligament - elastic and solid mechanics, stress models, etc.

**Wrinkling analyses**

Compression induced - calculation of first principal strains in, eg, paper crinkling, skin wrinkles in aging, etc. Note importance of interpolation techniques for this application.

**Bi-layer structure**

Importance of thickness and structure for results.

**Simulation of in-plane compression of the epidermis**

Another skin application; the importance of topography, swelling with moisture, etc.

**Oral implants**

Dentistry applications, in combination with imaging techniques.

**Artificial organs and implants**

Importance of mechanics, material, electronics and chemistry.

**Biomedical (stents)**

Combination of health and engineering aspects, with combinations of FEA and CFD.

**Consumer goods**

How the feel of the packaging affects the way you feel about the goods: make it feel expensive to trigger an emotional response.

**EM radiation**

Microwave effects on tissues, and UV-induced damage.

**Movies**

Physics-based computer graphics.

## FEM in a nutshell

1. Transform PDE into a variational problem
2. Introduce a piecewise approximation to the field variables (eg displacement, temperature, electric charge) in the equations.
3. Discretise the physical domain into elements and write approximate equations for each element (**meshing** process). Local equations in each element expressed in matrix form.
4. Assemble local equations into a global matrix.
5. Solve.

## Step 2.

(Step 1 later).

* In each element, choose **interpolation** (or **shape**) functions to approximate field within element in terms of **nodal values** (the **degrees of freedom**).
* In each element, value of each variable anywhere in the element is a linear combination of the shape functions and the nodal values
* Interpolation functions are usually polynomials (Largrange, Hermite, B-spline, ...)

## Step 3 and 4

Step 3:

* Field approximations are injected into the variational form.
* Ran out of time.

## Direct stiffness approach

### Equation for a single elastic spring

Equivalent to a single element.

Forces $f_1$ (left end) and $f_2$ (right end).

Equilibrium equation

$$
  f_1 + f_2 = 0.
$$

Combine with Hooke's law for the spring

$$
\begin{align}
  f_2 & = k (d_2 - d_1) \\ f_1 & = -k (d_2 - d_1) = f_2
\end{align}
$$

to get the equilibrium equations in matrix form

$$
\begin{pmatrix} f_1 \\ f_2 \end{pmatrix} = \begin{pmatrix} k & -k \\ -k & k \end{pmatrix} \begin{pmatrix} d_1 \\ d_2 \end{pmatrix}
$$

The stiffness matrix is symmetric but singular. The problem is that this doesn't constrain spatial translations; can take an infinite number of positions in space. Need to remember this and work around.

Go to a system with two elastic springs.

Equilibrium equation

$$
  F_1 + F_2 + F_3 = 0
$$

Constitutive equations

Split the original structure into elemental components

$$
{\bf f}^{(1)} = k^{(1)} {\bf d}^{(1)}
$$

and similarly for the second element.

Using the link between the displacements, that $d_2^{(1)} = d_1^{(2)}$, can set up a **connectivity matrix** linking the nodes in the different elements. Do something similarly for the forces, getting

$$
\begin{pmatrix} F_1 \\ F_2 \\ F_3 \end{pmatrix} = \begin{pmatrix} f_1^{(1)} \\ f_1^{(2)} + f_2^{(1)} \\ f_2^{(2)} \end{pmatrix}
$$

Expanded local equations

The location equations for each spring can be rewritten in matrix form, or they can be expanded into larger matrices and vectors. Local versions have size 2, global version has size 3.

Final version from adding up all the expanded versions

$$
  K {\bf d} = {\bf f}.
$$

Direct assembly of the global stiffness matrix $K$.

Use the connectivity to measure the contributions or connections.

Special case: grounded spring

Partition and apply boundary condition $d_1 = 0$.

Global equilibrium equations

**Q1** What is the physical meaning of $K$?

It denotes the force felt at node $i$ due to unit desplacement at node $j$ (all others fixed)

**Q2** Which elements contribute to $K$?

Those between $i$ and $j$, connected to $i$.

The stiffness matrix is invertible only when suitable boundary conditions are applied.

## Weak form

### Turning PDEs into variational problems

Multiply by an arbitrary **test function** and integrate over the domain. Perform integration by parts to get rid of second order derivatives.

Very powerful technique - not restricted to conservative systems.

Take the **strong form** of the initial boundary value problem.

$$
\nabla \cdot \sigma + b = \rho \dot{v} = \rho \ddot{u}.
$$

Add prescribed displacement $u = \bar{u}$ and traction $t = \bar{t}$ on the boundaries, together with initial condition on displacement and velocity at $t=0$.

Consider arbitrary vector valued function $\eta = \eta(x) = \etc(\chi(X, t))$. This is the **test** or **weighting** function.

* Time is assumed to be fixed
* $\eta$ vanishes on the boundary where displacements are fixed.

Write functional obtained by multiplying the strong form by the test function and integrating over the domain:

$$
  f(u, \eta) = \int_{\Omega} ( -\nabla \cdot \sigma - b + \rho \ddot{u} ) \cdot \eta \, dv = 0.
$$

Expand the divergence term as

$$
  \nabla \cdot \sigma \cdot \eta = \nabla (\sigma \eta) - \sigma : \nabla \eta = \nabla (\sigma \eta) - \sigma : \nabla_x \eta.
$$

Hence the functional becomes

$$
  f(u, \eta) = \int_{\Omega} \sigma : \nabla \eta - (b - \rho \ddot{u}) \cdot \eta - \int_{\partial \Omega} \sigma \eta \cdot n \, ds
$$

Using that $\eta$ vanishes on the boundary you get

$$
  \int_{\partial \Omega} \sigma \eta \cdot n \, ds = \int_{\partial \Omega} \bar{t} \cdot \eta \, ds.
$$

### Variational problem

Finally rewrite problem as

$$
  f(u, \eta) = \int_{\Omega} \left[ \sigma : \nabla \eta - (b - \rho \ddot{u})\cdot \eta \right] \, dv - \int_{\partial \Omega} \bar{t} \cdot \eta \, ds = 0,
$$

subject to conditions on the initial data.

### Special choice of the test function

Choose the test function to be a **virtual** displacement, $\eta = \delta u$.

The we get the **principle of virtual work**, and the first term in the volume integral is the **internal mechanical virtual work**, and the second term (less the acceleration term) and the surface integral is the **external mechanical virtual work**. Hence the variational principle is equivalent to the virtual work balancing off against the acceleration.

### Initial boundary value problem

Summarizing, the weak form has the integral plus the initial conditions (in integral form).

### Additional comments

Note that 80% of the time can be spent in meshing. In cases where the interpolation functions are NURBS or B-splines, the geometry is the mesh, so using these functions can greatly speed things up.

FEEG6016 Simulation and Modelling/2014/Finite Elements Lecture 1.ipynb


# Homework 5
## Due Date:  Tuesday, October 3rd at 11:59 PM

# Problem 1
We discussed documentation and testing in lecture and also briefly touched on code coverage.  You must write tests for your code for your final project (and in life).  There is a nice way to automate the testing process called continuous integration (CI).

This problem will walk you through the basics of CI and show you how to get up and running with some CI software.

### Continuous Integration
The idea behind continuous integration is to automate away the testing of your code.

We will be using it for our projects.

The basic workflow goes something like this:

1. You work on your part of the code in your own branch or fork
2. On every commit you make and push to GitHub, your code is automatically tested on a fresh machine on Travis CI. This ensures that there are no specific dependencies on the structure of your machine that your code needs to run and also ensures that your changes are sane
3. Now you submit a pull request to `master` in the main repo (the one you're hoping to contribute to). The repo manager creates a branch off `master`.
4. This branch is also set to run tests on Travis. If all tests pass, then the pull request is accepted and your code becomes part of master.

We use GitHub to integrate our roots library with Travis CI and Coveralls. Note that this is not the only workflow people use. Google git..github..workflow and feel free to choose another one for your group.

### Part 1:  Create a repo
Create a public GitHub repo called `cs207test` and clone it to your local machine.

**Note:** No need to do this in Jupyter.

### Part 2:  Create a roots library
Use the example from lecture 7 to create a file called `roots.py`, which contains the `quad_roots` and `linear_roots` functions (along with their documentation).

Also create a file called `test_roots.py`, which contains the tests from lecture.

All of these files should be in your newly created `cs207test` repo.  **Don't push yet!!!**

### Part 3:  Create an account on Travis CI and Start Building

#### Part A:
Create an account on Travis CI and set your `cs207test` repo up for continuous integration once this repo can be seen on Travis.

#### Part B:
Create an instruction to Travis to make sure that

1. python is installed
2. its python 3.5
3. pytest is installed

The file should be called `.travis.yml` and should have the contents:
```yml
language: python
python:
    - "3.5"
before_install:
    - pip install pytest pytest-cov
script:
    - pytest
```

You should also create a configuration file called `setup.cfg`:
```cfg
[tool:pytest]
addopts = --doctest-modules --cov-report term-missing --cov roots
```

#### Part C:
Push the new changes to your `cs207test` repo.

At this point you should be able to see your build on Travis and if and how your tests pass.

### Part 4:  Coveralls Integration
In class, we also discussed code coverage.  Just like Travis CI runs tests automatically for you, Coveralls automatically checks your code coverage.  One minor drawback of Coveralls is that it can only work with public GitHub accounts.  However, this isn't too big of a problem since your projects will be public.

#### Part A:
Create an account on [`Coveralls`](https://coveralls.zendesk.com/hc/en-us), connect your GitHub, and turn Coveralls integration on.

#### Part B:
Update your the `.travis.yml` file as follows:
```yml
language: python
python:
    - "3.5"
before_install:
    - pip install pytest pytest-cov
    - pip install coveralls
script:
    - py.test
after_success:
    - coveralls
```

Be sure to push the latest changes to your new repo.

### Part 5:  Update README.md in repo
You can have your GitHub repo reflect the build status on Travis CI and the code coverage status from Coveralls.  To do this, you should modify the `README.md` file in your repo to include some badges.  Put the following at the top of your `README.md` file:

```
[](https://travis-ci.org/dsondak/cs207testing.svg?branch=master)

[](https://coveralls.io/github/dsondak/cs207testing?branch=master)
```

Of course, you need to make sure that the links are to your repo and not mine.  You can find embed code on the Coveralls and Travis CI sites.

---

# Problem 2
Write a Python module for reaction rate coefficients.  Your module should include functions for constant reaction rate coefficients, Arrhenius reaction rate coefficients, and modified Arrhenius reaction rate coefficients.  Here are their mathematical forms:
\begin{align}
  &k_{\textrm{const}}   = k \tag{constant} \\
  &k_{\textrm{arr}}     = A \exp\left(-\frac{E}{RT}\right) \tag{Arrhenius} \\
  &k_{\textrm{mod arr}} = A T^{b} \exp\left(-\frac{E}{RT}\right) \tag{Modified Arrhenius}
\end{align}

Test your functions with the following paramters:  $A = 10^7$, $b=0.5$, $E=10^3$.  Use $T=10^2$.

A few additional comments / suggestions:
* The Arrhenius prefactor $A$ is strictly positive
* The modified Arrhenius parameter $b$ must be real
* $R = 8.314$ is the ideal gas constant.  It should never be changed (except to convert units)
* The temperature $T$ must be positive (assuming a Kelvin scale)
* You may assume that units are consistent
* Document each function!
* You might want to check for overflows and underflows

**Recall:** A Python module is a `.py` file which is not part of the main execution script.  The module contains several functions which may be related to each other (like in this problem).  Your module will be importable via the execution script.  For example, suppose you have called your module `reaction_coeffs.py` and your execution script `kinetics.py`.  Inside of `kinetics.py` you will write something like:
```python
import reaction_coeffs
# Some code to do some things
# :
# :
# :
# Time to use a reaction rate coefficient:
reaction_coeffs.const() # Need appropriate arguments, etc
# Continue on...
# :
# :
# :
```
Be sure to include your module in the same directory as your execution script.

---

# Problem 3
Write a function that returns the **progress rate** for a reaction of the following form:
\begin{align}
  \nu_{A} A + \nu_{B} B \longrightarrow \nu_{C} C.
\end{align}
Order your concentration vector so that
\begin{align}
  \mathbf{x} =
  \begin{bmatrix}
    \left[A\right] \\
    \left[B\right] \\
    \left[C\right]
  \end{bmatrix}
\end{align}

Test your function with
\begin{align}
  \nu_{i} =
  \begin{bmatrix}
    2.0 \\
    1.0 \\
    1.0
  \end{bmatrix}
  \qquad
  \mathbf{x} =
  \begin{bmatrix}
    1.0 \\
    2.0 \\
    3.0
  \end{bmatrix}
  \qquad
  k = 10.
\end{align}

You must document your function and write some tests in addition to the one suggested.  You choose the additional tests, but you must have at least one doctest in addition to a suite of unit tests.

### Answer:

Answers to Problems 3-6 are saved as .py files in the cs207test repo created in Problems 1-2.

```python
import doctest
from typing import Iterable, List, Sequence

def progress_rate (x: Sequence[float], v: Sequence[float], k: float) -> float:
    """ Calculates progress rate for single reaction represented by inputs.

    Args:
        x: Concentration vector.
        v: Stoichiometric coefficients of reactants.
        k: Forward reaction rate coefficient.

    Notes:
        Raises ValueError if sequences x and v are not the same length.

    Returns:
        Reaction progress rate.

    Examples:
    >>> progress_rate([1.0, 2.0, 3.0], [2.0, 1.0, 1.0], 10)
    60.0
    """
    if len(x) != len(v):
        raise ValueError('x and v must be same length.')

    result = 1
    for i, x_i in enumerate(x):
        result *= pow(x_i, v[i])
    return k * result
```

```python
import unittest

class TestProgressRate(unittest.TestCase):
    """ Tests progress_rate() function under various circumstances. """

    def test_unequal_dim_args (self):
        """ Ensures func raises ValueError with unequally-dimensioned args. """
        x = [1.0, 2.0, 3.0]
        v = [2.0, 1.0]
        k = 10

        try:
            rate = progress_rate(x, v, k)
        except Exception as e:
            if not isinstance(e, ValueError):
                self.fail('Expected ValueError but caught different exception.')
            else:
                pass
        else:
            self.fail('No exception raised even though args unequally-dimensioned.')

    def test_x_not_seq (self):
        """ Ensures that TypeError is raised when x is not a sequence. """
        x = 1.0
        v = [2.0, 1.0]
        k = 10
        try:
            rate = progress_rate(x, v, k)
        except Exception as e:
            if not isinstance(e, TypeError):
                self.fail('Expected TypeError but caught different exception.')
            else:
                pass
        else:
            self.fail('No exception raised even though x not sequence.')
```

---
# Problem 4
Write a function that returns the **progress rate** for a system of reactions of the following form:
\begin{align}
  \nu_{11}^{\prime} A + \nu_{21}^{\prime} B \longrightarrow \nu_{31}^{\prime\prime} C \\
  \nu_{12}^{\prime} A + \nu_{32}^{\prime} C \longrightarrow \nu_{22}^{\prime\prime} B + \nu_{32}^{\prime\prime} C
\end{align}
Note that $\nu_{ij}^{\prime}$ represents the stoichiometric coefficient of reactant $i$ in reaction $j$ and $\nu_{ij}^{\prime\prime}$ represents the stoichiometric coefficient of product $i$ in reaction $j$.  Therefore, in this convention, I have ordered my vector of concentrations as
\begin{align}
  \mathbf{x} =
  \begin{bmatrix}
    \left[A\right] \\
    \left[B\right] \\
    \left[C\right]
  \end{bmatrix}.
\end{align}

Test your function with
\begin{align}
  \nu_{ij}^{\prime} =
  \begin{bmatrix}
    1.0 & 2.0 \\
    2.0 & 0.0 \\
    0.0 & 2.0
  \end{bmatrix}
  \qquad
  \nu_{ij}^{\prime\prime} =
  \begin{bmatrix}
    0.0 & 0.0 \\
    0.0 & 1.0 \\
    2.0 & 1.0
  \end{bmatrix}
  \qquad
  \mathbf{x} =
  \begin{bmatrix}
    1.0 \\
    2.0 \\
    1.0
  \end{bmatrix}
  \qquad
  k = 10.
\end{align}

You must document your function and write some tests in addition to the one suggested.  You choose the additional tests, but you must have at least one doctest in addition to a suite of unit tests.

```python
def progress_rates (x: Sequence[float], v_react: Sequence[Sequence[float]],
                    k: float) -> List[float]:
    """ Calculates progress rates for system of reactions.

    Args:
        x: Concentration vector.
        v_react: Matrix of stoichiometric coefficients for reactants.
        k: Forward reaction rate coefficient.

    Notes:
        v_react is an m-by-n matrix, where the number of rows (m) must equal the number
        of species (i.e., length of concentration vector x) and the number of columns
        must equal the number of reactions.

    Returns:
        List of progress rates for system of reactions.

    Examples:
    >>> progress_rates([1.0, 2.0, 1.0], [[1.0, 2.0], [2.0, 0.0], [0.0, 2.0]], 10)
    [40.0, 10.0]
    """
    # Test inputs
    # Ensure all rows in v_react have same number of columns (corresponding to number
    # of reactions).
    M = len(v_react[0])
    for row in v_react:
        if len(row) != M:
            raise ValueError('Not all rows in v_react share the same dimension.')

    if len(x) != len(v_react):
        raise ValueError('Length of x != number of rows in v_react.')

    # Call progress_rate() on reaction formed from concentration vector x and the
    # reactant coefficients in each column of v_react.
    result = []
    for j in range(0, M):
        v = [row[j] for row in v_react]
        rate = progress_rate(x, v, k)
        result.append(rate)

    return result
```

```python
class TestProgressRates(unittest.TestCase):
    """ Tests progress_rates() function under various circumstances. """

    def test_unequal_dim_args (self):
        """ Ensures func raises ValueError when number of rows in stoichiometric
        coefficient matrix is fewer than len(x).
        """
        x = [1.0, 2.0, 1.0]
        v = [[1.0, 2.0], [2.0, 0.0]]
        k = 10

        try:
            rate = progress_rates(x, v, k)
        except Exception as e:
            if not isinstance(e, ValueError):
                self.fail('Expected ValueError but different exception caught.')
            else:
                pass
        else:
            self.fail('No exception raised even though args unequally-dimensioned.')

    def test_unequal_dim_v_cols (self):
        """ Ensures func raises ValueError when not all of the rows within v share the
        same dimension.
        """
        x = [1.0, 2.0, 1.0]
        # 3rd row has 1 fewer element than other 2 rows.
        v = [[1.0, 2.0], [2.0, 0.0], [0.0]]
        k = 10

        try:
            rate = progress_rates(x, v, k)
        except Exception as e:
            if not isinstance(e, ValueError):
                self.fail('Expected ValueError but different exception caught.')
            else:
                pass
        else:
            self.fail(
                  'No exception raised even though not all rows within v share same '
                  'dimension.')
```

---
# Problem 5
Write a function that returns the **reaction rate** of a system of irreversible reactions of the form:
\begin{align}
  \nu_{11}^{\prime} A + \nu_{21}^{\prime} B &\longrightarrow \nu_{31}^{\prime\prime} C \\
  \nu_{32}^{\prime} C &\longrightarrow \nu_{12}^{\prime\prime} A + \nu_{22}^{\prime\prime} B
\end{align}

Once again $\nu_{ij}^{\prime}$ represents the stoichiometric coefficient of reactant $i$ in reaction $j$ and $\nu_{ij}^{\prime\prime}$ represents the stoichiometric coefficient of product $i$ in reaction $j$.  In this convention, I have ordered my vector of concentrations as
\begin{align}
  \mathbf{x} =
  \begin{bmatrix}
    \left[A\right] \\
    \left[B\right] \\
    \left[C\right]
  \end{bmatrix}
\end{align}

Test your function with
\begin{align}
  \nu_{ij}^{\prime} =
  \begin{bmatrix}
    1.0 & 0.0 \\
    2.0 & 0.0 \\
    0.0 & 2.0
  \end{bmatrix}
  \qquad
  \nu_{ij}^{\prime\prime} =
  \begin{bmatrix}
    0.0 & 1.0 \\
    0.0 & 2.0 \\
    1.0 & 0.0
  \end{bmatrix}
  \qquad
  \mathbf{x} =
  \begin{bmatrix}
    1.0 \\
    2.0 \\
    1.0
  \end{bmatrix}
  \qquad
  k = 10.
\end{align}

You must document your function and write some tests in addition to the one suggested.  You choose the additional tests, but you must have at least one doctest in addition to a suite of unit tests.

```python
import numpy as np

def reaction_rate (x: Sequence[float], v_react: Sequence[Sequence[float]],
                   v_prod: Sequence[Sequence[float]], k: float) -> Iterable:
    """ Calculates reaction rates for species for a given system of reactions.

    Args:
        x: Concentration vector.
        v_react: Matrix of stoichiometric coefficients for reactants.
        v_prod: Matrix of stoichiometric coefficients for products.
        k: Forward reaction rate coefficient.

    Notes:
        v_react is an m-by-n matrix, where the number of rows (m) must equal the number
        of species (i.e., length of concentration vector x) and the number of columns
        must equal the number of reactions.

        v_react and v_prod must be of the same dimensions.

    Returns:
        List of reaction rates for species represented by concentration rates in x.

    Examples:
    >>> reaction_rate([1.0, 2.0, 1.0], [[1.0, 0.0], [2.0, 0.0], [0.0, 2.0]], [[0.0, 1.0], [0.0, 2.0], [1.0, 0.0]])
    [-30.0, -60.0, 20.0]
    """
    reaction_rates = progress_rates(x, v_react, k)

    # Convert stoichiometric coefficients for reactants and products to np matrices to
    # make matrix transformations simpler.
    if not isinstance(v_react, np.matrix):
        v_react = np.matrix(v_react)
    if not isinstance(v_prod, np.matrix):
        v_prod = np.matrix(v_prod)

    if v_react.shape != v_prod.shape:
        raise ValueError('Dimensions of v_react and v_prod not equal.')

    v_ij = v_prod - v_react
    result = v_ij.dot(reaction_rates)
    # Convert 1D matrix to a simple list.
    return np.array(result).flatten().tolist()
```

```python
class TestReactionRate(unittest.TestCase):
    """ Tests reaction_rate() function under various circumstances. """

    def test_unequal_dim_stoichiometric_coeff (self):
        """ Ensures func raises ValueError when v_react and v_prod do not share the
        same dimensions, both in terms of number of rows and columns.
        """
        x = [1.0, 2.0, 1.0]
        k = 10

        # First test with v_react and v_prod having different number of rows.
        v_react = [[1.0, 0.0], [2.0, 0.0], [0.0, 2.0]]
        v_prod = [[0.0, 1.0], [0.0, 2.0]] # 1 row less than v_react

        try:
            rate = reaction_rate(x, v_react, v_prod, k)
        except Exception as e:
            if not isinstance(e, ValueError):
                self.fail('Expected ValueError but different exception caught.')
            else:
                pass
        else:
            self.fail(
                  'No exception raised even though v_prod contains fewer rows than '
                  'v_react.')

        # Next test with v_react and v_prod having same number of rows but one of their
        # rows containing fewer elements than the rest.
        v_react = [[1.0, 0.0], [2.0], [0.0, 2.0]] # 2nd row only contains 1 element
        v_prod = [[0.0, 1.0], [0.0, 2.0], [1.0, 0.0]]

        try:
            rate = reaction_rate(x, v_react, v_prod, k)
        except Exception as e:
            if not isinstance(e, ValueError):
                self.fail('Expected ValueError but different exception caught.')
            else:
                pass
        else:
            self.fail(
                  'No exception raised even though v_react contains fewer elements than '
                  'required in 2nd row.')
```

---
# Problem 6
Put parts 3, 4, and 5 in a module called `chemkin`.

Next, pretend you're a client who needs to compute the reaction rates at three different temperatures ($T = \left\{750, 1500, 2500\right\}$) of the following system of irreversible reactions:
\begin{align}
  2H_{2} + O_{2} \longrightarrow 2OH + H_{2} \\
  OH + HO_{2} \longrightarrow H_{2}O + O_{2} \\
  H_{2}O + O_{2} \longrightarrow HO_{2} + OH
\end{align}

The client also happens to know that reaction 1 is a modified Arrhenius reaction with $A_{1} = 10^{8}$, $b_{1} = 0.5$, $E_{1} = 5\times 10^{4}$, reaction 2 has a constant reaction rate parameter $k = 10^{4}$, and reaction 3 is an Arrhenius reaction with $A_{3} = 10^{7}$ and $E_{3} = 10^{4}$.

You should write a script that imports your `chemkin` module and returns the reaction rates of the species at each temperature of interest.

You may assume that these are elementary reactions.

### Answer:

The order for the species I chose in my vectors/matrices is:

\begin{align}
  \mathbf{x} =
  \begin{bmatrix}
    H_2 \\
    O_2 \\
    OH \\
    HO_2 \\
    H_2O
  \end{bmatrix}
\end{align}

```python
"""
client.py
---------

Simulating client script that will use chemkin and reaction_coeffs in order to
calculate the reaction rates of species at various temperatures.
"""
import chemkin
import numpy as np
import reaction_coeffs

# Store params in lists so we can iterate through them simultaneously.
T = [750, 1500, 2500]

k1 = reaction_coeffs.modified_arrhenius(A=1E8, E=5 * 1E4, T=T[0], b=0.5)
k2 = 1E4
k3 = reaction_coeffs.arrhenius(A=1E7, E=1E4, T=T[2])
k = [k1, k2, k3]

v_react = [[2, 0, 0], [1, 0, 1], [0, 1, 0], [0, 1, 0], [0, 0, 1]]
v_prod = [[1, 0, 0], [0, 1, 0], [2, 0, 1], [0, 0, 1], [0, 1, 0]]

# Client must update concentration rates. Random numbers used here for demo.
x = np.random.randint(1, 5, size=len(v_react))

# Loop through temperatures and calculate reaction rate at each temperature and
# corresponding k value.
rates = []
for i, t in enumerate(T):
    rate = chemkin.reaction_rate(x, v_react, v_prod, k[i])
    rates.append(rate)
    print('At T = {0}, species reaction rates = {1}'.format(t, rate))
```

---
# Problem 7
Get together with your project team, form a GitHub organization (with a descriptive team name), and give the teaching staff access.  You can have has many repositories as you like within your organization.  However, we will grade the repository called **`cs207-FinalProject`**.

Within the `cs207-FinalProject` repo, you must set up Travis CI and Coveralls.  Make sure your `README.md` file includes badges indicating how many tests are passing and the coverage of your code.

```python

```


# Introduction

**Prerequisites**

- [Python Fundamentals](https://datascience.quantecon.org/../python_fundamentals/index.html)

**Outcomes**

- Understand the core pandas objects (`Series` and `DataFrame`)
- Index into particular elements of a Series and DataFrame
- Understand what `.dtype`/`.dtypes` do
- Make basic visualizations

**Data**

- US regional unemployment data from Bureau of Labor Statistics

## Outline

- [Introduction](#Introduction)
  - [pandas](#pandas)
  - [Series](#Series)
  - [DataFrame](#DataFrame)
  - [Data Types](#Data-Types)
  - [Changing DataFrames](#Changing-DataFrames)
  - [Exercises](#Exercises)

```python
# Uncomment following line to install on colab
! pip install qeds
#qeds = quantecon data science
```

    Requirement already satisfied: qeds in c:\users\asus\anaconda3\lib\site-packages (0.6.2)
    Requirement already satisfied: quandl in c:\users\asus\anaconda3\lib\site-packages (from qeds) (3.5.0)
    Requirement already satisfied: plotly in c:\users\asus\anaconda3\lib\site-packages (from qeds) (4.5.4)
    Requirement already satisfied: pandas-datareader in c:\users\asus\anaconda3\lib\site-packages (from qeds) (0.8.1)
    Requirement already satisfied: numpy in c:\users\asus\anaconda3\lib\site-packages (from qeds) (1.16.5)
    Requirement already satisfied: requests in c:\users\asus\anaconda3\lib\site-packages (from qeds) (2.22.0)
    Requirement already satisfied: scipy in c:\users\asus\anaconda3\lib\site-packages (from qeds) (1.3.1)
    Requirement already satisfied: pandas in c:\users\asus\anaconda3\lib\site-packages (from qeds) (0.25.1)
    Requirement already satisfied: matplotlib in c:\users\asus\anaconda3\lib\site-packages (from qeds) (3.1.1)
    Requirement already satisfied: quantecon in c:\users\asus\anaconda3\lib\site-packages (from qeds) (0.4.6)
    Requirement already satisfied: openpyxl in c:\users\asus\anaconda3\lib\site-packages (from qeds) (3.0.0)
    Requirement already satisfied: scikit-learn in c:\users\asus\anaconda3\lib\site-packages (from qeds) (0.21.3)
    Requirement already satisfied: seaborn in c:\users\asus\anaconda3\lib\site-packages (from qeds) (0.9.0)
    Requirement already satisfied: statsmodels in c:\users\asus\anaconda3\lib\site-packages (from qeds) (0.10.1)
    Requirement already satisfied: pyarrow in c:\users\asus\anaconda3\lib\site-packages (from qeds) (0.16.0)
    Requirement already satisfied: inflection>=0.3.1 in c:\users\asus\anaconda3\lib\site-packages (from quandl->qeds) (0.3.1)
    Requirement already satisfied: six in c:\users\asus\anaconda3\lib\site-packages (from quandl->qeds) (1.12.0)
    Requirement already satisfied: more-itertools in c:\users\asus\anaconda3\lib\site-packages (from quandl->qeds) (7.2.0)
    Requirement already satisfied: python-dateutil in c:\users\asus\anaconda3\lib\site-packages (from quandl->qeds) (2.8.0)
    Requirement already satisfied: retrying>=1.3.3 in c:\users\asus\anaconda3\lib\site-packages (from plotly->qeds) (1.3.3)
    Requirement already satisfied: lxml in c:\users\asus\anaconda3\lib\site-packages (from pandas-datareader->qeds) (4.4.1)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in c:\users\asus\anaconda3\lib\site-packages (from requests->qeds) (1.24.2)
    Requirement already satisfied: idna<2.9,>=2.5 in c:\users\asus\anaconda3\lib\site-packages (from requests->qeds) (2.8)
    Requirement already satisfied: certifi>=2017.4.17 in c:\users\asus\anaconda3\lib\site-packages (from requests->qeds) (2019.9.11)
    Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\users\asus\anaconda3\lib\site-packages (from requests->qeds) (3.0.4)
    Requirement already satisfied: pytz>=2017.2 in c:\users\asus\anaconda3\lib\site-packages (from pandas->qeds) (2019.3)
    Requirement already satisfied: cycler>=0.10 in c:\users\asus\anaconda3\lib\site-packages (from matplotlib->qeds) (0.10.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\asus\anaconda3\lib\site-packages (from matplotlib->qeds) (1.1.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in c:\users\asus\anaconda3\lib\site-packages (from matplotlib->qeds) (2.4.2)
    Requirement already satisfied: sympy in c:\users\asus\anaconda3\lib\site-packages (from quantecon->qeds) (1.4)
    Requirement already satisfied: numba>=0.38 in c:\users\asus\anaconda3\lib\site-packages (from quantecon->qeds) (0.45.1)
    Requirement already satisfied: jdcal in c:\users\asus\anaconda3\lib\site-packages (from openpyxl->qeds) (1.4.1)
    Requirement already satisfied: et-xmlfile in c:\users\asus\anaconda3\lib\site-packages (from openpyxl->qeds) (1.0.1)
    Requirement already satisfied: joblib>=0.11 in c:\users\asus\anaconda3\lib\site-packages (from scikit-learn->qeds) (0.13.2)
    Requirement already satisfied: patsy>=0.4.0 in c:\users\asus\anaconda3\lib\site-packages (from statsmodels->qeds) (0.5.1)
    Requirement already satisfied: setuptools in c:\users\asus\anaconda3\lib\site-packages (from kiwisolver>=1.0.1->matplotlib->qeds) (41.4.0)
    Requirement already satisfied: mpmath>=0.19 in c:\users\asus\anaconda3\lib\site-packages (from sympy->quantecon->qeds) (1.1.0)
    Requirement already satisfied: llvmlite>=0.29.0dev0 in c:\users\asus\anaconda3\lib\site-packages (from numba>=0.38->quantecon->qeds) (0.29.0)

## pandas

This lecture begins the material on `pandas`.

To start, we will import the pandas package and give it the alias
`pd`, which is conventional practice.

```python
import pandas as pd

# Don't worry about this line for now!
%matplotlib inline
# activate plot theme
import qeds
qeds.themes.mpl_style();
```

Sometimes, knowing which pandas version we are
using is helpful.

We can check this by running the code below.

```python
pd.__version__
```

    '0.25.1'

## Series

The first main pandas type we will introduce is called Series.

A Series is a single column of data, with row labels for each
observation.

pandas refers to the row labels as the *index* of the Series.

Below, we create a Series which contains the US unemployment rate every
other year starting in 1995.

```python
values = [5.6, 5.3, 4.3, 4.2, 5.8, 5.3, 4.6, 7.8, 9.1, 8., 5.7]
years = list(range(1995, 2017, 2))

unemp = pd.Series(data=values, index=years, name="Unemployment")
```

```python
unemp
```

    1995    5.6
    1997    5.3
    1999    4.3
    2001    4.2
    2003    5.8
    2005    5.3
    2007    4.6
    2009    7.8
    2011    9.1
    2013    8.0
    2015    5.7
    Name: Unemployment, dtype: float64

We can look at the index and values in our Series.

```python
unemp.index
```

    Int64Index([1995, 1997, 1999, 2001, 2003, 2005, 2007, 2009, 2011, 2013, 2015], dtype='int64')

```python
unemp.values
```

    array([5.6, 5.3, 4.3, 4.2, 5.8, 5.3, 4.6, 7.8, 9.1, 8. , 5.7])

### What Can We Do with a Series object?

#### `.head` and `.tail`

Often, our data will have many rows, and we won’t want to display it all
at once.

The methods `.head` and `.tail` show rows at the beginning and end
of our Series, respectively.

```python
unemp.head() #default 5, but we can put in brackets other number
```

    1995    5.6
    1997    5.3
    1999    4.3
    2001    4.2
    2003    5.8
    Name: Unemployment, dtype: float64

```python
unemp.tail() #default 5, but we can put in brackets other number
```

    2007    4.6
    2009    7.8
    2011    9.1
    2013    8.0
    2015    5.7
    Name: Unemployment, dtype: float64

#### Basic Plotting

We can also plot data using the `.plot` method.

```python
unemp.plot() #check if there's a way to do the graph for a period of time, etc.
```

>**Note**
>
>This is why we needed the `%matplotlib inline` — it tells the notebook
to display figures inside the notebook itself. Also, pandas has much greater visualization functionality than this, but we will study that later on.

#### Unique Values

Though it doesn’t make sense in this data set, we may want to find the
unique values in a Series – which can be done with the `.unique` method.

```python
unemp.unique()
```

    array([5.6, 5.3, 4.3, 4.2, 5.8, 4.6, 7.8, 9.1, 8. , 5.7])

#### Indexing

Sometimes, we will want to select particular elements from a Series.

We can do this using `.loc[index_items]`; where `index_items` is
an item from the index, or a list of items in the index.

We will see this more in-depth in a coming lecture, but for now, we
demonstrate how to select one or multiple elements of the Series.

```python
unemp.loc[1995]
```

    5.6

```python
unemp.loc[[1995, 2005, 2015]]
```

    1995    5.6
    2005    5.3
    2015    5.7
    Name: Unemployment, dtype: float64

## DataFrame

A DataFrame is how pandas stores one or more columns of data.

We can think a DataFrames a multiple Series stacked side by side as
columns.

This is similar to a sheet in an Excel workbook or a table in a SQL
database.

In addition to row labels (an index), DataFrames also have column labels.

We refer to these column labels as the columns or column names.

Below, we create a DataFrame that contains the unemployment rate every
other year by region of the US starting in 1995.

```python
data = {
    "NorthEast": [5.9,  5.6,  4.4,  3.8,  5.8,  4.9,  4.3,  7.1,  8.3,  7.9,  5.7],
    "MidWest": [4.5,  4.3,  3.6,  4. ,  5.7,  5.7,  4.9,  8.1,  8.7,  7.4,  5.1],
    "South": [5.3,  5.2,  4.2,  4. ,  5.7,  5.2,  4.3,  7.6,  9.1,  7.4,  5.5],
    "West": [6.6, 6., 5.2, 4.6, 6.5, 5.5, 4.5, 8.6, 10.7, 8.5, 6.1],
    "National": [5.6, 5.3, 4.3, 4.2, 5.8, 5.3, 4.6, 7.8, 9.1, 8., 5.7]
}

unemp_region = pd.DataFrame(data, index=years)
unemp_region
```

      NorthEast
      MidWest
      South
      West
      National

      1995
      5.9
      4.5
      5.3
      6.6
      5.6

      1997
      5.6
      4.3
      5.2
      6.0
      5.3

      1999
      4.4
      3.6
      4.2
      5.2
      4.3

      2001
      3.8
      4.0
      4.0
      4.6
      4.2

      2003
      5.8
      5.7
      5.7
      6.5
      5.8

      2005
      4.9
      5.7
      5.2
      5.5
      5.3

      2007
      4.3
      4.9
      4.3
      4.5
      4.6

      2009
      7.1
      8.1
      7.6
      8.6
      7.8

      2011
      8.3
      8.7
      9.1
      10.7
      9.1

      2013
      7.9
      7.4
      7.4
      8.5
      8.0

      2015
      5.7
      5.1
      5.5
      6.1
      5.7

We can retrieve the index and the DataFrame values as we
did with a Series.

```python
unemp_region.index
```

    Int64Index([1995, 1997, 1999, 2001, 2003, 2005, 2007, 2009, 2011, 2013, 2015], dtype='int64')

```python
unemp_region.values
```

    array([[ 5.9,  4.5,  5.3,  6.6,  5.6],
           [ 5.6,  4.3,  5.2,  6. ,  5.3],
           [ 4.4,  3.6,  4.2,  5.2,  4.3],
           [ 3.8,  4. ,  4. ,  4.6,  4.2],
           [ 5.8,  5.7,  5.7,  6.5,  5.8],
           [ 4.9,  5.7,  5.2,  5.5,  5.3],
           [ 4.3,  4.9,  4.3,  4.5,  4.6],
           [ 7.1,  8.1,  7.6,  8.6,  7.8],
           [ 8.3,  8.7,  9.1, 10.7,  9.1],
           [ 7.9,  7.4,  7.4,  8.5,  8. ],
           [ 5.7,  5.1,  5.5,  6.1,  5.7]])

### What Can We Do with a DataFrame?

Pretty much everything we can do with a Series.

#### `.head` and `.tail`

As with Series, we can use `.head` and `.tail` to show only the
first or last `n` rows.

```python
unemp_region.head()
```

      NorthEast
      MidWest
      South
      West
      National

      1995
      5.9
      4.5
      5.3
      6.6
      5.6

      1997
      5.6
      4.3
      5.2
      6.0
      5.3

      1999
      4.4
      3.6
      4.2
      5.2
      4.3

      2001
      3.8
      4.0
      4.0
      4.6
      4.2

      2003
      5.8
      5.7
      5.7
      6.5
      5.8

```python
unemp_region.tail(3)
```

      NorthEast
      MidWest
      South
      West
      National

      2011
      8.3
      8.7
      9.1
      10.7
      9.1

      2013
      7.9
      7.4
      7.4
      8.5
      8.0

      2015
      5.7
      5.1
      5.5
      6.1
      5.7

#### Plotting

We can generate plots with the `.plot` method.

Notice we now have a separate line for each column of data.

```python
unemp_region.plot()
```

#### Indexing

We can also do indexing using `.loc`.

This is slightly more advanced than before because we can choose
subsets of both row and columns.

```python
unemp_region.loc[1995, "NorthEast"]
```

    5.9

```python
unemp_region.loc[[1995, 2005], "South"]
```

    1995    5.3
    2005    5.2
    Name: South, dtype: float64

```python
unemp_region.loc[1995, ["NorthEast", "National"]]
```

    NorthEast    5.9
    National     5.6
    Name: 1995, dtype: float64

```python
unemp_region.loc[:, "NorthEast"]
```

    1995    5.9
    1997    5.6
    1999    4.4
    2001    3.8
    2003    5.8
    2005    4.9
    2007    4.3
    2009    7.1
    2011    8.3
    2013    7.9
    2015    5.7
    Name: NorthEast, dtype: float64

```python
# `[string]` with no `.loc` extracts a whole column
unemp_region["MidWest"]
```

    1995    4.5
    1997    4.3
    1999    3.6
    2001    4.0
    2003    5.7
    2005    5.7
    2007    4.9
    2009    8.1
    2011    8.7
    2013    7.4
    2015    5.1
    Name: MidWest, dtype: float64

### Computations with Columns

pandas can do various computations and mathematical operations on
columns.

Let’s take a look at a few of them.

```python
# Divide by 100 to move from percent units to a rate
unemp_region["West"] / 100
```

    1995    0.066
    1997    0.060
    1999    0.052
    2001    0.046
    2003    0.065
    2005    0.055
    2007    0.045
    2009    0.086
    2011    0.107
    2013    0.085
    2015    0.061
    Name: West, dtype: float64

```python
# Find maximum
unemp_region["West"].max()
```

    10.7

```python
# Find the difference between two columns
# Notice that pandas applies `-` to _all rows_ at once
# We'll see more of this throughout these materials
unemp_region["West"] - unemp_region["MidWest"]
```

    1995    2.1
    1997    1.7
    1999    1.6
    2001    0.6
    2003    0.8
    2005   -0.2
    2007   -0.4
    2009    0.5
    2011    2.0
    2013    1.1
    2015    1.0
    dtype: float64

```python
# Find correlation between two columns
unemp_region.West.corr(unemp_region["MidWest"])
```

    0.9006381255384481

```python
# find correlation between all column pairs
unemp_region.corr()
```

      NorthEast
      MidWest
      South
      West
      National

      NorthEast
      1.000000
      0.875654
      0.964415
      0.967875
      0.976016

      MidWest
      0.875654
      1.000000
      0.951379
      0.900638
      0.952389

      South
      0.964415
      0.951379
      1.000000
      0.987259
      0.995030

      West
      0.967875
      0.900638
      0.987259
      1.000000
      0.981308

      National
      0.976016
      0.952389
      0.995030
      0.981308
      1.000000

## Data Types

We asked you to run the commands `unemp.dtype` and
`unemp_region.dtypes` and think about the outputs.

You might have guessed that they return the type of the values inside
each column.

Occasionally, you might need to investigate what types you have in your
DataFrame when an operation isn’t behaving as expected.

```python
unemp.dtype
```

    dtype('float64')

```python
unemp_region.dtypes
```

    NorthEast    float64
    MidWest      float64
    South        float64
    West         float64
    National     float64
    dtype: object

DataFrames will only distinguish between a few types.

- Booleans (`bool`)
- Floating point numbers (`float64`)
- Integers (`int64`)
- Dates (`datetime`) — we will learn this soon
- Categorical data (`categorical`)
- Everything else, including strings (`object`)

In the future, we will often refer to the type of data stored in a
column as its `dtype`.

Let’s look at an example for when having an incorrect `dtype` can
cause problems.

Suppose that when we imported the data the `South` column was
interpreted as a string.

```python
str_unemp = unemp_region.copy()
str_unemp["South"] = str_unemp["South"].astype(str)
str_unemp.dtypes
```

    NorthEast    float64
    MidWest      float64
    South         object
    West         float64
    National     float64
    dtype: object

Everything *looks* ok…

```python
str_unemp.head()
```

      NorthEast
      MidWest
      South
      West
      National

      1995
      5.9
      4.5
      5.3
      6.6
      5.6

      1997
      5.6
      4.3
      5.2
      6.0
      5.3

      1999
      4.4
      3.6
      4.2
      5.2
      4.3

      2001
      3.8
      4.0
      4.0
      4.6
      4.2

      2003
      5.8
      5.7
      5.7
      6.5
      5.8

But if we try to do something like compute the sum of all the columns,
we get unexpected results…

```python
str_unemp.sum()
```

    NorthEast                                 63.7
    MidWest                                     62
    South        5.35.24.24.05.75.24.37.69.17.45.5
    West                                      72.8
    National                                  65.7
    dtype: object

This happened because `.sum` effectively calls `+` on all rows in
each column.

Recall that when we apply `+` to two strings, the result is the two
strings concatenated.

So, in this case, we saw that the entries in all rows of the South
column were stitched together into one long string.

## Changing DataFrames

We can change the data inside of a DataFrame in various ways:

- Adding new columns
- Changing index labels or column names
- Altering existing data (e.g. doing some arithmetic or making a column
  of strings lowercase)

Some of these “mutations” will be topics of future lectures, so we will
only briefly discuss a few of the things we can do below.

### Creating New Columns

We can create new data by assigning values to a column similar to how
we assign values to a variable.

In pandas, we create a new column of a DataFrame by writing:

```python
df["New Column Name"] = new_values
```

Below, we create an unweighted mean of the unemployment rate across the
four regions of the US — notice that this differs from the national
unemployment rate.

```python
unemp_region["UnweightedMean"] = (unemp_region["NorthEast"] +
                                  unemp_region["MidWest"] +
                                  unemp_region["South"] +
                                  unemp_region["West"])/4
```

```python
unemp_region.head()
```

      NorthEast
      MidWest
      South
      West
      National
      UnweightedMean

      1995
      5.9
      4.5
      5.3
      6.6
      5.6
      5.575

      1997
      5.6
      4.3
      5.2
      6.0
      5.3
      5.275

      1999
      4.4
      3.6
      4.2
      5.2
      4.3
      4.350

      2001
      3.8
      4.0
      4.0
      4.6
      4.2
      4.100

      2003
      5.8
      5.7
      5.7
      6.5
      5.8
      5.925

### Changing Values

Changing the values inside of a DataFrame should be done sparingly.

However, it can be done by assigning a value to a location in the
DataFrame.

`df.loc[index, column] = value`

```python
unemp_region.loc[1995, "UnweightedMean"] = 0.0
```

```python
unemp_region.head()
```

      NorthEast
      MidWest
      South
      West
      National
      UnweightedMean

      1995
      5.9
      4.5
      5.3
      6.6
      5.6
      0.000

      1997
      5.6
      4.3
      5.2
      6.0
      5.3
      5.275

      1999
      4.4
      3.6
      4.2
      5.2
      4.3
      4.350

      2001
      3.8
      4.0
      4.0
      4.6
      4.2
      4.100

      2003
      5.8
      5.7
      5.7
      6.5
      5.8
      5.925

### Renaming Columns

We can also rename the columns of a DataFrame, which is helpful because the names that sometimes come with datasets are
unbearable…

For example, the original name for the North East unemployment rate
given by the Bureau of Labor Statistics was `LASRD910000000000003`…

They have their reasons for using these names, but it can make our job
difficult since we often need to type it repeatedly.

We can rename columns by passing a dictionary to the `rename` method.

This dictionary contains the old names as the keys and new names as the
values.

See the example below.

```python
names = {"NorthEast": "NE",
         "MidWest": "MW",
         "South": "S",
         "West": "W"}
unemp_region.rename(columns=names)
```

      NE
      MW
      S
      W
      National
      UnweightedMean

      1995
      5.9
      4.5
      5.3
      6.6
      5.6
      0.000

      1997
      5.6
      4.3
      5.2
      6.0
      5.3
      5.275

      1999
      4.4
      3.6
      4.2
      5.2
      4.3
      4.350

      2001
      3.8
      4.0
      4.0
      4.6
      4.2
      4.100

      2003
      5.8
      5.7
      5.7
      6.5
      5.8
      5.925

      2005
      4.9
      5.7
      5.2
      5.5
      5.3
      5.325

      2007
      4.3
      4.9
      4.3
      4.5
      4.6
      4.500

      2009
      7.1
      8.1
      7.6
      8.6
      7.8
      7.850

      2011
      8.3
      8.7
      9.1
      10.7
      9.1
      9.200

      2013
      7.9
      7.4
      7.4
      8.5
      8.0
      7.800

      2015
      5.7
      5.1
      5.5
      6.1
      5.7
      5.600

```python
unemp_region.head()
```

      NorthEast
      MidWest
      South
      West
      National
      UnweightedMean

      1995
      5.9
      4.5
      5.3
      6.6
      5.6
      0.000

      1997
      5.6
      4.3
      5.2
      6.0
      5.3
      5.275

      1999
      4.4
      3.6
      4.2
      5.2
      4.3
      4.350

      2001
      3.8
      4.0
      4.0
      4.6
      4.2
      4.100

      2003
      5.8
      5.7
      5.7
      6.5
      5.8
      5.925

We renamed our columns… Why does the DataFrame still show the old
column names?

Many pandas operations create a copy of your data by
default to protect your data and prevent you from overwriting
information you meant to keep.

We can make these operations permanent by either:

1. Assigning the output back to the variable name
  `df = df.rename(columns=rename_dict)`
1. Looking into whether the method has an `inplace` option. For
  example, `df.rename(columns=rename_dict, inplace=True)`

Setting `inplace=True` will sometimes make your code faster
(e.g. if you have a very large DataFrame and you don’t want to copy all
the data), but that doesn’t always happen.

We recommend using the first option until you get comfortable with
pandas because operations that don’t alter your data are (usually)
safer.

```python
names = {"NorthEast": "NE",
         "MidWest": "MW",
         "South": "S",
         "West": "W"}

unemp_shortname = unemp_region.rename(columns=names)
unemp_shortname.head()
```

      NE
      MW
      S
      W
      National
      UnweightedMean

      1995
      5.9
      4.5
      5.3
      6.6
      5.6
      0.000

      1997
      5.6
      4.3
      5.2
      6.0
      5.3
      5.275

      1999
      4.4
      3.6
      4.2
      5.2
      4.3
      4.350

      2001
      3.8
      4.0
      4.0
      4.6
      4.2
      4.100

      2003
      5.8
      5.7
      5.7
      6.5
      5.8
      5.925


```python
from IPython.display import Image
Image('../../../python_for_probability_statistics_and_machine_learning.jpg')
```

[Python for Probability, Statistics, and Machine Learning](https://www.springer.com/fr/book/9783319307152)

```python
from __future__ import division
%pylab  inline
```

    Populating the interactive namespace from numpy and matplotlib

# Monte Carlo Sampling Methods

So far, we have studied analytical ways to transform random variables and how
to augment these methods using Python. In spite of all this, we frequently must
resort to purely numerical methods to solve real-world problems.  Hopefully,
now that we have seen the deeper theory, these numerical methods feel more
concrete.  Suppose we want to generate samples of a given density, $f(x)$,
given we already can generate samples from a uniform distribution,
$\mathcal{U}[0,1]$.  How do we know a random sample $v$ comes from the $f(x)$
distribution?  One approach is to look at how a histogram of samples of $v$
approximates $f(x)$. Specifically,

$$
\begin{equation}
\mathbb{P}( v \in N_{\Delta}(x) )  = f(x) \Delta x
\end{equation}
\label{eq:mc01} \tag{1}
$$

The histogram approximates the target probability density.

 which says that the probability that a sample is in some $N_\Delta$
neighborhood of $x$ is approximately $f(x)\Delta x$. [Figure](#fig:Sampling_Monte_Carlo_000) shows the target probability density function
$f(x)$ and a histogram that approximates it. The histogram is generated from
samples $v$. The hatched rectangle in the center illustrates Equation
ref{eq:mc01}. The area of this rectangle is approximately $f(x)\Delta x$ where
$x=0$, in this case. The width of the rectangle is $N_{\Delta}(x)$ The quality
of the approximation may be clear visually, but to know that $v$ samples are
characterized by $f(x)$, we need the statement of Equation ref{eq:mc01}, which
says that the proportion of samples $v$  that fill the hatched rectangle is
approximately equal to $f(x)\Delta x$.

Now that we know how to evaluate samples $v$ that are characterized by the density
$f(x)$, let's consider how to create these samples for both discrete and
continuous random variables.

## Inverse CDF Method for Discrete Variables

Suppose we want to generate samples from a fair six-sided die.  Our workhouse
uniform random variable is defined continuously over the unit interval  and the
fair six-sided die is discrete. We must first create a mapping between the
continuous random variable $u$ and the discrete outcomes of the die. This
mapping is shown in [Figure](#fig:Sampling_Monte_Carlo_0001) where the unit
interval is broken up into segments, each of length $1/6$. Each individual
segment is assigned to one of the die outcomes. For example, if $u \in
[1/6,2/6)$, then the outcome for the die is $2$. Because the die is fair, all
segments on the unit interval are the same length. Thus, our new random
variable $v$ is derived from $u$ by this assignment.

A uniform distribution random variable on the unit interval is assigned to the six outcomes of a fair die using these segements.

For example, for $v=2$, we have,

$$
\mathbb{P}(v=2) = \mathbb{P}(u\in [1/6,2/6)) = 1/6
$$

 where, in the language of the Equation ref{eq:mc01}, $f(x)=1$
(uniform distribution), $\Delta x = 1/6$, and $N_\Delta (2)=[1/6,2/6)$.
Naturally, this pattern holds for all the other die outcomes in
$\left\{1,2,3,..,6\right\}$.  Let's consider a quick simulation to make this
concrete. The following code generates uniform random samples and stacks them
in a Pandas dataframe.

```python
import pandas as pd
import numpy as np
from pandas import DataFrame
u= np.random.rand(100)
df = DataFrame(data=u,columns=['u'])
```

 The next block uses `pd.cut` to map the individual samples to
the set $\left\{1,2,\ldots,6\right\}$ labeled `v`.

```python
labels = [1,2,3,4,5,6]
df['v']=pd.cut(df.u,np.linspace(0,1,7),
               include_lowest=True,labels=labels)
```

 This is what the dataframe contains. The `v` column contains
the samples drawn from the fair die.

```python
>>> df.head()

```

      u
      v

      0
      0.605368
      4

      1
      0.525941
      4

      2
      0.141985
      1

      3
      0.729510
      5

      4
      0.240505
      2

 The following is a count of the number of samples in each group. There
should be roughly the same number of samples in each group because the die is fair.

```python
>>> df.groupby('v').count()

```

      u

      v

      1
      19

      2
      15

      3
      15

      4
      16

      5
      23

      6
      12

 So far, so good. We now have a way to simulate a fair
die from a uniformly distributed random variable.

To extend this to unfair die, we need only make some small adjustments to this
code.  For example, suppose that we want an unfair die so that
$\mathbb{P}(1)=\mathbb{P}(2)=\mathbb{P}(3)=1/12$ and
$\mathbb{P}(4)=\mathbb{P}(5)=\mathbb{P}(6)=1/4$. The only change we have to
make is with `pd.cut` as follows,

```python
df['v']=pd.cut(df.u,[0,1/12,2/12,3/12,2/4,3/4,1],
               include_lowest=True,labels=labels)
```

```python
>>> df.groupby('v').count()/df.shape[0]

```

      u

      v

      1
      0.05

      2
      0.14

      3
      0.07

      4
      0.23

      5
      0.32

      6
      0.19

u
v
1  0.08
2  0.10
3  0.09
4  0.23
5  0.19
6  0.31
 where now these are the individual probabilities of each digit. You
can take more than `100` samples to get a clearer view of the individual
probabilities but the mechanism for generating them is the same. The method is
called the inverse CDF [^CDF] method because the CDF
(namely,$\texttt{[0,1/12,2/12,3/12,2/4,3/4,1]}$) in the last example has been
inverted (using the `pd.cut` method) to generate the samples.
The inversion is easier to see for continuous variables, which we consider
next.

[^CDF]: Cumulative density function. Namely, $F(x)=\mathbb{P}(X < x)$.

## Inverse CDF Method for Continuous Variables

The method above applies to continuous random variables, but now we have to use
squeeze the  intervals down to individual points. In the example above, our
inverse function was a piecewise function that operated on uniform random
samples. In this case, the piecewise function collapses to a continuous inverse
function.  We want to generate random samples for a CDF that is invertible.
As before, the criterion for generating an appropriate sample $v$ is the
following,

$$
\mathbb{P}(F(x) < v < F(x+\Delta x)) =  F(x+\Delta x) - F(x) = \int_x^{x+\Delta x} f(u) du \approx  f(x) \Delta x
$$

 which is saying that the probability that the sample $v$ is contained
in a $\Delta x$ interval is approximately equal to the density function, $f(x)
\Delta x$, at that point. Once again, the trick is to use a uniform random
sample $u$ and an invertible CDF $F(x)$ to construct these samples.  Note
that for a uniform random variable $u \sim \mathcal{U}[0,1]$, we have,

$$
\begin{align*}
\mathbb{P}(x < F^{-1}(u) < x+\Delta x) & = \mathbb{P}(F(x) < u < F(x+\Delta x)) \\\
                                       & =  F(x+\Delta x) - F(x) \\\
                                       & = \int_x^{x+\Delta x} f(p) dp \approx  f(x) \Delta x
\end{align*}
$$

 This means that $ v=F^{-1}(u) $ is distributed according to $f(x)$,
which is what we want.

Let's try this to generate samples from the
exponential distribution,

$$
f_{\alpha}(x) = \alpha e^{ -\alpha x }
$$

 which has the following CDF,

$$
F(x) = 1-e^{ -\alpha x  }
$$

 and corresponding inverse,

$$
F^{-1}(u)  = \frac{1}{\alpha}\ln \frac{1}{(1-u)}
$$

 Now, all we have to do is generate some uniformly distributed
random samples and then feed them into $F^{-1}$.

```python
from numpy import array, log
import scipy.stats
alpha = 1.   # distribution parameter
nsamp = 1000 # num of samples
# define uniform random variable
u=scipy.stats.uniform(0,1)
# define inverse function
Finv=lambda u: 1/alpha*log(1/(1-u))
# apply inverse function to samples
v = array(map(Finv,u.rvs(nsamp)))
```

 Now, we have the samples from the exponential distribution, but how
do we know the method is correct with samples distributed accordingly?
Fortunately, `scipy.stats` already has a exponential distribution, so we can
check our work against the reference using a *probability plot* (i.e.,  also
known as a *quantile-quantile* plot).  The following code sets up the
probability plot from `scipy.stats`.

```python
%matplotlib inline

from matplotlib.pylab import setp, subplots
fig,ax = subplots()
fig.set_size_inches((7,5))
_=scipy.stats.probplot(v,(1,),dist='expon',plot=ax)
line=ax.get_lines()[0]
_=setp(line,'color','k')
_=setp(line,'alpha',.1)
line=ax.get_lines()[1]
_=setp(line,'color','gray')
_=setp(line,'lw',3.0)
_=setp(ax.yaxis.get_label(),'fontsize',18)
_=setp(ax.xaxis.get_label(),'fontsize',18)
_=ax.set_title('Probability Plot',fontsize=18)
_=ax.grid()
fig.tight_layout()
#fig.savefig('fig-probability/Sampling_Monte_Carlo_005.png')
```

```python
fig,ax=subplots()
scipy.stats.probplot(v,(1,),dist='expon',plot=ax)
```

 Note that we have to supply an axes object (`ax`) for it to draw on.
The result is [Figure](#fig:Sampling_Monte_Carlo_005). The more the samples
line match the diagonal line, the more they match the reference distribution
(i.e., exponential distribution in this case). You may also want to try
`dist=norm` in the code above To see what happens when the normal distribution
is the reference distribution.

The samples created using the inverse cdf method match the exponential reference distribution.

## Rejection Method

In some cases, inverting the CDF may be impossible. The *rejection*
method can handle this situation. The idea is to pick two uniform random
variables $u_1,u_2 \sim \mathcal{U}[a,b]$ so that

$$
\mathbb{P}\left(u_1 \in N_{\Delta}(x) \bigwedge u_2 < \frac{f(u_1)}{M} \right) \hspace{0.5em} \approx \frac{\Delta x}{b-a} \frac{f(u_1)}{M}
$$

 where we take $x=u_1$ and $f(x) < M $.  This is a two-step process.
First, draw $u_1$ uniformly from the interval $[a,b]$. Second,  feed it into
$f(x)$ and if $u_2 < f(u_1)/M$, then you have a valid sample for $f(x)$. Thus,
$u_1$ is the proposed sample from $f$ that may or may not be rejected depending
on $u_2$. The only job of the $M$ constant is to scale down the $f(x)$ so that
the $u_2$ variable can span the range.  The *efficiency* of this method is the
probability of accepting $u_1$ which comes from integrating out the above
approximation,

$$
\int \frac{f(x)}{M(b-a)} dx = \frac{1}{M(b-a)} \int f(x)dx =\frac{1}{M(b-a)}
$$

 This means that we don't want an unecessarily large $M$ because that
makes it more likely that samples will be discarded.

Let's try this method for a density that does not have a continuous inverse [^normalization].

[^normalization]: Note that this example density does not *exactly* integrate
out to one like a probability density function should, but the normalization
constant for this is distracting for our purposes here.

$$
f(x) = \exp\left(-\frac{(x-1)^2}{2x} \right) (x+1)/12
$$

 where $x>0$. The following code implements the rejection plan.

```python
import numpy as np
x = np.linspace(0.001,15,100)
f= lambda x: np.exp(-(x-1)**2/2./x)*(x+1)/12.
fx = f(x)
M=0.3                          # scale factor
u1 = np.random.rand(10000)*15  # uniform random samples scaled out
u2 = np.random.rand(10000)     # uniform random samples
idx,= np.where(u2<=f(u1)/M)    # rejection criterion
v = u1[idx]
```

```python
fig,ax=subplots()
fig.set_size_inches((9,5))
_=ax.hist(v,normed=1,bins=40,alpha=.3,color='gray')
_=ax.plot(x,fx,'k',lw=3.,label='$f(x)$')
_=ax.set_title('Estimated Efficency=%3.1f%%'%(100*len(v)/len(u1)),
             fontsize=18)
_=ax.legend(fontsize=18)
_=ax.set_xlabel('$x$',fontsize=24)
fig.tight_layout()
#fig.savefig('fig-probability/Sampling_Monte_Carlo_007.png')
```

 [Figure](#fig:Sampling_Monte_Carlo_007) shows a histogram of the
so-generated samples that nicely fits the probability density function. The
title in the figure shows the efficiency, which is poor. It means that we threw
away most of the proposed samples.  Thus, even though there is nothing
conceptually wrong with this result, the low efficiency must be fixed, as a
practical matter.  [Figure](#fig:Sampling_Monte_Carlo_008) shows where the
proposed samples were rejected. Samples under the curve were retained (i.e.,
$u_2 < \frac{f(u_1)}{M}$) but the vast majority of the samples are outside this
umbrella.

The rejection method generate samples in the histogram that nicely match the target distribution. Unfortunately, the efficiency is not so good.

```python
fig,ax=subplots()
fig.set_size_inches((9,5))
_=ax.plot(u1,u2,'+',label='rejected',alpha=.3,color='gray')
_=ax.plot(u1[idx],u2[idx],'.',label='accepted',alpha=.3,color='k')
_=ax.legend(fontsize=22)
fig.tight_layout()
#fig.savefig('fig-probability/Sampling_Monte_Carlo_008.png')
```

The proposed samples under the curve were accepted and the others were not. This shows the majority of samples were rejected.

The rejection method uses $u_1$ to select along the domain of $f(x)$ and the
other $u_2$ uniform random variable  decides whether to accept or not. One idea
would be to choose $u_1$ so that $x$ values are coincidentally those that are
near the peak of $f(x)$, instead of uniformly anywhere in the domain,
especially near the tails, which are low probability anyway. Now, the trick is
to find a new density function $g(x)$ to sample from that has a similiar
concentration of probability density. One way it to familiarize oneself with
the probability density functions that have adjustable parameters and fast random
sample generators already. There are lots of places to look and, chances are,
there is likely already such a generator for your problem. Otherwise, the
family of $\beta$ densities is a good place to start.

To be explicit, what we want is $u_1 \sim g(x)$ so that, returning to our
earlier argument,

$$
\mathbb{P}\left( u_1 \in N_{\Delta}(x) \bigwedge u_2 < \frac{f(u_1)}{M} \right) \approx g(x) \Delta x \frac{f(u_1)}{M}
$$

 but this is *not* what we need here. The problem is with the
second part of the logical $\bigwedge$ conjunction. We need to put
something there that will give us something proportional to $f(x)$.
Let us  define the following,

$$
\begin{equation}
 h(x) = \frac{f(x)}{g(x)}
\end{equation}
\label{eq:rej01} \tag{2}
$$

 with corresponding maximum on the domain as $h_{\max}$ and
then go back and construct the second part of the clause as

$$
\mathbb{P}\left(u_1 \in N_{\Delta}(x) \bigwedge u_2 < \frac{h(u_1)}{h_{\max}} \right) \approx g(x) \Delta x \frac{h(u_1)}{h_{\max}} = f(x)/h_{\max}
$$

 Recall that satisfying this criterion means that $u_1=x$. As before,
we can estimate the probability of acceptance of the $u_1$ as $1/h_{\max}$.

Now, how to construct the $g(x)$ function in the denominator of Equation
ref{eq:rej01}? Here's where familarity with some standard probability densities
pays off. For this case, we choose the chi-squared distribution. The following
plots the $g(x)$ and $f(x)$ (left plot) and the corresponding $h(x)=f(x)/g(x)$
(right plot). Note that $g(x)$ and $f(x)$ have peaks that almost coincide,
which is what we are looking for.

```python
ch=scipy.stats.chi2(4) # chi-squared
h = lambda x: f(x)/ch.pdf(x) # h-function
```

```python
fig,axs=subplots(1,2,sharex=True)
fig.set_size_inches(12,4)
_=axs[0].plot(x,fx,label='$f(x)$',color='k')
_=axs[0].plot(x,ch.pdf(x),'--',lw=2,label='$g(x)$',color='gray')
_=axs[0].legend(loc=0,fontsize=24)
_=axs[0].set_xlabel(r'$x$',fontsize=22)
_=axs[1].plot(x,h(x),'-k',lw=3)
_=axs[1].set_title('$h(x)=f(x)/g(x)$',fontsize=24)
_=axs[1].set_xlabel(r'$x$',fontsize=22)
fig.tight_layout()
#fig.savefig('fig-probability/Sampling_Monte_Carlo_009.png')
```

The plot on the right shows $h(x)=f(x)/g(x)$ and the one on the left shows $f(x)$ and $g(x)$ separately.

 Now, let's generate  some samples from this $\chi^2$
distribution with the rejection method.

```python
hmax=h(x).max()
u1 = ch.rvs(5000)        # samples from chi-square distribution
u2 = np.random.rand(5000)# uniform random samples
idx = (u2 <= h(u1)/hmax) # rejection criterion
v = u1[idx]              # keep these only
```

```python
fig,ax=subplots()
fig.set_size_inches((7,3))
_=ax.hist(v,normed=1,bins=40,alpha=.3,color='gray')
_=ax.plot(x,fx,color='k',lw=3.,label='$f(x)$')
_=ax.set_title('Estimated Efficency=%3.1f%%'%(100*len(v)/len(u1)))
_=ax.axis(xmax=15)
_=ax.legend(fontsize=18)
#fig.savefig('fig-probability/Sampling_Monte_Carlo_010.png')
```

Using the updated method, the histogram matches the target probability density function with high efficiency.

Using the $\chi^2$ distribution with the rejection method results in throwing
away less than 10% of the generated samples compared with our prior example
where we threw out at least 80%. This is dramatically more
efficient.  [Figure](#fig:Sampling_Monte_Carlo_010) shows that the histogram
and the probability density function match.  For completeness, [Figure](#fig:Sampling_Monte_Carlo_011) shows the samples with the corresponding
threshold $h(x)/h_{\max}$ that was used to select them.

```python
fig,ax=subplots()
fig.set_size_inches((7,4))
_=ax.plot(u1,u2,'+',label='rejected',alpha=.3,color='gray')
_=ax.plot(u1[idx],u2[idx],'g.',label='accepted',alpha=.3,color='k')
_=ax.plot(x,h(x)/hmax,color='k',lw=3.,label='$h(x)$')
_=ax.legend(fontsize=16,loc=0)
_=ax.set_xlabel('$x$',fontsize=24)
_=ax.set_xlabel('$h(x)$',fontsize=24)
_=ax.axis(xmax=15,ymax=1.1)
fig.tight_layout()
#fig.savefig('fig-probability/Sampling_Monte_Carlo_011.png')
```

Fewer proposed points were rejected in this case, which means better efficiency.

In this section, we investigated how to generate random samples from a given
distribution, beit discrete or continuous. For the continuous case, the key
issue was whether or not the cumulative density function had a continuous
inverse. If not, we had to turn to the rejection method, and find an
appropriate related density that we could easily sample from to use as part of
a rejection threshold. Finding such a function is an art, but many families of
probability densities have been studied over the years that already have fast
random number generators.

The rejection method has many complicated extensions that involve careful
partitioning of the domains and lots of special methods for corner cases.
Nonetheless, all of these advanced techniques are still variations on the same
fundamental theme we illustrated here [[dunn2011exploring]](#dunn2011exploring),[[johnson1995continuous]](#johnson1995continuous).