## Data-driven Design and Analyses of Structures and Materials (3dasm)

## Lecture 13

### Miguel A. Bessa | M.A.Bessa@tudelft.nl  | Associate Professor

**What:** A lecture of the "3dasm" course

**Where:** This notebook comes from this [repository](https://github.com/bessagroup/3dasm_course)

**Reference for entire course:** Murphy, Kevin P. *Probabilistic machine learning: an introduction*. MIT press, 2022. Available online [here](https://probml.github.io/pml-book/book1.html)

**How:** We try to follow Murphy's book closely, but the sequence of Chapters and Sections is different. The intention is to use notebooks as an introduction to the topic and Murphy's book as a resource.
* If working offline: Go through this notebook and read the book.
* If attending class in person: listen to me (!) but also go through the notebook in your laptop at the same time. Read the book.
* If attending lectures remotely: listen to me (!) via Zoom and (ideally) use two screens where you have the notebook open in 1 screen and you see the lectures on the other. Read the book.

**Optional reference (the "bible" by the "bishop"... pun intended 😆) :** Bishop, Christopher M. *Pattern recognition and machine learning*. Springer Verlag, 2006.

**References/resources to create this notebook:**
* Chapter 11 of Murphy's book.

Apologies in advance if I missed some reference used in this notebook. Please contact me if that is the case, and I will gladly include it here.

## **OPTION 1**. Run this notebook **locally in your computer**:
1. Confirm that you have the 3dasm conda environment (see Lecture 1).

2. Go to the 3dasm_course folder in your computer and pull the last updates of the [repository](https://github.com/bessagroup/3dasm_course):
```
git pull
```
3. Open command window and load jupyter notebook (it will open in your internet browser):
```
conda activate 3dasm
jupyter notebook
```
4. Open notebook of this Lecture.

## **OPTION 2**. Use **Google's Colab** (no installation required, but times out if idle):

1. go to https://colab.research.google.com
2. login
3. File > Open notebook
4. click on Github (no need to login or authorize anything)
5. paste the git link: https://github.com/bessagroup/3dasm_course
6. click search and then click on the notebook for this Lecture.

```python
# Basic plotting tools needed in Python.

import matplotlib.pyplot as plt # import plotting tools to create figures
import numpy as np # import numpy to handle a lot of things!
from IPython.display import display, Math # to print with Latex math

%config InlineBackend.figure_format = "retina" # render higher resolution images in the notebook
#plt.style.use("seaborn") # style for plotting that comes from seaborn
plt.rcParams["figure.figsize"] = (8,4) # rescale figure size appropriately for slides
```

## Outline for today

* Derivation of different Linear Regression models
    - Picking up where we left off in Lecture 8.

**Reading material**: This notebook + Chapter 11 of the book.

## Recap of Lectures 8 and 9

Recall our view of Linear regression models from a Bayesian perspective: it's all about the choice of **likelihood** and **prior**!

| Likelihood | Prior (on the weights)    | Posterior      | Name of the model | Book section  |
|---        |---         |---             |---              |---            |
| Gaussian  | Uniform    | Point estimate | Least Squares regression  | 11.2.2  |
| Gaussian  | Gaussian    | Point estimate | Ridge regression   | 11.3  |
| Gaussian  | Laplace    | Point estimate | Lasso regression  | 11.4  |
| Student-$t$  | Uniform    | Point estimate | Robust regression   | 11.6.1  |
| Laplace  | Uniform    | Point estimate | Robust regression   | 11.6.2  |
| Gaussian  | Gaussian    | Gaussian | Bayesian linear regression   | 11.7 |

Let's continue along the lines of the Homework of Lecture 8, and derive a few of these models for the multidimensional case.

We are now totally prepared to derive any ML model in any dimension!

In Lecture 8 and its Homework we derived linear regression models using 1D input $x$, 1D output $y$, and a polynomial basis function $\boldsymbol{\phi}(x)$.

We will quickly recap what we did then, and then show how this generalizes to multidimensional inputs $\mathbf{x}$ and for any kind of basis function $\boldsymbol{\phi}(\mathbf{x})$.

* Note: without loss of generality, we will keep considering a single output $y$.

## Linear Least Squares: Linear regression with Gaussian likelihood, Uniform prior and posterior via Point estimate

| Likelihood | Prior (on the weights)    | Posterior      | Name of the model | Book section  |
|---        |---         |---             |---              |---            |
| Gaussian  | Uniform    | Point estimate | Least Squares regression  | 11.2.2  |

This model assumes a Gaussian observation distribution with constant variance and "linear" mean (recall: linear in the unknowns $\mathbf{z}$). If considering 1D input $x$ and 1D output $y$ the model is written as:
1. Gaussian observation distribution: $p(y|x, \mathbf{z}) = \mathcal{N}(y| \mu_{y|z} = \mathbf{w}^T \boldsymbol{\phi}(x), \sigma_{y|z}^2 = \sigma^2)$

where $\mathbf{z} = (\mathbf{w}, \sigma)$ are all the unknown model parameters (hidden rv's).

2. Uniform prior distribution for each hidden rv in $\mathbf{z}$: $p(\mathbf{z}) \propto 1$

3. MLE point estimate for posterior: $\hat{\mathbf{z}}_{\text{mle}} = \underset{z}{\mathrm{argmin}}\left[-\sum_{i=1}^{N}\log{ p(y=y_i|x=x_i, \mathbf{z})}\right]$

Final prediction is given by the PPD: $\require{color}
{\color{orange}p(y|x, \mathcal{D})} = \int p(y|x,\mathbf{z}) \delta(\mathbf{z}-\hat{\mathbf{z}}) dz = p(y|x, \mathbf{z}=\hat{\mathbf{z}})$

#### Notes

Compared to the previous lectures, pay attention to the following updates in the notation of our 1D linear regression model:

1. We are explicitly including the input $x$ in the probability densities, as we will no longer fix $x$ to a particular value like we did up to now

2. We are now considering more than one unknown rv and grouping them in the vector $\mathbf{z}$.

### Recall the car stopping distance problem

Let's focus (again) on our favorite problem, but now we will not keep the velocity of the car $x$ fixed.

If we knew the "ground truth" of this problem, then it would be given by:

$\require{color}y = {\color{red}z_1}\cdot x + {\color{red}z_2}\cdot x^2$

- $y$ is the **output**: the car stopping distance (in meters)
- ${\color{red}z_1}$ is a hidden variable: an rv representing the driver's reaction time (in seconds)
- ${\color{red}z_2}$ is another hidden variable: an rv that depends on the coefficient of friction, the inclination of the road, the weather, etc. (in m$^{-1}$s$^{-2}$).
- $x$ is the **input**: constant car velocity (in m/s).

where $z_1 \sim \mathcal{N}(\mu_{z_1}=1.5,\sigma_{z_1}^2=0.5^2)$, and $z_2 \sim \mathcal{N}(\mu_{z_2}=0.1,\sigma_{z_2}^2=0.01^2)$.

Unsurprisingly, in Exercise 1 of Lecture 9 we saw that a linear model with a **quadratic polynomial basis function** predicts the stopping distance for this problem very well:

1. Gaussian observation distribution: $p(y|x, \mathbf{z}) = \mathcal{N}(y| \mu_{y|z} = \mathbf{w}^T \boldsymbol{\phi}(x), \sigma_{y|z}^2 = \sigma^2)$

where $\mathbf{z} = (\mathbf{w}, \sigma)$ are all the hidden rv's of the model, i.e. the model parameters.
* the vector $\mathbf{w} = [w_0, w_1, w_2 ..., w_{M-1}]^T$ includes the **bias** term $w_0$ and the remaining **weights** $w_m$ with $m=0,..., M-1$.
* the vector $\boldsymbol{\phi}(x) = [1, x, x^2, ..., x^{M-1}]^T$ includes the **basis functions**, which now correspond to a polynomial of degree $M-1$.  When $M=3$ we have a quadratic polynomial basis (3 unknowns).

2. Uniform prior distribution for each hidden rv in $\mathbf{z}$: $p(\mathbf{z}) \propto 1$

3. MLE point estimate for posterior: $\hat{\mathbf{z}}_{\text{mle}} = \underset{z}{\mathrm{argmin}}\left[-\sum_{n=1}^{N}\log{ p(y=y_n|x_n, \mathbf{z})}\right]$

For other problems, the polynomial degree $M-1$ of the basis functions may need to be different.

* For example, also in Lecture 9 we saw that for a problem whose ground truth is $x\sin{x}$ then the polynomial basis function needs to have a higher degree. However, even then the approximation is not brilliant because the ground truth is not really a polynomial!

There are other basis functions that can be adopted. For example, spline basis functions (Section 11.5 in the book), among many other possibilities (kernels!).

As we also mentioned, as long as the basis functions $\boldsymbol{\phi}(x)$ do not depend on any rv $\mathbf{z}$ and the mean of observation distribution is defined linearly as a function of the rv's, then we still have a linear regression model.

But now let's consider problems that still have only one output $y$ but that can have multiple inputs $\mathbf{x} = [x_1, x_2, ..., x_D]^T$ where $x_d$ is feature $d$ and where $d=1, ..., D$.

In this case, we can write the multidimensional linear regression model as:

1. Gaussian observation distribution: $p(y|\mathbf{x}, \mathbf{z}) = \mathcal{N}(y| \mu_{y|z} = \mathbf{w}^T \boldsymbol{\phi}(\mathbf{x}), \sigma_{y|z}^2 = \sigma^2)$

where $\mathbf{z} = (\mathbf{w}, \sigma)$ are all the hidden rv's of the model, i.e. the model parameters.
* the vector $\mathbf{w} = [w_0, w_1, w_2 ..., w_{M-1}]^T$ includes the **bias** term $w_0$ and the remaining **weights** $w_m$ with $m=0,..., M-1$.
* and the basis functions remain a vector but where each element also acts on a vector $\mathbf{x}$, where $x_d$ has $D$ features: $\boldsymbol{\phi}(\mathbf{x}) = [\phi_0(\mathbf{x}), \phi_1(\mathbf{x}), \phi_2(\mathbf{x}) ..., \phi_{M-1}(\mathbf{x})]^T$

and where the remaining choices for the linear regression model remain the same:

2. Uniform prior distribution for each hidden rv in $\mathbf{z}$: $p(\mathbf{z}) \propto 1$

3. MLE point estimate for posterior: $\hat{\mathbf{z}}_{\text{mle}} = \underset{z}{\mathrm{argmin}}\left[-\sum_{n=1}^{N}\log{ p(y=y_n|\mathbf{x} = \mathbf{x}_n, \mathbf{z})}\right]$

Final prediction is given by the PPD:

$$\require{color}
{\color{orange}p(y|x, \mathcal{D})} = \int p(y|x,\mathbf{z}) \delta(\mathbf{z}-\hat{\mathbf{z}}) dz = p(y|x, \mathbf{z}=\hat{\mathbf{z}})$$

Therefore, we are capable of predicting the PPD by discovering the unknowns $\mathbf{z}$ via the point estimate of the posterior, which requires solving the $\mathrm{argmin}$ of the negative log likelihood.

Now, let's focus on estimating the unknowns $\mathbf{z}$ via the MLE point estimate of the posterior (maximum likelihood estimation).

As we saw in Lecture 8, finding the MLE is the same as finding the location of the minimum of the negative log likelihood.

Since our observation distribution is a multivariate Gaussian,

$p(y|x, \mathbf{z}) = \mathcal{N}(y| \mu_{y|z} = \mathbf{w}^T \boldsymbol{\phi}(\mathbf{x}), \sigma_{y|z}^2 = \sigma^2)$

then the likelihood is given by (Lecture 5 but now with vectors):

$$
\begin{align}
p(y=\mathcal{D}_y | \mathbf{x}=\mathcal{D}_x, \mathbf{z}) &= \prod_{n=1}^{N} p(y=y_n|\mathbf{x}=\mathbf{x}_n, \mathbf{z}) \\
&= p(y=y_1|\mathbf{x}=\mathbf{x}_1, \mathbf{z})p(y=y_2|\mathbf{x}=\mathbf{x}_2, \mathbf{z}) \cdots p(y=y_N|\mathbf{x}=\mathbf{x}_N, \mathbf{z})
\end{align}
$$

which we already know that is also a multivariate Gaussian (unnormalized).

But, since we are not going fully Bayesian, the only thing we need to estimate is the location of the maximum of the likelihood (point estimate!):

$$\begin{align}
\hat{\mathbf{z}}_{\text{mle}} &= \underset{z}{\mathrm{argmin}}\left[\text{NLL}(\mathbf{z})\right]
\\
&= \underset{z}{\mathrm{argmin}}\left[-\sum_{n=1}^{N}\log{ p(y=y_n|\mathbf{x}=\mathbf{x}_n, \mathbf{z})}\right]
\end{align}
$$

In Lecture 9 we allowed scikit-learn to find the minimum for us! But today we will actually determine this minimum...

You already did this in the Homework of Lecture 8 for the 1D case with a linear polynomial basis and fixing $x$. The multivariate case for a general basis function and for different $\mathbf{x}$ is just as easy! Especially when considering the variance of the observation distribution to be the same everywhere!

$$
\begin{align}
\hat{\mathbf{z}}_{\text{mle}} &= \underset{z}{\mathrm{argmin}}\left[-\sum_{n=1}^{N}\log{ p(y=y_n|\mathbf{x}=\mathbf{x}_n, \mathbf{z})}\right] \\
&= \underset{z}{\mathrm{argmin}}\left[-\sum_{n=1}^{N}\log{\left( \frac{1}{\sqrt{2\pi \sigma^2}} \exp\left\{ -\frac{1}{2\sigma^2}\left[y_n-\mathbf{w}^T \boldsymbol{\phi}(\mathbf{x}_n)\right]^2\right\}\right)}\right]\\
&= \underset{z}{\mathrm{argmin}}\left[\frac{N}{2}\log{\left(2\pi \sigma^2\right)}+\frac{1}{2 \sigma^2}\sum_{n=1}^{N}\left[y_n-\mathbf{w}^T \boldsymbol{\phi}(\mathbf{x}_n)\right]^2 \right]\\
\end{align}
$$

where we recall that the unknowns are $\mathbf{z} = (\mathbf{w}, \sigma)$.

To find the minimum location we need to take the gradient of the $\text{NLL}(\mathbf{z})$ wrt $\mathbf{z}$ and equal it to zero:

$$
\nabla_{\mathbf{z}} \text{NLL}(\mathbf{z}) = \mathbf{0}
$$

which can be written as,

$$
\begin{bmatrix}
\frac{\partial \text{NLL}(\mathbf{z})}{\partial w_0}\\
\frac{\partial \text{NLL}(\mathbf{z})}{\partial w_1}\\
\vdots \\
\frac{\partial \text{NLL}(\mathbf{z})}{\partial w_M}\\
\frac{\partial \text{NLL}(\mathbf{z})}{\partial \sigma^2}\\
\end{bmatrix} =
\begin{bmatrix}0\\
0\\
\vdots \\
0\\
0\\
\end{bmatrix}
$$

We can first solve this system of equations wrt $\mathbf{w}$, and then solve wrt $\sigma$

Then, solving first for the weights $\mathbf{w}$:

$$
\nabla_{\mathbf{w}} \text{NLL}(\mathbf{w}, \sigma^2) = \mathbf{0}
$$

we note that,

$$\begin{align}
\nabla_{\mathbf{w}} \text{NLL}(\mathbf{w}, \sigma^2) = \mathbf{0} \\
\nabla_{\mathbf{w}} \left[\frac{N}{2}\log{\left(2\pi \sigma^2\right)}+\frac{1}{2 \sigma^2}\sum_{n=1}^{N}\left[y_n-\mathbf{w}^T \boldsymbol{\phi}(\mathbf{x}_n)\right]^2 \right] = \mathbf{0} \\
\nabla_{\mathbf{w}} \left[\underbrace{\frac{1}{2}\sum_{n=1}^{N}\left[y_n-\mathbf{w}^T \boldsymbol{\phi}(\mathbf{x}_n)\right]^2}_{\text{RSS}(\mathbf{w})} \right] = \mathbf{0}
\end{align}
$$

Note: in Statistics the term in the argument is called **residual sum of squares**.

We can rewrite the above expression in a simpler form:

$$\begin{align}
\nabla_{\mathbf{w}} \left[\frac{1}{2}\sum_{n=1}^{N}\left[y_n-\mathbf{w}^T \boldsymbol{\phi}(\mathbf{x}_n)\right]^2 \right] = 0 \\
\nabla_{\mathbf{w}} \left[\frac{1}{2}\left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right)^T \left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right) \right] = 0
\end{align}
$$

where we group all output measurements $y_n$ into a $N\times 1$ vector $\mathbf{y}$ and where we group all $N$ evaluations of the basis functions into the $N\times M$ matrix:

$$
\boldsymbol{\Phi} = \begin{bmatrix} \phi_0(\mathbf{x}_1) & \phi_1(\mathbf{x}_1) & \cdots & \phi_{M-1}(\mathbf{x}_1) \\
\phi_0(\mathbf{x}_2) & \phi_1(\mathbf{x}_2) & \cdots & \phi_{M-1}(\mathbf{x}_2) \\
\vdots & \vdots & \ddots & \vdots \\
\phi_0(\mathbf{x}_N) & \phi_1(\mathbf{x}_N) & \cdots & \phi_{M-1}(\mathbf{x}_N) \\
\end{bmatrix}
$$

Setting the gradient wrt all $\mathbf{w}$ to zero gives,

$$\begin{align}
\nabla_{\mathbf{w}} \left[\frac{1}{2}\left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right)^T \left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right) \right] = \mathbf{0}\\
\frac{1}{2} \left[ \left( \boldsymbol{\Phi}^T\boldsymbol{\Phi}+\boldsymbol{\Phi}^T\boldsymbol{\Phi} \right)\mathbf{w} -\boldsymbol{\Phi}^T\mathbf{y}-\boldsymbol{\Phi}^T\mathbf{y} \right] = \mathbf{0}
\end{align}
$$

where we used the identity $ \frac{\partial \mathbf{x}^T\mathbf{A}\mathbf{x}}{\partial \mathbf{x}} = \left( \mathbf{A}+\mathbf{A}^T\right)\mathbf{x}$. (See Section 7.8 of Murphy's book if you need to revise matrix calculus).

From which we reach the MLE prediction for the weights:

$$
\hat{\mathbf{w}}_{\text{mle}} = \left(\boldsymbol{\Phi}^T \boldsymbol{\Phi} \right)^{-1} \boldsymbol{\Phi}^T \mathbf{y}
$$

We conclude that our point estimate for the posterior (MLE) is: $\hat{\mathbf{w}}_{\text{MLE}} = \left(\boldsymbol{\Phi}^T \boldsymbol{\Phi} \right)^{-1} \boldsymbol{\Phi}^T \mathbf{y}$

where the quantity

$$\boldsymbol{\Phi}^{\dagger} = \left(\boldsymbol{\Phi}^T \boldsymbol{\Phi} \right)^{-1} \boldsymbol{\Phi}^T $$

is known as the Moore-Penrose pseudo-inverse of the matrix $\boldsymbol{\Phi}$. It can be regarded as a generalization of the notion of matrix inverse to **nonsquare matrices**. In the special case of $\boldsymbol{\Phi}$ being square and invertible, then using the property $\left( \mathbf{A}\mathbf{B}\right)^{-1}=\mathbf{B}^{-1}\mathbf{A}^{-1}$ we see that $\boldsymbol{\Phi}^{\dagger}=\boldsymbol{\Phi}^{-1}$.

Also note that we could have calculated separetely the bias term $w_0$ (which is convenient because for other models the bias usually has a uniform prior, unlike the remaining weights). If we do that we obtain:

$$
\hat{w}_0 = \bar{y}-\sum_{m=1}^{M-1} w_m \bar{\phi}_m
$$

where we defined $\bar{y} = \frac{1}{N}\sum_{n=1}^N y_n$ and $\bar{\phi}_m = \frac{1}{N}\sum_{n=1}^{N} \phi_m(\mathbf{x}_n)$.

Having found the solution for all $\mathbf{w}$, we just need to find one last unknown from the point estimate of the posterior:

$$
\nabla_{\mathbf{\sigma^2}} \text{NLL}(\mathbf{w}, \sigma^2) = 0
$$

which is particularly simple:

$$
\hat{\sigma}_{\text{mle}} = \frac{1}{N}\sum_{n=1}^{N} \left[y_n -\hat{\mathbf{w}}^T_{\text{mle}}\boldsymbol{\phi}(\mathbf{x}_n)\right]^2
$$

In summary, the MLE point estimate of the posterior leads to the following estimation of parameters:

$$
\hat{\mathbf{w}}_{\text{mle}} = \left(\boldsymbol{\Phi}^T \boldsymbol{\Phi} \right)^{-1} \boldsymbol{\Phi}^T \mathbf{y}
$$

$$
\hat{\sigma}_{\text{mle}} = \frac{1}{N}\sum_{n=1}^{N} \left[y_n -\hat{\mathbf{w}}^T_{\text{mle}}\boldsymbol{\phi}(\mathbf{x}_n)\right]^2
$$

where the Moore-Penrose pseudo inverse needs to be calculated $\boldsymbol{\Phi}^{\dagger} = \left(\boldsymbol{\Phi}^T \boldsymbol{\Phi} \right)^{-1} \boldsymbol{\Phi}^T $.

This calculation can be done efficiently by many libraries, including Numpy.

* For example, scikit-learn uses a solver based on SVD (Single Value Decomposition: the most common dimensionality reduction method) which is efficient when $N > M$ (overdetermined system). Book section 7.5 has an excellent summary of SVD, if you are curious.

* Of course, if $N = M$ then there is a unique solution (you know that from the Midterm!) and the error on the training set becomes zero (linear regression becomes fully interpolatory).

## Ridge regression: Linear regression with Gaussian likelihood, Gaussian prior and posterior via Point estimate

| Likelihood | Prior (on the weights)    | Posterior      | Name of the model | Book section  |
|---        |---         |---             |---              |---            |
| Gaussian  | Gaussian    | Point estimate | Ridge regression   | 11.3  |

1. Gaussian observation distribution: $p(y|\mathbf{x}, \mathbf{z}) = \mathcal{N}(y| \mu_{y|z} = \mathbf{w}^T \boldsymbol{\phi}(\mathbf{x}), \sigma_{y|z}^2 = \sigma^2)$

where $\mathbf{z} = (\mathbf{w}, \sigma)$ are all the unknown model parameters (hidden rv's).

2. But using a Gaussian prior for the weights $\mathbf{w}$: $p(\mathbf{w}) = \mathcal{N}(\mathbf{w}| \mathbf{0}, \overset{\scriptscriptstyle <}{\sigma}_w \mathbf{I})$

3. MAP point estimate for posterior: $\hat{\mathbf{z}}_{\text{map}} = \underset{z}{\mathrm{argmin}}\left[-\sum_{i=1}^{N}\log{ p(y=y_i|z)} - \log{p(\mathbf{w})}\right]$

Final prediction is given by the PPD: $\require{color}
{\color{orange}p(y|\mathcal{D}_y)} = \int p(y|z) \delta(z-\hat{z}_{\text{map}}) dz = p(y|z=\hat{z}_{\text{map}})$

#### Note on the choice of prior for linear regression

The Gaussian prior is usually imposed only on the weights. The bias and variance term in the observation distribution still have a Uniform prior because they do not contribute to overfitting.

You can see this from the expressions for $\hat{w}_{0, \text{mle}}$ and $\sigma^2_{\text{mle}}$, as they act on the global mean and MSE (mean squared error) of the residuals, respectively.

Computing the MAP estimate is very similar to what we did for the MLE:

$$
\begin{align}
\hat{\mathbf{w}}_{\text{map}} &= \underset{w}{\mathrm{argmin}}\left[\frac{1}{2\sigma^2}\left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right)^T \left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right) + \frac{1}{2\overset{\scriptscriptstyle <}{\sigma}_w^2}\mathbf{w}^T\mathbf{w}\right] \\
&= \underset{w}{\mathrm{argmin}}\left[\text{RSS}(\mathbf{w}) + \lambda ||\mathbf{w}||_2^2\right]
\end{align}
$$

where $\lambda = \frac{\sigma^2}{\overset{\scriptscriptstyle <}{\sigma}_w^2}$ is proportional to the strength of the prior, and

$||\mathbf{w}||_2^2 = \sqrt{\sum_{m=1}^{M-1} |w_m|^2} = \sqrt{\mathbf{w}^T\mathbf{w}}$

is called the $l_2$ norm of the vector $\mathbf{w}$. Thus, we are penalizing weights that become too large in magnitude. In ML literature this is usually called $l_2$ **regularization** or **weight decay**, and is very widely used.

In the midterm, you will experience the difference between Linear Least Squares and Ridge regression, reporting on the influence of the prior strength for the latter.

Then, solving the MAP first for the weights $\mathbf{w}$, as we did for the MLE:

$$\begin{align}
\nabla_{\mathbf{w}} \left[\frac{1}{2\sigma^2}\left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right)^T \left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right) + \frac{1}{2\overset{\scriptscriptstyle <}{\sigma}_w^2}\mathbf{w}^T\mathbf{w}\right] &= \mathbf{0} \\
\nabla_{\mathbf{w}} \left[\left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right)^T \left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right) + \lambda\mathbf{w}^T\mathbf{w}\right] = \mathbf{0}
\end{align}
$$

from which we determine the MAP estimate for the the weights $\mathbf{w}$ as:

$$
\hat{\mathbf{w}}_{\text{map}} = \left( \boldsymbol{\Phi}^T\boldsymbol{\Phi} + \lambda \mathbf{I}_M \right)^{-1} \boldsymbol{\Phi}^T \mathbf{y} = \left( \sum_{n=1}^{N} \boldsymbol{\phi}(\mathbf{x}_n)\boldsymbol{\phi}(\mathbf{x}_n)^T + \lambda \mathbf{I}_M \right)^{-1} \left( \sum_{n=1}^{N} \boldsymbol{\phi}(\mathbf{x}_n) y_n \right)
$$

Once again, this can be solved using SVD or other methods to ensure that the Moore-Penrose pseudoi inverse is calculated properly.

## Lasso regression: Linear regression with Gaussian likelihood, Laplace prior and posterior via Point estimate

| Likelihood | Prior (on the weights)    | Posterior      | Name of the model | Book section  |
|---        |---         |---             |---              |---            |
| Gaussian  | Laplace    | Point estimate | Lasso regression  | 11.4  |

1. Gaussian observation distribution: $p(y|\mathbf{x}, \mathbf{z}) = \mathcal{N}(y| \mu_{y|z} = \mathbf{w}^T \boldsymbol{\phi}(\mathbf{x}), \sigma_{y|z}^2 = \sigma^2)$

where $\mathbf{z} = (\mathbf{w}, \sigma)$ are all the unknown model parameters (hidden rv's).

2. But using a **Laplace** prior for the weights $\mathbf{w}$: $p(\mathbf{w}) = \prod_{m=1}^{M-1}\text{Lap}\left(w_m| 0, 1/\overset{\scriptscriptstyle <}{\lambda}_w\right) \propto \prod_{m=1}^{M-1} \exp{\left[ -\overset{\scriptscriptstyle <}{\lambda}_w |w_m|\right]}$

3. MAP point estimate for posterior: $\hat{\mathbf{z}}_{\text{map}} = \underset{z}{\mathrm{argmin}}\left[-\sum_{i=1}^{N}\log{ p(y=y_i|z)} - \log{p(\mathbf{w})}\right]$

Final prediction is given by the PPD: $\require{color}
{\color{orange}p(y|\mathcal{D}_y)} = \int p(y|z) \delta(z-\hat{z}_{\text{map}}) dz = p(y|z=\hat{z}_{\text{map}})$

#### Note about the sparsity parameter $\overset{\scriptscriptstyle <}{\lambda}_w$

Please note that the $\overset{\scriptscriptstyle <}{\lambda}_w$ parameter defining the strength of the Laplace prior is different from the $\lambda$ parameter defined in Ridge regression.

#### Note about number of parameters $M$ and number of input dimensions $D$

If $M=D$ the method is called Lasso, but if there are more parameters than input variables $M>D$ then it is called Group Lasso (Section 11.4.7).

* The next cells describe Lasso, which introduces sparsity by making a weight associated to a particular variable tending to zero.

* Group Lasso leads to a sparsity of more than one parameter that is associated to a given variable (which is an interesting way to induce sparsity in overparameterized models such as Artificial Neural Networks). It is derived in a very similar manner.

Computing the MAP estimate:

$$
\begin{align}
\hat{\mathbf{w}}_{\text{map}} &= \underset{w}{\mathrm{argmin}}\left[\left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right)^T \left(\boldsymbol{\Phi}\mathbf{w} - \mathbf{y}\right) + \overset{\scriptscriptstyle <}{\lambda}_w||\mathbf{w}||_1\right]
\end{align}
$$

where $||\mathbf{w}||_1 = \sum_{m=1}^{M-1}|w_m|$ is called the $l_1$ norm of $\mathbf{w}$. In ML literature this is called $l_1$ regularization.

* Calculating the MAP for Lasso is not done the same way as for Ridge because the term $||\mathbf{w}||_1$ is not differentiable whenever $w_m = 0$. In this case, the solution is found using hard- or soft-thresholding (See Section 11.4.3 in the book) because the gradient becomes a branch function.

* A more important point is to see that **different types of prior distributions** introduce **different regularizations** on the weights, aleviating overfitting in a different manner.
    - For example, in the case of Lasso, since the Laplace prior puts more density around the mean (which is zero here) than the Gaussian prior, then it has a tendency to lead the weights to zero, i.e. it introduces **sparsity** when estimating the weights via MAP. The book has a beautiful discussion about this.

## Bayesian linear regression: Linear regression with Gaussian likelihood, Gaussian prior and Gaussian posterior (Bayesian solution)

As we saw in the beginning of the Lecture, there are many more models we can define! The book covers quite a few!

| Likelihood | Prior (on the weights)    | Posterior      | Name of the model | Book section  |
|---        |---         |---             |---              |---            |
| Gaussian  | Gaussian    | Gaussian | Bayesian linear regression   | 11.7 |

1. Gaussian observation distribution (with known variance): $p(y|x, \mathbf{z}) = \mathcal{N}(y| \mu_{y|z} = \mathbf{w}^T \boldsymbol{\phi}(x), \sigma_{y|z}^2 = \sigma^2)$

where $\mathbf{z} = (\mathbf{w}, \sigma)$ are all the unknown model parameters (hidden rv's).

2. Gaussian prior for the weights $\mathbf{w}$: $p(\mathbf{w}) = \mathcal{N}(\mathbf{w}| \overset{\scriptscriptstyle <}{\boldsymbol{\mu}}_w, \overset{\scriptscriptstyle <}{\boldsymbol{\Sigma}}_w)$

3. Gaussian posterior (obtained from Bayes rule)

Final prediction is given by the PPD: $\require{color}
{\color{orange}p(y|x, \mathcal{D})} = \int p(y|x,\mathbf{z}) p(\mathbf{z}|\mathcal{D}) dz$

At this point you may notice that we have already derived this model in Lecture 7.

The only differences are that now we have multiple weights $\mathbf{z}$, multidimensional inputs $\mathbf{x}$ and that we allow them to have different values.

Yet, the derivation is the same! We just need to bold the letters. Let's do it:

The likelihood is a (multivariate) Gaussian distribution (product of MVN evaluated at each data point):

$$
p(\mathcal{D}|\mathbf{w}, \sigma^2) = \prod_{n=1}^N p(y_n | \mathbf{w}^T\boldsymbol{\phi}(\mathbf{x}), \sigma^2) = \mathcal{N}(\mathbf{y} | \boldsymbol{\Phi}\mathbf{w}, \sigma^2 \mathbf{I}_N)
$$

where $\mathbf{I}_N$ is the $N\times N$ identity matrix, as defined previously.

To calculate the posterior, we also use the product of Gaussians rule (Lecture 5 in the cell after the Homework we also defined this rule for multivariate Gaussians!):

$$
p(\mathbf{w}| \boldsymbol{\Phi}, \mathbf{y}, \sigma^2) \propto \mathcal{N}(\mathbf{y} | \boldsymbol{\Phi}\mathbf{w}, \sigma^2 \mathbf{I}_N)    \mathcal{N}(\mathbf{w}| \overset{\scriptscriptstyle <}{\boldsymbol{\mu}}_w, \overset{\scriptscriptstyle <}{\boldsymbol{\Sigma}}_w) = \mathcal{N}(\mathbf{w}| \overset{\scriptscriptstyle >}{\boldsymbol{\mu}}_w, \overset{\scriptscriptstyle >}{\boldsymbol{\Sigma}}_w)
$$

where the mean and covariance of the posterior are given by:

$$
\overset{\scriptscriptstyle >}{\boldsymbol{\mu}}_w = \overset{\scriptscriptstyle >}{\boldsymbol{\Sigma}}_w \left( \overset{\scriptscriptstyle <}{\boldsymbol{\Sigma}}_w \overset{\scriptscriptstyle <}{\boldsymbol{\mu}}_w + \frac{1}{\sigma^2}\boldsymbol{\Phi}^T\mathbf{y}\right)
$$

$$
\overset{\scriptscriptstyle >}{\boldsymbol{\Sigma}}_w = \left( \overset{\scriptscriptstyle <}{\boldsymbol{\Sigma}}_w + \frac{1}{\sigma^2}\boldsymbol{\Phi}^T\boldsymbol{\Phi}\right)^{-1}
$$

Often, we use a prior with zero mean $\overset{\scriptscriptstyle <}{\boldsymbol{\mu}}_w =\mathbf{0}$ and diagonal covariance $\overset{\scriptscriptstyle <}{\boldsymbol{\Sigma}}_w = \overset{\scriptscriptstyle <}{\sigma}_w^2 \mathbf{I}_M$ like the prior we used in Ridge regression. In this case, the posterior mean becomes the same as the MAP estimate obtained from Ridge regression: $\overset{\scriptscriptstyle >}{\boldsymbol{\mu}}_w =\left( \lambda \mathbf{I}_M  + \boldsymbol{\Phi}^T\boldsymbol{\Phi}\right)^{-1} \boldsymbol{\Phi}^T \mathbf{y}$.

#### Note on the reduction of the posterior mean for Bayesian linear regression to the MAP estimate of Ridge regression

If we use a prior with zero mean $\overset{\scriptscriptstyle <}{\boldsymbol{\mu}}_w =\mathbf{0}$ and diagonal covariance $\overset{\scriptscriptstyle <}{\boldsymbol{\Sigma}}_w = \overset{\scriptscriptstyle <}{\sigma}_w^2 \mathbf{I}_M$ then the posterior mean becomes

$$\overset{\scriptscriptstyle >}{\boldsymbol{\mu}}_w = \frac{1}{\overset{\scriptscriptstyle <}{\sigma}_w^2} \overset{\scriptscriptstyle >}{\boldsymbol{\Sigma}}_w \boldsymbol{\Phi}^T\mathbf{y}
$$

which is the same as the Ridge regression estimate when we define $\lambda = \frac{\sigma^2}{\overset{\scriptscriptstyle <}{\sigma}_w^2}$,

$\overset{\scriptscriptstyle >}{\boldsymbol{\mu}}_w =\left( \lambda \mathbf{I}_M  + \boldsymbol{\Phi}^T\boldsymbol{\Phi}\right)^{-1} \boldsymbol{\Phi}^T \mathbf{y}$

Having determined the posterior, we can determine what we really want: the PPD.

$$
\begin{align}
{\color{orange}p(y|x, \mathcal{D})} &= \int p(y|x,\mathbf{z}) p(\mathbf{z}|\mathcal{D}) d\mathbf{z} \\
p(y|x, \mathcal{D}, \sigma^2) &= \int p(y|x,\mathbf{w}, \sigma^2) p(\mathbf{w}|\mathcal{D}) d\mathbf{w} \\
&= \int \mathcal{N}(y | \boldsymbol{\phi}(\mathbf{x})^T\mathbf{w}, \sigma^2)    \mathcal{N}(\mathbf{w}| \overset{\scriptscriptstyle >}{\boldsymbol{\mu}}_w, \overset{\scriptscriptstyle >}{\boldsymbol{\Sigma}}_w) dw \\
&= \mathcal{N}\left(y \mid \overset{\scriptscriptstyle >}{\boldsymbol{\mu}}^T_w \boldsymbol{\phi}(\mathbf{x}) \,,\, \sigma^2 + \boldsymbol{\phi}(\mathbf{x})^T \overset{\scriptscriptstyle >}{\boldsymbol{\Sigma}}_w \boldsymbol{\phi}(\mathbf{x})\right)
\end{align}
$$

where we recall the meaning of each term:

* ($\mathbf{x}$, $y$) is the point were we want to make a prediction
* $\boldsymbol{\phi}(\mathbf{x})$ is an $M\times 1$ vector of basis functions
* $\overset{\scriptscriptstyle <}{\boldsymbol{\mu}}_w$ is the $M\times 1$ vector with the mean of the posterior for the weights
* and $\overset{\scriptscriptstyle >}{\boldsymbol{\Sigma}}_w$ is the $M \times M$ covariance matrix of the posterior for the weights.

#### Note about integration of the PPD

The book uses the following notation:

$$\begin{align}
p(y|x, \mathcal{D}, \sigma^2) &= \int p(y|x,\mathbf{w}, \sigma^2) p(\mathbf{w}|\mathcal{D}) d\mathbf{w}
\end{align}
$$

We could be more explicit and use the following notation:

$$\begin{align}
p(y|x, \mathcal{D}, \sigma^2) &= \int p(y|x,\mathbf{w}, \sigma^2) p(\mathbf{w}|\mathcal{D}) d^{M-1}\mathbf{w} \\
&= \int\int\cdots \int p(y|x,\mathbf{w}, \sigma^2) p(\mathbf{w}|\mathcal{D}) dw_1 dw_2 \cdots dw_{M-1}
\end{align}
$$

where it is clear that we are integrating for all variables (not integrating to get a vector).

However, despite this notation being more precise, it is also less appealing. Just make sure you realize the type of integral that we are calculating when we are finding the PPD.

Observing the obtained PPD,

$$
\begin{align}
p(y|x, \mathcal{D}, \sigma^2) &= \mathcal{N}\left(y \mid \overset{\scriptscriptstyle >}{\boldsymbol{\mu}}^T_w \boldsymbol{\phi}(\mathbf{x}) \,,\, \sigma^2 + \boldsymbol{\phi}(\mathbf{x})^T \overset{\scriptscriptstyle >}{\boldsymbol{\Sigma}}_w \boldsymbol{\phi}(\mathbf{x})\right)
\end{align}
$$

we see something very interesting: the variance of the PPD at a point $\mathbf{x}$ after seeing $N$ data points depends on two terms:

1. the variance of the observation noise, $\sigma^2$ that we defined to be constant

2. and the variance in the parameters obtained by the posterior $\overset{\scriptscriptstyle >}{\boldsymbol{\Sigma}}_w$

This means that the predicted uncertainty increases when $\mathbf{x}$ is located far from the training data $\mathcal{D}$, just like we want it to be! We are less certain about points **away** from our observations (training data).

We have seen this happening in Gaussian processes too... You will see that Gaussian processes are not too different from Bayesian linear regression...

## Other linear regression models

As we saw in the beginning of the Lecture, there are many more models we can define! The book covers quite a few!

| Likelihood | Prior (on the weights)    | Posterior      | Name of the model | Book section  |
|---        |---         |---             |---              |---            |
| Gaussian  | Uniform    | Point estimate | Least Squares regression  | 11.2.2  |
| Gaussian  | Gaussian    | Point estimate | Ridge regression   | 11.3  |
| Gaussian  | Laplace    | Point estimate | Lasso regression  | 11.4  |
| Gaussian  | Gaussian$\times$Laplace    | Point estimate | Elastic net  | 11.4.8  |
| Student-$t$  | Uniform    | Point estimate | Robust regression   | 11.6.1  |
| Laplace  | Uniform    | Point estimate | Robust regression   | 11.6.2  |
| Gaussian  | Gaussian    | Gaussian | Bayesian linear regression   | 11.7 |

For example, the "Elastic net" is literally the combination of Ridge regression and Lasso by defining a prior that depends on both a Gaussian and a Laplace distribution. This model was proposed in 2005. Five years later the "Bayesian Elastic Net" was also proposed, where the posterior is calculated in a Bayesian way (just like what we did for Bayesian linear regression).

## In summary

Almost every ML model is derived following 4 steps:

1. Define the Observation distribution and compute the likelihood.

2. Define the prior and its parameters.

3. Compute the posterior to estimate the unknown parameters (whether via a Bayesian approach or via a Point estimate).

4. Compute the PPD.

Done.

If you only care about the mean of the PPD (the mean of your prediction), then you can compute it directly without even thinking about uncertainty... But then, make sure you characterize the quality of your predictions using an appropriate **error metric** and use strategies like cross-validation, like you did for your Midterm Project!

### See you next class

Have fun!


In order to successfully complete this assignment you need to participate both individually and in groups during class on **Wednesday February 19**.

# In-Class Assignment: Instructor template

Figure From: https://www.researchgate.net/publication/221561415_Towards_energy-aware_scheduling_in_data_centers_using_machine_learning

### Agenda for today's class (80 minutes)

1. [(20 minutes) Anthill Review](#anthill)
1. [(20 minutes) Pre-class Assignment Review](#class_Assignment_Review)
2. [(20 minutes) Machine Learning Rules of thumb](#Machine_Learning_Rules_of_thumb)
3. [(20 minutes) Example Application: The Skin Cancer data set](#The_cancer_data_set)

---

# 1. Anthill Review

git clone https://gitlab.msu.edu/colbrydi/anthill.git

----

# 2. Pre-class Assignment Review

* [0218--ML-pre-class-assignment](0218--ML-pre-class-assignment.ipynb)

---

# 3. Machine Learning Rules of thumb

- [Ugly duckling Theorem](https://en.wikipedia.org/wiki/Ugly_duckling_theorem)
- [Curse of Dimentionality](https://en.wikipedia.org/wiki/Curse_of_dimensionality)

✅ **DO THIS:** The above two properties can dominate how machine learning can be used. Briefly review both and discuss with your group how you think they relate to Machine Learning. Be prepaired to discuss with the rest of the class.

---

# 4. Example Application: The Skin Cancer data set

In this example we will do the same calculation steps but using different dataset provided by scikit learn called the "cancer" dataset.

The following commands loads a dataset of measurements computed from a digitized image of a fine needle aspirate (FNA) of a breast mass.

```python
%matplotlib inline
import matplotlib.pylab as plt
import numpy as np
import sympy as sym
sym.init_printing()
```

```python
from sklearn.datasets import load_breast_cancer
cancer = load_breast_cancer()
data = cancer.data
target = cancer.target
data.shape

# Variables used for plotting
labels=cancer.target
cdict={0:'red',1:'green'}
labl={0:'Malignant',1:'Benign'}
marker={0:'*',1:'o'}
alpha={0:.5, 1:.5}
```

```python
print(cancer.DESCR)
```

## Step A: Feature Extraction

The Following is a plot of just the first two features:

```python
plt.scatter(data[:,0],data[:,1], c=labels, s=30, cmap=plt.cm.rainbow);
```

## Step B: Splitting the dataset for model into training and testing sets
✅ **DO THIS:** Split the iris data into a training and testing set like we did in the previous example:

```python
##Put your code here
```

## Step C: Select and train a Classifier using the training dataset

✅ **DO THIS:** Use the train_vectors set and train_labels to train a Support Vector Machine. *Hint:* You should be able to use the same code and parameters we used in the previous example:

```python
##Put your code here
```

## Step D. Show the results of the classification on the testing dataset

✅ **DO THIS:** Test the predictive capabilities of your SVM using the test_vectors and compare the predicted labels to the actual labels.

```python
##Put your code here
```

-----
### Congratulations, we're done!

### Course Resources:

- [Syllabus](https://docs.google.com/document/d/e/2PACX-1vTW4OzeUNhsuG_zvh06MT4r1tguxLFXGFCiMVN49XJJRYfekb7E6LyfGLP5tyLcHqcUNJjH2Vk-Isd8/pub)
- [Preliminary Schedule](https://docs.google.com/spreadsheets/d/e/2PACX-1vRsQcyH1nlbSD4x7zvHWAbAcLrGWRo_RqeFyt2loQPgt3MxirrI5ADVFW9IoeLGSBSu_Uo6e8BE4IQc/pubhtml?gid=2142090757&single=true)
- [D2L Page](https://d2l.msu.edu/d2l/home/912152)
- [Git Repository](https://gitlab.msu.edu/colbrydi/cmse802-s20)

© Copyright 2020,  Michigan State University Board of Trustees


    Run in Google Colab

    View source on GitHub

# Explainable Recommendations

While the main objective of a recommender system is to identify the items to be recommended to a user, providing explanations to accompany the recommendations would be more persuasive as well as engender trust and transparency.  There are different types of explanations.  In this tutorial, we explore explainable recommendation approaches that rely on user product aspect-level sentiment for modeling explanations.

## 1. Setup

```
!pip install --quiet cornac==1.6.1
```

         |████████████████████████████████| 10.1MB 3.1MB/s

```
import os
import sys
from collections import defaultdict

import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline

import cornac
from cornac.utils import cache
from cornac.datasets import amazon_toy
from cornac.eval_methods import RatioSplit
from cornac.data import Reader, SentimentModality
from cornac.models import EFM, MTER, NMF, BPR

print(f"System version: {sys.version}")
print(f"Cornac version: {cornac.__version__}")

SEED = 42
VERBOSE = False
```

    System version: 3.6.9 (default, Apr 18 2020, 01:56:04)
    [GCC 8.4.0]
    Cornac version: 1.6.1

## 2. Aspect-Level Sentiments
To model fine-grained product aspect-ratings. Several works rely on sentiment analysis to extract aspect sentiment from product reviews. In other words, each review is now a list of aspect sentiments. Along with product rating, we also have aspect sentiments expressed in users' reviews. Here, we work with Toys and Games dataset, a sub-category of [Amazon reviews](http://jmcauley.ucsd.edu/data/amazon/).

Below are some examples of aspect-level sentiments that have been extracted from users' reviews of items.

```
sentiment = amazon_toy.load_sentiment()
samples = sentiment[:10]
pd.DataFrame.from_dict({
  "user": [tup[0] for tup in samples],
  "item": [tup[1] for tup in samples],
  "aspect-level sentiment": [tup[2] for tup in samples]
})
```

      user
      item
      aspect-level sentiment

      0
      A012468118FTQAINEI0OQ
      B00005BZM6
      [(paint, great, 1)]

      1
      A012468118FTQAINEI0OQ
      B001HA9JOA
      [(game, great, 1), (money, worth, 1)]

      2
      A012468118FTQAINEI0OQ
      B002BY2BVE
      [(paint, fun, 1), (item, well, 0)]

      3
      A012468118FTQAINEI0OQ
      B007U7M0LI
      [(price, sturdy, 1)]

      4
      A012468118FTQAINEI0OQ
      B00804BCO6
      [(gift, great, 1)]

      5
      A0182108CPDLPRCXQUZQ
      B002IUNLLK
      [(toy, best, 1), (heavy, cool, 1)]

      6
      A0182108CPDLPRCXQUZQ
      B007WYU7R8
      [(toy, great, 1)]

      7
      A0182108CPDLPRCXQUZQ
      B00ABY8WVO
      [(toy, love, 1)]

      8
      A0182108CPDLPRCXQUZQ
      B00AFP86KG
      [(toy, love, 1)]

      9
      A0182108CPDLPRCXQUZQ
      B00BJT861Q
      [(figure, well, 1), (toy, well, 1)]

```
# Load rating and sentiment information
reader = Reader(min_item_freq=20)
rating = amazon_toy.load_feedback(reader)

# Use Sentiment Modality for aspect-level sentiment data
sentiment_modality = SentimentModality(data=sentiment)

rs = RatioSplit(
    data=rating,
    test_size=0.2,
    exclude_unknowns=True,
    sentiment=sentiment_modality,
    verbose=VERBOSE,
    seed=SEED,
)
print("Total number of aspects:", rs.sentiment.num_aspects)
print("Total number of opinions:", rs.sentiment.num_opinions)

id_aspect_map = {v:k for k, v in rs.sentiment.aspect_id_map.items()}
id_opinion_map = {v:k for k, v in rs.sentiment.opinion_id_map.items()}
```

    rating_threshold = 1.0
    exclude_unknowns = True
    ---
    Training data:
    Number of users = 17433
    Number of items = 2180
    Number of ratings = 66758
    Max rating = 5.0
    Min rating = 1.0
    Global mean = 4.3
    ---
    Test data:
    Number of users = 8955
    Number of items = 2167
    Number of ratings = 15777
    Number of unknown users = 0
    Number of unknown items = 0
    ---
    Total users = 17433
    Total items = 2180
    Total number of aspects: 429
    Total number of opinions: 2604

## 3. Explicit Factor Model (EFM)

EFM model extends Non-negative Matrix Factorization (NMF) with the additional information from **aspect-level sentiments**.  The objective is to learn user, item, and aspect latent factors to explain user-item ratings, users' interest in certain aspects of the items, as well as the quality of items according to those aspects.  In a nutshell, EFM factorizes three matrices: *rating matrix*, *user-aspect attention matrix*, and *item-aspect quality matrix*.  Let's take a look at what the later two matrices are.

```
efm = EFM()
efm.train_set = rs.train_set
_, X, Y = efm._build_matrices(rs.train_set)
```

### User-Aspect Attention Matrix

Let $\mathcal{F} = \{f_1, f_2, \dots, f_F\}$ be the set of aspects (e.g., screen, earphone).

Let $\mathbf{X} \in \mathbb{R}^{N \times F}$ be a sparse aspect matrix for $N$ users and $F$ aspects, whereby each element $x_{if} \in \mathbf{X}$  indicates the degree of **attention** by user $i$ on aspect $f$, defined as follows:

\begin{equation}
x_{if} = \
\begin{cases}
0, & \text{if user $i$ never mentions aspect $f$} \\
1 + (N-1)\left(\frac{2}{1+\exp(-t_{if})}-1\right), & \text{otherwise}
\end{cases}
\end{equation}

where $N=5$ is the highest rating score, $t_{if}$ is the frequency of user $i$ mentions aspect $f$ across all her reviews.

For illustration purpose, we show a small matrix $\mathbf{X}$ of 5 users and 5 aspects below.

```
n_users = 5
n_aspects = 5
pd.DataFrame(
  data=X[:n_users, :n_aspects].A,
  index=[f"User {u + 1}" for u in np.arange(n_users)],
  columns=[f"{id_aspect_map[i]}" for i in np.arange(n_aspects)]
)
```

      paint
      game
      money
      item
      price

      User 1
      0.0
      0.000000
      0.0
      0.000000
      2.848469

      User 2
      0.0
      0.000000
      0.0
      0.000000
      0.000000

      User 3
      0.0
      4.620593
      0.0
      2.848469
      4.620593

      User 4
      0.0
      4.856110
      0.0
      0.000000
      0.000000

      User 5
      0.0
      0.000000
      0.0
      0.000000
      0.000000

In the example below, we can see that *User 4* finds the aspect *game* important, whereas *User 3* is concerned with *game* as well as *price*.

### Item-Aspect Quality Matrix

Let $\mathbf{Y} \in \mathbb{R}^{M \times F}$ be a sparse aspect matrix for $M$ items and $F$ aspects, whereby $y_{jf} \in \mathbf{Y}$ indicates the **quality** of item $j$ on aspect $f$, defined as follows:

\begin{equation}
y_{jf} = \
\begin{cases}
0, & \text{if item $j$ was never reviewed on aspect $f$} \\
1 + (N - 1) \left( \frac{1}{1+\exp(-s_{jf})} \right), & \text{otherwise}
\end{cases}
\end{equation}

where $s_{jf}$ is the sum of sentiment values with which item $j$ has been mentioned with regards to aspect $f$ across all its reviews.

We show a small matrix $Y$ of 5 items and 5 aspects below:

```
n_items = 5
n_aspects = 5
pd.DataFrame(
  data=Y[:n_items, :n_aspects].A,
  index=[f"Item {u + 1}" for u in np.arange(n_items)],
  columns=[f"{id_aspect_map[i]}" for i in np.arange(n_aspects)]
)
```

      paint
      game
      money
      item
      price

      Item 1
      3.924234
      0.000000
      3.000000
      0.000000
      3.924234

      Item 2
      0.000000
      0.000000
      0.000000
      4.523188
      4.523188

      Item 3
      0.000000
      4.523188
      0.000000
      0.000000
      0.000000

      Item 4
      0.000000
      0.000000
      0.000000
      0.000000
      3.924234

      Item 5
      3.924234
      0.000000
      3.924234
      4.523188
      4.523188

We see from the example above that *Item 3* has a positive quality in the aspect *game*, whereas *Item 5* has positive quality on the other 4 aspects.

### Optimization

As these matrices are sparse, for prediction, EFM jointly factorizes $X$ and $Y$ matrices along with rating matrix $R$.   Learning the latent factors can be done via minimizing the following loss function:

\begin{align}
&\mathcal{L}(\mathbf{U_1, U_2, V, H_1, H_2} | \lambda_x, \lambda_y, \lambda_u, \lambda_h, \lambda_v) = ||\mathbf{U_1} \mathbf{U_2}^T + \mathbf{H_1} \mathbf{H_2}^T - \mathbf{R}||_F^2 + \lambda_x ||\mathbf{U_1} \mathbf{V}^T - \mathbf{X}||_F^2 + \lambda_y ||\mathbf{U_2} \mathbf{V}^T - \mathbf{Y}||_F^2 + \lambda_u(||\mathbf{U_1}||_F^2+||\mathbf{U_2}||_F^2) + \lambda_h(||\mathbf{H_1}||_F^2+||\mathbf{H_2}||_F^2) + \lambda_v ||\mathbf{V}||_F^2 \\
&\text{such that: }  \forall_{i, k} u_{ik} \ge 0, \forall_{j, k} v_{jk} \ge 0
\end{align}

The can be solved as a constrained optimization problem.

Let's conduct an experiment with EFM model and compare with NMF as a baseline.

```
efm = EFM(
  num_explicit_factors=40,
  num_latent_factors=60,
  num_most_cared_aspects=15,
  rating_scale=5.0,
  alpha=0.85,
  lambda_x=1,
  lambda_y=1,
  lambda_u=0.01,
  lambda_h=0.01,
  lambda_v=0.01,
  max_iter=100,
  verbose=VERBOSE,
  seed=SEED,
)

# compare to baseline NMF
nmf = NMF(k=100, max_iter=100, verbose=VERBOSE, seed=SEED)

eval_metrics = [
  cornac.metrics.RMSE(),
  cornac.metrics.NDCG(k=50),
  cornac.metrics.AUC()
]

cornac.Experiment(
  eval_method=rs, models=[nmf, efm], metrics=eval_metrics
).run()
```

    TEST:
    ...
        |   RMSE |    AUC | NDCG@50 | Train (s) | Test (s)
    --- + ------ + ------ + ------- + --------- + --------
    NMF | 0.8027 | 0.5418 |  0.0093 |    4.6980 |   8.4513
    EFM | 0.7315 | 0.5536 |  0.0105 |   11.2728 |  12.2117

### Refining Ranking Prediction

With EFM model, you can refine the recommendation after training by experimenting with different values of:
*   `num_most_cared_aspects` ($k$): integer, value range $\in[0, 429]$ as we have $429$ aspects in total
*   `alpha` $\in [0,1]$

These parameters will affect ranking performance of the EFM model, as the ranking score is predicted as follow:

$$
ranking\_score = \alpha \cdot \frac{\sum_{c \in C_i}{\hat{x}_{if}\cdot\hat{y}_{jf}}}{k \cdot N} + (1-\alpha)\cdot\hat{r}_{ij}
$$

```
alpha = 0.9 # alpha value in range [0,1]
num_most_cared_aspects = 100

eval_metrics = [
  cornac.metrics.NDCG(k=50),
  cornac.metrics.AUC()
]

cornac.Experiment(
  eval_method=rs,
  models=[
    EFM(
      alpha=alpha,
      num_most_cared_aspects=num_most_cared_aspects,
      init_params={'U1': efm.U1, 'U2': efm.U2, 'H1': efm.H1, 'H2': efm.H2, 'V': efm.V},
      trainable=False,
      verbose=VERBOSE,
      seed=SEED
    )
  ],
  metrics=eval_metrics
).run()
```

    TEST:
    ...
        |    AUC | NDCG@50 | Train (s) | Test (s)
    --- + ------ + ------- + --------- + --------
    EFM | 0.5549 |  0.0107 |    0.0008 |  15.3882

### Recommendation Explanation with EFM

Given a user and an item, EFM model is able of predicting **user's attention scores** as well as **item's quality scores** regarding the aspects.  Those scores with the corresponding aspects will be the explanation on why a user *likes* or *dislikes* an item.

Let's take a look at an example below. Feel free to explore other users and items!

```
UIDX = 1
IIDX = 4
num_top_cared_aspects = 10

id_aspect_map = {v:k for k, v in rs.sentiment.aspect_id_map.items()}

predicted_user_aspect_scores = np.dot(efm.U1[UIDX], efm.V.T)
predicted_item_aspect_scores = np.dot(efm.U2[IIDX], efm.V.T)

top_cared_aspect_ids = (-predicted_user_aspect_scores).argsort()[:num_top_cared_aspects]
top_cared_aspects = [id_aspect_map[aid] for aid in top_cared_aspect_ids]
pd.DataFrame.from_dict({
  "aspect": top_cared_aspects,
  "user_aspect_attention_score": predicted_user_aspect_scores[top_cared_aspect_ids],
  "item_aspect_quality_score": predicted_item_aspect_scores[top_cared_aspect_ids]
})

```

      aspect
      user_aspect_attention_score
      item_aspect_quality_score

      0
      toy
      4.061583
      4.679122

      1
      pieces
      3.610668
      3.844794

      2
      game
      3.571418
      4.211304

      3
      furby
      3.571004
      4.787230

      4
      doll
      3.480351
      4.326976

      5
      quality
      3.468948
      3.952711

      6
      really
      3.368340
      4.586975

      7
      gift
      3.364289
      4.410737

      8
      also
      3.360865
      4.076249

      9
      puzzle
      3.294265
      4.726530

EFM takes an aspect with the **highest score** in `item_aspect_quality_score` as the well-performing aspect, and an aspect with the **lowest score** in `item_aspect_quality_score` as the poorly-performing aspect. See example explanations in their templates below.

```
perform_well_aspect = top_cared_aspects[predicted_item_aspect_scores[top_cared_aspect_ids].argmax()]
perform_poorly_aspect = top_cared_aspects[predicted_item_aspect_scores[top_cared_aspect_ids].argmin()]

explanation = \
f"You might interested in [{perform_well_aspect}], on which this product perform well. \n\
You might interested in [{perform_poorly_aspect}], on which this product perform poorly."
print("EFM explanation:")
print(explanation)
```

    EFM explanation:
    You might interested in [furby], on which this product perform well.
    You might interested in [pieces], on which this product perform poorly.

## 4. Multi-Task Explainable Recommendation (MTER)

MTER model extends the concept of exploiting information from *Aspect-Level Sentiments* with tensor factorization (using Tucker Decomposition).   The model takes in the input of three tensors.  Let's go through each of them and see how they are constructed.

### Tensor\#1: User by Item by Aspect ($\mathbf{X}$)

Let $\mathbf{R} \in \mathbb{R}^{N \times M}$ be a sparse rating matrix of $N$ users and $M$ items.

Let $\mathbf{X} \in \mathbb{R}_{+}^{N \times M \times F}$ be a 3-dimensional tensor, each element $x_{ijf}$ indicates a relationship between user $i$, item $j$, and aspect $f$:

\begin{equation}
x_{ijf} = \
\begin{cases}
0, & \text{if aspect $f$ has not been mentioned by user $i$ about item $j$} \\
1 + (N-1)\left(\frac{1}{1+\exp(-s_{ijf})}\right), & \text{otherwise}
\end{cases}
\end{equation}

where $s_{ijf}$ is the sum of sentiment values with which item $j$ has been mentioned by user $i$ with regards to aspect $f$.

We can extend $\mathbf{X}$ into $\mathbf{\tilde{X}}$ with the rating matrix $\mathbf{R}$ as the last slice or the $(F + 1)^{\mathrm{th}}$ aspect (i.e., $\tilde{x}_{ij(F+1)} = r_{ij}$).

### Tensor\#2: User by Aspect by Opinion ($\mathbf{Y}^{U}$)

Let $\mathbf{Y}^{U} \in \mathbb{R}_{+}^{N \times F \times O}$ be a 3-dimensional tensor, each element $y^U_{ifo}$ indicates a relationship between user $i$, aspect $f$, and opinion $o$:

\begin{equation}
y^U_{ifo} = \
\begin{cases}
0, & \text{if user $i$ has not been used opinion $o$ to describe aspect $f$ positively} \\
1 + (N-1)\left(\frac{1}{1+\exp(-t_{ifo})}\right), & \text{otherwise}
\end{cases}
\end{equation}

where $t_{ifo}$ is the frequency with which user $i$ employs opinion $o$ to describe aspect $f$ positively across all her reviews.

### Tensor\#3: Item by Aspect by Opinion ($\mathbf{Y}^{I}$)

Let $\mathbf{Y}^{I} \in \mathbb{R}_{+}^{M \times F \times O}$ be a 3-dimensional tensor, each element $y^I_{jfo}$ indicates a relationship between item $j$, aspect $f$, and opinion $o$:

\begin{equation}
y^I_{jfo} = \
\begin{cases}
0, & \text{if item $j$ has not been described positively with opinion $o$ on aspect $f$} \\
1 + (N-1)\left(\frac{1}{1+\exp(-t_{jfo})}\right), & \text{otherwise}
\end{cases}
\end{equation}

where $t_{jfo}$ is the frequency with which item $j$ has been described positively with opinion $o$ on aspect $f$ positively across all its reviews.

### Optimization

MTER employs Tucker Decomposition to jointly factorize three tensors $\mathbf{\tilde{X}}$, $\mathbf{Y}^U$, and $\mathbf{Y}^I$.   In addition, MTER also optimizes for a ranking objective akin to BPR where:
*  Positive triples $\mathbf{T} = \{ j >_{i} j' | x_{ij(F+1)} \in \mathbf{R}^+ \land x_{ij'(F+1)} \in \mathbf{R}^- \}$
*  For aspect (F + 1), which is the overall rating, user $i$ prefers item $j$ to item $j'$

Learning the latent factors can be done via minimizing the following loss function:

\begin{align}
&\mathcal{L}(\mathbf{U, V, Z, W, C_1, C_2, C_3} | \lambda_B, \lambda) = ||\mathbf{\tilde{X}} - \mathbf{\hat{X}}||_F^2 + ||\mathbf{Y}^U - \hat{\mathbf{Y}}^U||_F^2 + ||\mathbf{Y}^I - \hat{\mathbf{Y}}^I||_F^2 - \lambda_B \sum_{j >_i j'} \ln(1 + \exp{(-(\hat{x}_{ij(F+1)} - \hat{x}_{ij'(F+1)}))}) + \lambda(||\mathbf{U}||_F^2+||\mathbf{V}||_F^2+||\mathbf{Z}||_F^2+||\mathbf{W}||_F^2 +||\mathbf{C_1}||_F^2 +||\mathbf{C_2}||_F^2 +||\mathbf{C_3}||_F^2) \\
&\text{such that: }  \mathbf{U} \ge 0, \mathbf{V} \ge 0, \mathbf{Z} \ge 0, \mathbf{W} \ge 0, \mathbf{C_1} \ge 0, \mathbf{C_2} \ge 0, \mathbf{C_3} \ge 0
\end{align}

The can be solved as a constrained optimization problem.

Let's conduct an experiment with MTER model and compare with the BPR baseline.

```
mter = MTER(
  n_user_factors=10,
  n_item_factors=10,
  n_aspect_factors=10,
  n_opinion_factors=10,
  n_bpr_samples=1000,
  n_element_samples=50,
  lambda_reg=0.1,
  lambda_bpr=10,
  max_iter=3000,
  lr=0.5,
  verbose=VERBOSE,
  seed=SEED,
)

# compare to baseline BPR
bpr = BPR(k=10, verbose=VERBOSE, seed=SEED)

eval_metrics = [
  cornac.metrics.NDCG(k=50),
  cornac.metrics.AUC()
]

# Instantiate and run an experiment
cornac.Experiment(
  eval_method=rs, models=[bpr, mter], metrics=eval_metrics,
).run()
```

    TEST:
    ...
         |    AUC | NDCG@50 | Train (s) | Test (s)
    ---- + ------ + ------- + --------- + --------
    BPR  | 0.6271 |  0.0314 |    1.9081 |   5.7188
    MTER | 0.7185 |  0.0357 |   51.1022 |   6.8672

### Recommendation Explanation with MTER

* To provide recommendation to user $i$, we rank items $j$ in terms of the predicted rating scores: $\hat{x}_{ij(F+1)}$

* To determine which aspect $f$ of product $j$ a user $i$ cares about, we rank aspects $f$ in terms of: $\hat{x}_{ijf}$

* To determine which opinion phrases $o$ to use when describing aspect $f$ while recommending item $j$ to user $i$, we rank phrases in terms of: $\hat{y}^U_{ifo} \times \hat{y}^I_{jfo}$

Let's explore an example below on how we can generate explanations for recommendation by MTER model.

```
UIDX = 10
IIDX = 10
num_top_aspects = 2
num_top_opinions = 3

item_aspect_ids = np.array(list(set([
    tup[0]
    for idx in rs.sentiment.item_sentiment[IIDX].values()
    for tup in rs.sentiment.sentiment[idx]
])))

item_opinion_ids = np.array(list(set([
  tup[1]
  for idx in rs.sentiment.item_sentiment[IIDX].values()
  for tup in rs.sentiment.sentiment[idx]
])))

item_aspects = [id_aspect_map[idx] for idx in item_aspect_ids]

ts1 = np.einsum("abc,a->bc", mter.G1, mter.U[UIDX])
ts2 = np.einsum("bc,b->c", ts1, mter.I[IIDX])
predicted_aspect_scores = np.einsum("c,Mc->M", ts2, mter.A)

top_aspect_ids = item_aspect_ids[(-predicted_aspect_scores[item_aspect_ids]).argsort()[:num_top_aspects]]
top_aspects = [id_aspect_map[idx] for idx in top_aspect_ids]

top_aspect_opinions = []
mter_explanations = []
for top_aspect_id, top_aspect in zip(top_aspect_ids, top_aspects):
  ts1_G2 = np.einsum("abc,a->bc", mter.G2, mter.U[UIDX])
  ts2_G2 = np.einsum("bc,b->c", ts1_G2, mter.A[top_aspect_id])
  predicted_user_aspect_opinion_scores = np.einsum("c,Mc->M", ts2_G2, mter.O)

  ts1_G3 = np.einsum("abc,a->bc", mter.G3, mter.I[IIDX])
  ts2_G3 = np.einsum("bc,b->c", ts1_G3, mter.A[top_aspect_id])
  predicted_item_aspect_opinion_scores = np.einsum("c,Mc->M", ts2_G3, mter.O)

  predicted_aspect_opinion_scores = np.multiply(predicted_user_aspect_opinion_scores, predicted_item_aspect_opinion_scores)
  top_opinion_ids = item_opinion_ids[(-predicted_aspect_opinion_scores[item_opinion_ids]).argsort()[:num_top_opinions]]
  top_opinions = [id_opinion_map[idx] for idx in top_opinion_ids]
  top_aspect_opinions.append(top_opinions)

  # Generate explanation for top-1 aspect
  mter_explanations.append(f"Its {top_aspect} is [{'] ['.join(top_opinions)}].")

pd.DataFrame.from_dict({"aspect": top_aspects, "top_opinions": top_aspect_opinions, "explanation": mter_explanations})
```

      aspect
      top_opinions
      explanation

      0
      really
      [disappointed, great, like]
      Its really is [disappointed] [great] [like].

      1
      addition
      [disappointed, great, fun]
      Its addition is [disappointed] [great] [fun].

## References

1.   Zhang, Y., Lai, G., Zhang, M., Zhang, Y., Liu, Y., & Ma, S. (2014). Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In SIGIR (pp. 83-92).
2. Wang, N., Wang, H., Jia, Y., & Yin, Y. (2018). Explainable recommendation via multi-task learning in opinionated text data. In SIGIR (pp. 165-174).
4.   Cornac - A Comparative Framework for Multimodal Recommender Systems (https://cornac.preferred.ai/)


```python
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

### Library import

```python
from sympy import init_printing, Matrix, symbols, eye, Rational
init_printing()
```

# Matrix multiplication, inverse and transpose

## Multiplying matrices

In this section we approach matrix multiplication on four ways.  Each method provides its own unique insight into the process of multiplication.  We also take a look at _block multiplication_, where a matrix can be _broken up_ into specifically sized blocks that make multiplication easier.

### Method 1

Consider multiplying matrices $A$ and $B$ to result in $C$. We have already seen that the column size of the first must equal the row size of the second, $A_{n \times m} B_{m \times p} = C_{n \times p}$.

Every element in $C$ (with a row value of $i$ and a column value of $j$) is the dot product of the corresponding row vector in $A$ and the column vector in $B$.  In (1) we take the dot product of row `2` in the first matrix and column `1` in the second matrix to get $C_{21}$.

$$ { \begin{bmatrix} \cdots  & \cdots  & \cdots  \\ 3 & 2 & -1 \\ \cdots  & \cdots  & \cdots  \\ \cdots  & \cdots  & \cdots  \end{bmatrix} }_{ 4\times 3 }{ \begin{bmatrix} 1 & \vdots  \\ 2 & \vdots  \\ 1 & \vdots  \end{bmatrix} }_{ 3\times 2 }={ \begin{bmatrix} { c }_{ 11 } & { c }_{ 12 } \\ \left( 3\times 1 \right) +\left( 2\times 2 \right) +\left( -1\times 1 \right)  & { c }_{ 22 } \\ { c }_{ 31 } & { c }_{ 32 } \\ { c }_{ 41 } & { c }_{ 42 } \end{bmatrix} }_{ 4\times 2 } \tag{1}$$

### Method 2

Here we are concerned with the columns of $B$.  We note that each column in $C$ is the result of the matrix $A$ times the corresponding column in $B$.  In this example, it would be akin to a matrix multiplied by a vector $A \underline{x} = \underline{b}$.  $B$ is then made up from four column vectors.

### Method 3

Here every row in $A$ produces the same numbered row in $C$ by multiplying it with the matrix $B$.  The rows of $C$ are linear combinations of $B$.

### Method 4

In method 1 we looked at $\text{row}_A \times \text{col}_B$, which produced a single number in $C$.  What if we did $\text{col} \times \text{row}$?

The size of a column in $A$ might be written as $r_A \times 1$ and a row in $B$ as $1 \times s_B$.  The result in $C$ would then be $r_A \times s_B$.  Let's use `sympy` to investigate a simple example.  Here we have $A_{3 \times 1}$ and $B_{1 \times 2}$, with $C_{3 \times 2}$.

```python
A = Matrix([[2], [3], [4]])
B = Matrix([[1, 6]])
A, B
```

```python
C = A * B
C
```

So in method 4, for the example below, $C$ is the sum of *the columns of* $A$ times *the rows of* $B$.

$$ \begin{bmatrix} { a }_{ 11 } & { a }_{ 12 } \\ { a }_{ 21 } & { a }_{ 22 } \\ { a }_{ 31 } & { a }_{ 32 } \end{bmatrix}\begin{bmatrix} { b }_{ 11 } & { b }_{ 12 } \\ b_{ 21 } & { b }_{ 22 } \end{bmatrix}=\begin{bmatrix} { a }_{ 11 } \\ { a }_{ 21 } \\ { a }_{ 31 } \end{bmatrix}\begin{bmatrix} { b }_{ 11 } & { b }_{ 12 } \end{bmatrix}+\begin{bmatrix} { a }_{ 12 } \\ { a }_{ 22 } \\ { a }_{ 32 } \end{bmatrix}\begin{bmatrix} { b }_{ 21 } & { b }_{ 22 } \end{bmatrix}  \tag{2}$$

### Block multiplication

This is in essence, a combination of the above.  We do the following:
+ Both $A$ and $B$ can be broken into block of sizes that allow for multiplication

We see an example of this in (3).

$$ \begin{bmatrix} { A }_{ 1 } & { A }_{ 2 } \\ { A }_{ 3 } & { A }_{ 4 } \end{bmatrix}\begin{bmatrix} { B }_{ 1 } & { B }_{ 2 } \\ { B }_{ 3 } & { B }_{ 4 } \end{bmatrix}=\begin{bmatrix} { A }_{ 1 }{ B }_{ 1 }+{ A }_{ 2 }{ B }_{ 3 } & { A }_{ 1 }{ B }_{ 2 }+{ A }_{ 2 }{ B }_{ 4 } \\ { A }_{ 3 }{ B }_{ 1 }+{ A }_{ 4 }{ B }_{ 3 } & { A }_{ 3 }{ B }_{ 2 }+{ A }_{ 4 }{ B }_{ 4 } \end{bmatrix} \tag{3}$$

## Inverses

W e know by now that **if** the inverse of a matrix $A$ exists then $A^{-1} A = I$, the identity matrix.  This is a _left inverse_, but what about a _right inverse_, $A A^{-1}$?  This is also equal to the identity matrix (given that $A$ is invertible).  Invertible matrices are also called *non-singular* matrices.

This brings the question: "_What is a non-invertible matrix?_".  Non-invertible matrices are called *singular* matrices.  An example is shown in (4).

$$ \begin{bmatrix}1&3\\2&6\end{bmatrix} \tag{4}$$

Note how the elements of row `2` are just $2$ times the elements in row `1`.  The elements of row `2` are linear combinations of the elements in row `1`.  The same go for the columns, the first being a linear combination of the second, multiplying each element by $3$.  More profoundly, note that you could find a column vector $\underline{x}$ such that $A \times \underline{x}= \underline{0}$.

In (5) below we see that $3$ times column `1` in $A$ plus $-1$ times column `2` gives *nothing*, the zero vector.

$$ \begin{bmatrix}1&3\\2&6\end{bmatrix}\begin{bmatrix}3\\-1\end{bmatrix}=\begin{bmatrix}0\\0\end{bmatrix} \tag{5}$$

Let's construct as example, shown in (6) below as $A A^{-1} = I$ (given that $A^{-1}$ exists).

$$ \begin{bmatrix} 1 & 3 \\ 2 & 7 \end{bmatrix}\begin{bmatrix} a & c \\ b & d \end{bmatrix}=\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \tag{6}$$

In essence, we have to solve two systems.  $A$ times column `1` of $A^{-1}$ is column `1` of $I$.  This is the Gauss-Jordan idea of solving two systems at once.

$$ \begin{bmatrix} 1 & 3 \\ 2 & 7 \end{bmatrix}\begin{bmatrix} a \\ b \end{bmatrix}=\begin{bmatrix} 1 \\ 0 \end{bmatrix}\\ \begin{bmatrix} 1 & 3 \\ 2 & 7 \end{bmatrix}\begin{bmatrix} c \\ d \end{bmatrix}=\begin{bmatrix} 0 \\ 1 \end{bmatrix} \tag{7}$$

This will give us the two columns of $I$.  We proceed by creating an augmented matrix of the coefficients (note carefully!).

$$ \begin{bmatrix} 1 & 3 & 1 & 0 \\ 2 & 7 & 0 & 1 \end{bmatrix} \tag{8}$$

Now we use elementary row operations to reduced row-echelon form (leading $1$'s in the pivot positions, with $0$'s below and above each).

$$ \begin{bmatrix} 1 & 3 & 1 & 0 \\ 2 & 7 & 0 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} 1 & 3 & 1 & 0 \\ 0 & 1 & -2 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} 1 & 0 & 7 & -3 \\ 0 & 1 & -2 & 1 \end{bmatrix} \tag{9}$$

We now read off the two columns of $A^{-1}$.

 $$ \begin{bmatrix}7&-3\\-2&1\end{bmatrix} \tag{10}$$

## Example problems

### Example problem 1

Find the conditions on $a$ and $b$ that makes the matrix $A$ invertible and find $A^{-1}$.|

$$ A=\begin{bmatrix} a & b & b \\ a & a & b \\ a & a & a \end{bmatrix} \tag{11} $$

#### Solution

1. A matrix is singular (non-invertible) if we have a row or column of zeros, so $a \ne 0$
2. We can also not have similar columns, so $a \ne b$

Using Gauss-Jordan elimination we will have the following.

$$ \begin{bmatrix} a & b & b & 1 & 0 & 0 \\ a & a & b & 0 & 1 & 0 \\ a & a & a & 0 & 0 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \\ 0 & a-b & 0 & -1 & 1 & 0 \\ 0 & a-b & a-b & -1 & 0 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \\ 0 & a-b & 0 & -1 & 1 & 0 \\ 0 & 0 & a-b & 0 & -1 & 1 \end{bmatrix}\\ \rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \\ 0 & \frac { a-b }{ a-b }  & 0 & \frac { -1 }{ a-b }  & \frac { 1 }{ a-b }  & 0 \\ 0 & 0 & \frac { a-b }{ a-b }  & 0 & \frac { -1 }{ a-b }  & \frac { 1 }{ a-b }  \end{bmatrix}\rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \\ 0 & 1 & 0 & \frac { -1 }{ a-b }  & \frac { 1 }{ a-b }  & 0 \\ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b }  & \frac { 1 }{ a-b }  \end{bmatrix}\\ \rightarrow \begin{bmatrix} a & b & 0 & 1 & \frac { 1 }{ a-b } \left( b \right)  & -\frac { 1 }{ a-b } \left( b \right)  \\ 0 & 1 & 0 & \frac { -1 }{ a-b }  & \frac { 1 }{ a-b }  & 0 \\ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b }  & \frac { 1 }{ a-b }  \end{bmatrix}\rightarrow \begin{bmatrix} a & 0 & 0 & 1+\frac { b }{ a-b }  & 0 & -\frac { 1 }{ a-b } \left( b \right)  \\ 0 & 1 & 0 & \frac { -1 }{ a-b }  & \frac { 1 }{ a-b }  & 0 \\ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b }  & \frac { 1 }{ a-b }  \end{bmatrix}\\ \rightarrow \begin{bmatrix} 1 & 0 & 0 & \frac { 1 }{ a-b }  & 0 & -\frac { 1 }{ a\left( a-b \right)  } \left( b \right)  \\ 0 & 1 & 0 & \frac { -1 }{ a-b }  & \frac { 1 }{ a-b }  & 0 \\ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b }  & \frac { 1 }{ a-b }  \end{bmatrix}\\ { A }^{ -1 }=\frac { 1 }{ a-b } \begin{bmatrix} 1 & 0 & \frac { -b }{ a }  \\ -1 & 1 & 0 \\ 0 & -1 & 1 \end{bmatrix} \tag{12}$$

If we were to construct this matrix using `sympy` and invert it using the `.inv()` method, we can deduce our findings through simple algebra, i.e. $a \ne b$, $a \ne 0$.

```python
a, b = symbols('a b')
```

```python
A = Matrix([[a, b, b], [a, a, b], [a, a, a]])
A
```

```python
A.inv()
```

```python

```

Mathematics/Linear Algebra/0.0 MIT-18.06 - Jupyter/Lecture_3_Matrix_multiplication_Inverses.ipynb


```python
from IPython.display import HTML

HTML('''
<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code for QISKit exercises."></form>''')
```

<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code for QISKit exercises."></form>

# 8. Shor's algorithm

Shor's factoring algorithm [1] is the most well-known example of a quantum algorithm outperforming the best known classical algorithms. The algorithm allows to factor a number $N_0$ which is the product of two prime numbers $N_0 = p \cdot q$, in polynomial time. This is possible thanks to a theorem in number theory which turns the problem of finding factors into the problem of finding the period of a periodic function. Using the quantum Fourier transform (QFT), one can then find the period of this function with high probability and complete the factorization.

The problem of factoring numbers has been studied for centuries and no efficient (polynomial time) algorithm has ever been found. The difficulty of factoring numbers is the basis on which the most widespread encryption standard, the RSA, is founded. Therefore, an algorithm which is capable of factoring numbers efficiently could have a huge impact on the security of electronic based interactions. The discovery of this algorithm by Peter Shor in 1994, led to an explosion of the field of quantum computation because of its important application.

However, experimental implementation of the algorithm still remains a challenge because of the errors introduced by the large number of physical qubits and gates required to execute the algorithm. Proof of principle demonstration of Shor's factoring algorithm to factor the smallest number $N_0=15$ using five superconducting qubits have been done for setups like NMR [2], trapped ions [3], photons [4-6], photonic chips [7] and superconducting qubits [8,9]. These experiments show how complex is to implement the actual algorithm, as even for the smallest number which allows the algorithm to be run ($N_0=15$), extreme simplifications must be made to run the algorithm [10].

## 8.1 Mathematical preliminaries

### 8.1.1 Modular arithmetic
When division between integers $a$ (the dividend) and $b$ (the divisor) is taught for the first time, one is usually introduced to the idea of quotient $q$ and remainder $r$. The quotient is the number of times the dividend $a$ contains the divisor $b$, the remainder is any left-over that added to the product of the quotient by the divisor returns the dividend, $r + q\times b = a$ or $r = a - q\times b$.

#### Example:
Consider the case $a=40$ and $b=30$, let us find the quotient and the remainder of the division of $a$ by $b$. $40/30 \rightarrow q = 1, \, r = 13$,

Therefore, the quotient is $q=1$ and the remainder is $r = 13$.

By looking at the division between integers in this way, one can easily understand the fundamentals of modular arithmetic. Modular arithmetic is a set of rules for handling operations between integer numbers. In modular arithmetic, integers multiple of a fixed number $N_0$ are considered equivalent. Thus, by selecting an integer $N_0$, the set of all integers is restricted to the integers in the interval $\left[ 0 , N_0-1 \right]$.
To visualize this, one can think about a twelve-hours clock. In a clock, the only integers allowed lie in the interval $\left[ 0 , 11 \right]$, once this interval is exceeded one goes back at the beginning of the interval. Thus, $9+5 = 2 \, \left( mod \, 12 \right)$.
More formally, two integers $a$ and $b$ are congruent modulo $N_0$ if their difference $a-b$ is an integer multiple of $N_0$. That is, $a-b = r \cdot N_0$, where $r$ is an integer, then $a \equiv b  \, \left( mod \, N_0 \right)$.
Equivalently, if $a \equiv b  \, \left( mod \, N_0 \right)$, then $\frac{a-b}{N_0}$ has zero remainder. Or, $b  \, \left( mod \, N_0 \right)$ is the remainder of the division $\frac{b}{N_0}=a$. Thus, both numbers have the same remainder when divided by $N_0$.

#### Example:
Equivalence of two integers modulo $N_0$:
$38 \equiv 14  \, \left( mod \, 12 \right)$
$38 - 14  = 24 = 2 \cdot 12 $

Both $\frac{38}{12}$ and $\frac{14}{12}$ have remainder $2$.

### 8.1.2 Continued Fraction Algorithm

The continued fraction algorithm is used to reduce a fraction $\frac{m}{n}$ to another fraction $\frac{u}{t}$. In general, the algorithm allows to rewrite any irrational number as a finite/infinite sum of an integer part plus the reciprocal of a number.
So, consider the fraction $\frac{m}{n}$, its continued fraction expansion is

$$ \frac{m}{n} = a_0 + \frac{1}{a_1 + \frac{1}{a_2+ \frac{1}{a_3+\frac{1}{1+\frac{1}{a_4+\frac{1}{a_5+\frac{1}{...}}}}}}} $$

Using the integers $a_i$, one can rewrite the fraction $\frac{m}{n}$ as $\frac{u}{t}$. To find $u_i$ and $t_i$ one can use the following formulas:

$u_0 = a_0$, $u_1=1+a_0a_1$, ..., $u_n = a_n d_n + d_{n-2}$
$t_0=1$, $t_1 = a_1$, ..., $t_n =a_n t_{n−1} +t_{n−2}$

Which then give the possible approximations for $\frac{m}{n}$:

$\frac{m}{n} \approx \frac{u_0}{t_0}$, $\frac{m}{n} \approx \frac{u_1}{t_1}$, ..., $\frac{m}{n} \approx \frac{u_n}{t_n}$

## 8.2 Factoring and order finding

The possibility of running an efficient algorithm for factoring a product of two prime number arises from: i. the connection between factoring and order finding, ii. the ability of quantum computers to deal efficiently with periodic functions.
In this Section, the connection between the problem of factoring and the problem of finding the period of a periodic function, also called $\textit{order finding}$ is explained.
It is important to note that this equivalency holds in the realm of modular arithmetic.
Let us start from the concept of order of a number. Given an integer $a$, the order of $a$ is the smallest integer number $r$ for which the following condition holds

\begin{equation}
a^r \text{mod} N_0 = 1 \tag{1}
\end{equation}

#### Example:
Consider the case $a=2$ and $N_0=21$, let us find the order of $a$ by trying different exponents $r$ sequentially until 	we find the one for which condition (1) is satisfied:
	$2^1 (\text{mod}  21) = 2$,
	$2^2 (\text{mod}  21) = 4$,
	$2^3 (\text{mod} 21) = 8$,
	$2^4 (\text{mod} 21) = 16$,
	$2^5 (\text{mod} 21) = 11$,
	$2^6 (\text{mod} 21) = 1$.

Therefore, the order of $a=2$ is $r=6$.

Now that we have clarified what is the order of a number, let us explore the connection between order finding and factoring.
Start by rewriting Eq. (1) as $a^r - 1  \text{mod} \, N_0 = 0$, which means that $a^r - 1$ is a multiple of $N_0$. Assuming that $r$ is even, we can write $ a^r - 1  \text{mod} \, N_0  = \left( a^{\frac{r}{2}} - 1\right)\left( a^{\frac{r}{2}} + 1\right)  \text{mod} \, N_0 $. Finally, we check if either $a^{\frac{r}{2}} - 1$ or $a^{\frac{r}{2}} + 1$ shares any factors with $N_0$. This can be checked tacking the greatest common divisor (gcd) of them, which outputs the biggest number that divides both inputs. If the output of $\text{gcd}\left( a^{\frac{r}{2}} \pm 1,N_0 \right)$ is greater than one, than that's one of the factors. The other one being $N_0/ \text{gcd}\left( a^{\frac{r}{2}} \pm 1,N_0 \right)$.

#### Example:
Again consider the case $a=2$ and $N_0=21$, in the previous example we found that the order is $r = 6$. Let us now show how to find the two prime factors whose product gives $21$.
$2^6 - 1 (\text{mod} \, 21) = \left( 2^3 +1 \right)\left( 2^3 -1 \right) (\text{mod} \, 21) $.
We need to check if the two numbers $ \left( 2^3 +1 \right) = 9$ and $ \left( 2^3 -1 \right)=7$ have any factors in common with $21$.
Let's start by checking $ \left( 2^3 +1 \right)$: $\text{gcd}\left( 9, 21\right) = 3$.
We obtained a $gcd \neq 1$, which means that there is a common factor between $a^{\frac{r}{2}} + 1$ and $N_0$. We can now calculate the first factor of $21$.
$\text{factor}_1 = \frac{21}{\text{gcd}\left( 9, 21\right)} = \frac{21}{3} = 7$.
The second factor follows immediately from $\text{factor}_2 = \frac{21}{\text{factor}_1} = \frac{21}{7} = 3$.

Therefore, we find the two prime factors whose product is $N_0=21$, $\text{factor}_1 = 7$ and $\text{factor}_2 = 3$. Thus, the order of $a=2$ is $r=6$.

It's important to notice that several assumptions are needed to translate the problem of factoring to the one of order finding. These are "weak points" of the algorithm, which fails to produce the correct result each time one of these assumptions is violated.

## 8.3 Quantum Fourier transform

The Quantum Fourier Transform (QFT) is the heart of Shor's factoring algorithm. Similarly to the discrete Fourier transform, which allows to compute the Fourier transform of a complex number, the QFT allows to compute the Fourier transform of a quantum state. This means that we can rewrite a generic state of a qubit register as a superposition of all the possible basis state vectors of the register, with a certain phase.

The QFT of one of the basis states $\lvert i \rangle$ of an $n$ qubit register is

\begin{equation}
QFT \lvert j \rangle = \frac{1}{2^{n/2}} \sum_{k=0}^{2^{n}-1} e^{\frac{2 \pi i}{2^n} j k} \lvert k \rangle
\end{equation}

The QFT on a generic state can then be derived from this definition

\begin{equation}
QFT \sum_{j=0}^{2^n-1} x_j \lvert j \rangle = \sum_{k=0}^{2^{n}-1}  \sum_{l=0}^{2^{n}-1} x_l e^{\frac{2 \pi i}{2^n} l k} \lvert k \rangle
\end{equation}

By writing the basis state $\lvert j \rangle$ in binary representation we adopt the following convention: $\lvert j_1 j_2 ... j_n \rangle$ correspond to $j= j_1 2^{n-1}+j_2 2^{n-2} ... j_n 2^{0}$. Following the same convention, we can also write numbers smaller than one as $0.j_1 j_2 ... j_n$  meaning $j_1/2^{0} + j_2/2^{1} +...+ j_n/2^{n}$ a  it is possible to rewrite the QFT in a very simple form

\begin{aligned}
QFT \lvert j \rangle & = \frac{1}{2^{n/2}} \sum_{k=0}^{2^{n}-1} e^{2 \pi i jk / 2^n} \lvert k \rangle \\
& = \frac{1}{2^{n/2}} \sum_{k_1=0}^{1}...\sum_{k_n=0}^{1} e^{2 \pi i \left(\sum_{l=1}^{n} k_l 2^{-l}\right) j} \lvert k_1 ... k_n \rangle \\
& = \frac{1}{2^{n/2}}\sum_{k_1=0}^{1}...\sum_{k_n=0}^{1} \bigotimes_{l=0}^n e^{2 \pi i j k_l2^{-l} } \lvert k_l \rangle \\
& = \frac{1}{2^{n/2}} \bigotimes_{l=1}^n  \left(\lvert 0 \rangle + e^{2 \pi i j 2^{-l} } \lvert 1 \rangle \right) \\
& = \frac{1}{2^{n/2}} \left( \lvert 0 \rangle + e^{2 \pi i[0.j_n]} \lvert 1 \rangle \right) \otimes \left( \lvert 0 \rangle + e^{2 \pi i[0.j_{n-1} j_{n}]} \lvert 1 \rangle \right) \otimes ...\otimes  \left(\lvert 0 \rangle + e^{2 \pi i[0.j_1j_2...j_{n-1}j_n]} \lvert 1 \rangle \right)
\end{aligned}

By writing the QFT in this way, it is easy to find a circuit which implements it. The QFT turns out to be composed of very simple gates such as Hadamard and control-rotations around the z-axis between pairs of qubits $CR_k$.

$$ CR_k =
\begin{pmatrix}
1 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & 1 & 0 \\
0 & 0 & 0 & e^{2 \pi i/ 2^k}
\end{pmatrix}$$

$$\text{1. Quantum circuit for the Quantum Fourier Transform.}$$

#### Example: QFT on two qubits
Let us calculate the QFT in the case of $n=2$ qubits

$$QFT\lvert j_1j_2\rangle = \frac{1}{\sqrt{8}} \left(\lvert 0 \rangle + e^{2 \pi i[0.j_2]} \lvert 1 \rangle\right) \otimes \left(\lvert 0 \rangle + e^{2 \pi i[0.j_1 j_2]} \lvert 1 \rangle \right) $$

The steps to creating the circuit for $\lvert k_1k_2\rangle = QFT\lvert j_1j_2\rangle$ would be:

Apply an Hadamard to $\lvert j_2 \rangle$, giving the state $\frac{1}{\sqrt{2}}\left(\lvert 0 \rangle + e^{2 \pi i 0.j_2} \lvert 1\rangle\right) = \frac{1}{\sqrt{2}}\left(\lvert 0 \rangle + (-1)^{j_2} \lvert 1 \rangle\right)$

Apply a $CR_2$ from $\lvert j_1 \rangle$ to $\lvert j_2 \rangle$ and an Hadamard to $\lvert j_1 \rangle$, giving the state $\frac{1}{\sqrt{2}}\left(\lvert0\rangle + e^{2 \pi i[0.j_1j_2]} \lvert1\rangle\right)$

Measure the qubits in reverse order, that is $k_2$ corresponds to the first qubit, $k_1$ to the second qubit.

$$\text{2. Quantum Fourier Transform for two qubits.}$$

```python
# QISKIT: QFT code

# calculate qft of two qubit as the example above
```

###  QISKit: implement the quantum Fourier transform

####  1) Fourier transform of two qubits in the $\lvert 00 \rangle $ state

```python
from initialize import *

def qft(circ, q, n):
    """n-qubit QFT on q in circ."""
    for j in range(n):
        for k in range(j):
            circ.cu1(3.14/float(2**(j-k)), q[j], q[k])
        circ.h(q[j])

#initialize quantum program
my_alg = initialize(circuit_name = 'fourier', qubit_number=2, bit_number=2, backend = 'local_qasm_simulator', shots = 1024)

#add gates to the circuit
qft(my_alg.q_circuit, my_alg.q_reg, 2) # qft of input state
my_alg.q_circuit.measure(my_alg.q_reg[0], my_alg.c_reg[0]) # measures first qubit
my_alg.q_circuit.measure(my_alg.q_reg[1], my_alg.c_reg[1]) # measures second qubit

# print list of gates in the circuit
print('List of gates:')
for circuit in my_alg.q_circuit:
    print(circuit.name)

#Execute the quantum algorithm
result = my_alg.Q_program.execute(my_alg.circ_name, backend=my_alg.backend, shots= my_alg.shots)

#Show the results obtained from the quantum algorithm
counts = result.get_counts(my_alg.circ_name)

print('\nThe measured outcomes of the circuits are:',counts)

# credits to:  https://github.com/QISKit/qiskit-tutorial
```

    List of gates:
    h
    cu1
    h
    measure
    measure

    The measured outcomes of the circuits are: {'00': 243, '01': 241, '10': 272, '11': 268}

## 8.4 Description of the algorithm

Shor's algorithm exploits both classical and quantum computation. Classical operations are carried out in steps where an efficient classical algorithm exists, while quantum operations are used to find the periodicity of the function needed to factor $N_0$. Here we present an outline of the algorithm.

$$\text{3. Quantum circuit for Shor's algorithm.}$$

To run the algorithm, we need two quantum registers. One contains the order or period, called period register, and the other contains the results of the computation, called computational register. The size of both registers depends on the number $N_0$ to be factored. In particular, the period register should contain a number of qubits $n_p$ in the interval $2 \text{log} N_0 \leq n_p < 2 \text{log} (2N_0) $ and the computational register should be large enough to be able to represent the number $N_0-1$, thus $n_q = \text{log} (N_0-1)$.

First, one needs to check if $N_0$ is even. If $N_0$ is even, one of the factor is $2$ and the other is $N_0/2$.

If $N_0$ is odd, a base $a$ is picked randomly among the numbers from $0$ to $N_0-1$. Then, check if $a$ is a factor of $N_0$, by checking if gcd$\left(a,N_0 \right) \neq 1$. If $a$ is has a common divisor with $N_0$, then one factor is given by gcd$\left(a,N_0 \right)$ and the other by $N_0/$gcd$\left(a,N_0 \right)$. If $a$ is co-prime with $N_0$ ( gcd$\left(a,N_0 \right) = 1$ ) then one needs to compute the function $a^x$ mod $N_0$, the modular exponentiation function (MEF), for $x=1,2,3,..Q-1$ where $N_0^2 \leq Q < 2N_0^2$ and $Q=2^{n_p}$.

At the beginning of the quantum algorithm used to compute the MEF we need two registers initialized to zero $\lvert 00...0 \rangle \lvert 00...0 \rangle $. The first register also known as the period register will stores all the possible values of the exponent $x$ by creating a uniform superposition of all possible bit strings through Hadamard gates on all qubits $\frac{1}{\sqrt{Q}} \sum_{x=0}^{Q-1} \lvert x \rangle $, and the second register, called the computational register will store the results of the MEF $a^x$ mod $N_0$, $\lvert a^x $ mod $ N_0 \rangle$. Thus, after the first step, one has

\begin{equation}
	\frac{1}{\sqrt{Q}} \sum_{x=0}^{Q-1} \lvert x \rangle \lvert a^x mod N_0 \rangle
\end{equation}

Then, we apply the quantum Fourier transform (QFT) to the first register, so that $\lvert x \rangle \rightarrow \frac{1}{\sqrt{Q}} \sum_{s=0}^{Q-1} e^{\frac{2\pi i s x}{Q}}\lvert s \rangle$. As a result of the QFT, interference between all the possible states occurs and only the periodic ones survive. That is, if one measures the first register, one will see a value of $s$ such that $\frac{sx}{Q}$ is an integer $d$ when $x$ a multiple of the period $r$. Which means that $\frac{s}{Q}=\frac{d}{r}$. By knowing the fraction $\frac{s}{Q}$ one can find the value of $r$ through the continued fraction algorithm.
Now, if $r$ is odd or $r=0$, the algorithm fails and one needs to restart by picking a different base $a$. If $r$ is even, one can factorize $a^r - 1$  mod $N_0$ into $\left( a^{\frac{r}{2}} - 1\right)\left( a^{\frac{r}{2}} + 1\right)$ mod $N_0$. The final step is to check if $a^{\frac{r}{2}} + 1 \, \text{mod} \, N_0 \neq -1$. If that's true, then $\text{gcd}\left(a^{\frac{r}{2}} + 1,N_0 \right)$ will be the first factor and $\text{gcd}\left(a^{\frac{r}{2}} - 1,N_0 \right)$ the other.

The execution of this version of the algorithm requires $n=log_2 \left( N_0 \right)$ qubits in the computational register to perform the modular exponentiation and another $2n$ qubits in the period register to perform the QFT. Thus, the algorithm requires a total number of $3n$ qubits.

## 8.5 Example of Shor's factoring algorithm

Let us see here an example of Shor's factoring algorithm for $N_0=21$. Since $21$ needs five bits to be represented in binary, we need at least $\text{log}_2Q = 2*5$ qubits for factoring.

 Check if $N_0=21$ is even: $21 \, \text{mod} \, 2=1$, $N_0$ is not even.

 Pick a base $a$ at random. Let's say $a=2$.

 Check if $a$ has any common factors with $N_0$: $\text{gcd}\left(a,N_0 \right) = 1$, it doesn't.

 Initialize two qubit registers with $n_p = 2\times5=10$ qubits, as $5$ qubits are needed to represent $21$ in binary. We call the first register the $period$ register and the second register the $computational$ register
    $$ \lvert \psi \rangle = \lvert 0 \rangle^{\otimes 10}_p \lvert 0 \rangle^{\otimes 10}_c $$

 Apply Hadamard gate on all ther qubits of the first register to create a uniform superposition of all $2^{10}$ possible values
    $$ \lvert \psi \rangle =\frac{1}{\sqrt{2^{10}}} \sum_{x=0}^{2^{10}-1} \lvert x \rangle_p \lvert 0 \rangle_c $$

 Apply the modular exponentiation function $a^x \, \text{mod} \, N_0$ on the second register, for each of the stored value of $x$ in the first register.

$$ \lvert \psi \rangle =\frac{1}{\sqrt{1024}} \sum_{x=0}^{1023} \lvert x \rangle_p \lvert 2^x \, \text{mod} \, 21 \rangle_c
        = \\ = \frac{1}{\sqrt{1024}} \left( \lvert 0 \rangle_p \lvert 1 \rangle_c  + \lvert 1 \rangle_p \lvert 2 \rangle_c + \lvert 2 \rangle_p \lvert 4 \rangle_c + \lvert 3 \rangle_p \lvert 8 \rangle_c + \lvert 4 \rangle_p \lvert 16 \rangle_c  + \lvert 5 \rangle_p \lvert 11 \rangle_c + \lvert 6 \rangle_p \lvert 1 \rangle_c + \lvert 7 \rangle_p \lvert 2 \rangle_c + \lvert 8 \rangle_p \lvert 4 \rangle_c ... \right) $$
        By looking at the values stored in the second register, we can find out what is their periodicity. In particular, it can be seen that the values start repeating with order $r=6$ (value of the first register). However, we need to do a few more steps to allow the quantum computer to find the answer by itself.

 To simplify the example, we will adopt the "Principle of implicit measurement": Without loss of generality, any qubits which are not measured at the end of the quantum circuit may be assumed to be measured. Thus, let us use the principle of implicit measurement on the second register. Since each term of the superposition has equal weight, each outcome is equally likely, therefore one will see one of the following values: $ \lvert 1 \rangle_c$ , $\lvert 2 \rangle_c$, $ \lvert 4 \rangle_c$, $\lvert 8 \rangle_c$, $\lvert 16 \rangle_c$, $\lvert 11 \rangle_c$; with probability $1/6$.
     Assume that the state $ \lvert 4 \rangle_c$ is measured, then we are left with the composite state:

     $$ \lvert \psi \rangle = \frac{\sqrt{6}}{\sqrt{1024}} \left( \lvert 2 \rangle_p \lvert 4 \rangle_c + \lvert 8 \rangle_p \lvert 4 \rangle_c + \lvert 14 \rangle_p \lvert 4 \rangle_c + \lvert 20 \rangle_p \lvert 4 \rangle_c  ... \right) $$

 Apply the QFT to the first register
     $$ \lvert x \rangle_p \rightarrow \frac{1}{\sqrt{1024}} \sum_{s=0}^{1023} e^{2\pi i \frac{s x}{1024}}\lvert s \rangle_p $$
     Which transform each of the terms in the period register as:
     $$ \lvert 2 \rangle_p \rightarrow \frac{1}{\sqrt{1024}} \sum_{s=0}^{1023} e^{2\pi i \frac{2 s}{1024}}\lvert s \rangle_p $$
     $$ \lvert 8 \rangle_p \rightarrow \frac{1}{\sqrt{1024}} \sum_{s=0}^{1023} e^{2\pi i \frac{8 s}{1024}}\lvert s \rangle_p $$
     $$ \lvert 14 \rangle_p \rightarrow \frac{1}{\sqrt{1024}} \sum_{s=0}^{1023} e^{2\pi i \frac{14 s}{1024}}\lvert s \rangle_p $$
    $$ \lvert 20 \rangle_p \rightarrow \frac{1}{\sqrt{1024}} \sum_{s=0}^{1023} e^{2\pi i \frac{20 s}{1024}}\lvert s \rangle_p $$
    $$...$$

    So, we can write:
    $$ \lvert \psi \rangle = \frac{\sqrt{6}}{1024} \sum_{s=0}^{1023} \left( e^{2\pi i \frac{2s}{1024}} + e^{2\pi i \frac{8s}{1024}} + e^{2\pi i \frac{14s}{1024}} + e^{2\pi i \frac{20s}{1024}}  ... \right)  \lvert s \rangle_p \lvert 4 \rangle_c  = \\
    = \frac{\sqrt{6}}{1024} \sum_{s=0}^{1023} e^{2\pi i \frac{2s}{1024}} \left( 1 + e^{2\pi i \frac{6s}{1024}} + e^{2\pi i \frac{12s}{1024}} + e^{2\pi i \frac{18s}{1024}}  ... \right)  \lvert s \rangle_p \lvert 4 \rangle_c$$

 Measure the period register. The probability of finding a certain value $\lvert s \rangle_p$ is:
    $$ P(s) = \lvert \frac{\sqrt{6}}{1024} e^{2\pi i \frac{2s}{1024}} \left( 1 + e^{2\pi i \frac{6s}{1024}} + e^{2\pi i \frac{12s}{1024}} + e^{2\pi i \frac{18s}{1024}}  ... \right) \rvert^2 $$

    Because of the possible sign difference between all different terms, the values of $s$ which have the highest likelyhood to be observed are the ones for which the phase terms all have the same sign and they add up. That is, $\frac{s}{1024}=\frac{d}{6}$, where $d$ is an integer. Therefore, a value $s=\frac{1024 \cdot d}{6}$ where $d=1,2,3,...$ will most likely be observed.
    Let us assume that the value $s=853$ is measured.

 To find the period from the value of $s$ measured, one then uses the continued fraction algorithm in the following way: we find the fraction $\frac{d}{r}$ which approximates the fraction $\frac{s}{Q}$ to a fixed precision $2*Q$
    $$\lvert \frac{853}{1024} - \frac{d}{r} \rvert < \frac{1}{2048} $$
    So, let's find $d$ and $r$ with the continued fraction algorithm:
    $$ \frac{853}{1024} = 0 + \frac{1}{1 + \frac{1}{4+ \frac{1}{1+\frac{1}{1+\frac{1}{1+\frac{1}{84+\frac{1}{2}}}}}}} $$
   which gives as possible fractions $\frac{d}{r}$: $\frac{1}{1}$, $\frac{4}{5}$, $\frac{5}{6}$, ...

   imposing the condition written above, we find that the only fraction satisfying it is $\frac{d}{r} = \frac{5}{6}$. Therefore, the period is $r=6$!

 Once the period has been found, we can find the factors of $N_0=21$ almost immediately. First check if $r$ is even then check if $a^{\frac{r}{2}} + 1 \, \text{mod} \, N_0 \neq -1 $ :
    $$ 6 \, \text{mod} \, 2 = 0 $$
    $$  2^{\frac{6}{2}} + 1 \, \text{mod} \, 21 = 9$$
    Therefore the two factors are: $ p = \text{gcd}\left(a^{\frac{r}{2}} + 1,N_0 \right)$ and $q = \text{gcd}\left(a^{\frac{r}{2}} - 1,N_0 \right)$
    $$p = \text{gcd}\left(9,21 \right) = 3$$
    $$q = \text{gcd}\left(7,21 \right) = 7$$

## Exercises

Calculate the following modular expression:

$ 54 \, \text{(mod 5)}$

$ 54 \times 35 \, \text{(mod 17)}$

$ 9^{2} \, \text{(mod 6 )}$

Rewrite the following fractions using the continued fraction algorithm

$ \frac{45}{251}$

$ \frac{71}{385}$

$ \frac{512}{1027}$

What is the minimum number of qubits needed to factor:

$ 15 $

$ 4,367,398 $

Derive the expression of the Quantum Fourier Transform of three qubits, similarly to how it is done in the example shown in Section 8.3

Following the steps highlighted in the example given in Section 8.5, show the factoring of $N_0=15$ with $a=2$.

Write a QISKit program that calculates the QFT of three qubits in the state $\lvert 010 \rangle$ .

## References

[1] P. Shor, in Proc. 35th Annu. Symp. on the Foundations of
Computer Science, edited by S. Goldwasser (IEEE Computer
Society Press, Los Alamitos, California, 1994),
p. 124-134.

[2] L. M. K. Vandersypen, M. Steffen, G. Breyta, C. S. Yannoni,
M. H. Sherwood, and I. L. Chuang, Nature 414,
883 (2001).

[3] Monz, T. et al., Science 351, 1068-1070 (2016).

[4] C. Y. Lu, D. E. Browne, T. Yang, and J. W. Pan, Physical
Review Letters 99, 250504 (2007).

[5] E. Martin-Lopez, A. Laing, T. Lawson, R. Alvarez, X.-Q.
Zhou, and J. L. O'Brien, Nat Photon advance online publication (2012).

[6] Lanyon, B. P. et al., Phys. Rev. Lett. 99, 250505 (2007).

[7] A. Politi, J. C. F. Matthews, and J. L. O' Brien, Science 25, 1221 (2009).

[8] E. Lucero, R. Barends, Y. Chen, J. Kelly, M. Mariantoni, A. Megrant, P. O'Malley, D. Sank, A. Vainsencher, J. Wenner, T. White, Y. Yin, A. N. Cleland and John M. Martinis, Nature Physics volume 8, 719-723 (2012).

[9] P. J. Coles et al. (2018), arXiv:1804.03719.

[10] J. A. Smolin, G. Smith, and A. Vargo, Nature (London) 499, 163-165 (2013).