|
Name
               |
Date
 |
| ---------------------------------------------------| ------------------------------------- |
|
Diaaeldin SHALABY
   | 07.052021                            |

Hands-on AI II

Unit 2 — The Vanishing Gradient Problem (Assignment)

Authors: S. Lehner, J. Brandstetter, B. Schäfl

Date: 16-04-2021

This file is part of the "Hands-on AI II" lecture material. The following copyright statement applies to all code within this file.

Copyright statement:

This  material,  no  matter  whether  in  printed  or  electronic  form,  may  be  used  for personal  and non-commercial educational use only.  Any reproduction of this manuscript, no matter whether as a whole or in parts, no matter whether in printed or in electronic form, requires explicit prior acceptance of the authors.

Table of contents

Definition of Auxiliaries

Loading and visualizing

Downprojecting and interpreting

Loading and preparing

Training of a Neural Network

Constructing an FNN

Forward pass

Backward pass

Analyzing Gradients

Collecting and visualizing

Countermeasure and re-train

Comparing gradients

Deriving Derivatives

Case <tt>hardsigmoid</tt>

Case <tt>leaky_relu</tt>

How to use this notebook

This notebook is designed to run from start to finish. There are different tasks (displayed in orange boxes) which require your contribution (in form of code, plain text, ...). Most/All of the supplied functions are imported from the file u2_utils.py which can be seen and treated as a black box. However, for further understanding, you can look at the implementations of the helper functions. In order to run this notebook, the packages which are imported at the beginning of u2_utils.py need to be installed.

```python
# Import pre-defined utilities specific to this notebook.
import u2_utils as u2

# Import additional utilities needed in this notebook.
import math
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
import torch

from typing import Dict, Sequence

# Setup Jupyter notebook (warning: this may affect all Jupyter notebooks running on the same Jupyter server).
u2.setup_jupyter()
```

Setting up notebook ... finished.

Module versions

As mentioned in the introductiory slides, specific minimum versions of Python itself as well as of used modules is recommended.

```python
u2.check_module_versions()
```

    Installed Python version: 3.8 (✓)
    Installed numpy version: 1.19.1 (✓)
    Installed pandas version: 1.1.3 (✓)
    Installed PyTorch version: 1.7.1 (✓)
    Installed scikit-learn version: 0.23.2 (✓)
    Installed scipy version: 1.5.0 (✓)
    Installed matplotlib version: 3.3.1 (✓)
    Installed seaborn version: 0.11.0 (✓)
    Installed PIL version: 8.0.0 (✓)

Definition of Auxiliaries

In this exercise you will be working with a data set composed of images of various handwritten digits. It is probably the most prominent data set in the domain of machine learning: the MNIST data set. The data set distinguishes ten different classes, one for each digit (zero to nine). For curious minds, more information regarding this data set can be found at:

    LeCun, Y., 1998. The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/.

Before analyzing and tackling the vanishing gradient problem, the data sets needs to be inspected.

    Execute the notebook until here and try to solve the following tasks:

Load the MNIST data set using the appropriate function as supplied by us.

Divide the data set between the training set and the test set in a ratio of $7:1$.

Visualize the MNIST training set in tabular form. What is the size of both subsets with respect to sample and feature counts?

```python
data_mnist = u2.load_mnist()
data_mnist
```

      PX_0
      PX_1
      PX_2
      PX_3
      PX_4
      PX_5
      PX_6
      PX_7
      PX_8
      PX_9
      ...
      PX_775
      PX_776
      PX_777
      PX_778
      PX_779
      PX_780
      PX_781
      PX_782
      PX_783
      digit

      0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      5

      1
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0

      2
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      4

      3
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1

      4
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      9

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      69995
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      2

      69996
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      3

      69997
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      4

      69998
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      5

      69999
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      6

70000 rows × 785 columns

```python
# Set default plotting style and random seed for reproducibility.
sns.set()
np.random.seed(seed=42)

# Split the Fashion-MNIST data set into training as well as test set and print their respective size.
data_mnist_train, data_mnist_test = u2.split_data(data_mnist, test_size=1.0 / 8.0)
print(f'{"Full data set is of size:":>27} {data_mnist.shape[0]:>5}')
print(f'Training subset is of size: {data_mnist_train.shape[0]:>5}')
print(f'{"Testing subset is of size:":>27} {data_mnist_test.shape[0]:>5}')
```

      Full data set is of size: 70000
    Training subset is of size: 61250
     Testing subset is of size:  8750

    Execute the notebook until here and try to solve the following tasks:

Reduce the dimensionality of the MNIST training set using PCA and visualize the downprojection.

Comment on the separability of the MNIST training set with respect to the downprojection.

```python
# Set default plotting style and random seed for reproducibility.
sns.set()
np.random.seed(seed=42)

data_mnist_pca = u2.apply_pca(data=data_mnist,n_components=2)
u2.plot_points_2d(data=data_mnist_pca, figsize=(14, 7))
```

    Execute the notebook until here and try to solve the following tasks:

Create a corresponding <tt>TensorDataset</tt> for the training as well as the test set.

Wrap the previously defined <tt>TensorDataset</tt> instances in separate <tt>DataLoader</tt> instances with a batch size of $64$ (shuffle the training data set).

Scale the features of the training as well as test set by a factor of $1\,/\,255$.

```python
# Set random seed for reproducibility.
np.random.seed(seed=42)
torch.manual_seed(seed=42)

# Create data loader for iterating the Fashion-MNIST training data set.
loader_mnist_train = torch.utils.data.DataLoader(torch.utils.data.TensorDataset(
    torch.from_numpy(data_mnist_train.drop(columns=[r'digit']).values.astype(
        dtype=np.float32) / 255.0).unsqueeze(1).reshape(len(data_mnist_train), 28 * 28),
    torch.from_numpy(data_mnist_train[r'digit'].values.astype(dtype=np.long))
), batch_size=64, shuffle=True, drop_last=False)

# Create data loader for iterating the Fashion-MNIST test data set.
loader_mnist_test = torch.utils.data.DataLoader(torch.utils.data.TensorDataset(
    torch.from_numpy(data_mnist_test.drop(columns=[r'digit']).values.astype(
        dtype=np.float32) / 255.0).unsqueeze(1).reshape(len(data_mnist_test), 28 * 28),
    torch.from_numpy(data_mnist_test[r'digit'].values.astype(dtype=np.long))
), batch_size=64, shuffle=False, drop_last=False)
```

    The following code snippet is taken from the accompanying exercise notebook. You do not need to modify it for this assignment.

```python
def train_and_evaluate(model: torch.nn.Module, optimizer: torch.optim.Optimizer,
                       device: torch.device, num_epochs: int,
                       loader_train: torch.utils.data.DataLoader,
                       loader_test: torch.utils.data.DataLoader) -> None:
    """
    Auxiliary function for training and evaluating a corresponding model.

    :param model: model instance to train and evaluate
    :param optimizer: optimizer to use for model training
    :param device: device to use for model training and evaluation
    :param num_epochs: amount of epochs for model training
    :param loader_train: data loader supplying the training samples
    :param loader_test: data loader supplying the test samples
    :return: None
    """
    for epoch in range(num_epochs):

        # Train model instance for one epoch.
        u2.train_network(
            model=model, data_loader=loader_train, device=device, optimizer=optimizer)

        # Evaluate current model instance.
        performance = u2.test_network(
            model=model, data_loader=loader_train, device=device)

        # Print result of current epoch to standard out.
        print(f'Epoch: {str(epoch + 1).zfill(len(str(num_epochs)))} ' +
              f'/ Loss: {performance[0]:.4f} / Accuracy: {performance[1]:.4f}')

    # Evaluate final model on test data set.
    performance = u2.test_network(model=model, data_loader=loader_test, device=device)
    print(f'\nFinal loss: {performance[0]:.4f} / Final accuracy: {performance[1]:.4f}')
```

Training of a Neural Network

Loading and inspecting a new data set is always an exciting moment, but even more exciting is the implementation of a corresponding neural network and applying it to said data set. Hence, in this section you will have to implement and train an appropriate neural network model and revisit your knowledge about the forward as well as the backward pass.

    Execute the notebook until here and try to solve the following tasks:

Implement a class <tt>FNN_0</tt> with the following architecture:

        Position
        Element
        Comment

            0
            input
            -

            1
            fully connected
            square weight matrix

            2
            sigmoid
            -

            2
            fully connected
            square weight matrix

            3
            sigmoid
            -

            4
            fully connected
            square weight matrix

            5
            sigmoid
            -

            6
            fully connected
            square weight matrix

            7
            sigmoid
            -

            8
            fully connected
            square weight matrix

            9
            sigmoid
            -

            10
            fully connected
            $10$ output features

            11
            output
            -

Create an instance of <tt>FNN_0</tt> as well as of a corresponding <tt>Adam</tt> optimizer with a learning rate of $0.0001$.

Print the resulting model and verify the architecture by inspecting the output.

Train an <tt>FNN_0</tt> network for $5$ epochs, print the training accuracy as well as the loss per epoch and report the final test set loss and accuracy.

```python
class FNN_0 (torch.nn.Module):

    def __init__(self):
        super(FNN_0, self).__init__()
        self.fc1 = torch.nn.Linear(28 * 28, 28 * 28)
        self.ac1 = torch.nn.Sigmoid()
        self.fc2 = torch.nn.Linear(self.fc1.out_features, self.fc1.out_features)
        self.ac2 = torch.nn.Sigmoid()
        self.fc3 = torch.nn.Linear(self.fc2.out_features, self.fc2.out_features)
        self.ac3 = torch.nn.Sigmoid()
        self.fc4 = torch.nn.Linear(self.fc3.out_features, self.fc3.out_features)
        self.ac4 = torch.nn.Sigmoid()
        self.fc5 = torch.nn.Linear(self.fc4.out_features, self.fc4.out_features)
        self.ac5 = torch.nn.Sigmoid()

        self.fc6 = torch.nn.Linear(self.fc5.out_features, 10)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        x = self.fc1(x)
        x = self.ac1(x)
        x = self.fc2(x)
        x = self.ac2(x)
        x = self.fc3(x)
        x = self.ac3(x)
        x = self.fc4(x)
        x = self.ac4(x)
        x = self.fc5(x)
        x = self.ac5(x)
        return self.fc6(x)
```

```python
# Set random seed for reproducibility.
np.random.seed(seed=42)
torch.manual_seed(seed=42)

# Create FNN_0 instance and the corresponding optimizer to use.
target_device = torch.device(r'cpu')
fnn_model = FNN_0().to(target_device)
optimizer = torch.optim.Adam(fnn_model.parameters(), lr=1e-4)

# Show the architecture of the FNN_0 model.
print(fnn_model)
```

    FNN_0(
      (fc1): Linear(in_features=784, out_features=784, bias=True)
      (ac1): Sigmoid()
      (fc2): Linear(in_features=784, out_features=784, bias=True)
      (ac2): Sigmoid()
      (fc3): Linear(in_features=784, out_features=784, bias=True)
      (ac3): Sigmoid()
      (fc4): Linear(in_features=784, out_features=784, bias=True)
      (ac4): Sigmoid()
      (fc5): Linear(in_features=784, out_features=784, bias=True)
      (ac5): Sigmoid()
      (fc6): Linear(in_features=784, out_features=10, bias=True)
    )

```python
# Set random seed for reproducibility.
np.random.seed(seed=42)
torch.manual_seed(seed=42)

# Train and evaluate FNN_0 instance on the MNIST training set.
train_and_evaluate(
    model=fnn_model,
    optimizer=optimizer,
    device=target_device,
    num_epochs=5,
    loader_train=loader_mnist_train,
    loader_test=loader_mnist_test)
```

    Epoch: 1 / Loss: 0.0149 / Accuracy: 0.6675
    Epoch: 2 / Loss: 0.0089 / Accuracy: 0.8292
    Epoch: 3 / Loss: 0.0067 / Accuracy: 0.8728
    Epoch: 4 / Loss: 0.0056 / Accuracy: 0.8949
    Epoch: 5 / Loss: 0.0053 / Accuracy: 0.8997

    Final loss: 0.0055 / Final accuracy: 0.8951

    Execute the notebook until here and try to solve the following tasks:

Write down a formula for the corresponding forward pass of <tt>FNN_0</tt>. Use the same notation as presented during the exercise.

    \begin{equation}
        \hat{y} = f\left(h_6(h_5(h_4(h_3(h_2(h_1(\mathbf{x};\mathbf{W}_1);\mathbf{W}_2);\mathbf{W}_3);\mathbf{W}_4);\mathbf{W}_5);\mathbf{W}_6 \right )
    \end{equation}

    Execute the notebook until here and try to solve the following tasks:

Write down a formula for the corresponding backward pass of <tt>FNN_0</tt>. Use the same notation as presented during the exercise.

    \begin{align*}
        \mathbf{W}_6 & \leftarrow \mathbf{W}_6 - \eta \frac{\partial L}{\partial \mathbf{W}_5} \\
        \mathbf{W}_5 & \leftarrow \mathbf{W}_5 - \eta \frac{\partial L}{\partial h_5}\frac{\partial h_5}{\partial \mathbf{W}_5} \\
        \mathbf{W}_4 & \leftarrow \mathbf{W}_4 - \eta \frac{\partial L}{\partial h_5}\frac{\partial h_5}{\partial h_4}\frac{\partial h_4}{\partial \mathbf{W}_4} \\
        \mathbf{W}_3 & \leftarrow \mathbf{W}_3 - \eta \frac{\partial L}{\partial h_5}\frac{\partial h_5}{\partial h_4}\frac{\partial h_4}{\partial h_3}\frac{\partial h_3}{\partial \mathbf{W}_3} \\
        \mathbf{W}_2 & \leftarrow \mathbf{W}_2 - \eta \frac{\partial L}{\partial h_5}\frac{\partial h_5}{\partial h_4}\frac{\partial h_4}{\partial h_3}\frac{\partial h_3}{\partial h_2}\frac{\partial h_2}{\partial \mathbf{W}_2} \\
        \mathbf{W}_1 & \leftarrow \mathbf{W}_1 - \eta \frac{\partial L}{\partial h_5}\frac{\partial h_5}{\partial h_4}\frac{\partial h_4}{\partial h_3}\frac{\partial h_3}{\partial h_2}\frac{\partial h_2}{\partial h_1}\frac{\partial h_1}{\partial \mathbf{W}_1} \\
    \end{align*}

Analyzing Gradients

Actually, the results of <tt>FNN_0</tt> do not look that bad. Is there really a problem with a vanishing gradient? This is exactly the point you're going to figure out in this exercise. As a first step, the gradients of a freshly initialized model needs to be collected and analyzed. Afterwards, in case of a vanishing gradient problem, countermeasures need to get deployed.

    The following code snippet is taken from the accompanying exercise notebook. You do not need to modify it for this assignment.

```python
def collect_gradients(model: torch.nn.Module, device: torch.device,
                      loader: torch.utils.data.DataLoader) -> Sequence[Dict[str, np.array]]:
    """
    Auxiliary function for collecting gradients of a corresponding model.

    :param model: model instance to be used for collecting gradients
    :param device: device to use for gradient collection
    :param loader: data loader supplying the samples used for collecting gradients
    :return: sequence of parameter names and gradients, averaged over all parameter elements
    """
    model_state = model.training
    model.train()
    model.zero_grad()

    # Iterating over the data set and computing the corresponding gradients.
    gradients = {}
    criterion = torch.nn.CrossEntropyLoss()
    for batch_index, (data, target) in enumerate(loader):
        data, target = data.float().to(device), target.long().to(device)
        optimizer.zero_grad()
        output = model(data)
        loss = criterion(output, target)
        loss.backward()

        # Collecting the gradients from the current model.
        for name, parameter in model.named_parameters():
            if parameter.grad is not None:
                gradients.setdefault(name, []).append(parameter.grad.view(-1).abs().mean().item())
        model.zero_grad()

    # Reset model state and return collected gradients.
    model.train(mode=model_state)
    return gradients
```

    Execute the notebook until here and try to solve the following tasks:

Create a fresh instance of <tt>FNN_0</tt> and collect its gradients using the MNIST training set.

Visualize the gradients of each weight parameter accordingly.

Do the gradients vanish?

```python
fnn_model = FNN_0()
gradient = collect_gradients(
    model=fnn_model, device=target_device, loader=loader_mnist_train
)
```

```python
# Set default plotting style.
sns.set()

# Prepare collected gradients for plotting.

gradient_data = pd.DataFrame([
    v for k, v in sorted(gradient.items(), key=lambda _: _[0]) if r'weight' in k
]).transpose().rename(columns=lambda _: f'Layer {_}')
gradient_data = pd.melt(gradient_data, value_vars=gradient_data.columns)
gradient_data[r'Model'] = type(fnn_model).__name__

# Combine all gradients in a single data frame.
gradient_data = gradient_data.rename(
    columns={r'variable': r'Layer', r'value': r'Gradient Magnitude'})

# Define plotting figure and corresponding attributes.
fig, ax = plt.subplots(1, 1, figsize=(18, 7))
ax.set(yscale=r'log')

# Plot pre-processed gradients.
_ = sns.boxplot(x=r'Model', y=r'Gradient Magnitude', hue=r'Layer', data=gradient_data, ax=ax)
```

As shown by the graph, the gradient does vanish.

    Execute the notebook until here and try to solve the following tasks:

Assume a vanishing gradient. Apply the countermeasure presented during the accompanying exercise by implementing a corresponding <tt>FNN_1</tt>.

Create an instance of <tt>FNN_1</tt> as well as of a corresponding <tt>Adam</tt> optimizer with a learning rate of $0.0001$.

Print the resulting model and verify the architecture by inspecting the output.

Train an <tt>FNN_1</tt> network for $5$ epochs, print the training accuracy as well as the loss per epoch and report the final test set loss and accuracy.

```python
class FNN_1 (torch.nn.Module):

    def __init__(self):
        super(FNN_1, self).__init__()
        self.fc1 = torch.nn.Linear(28 * 28, 28 * 28)
        self.ac1 = torch.nn.ReLU()
        self.fc2 = torch.nn.Linear(self.fc1.out_features, self.fc1.out_features)
        self.ac2 = torch.nn.ReLU()
        self.fc3 = torch.nn.Linear(self.fc2.out_features, self.fc2.out_features)
        self.ac3 = torch.nn.ReLU()
        self.fc4 = torch.nn.Linear(self.fc3.out_features, self.fc3.out_features)
        self.ac4 = torch.nn.ReLU()
        self.fc5 = torch.nn.Linear(self.fc4.out_features, self.fc4.out_features)
        self.ac5 = torch.nn.ReLU()

        self.fc6 = torch.nn.Linear(self.fc5.out_features, 10)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        x = self.fc1(x)
        x = self.ac1(x)
        x = self.fc2(x)
        x = self.ac2(x)
        x = self.fc3(x)
        x = self.ac3(x)
        x = self.fc4(x)
        x = self.ac4(x)
        x = self.fc5(x)
        x = self.ac5(x)
        return self.fc6(x)
```

```python
# Set random seed for reproducibility.
np.random.seed(seed=42)
torch.manual_seed(seed=42)

# Create FNN_1 instance and the corresponding optimizer to use.
target_device = torch.device(r'cpu')
fnn_model = FNN_1().to(target_device)
optimizer = torch.optim.Adam(fnn_model.parameters(), lr=1e-4)

# Show the architecture of the FNN_1 model.
print(fnn_model)
```

    FNN_1(
      (fc1): Linear(in_features=784, out_features=784, bias=True)
      (ac1): ReLU()
      (fc2): Linear(in_features=784, out_features=784, bias=True)
      (ac2): ReLU()
      (fc3): Linear(in_features=784, out_features=784, bias=True)
      (ac3): ReLU()
      (fc4): Linear(in_features=784, out_features=784, bias=True)
      (ac4): ReLU()
      (fc5): Linear(in_features=784, out_features=784, bias=True)
      (ac5): ReLU()
      (fc6): Linear(in_features=784, out_features=10, bias=True)
    )

```python
# Set random seed for reproducibility.
np.random.seed(seed=42)
torch.manual_seed(seed=42)

# Train and evaluate FNN_1 instance on the MNIST training set.
train_and_evaluate(
    model=fnn_model,
    optimizer=optimizer,
    device=target_device,
    num_epochs=5,
    loader_train=loader_mnist_train,
    loader_test=loader_mnist_test)
```

    Epoch: 1 / Loss: 0.0032 / Accuracy: 0.9373
    Epoch: 2 / Loss: 0.0016 / Accuracy: 0.9679
    Epoch: 3 / Loss: 0.0009 / Accuracy: 0.9820
    Epoch: 4 / Loss: 0.0006 / Accuracy: 0.9877
    Epoch: 5 / Loss: 0.0009 / Accuracy: 0.9813

    Final loss: 0.0020 / Final accuracy: 0.9635

    Execute the notebook until here and try to solve the following tasks:

Create a fresh instance of <tt>FNN_1</tt> and collect its gradients using the MNIST training set.

Visualize the gradients of each weight parameter accordingly (include the gradient visualization of <tt>FNN_0</tt>).

Do the gradients vanish?

```python
fnn_model = FNN_1()
gradient = collect_gradients(
    model=fnn_model, device=target_device, loader=loader_mnist_train
)
```

```python
# Set default plotting style.
sns.set()

# Prepare collected gradients for plotting.

gradient_data = pd.DataFrame([
    v for k, v in sorted(gradient.items(), key=lambda _: _[0]) if r'weight' in k
]).transpose().rename(columns=lambda _: f'Layer {_}')
gradient_data = pd.melt(gradient_data, value_vars=gradient_data.columns)
gradient_data[r'Model'] = type(fnn_model).__name__

# Combine all gradients in a single data frame.
gradient_data = gradient_data.rename(
    columns={r'variable': r'Layer', r'value': r'Gradient Magnitude'})

# Define plotting figure and corresponding attributes.
fig, ax = plt.subplots(1, 1, figsize=(18, 7))
ax.set(yscale=r'log')

# Plot pre-processed gradients.
_ = sns.boxplot(x=r'Model', y=r'Gradient Magnitude', hue=r'Layer', data=gradient_data, ax=ax)
```

As shown by the graph above, diffrence between the gradient magnitude of each layer are much smaller now after applying ReLU activation layers instead of Sigmoid. Also the accuracies after each epoch and the final accuraccy have improved drastecally. Hence, we could say that this technique solved or at least minimized the vanishing gradient problem.

Deriving Derivatives

It is already known from the lecture as well as the exercise that activation functions are the primary culprit of the Vanishing Gradient Problem. Hence, it is important to know how the chosen activation functions activate the input and consequently what the derivative is.

    Execute the notebook until here and try to solve the following tasks (hint: have a look at the official PyTorch documentation):

Implement the <tt>hardsigmoid</tt> activation function as it was done for <tt>relu</tt> in the exercise.

Implement the derivative of the <tt>hardsigmoid</tt> activation function accordingly.

Find $3$ different inputs showing the value range of the <tt>hardsigmoid</tt> activation function.

Plot the <tt>hardsigmoid</tt> activation function indluding its derivative for the input range $[-6; 6]$.

```python
def hardsigmoid(x: float) -> float:
    """
    Compute the hardsigmoid function.

    :param x: the input on which to apply the hardsigmoid function
    :return: the result of the hardsigmoid function applied to its input
    """
    if x <= -3:
        return 0
    elif x >= 3:
        return 1
    else:
        return (x/6)+0.5

def hardsigmoid_d(x: float) -> float:
    """
    Compute the derivate of the hardsigmoid function.

    :param x: the input to the hardsigmoid function for computing its derivative
    :return: the derivative of the hardsigmoid function with respect to its input
    """
    if x <= -3:
        return 0
    elif x >= 3:
        return 0
    else:
        return (1/6)

# Crudly check the value range of the sigmoid function and its derivative.
print(f'hardsigmoid(-10): {hardsigmoid(-10.0):.4f} | hardsigmoid\'(-10): {hardsigmoid_d(-10.0):.4f}')
print(f'hardsigmoid(  0): {hardsigmoid(0.0):.4f} | hardsigmoid\'(  0): {hardsigmoid_d(0.0):.4f}')
print(f'hardsigmoid(+10): {hardsigmoid(10.0):.4f} | hardsigmoid\'(+10): {hardsigmoid_d(10.0):.4f}')
```

    hardsigmoid(-10): 0.0000 | hardsigmoid'(-10): 0.0000
    hardsigmoid(  0): 0.5000 | hardsigmoid'(  0): 0.1667
    hardsigmoid(+10): 1.0000 | hardsigmoid'(+10): 0.0000

```python
def plot_hardsigmoid_with_derivative(x_min: float = -6.0, x_max: float = 6.0, granularity: int = 100) -> None:
    """
    Plot the hardsigmoid function including its derivative.

    :param x_min: minimum value of the input value range
    :param x_max: maximum value of the input value range
    :param granularity: granularity controlling the stepsize of the input value range
    :return: None
    """
    data = np.linspace(x_min, x_max, granularity)

    fig, ax = plt.subplots(figsize=(14, 7))
    ax.spines[r'left'].set_position(r'center')
    ax.spines[r'right'].set_color(None)
    ax.spines[r'top'].set_color(None)

    plt.plot(data, tuple(map(hardsigmoid, data)), color=r'#307EC7', linewidth=3, label=r'hardsigmoid')
    plt.plot(data, tuple(map(hardsigmoid_d, data)), color=r'#accbe8', linewidth=3, label=r"hardsigmoid_d'")
    plt.locator_params(axis=r'y', nbins=6)
    plt.title(r'Hard Sigmoid function', fontsize=20)
    plt.legend(prop={'size': 15})
    plt.show()
```

```python
# Set default plotting style.
sns.set()

# Plot logistic function including its derivative.
plot_hardsigmoid_with_derivative()
```

    Execute the notebook until here and try to solve the following tasks (hint: have a look at the official PyTorch documentation):

Implement the <tt>leaky_relu</tt> activation function as it was done for <tt>relu</tt> in the exercise. Use a negative slope of $0.37$.

Implement the derivative of the <tt>leaky_relu</tt> activation function accordingly.

Find $3$ different inputs showing the value range of the <tt>leaky_relu</tt> activation function.

Plot the <tt>leaky_relu</tt> activation function indluding its derivative for the input range $[-6; 6]$.

```python
def leakyReLU(x: float) -> float:
    """
    Compute the leakyReLU function.

    :param x: the input on which to apply the leakyReLU function
    :return: the result of the leakyReLU function applied to its input
    """
    return x if x >= 0 else 0.01 * x

def leakyReLU_d(x: float) -> float:
    """
    Compute the derivate of the leakyReLU function.

    :param x: the input to the leakyReLU function for computing its derivative
    :return: the derivative of the leakyReLU function with respect to its input
    """
    return 1 if x >= 0 else 0.01

# Crudly check the value range of the sigmoid function and its derivative.
print(f'leakyReLU(-10): {hardsigmoid(-10.0):.4f} | leakyReLU\'(-10): {hardsigmoid_d(-10.0):.4f}')
print(f'leakyReLU(  0): {hardsigmoid(0.0):.4f} | leakyReLU\'(  0): {hardsigmoid_d(0.0):.4f}')
print(f'leakyReLU(+10): {hardsigmoid(10.0):.4f} | leakyReLU\'(+10): {hardsigmoid_d(10.0):.4f}')
```

    leakyReLU(-10): 0.0000 | leakyReLU'(-10): 0.0000
    leakyReLU(  0): 0.5000 | leakyReLU'(  0): 0.1667
    leakyReLU(+10): 1.0000 | leakyReLU'(+10): 0.0000

```python
def plot_leakyReLU_with_derivative(x_min: float = -6.0, x_max: float = 6.0, granularity: int = 100) -> None:
    """
    Plot the leakyReLU function including its derivative.

    :param x_min: minimum value of the input value range
    :param x_max: maximum value of the input value range
    :param granularity: granularity controlling the stepsize of the input value range
    :return: None
    """
    data = np.linspace(x_min, x_max, granularity)

    fig, ax = plt.subplots(figsize=(14, 7))
    ax.spines[r'left'].set_position(r'center')
    ax.spines[r'right'].set_color(None)
    ax.spines[r'top'].set_color(None)

    plt.plot(data, tuple(map(leakyReLU, data)), color=r'#307EC7', linewidth=3, label=r'leakyReLU')
    plt.plot(data, tuple(map(leakyReLU_d, data)), color=r'#accbe8', linewidth=3, label=r"leakyReLU_d'")
    plt.locator_params(axis=r'y', nbins=6)
    plt.title(r'Leaky ReLU function', fontsize=20)
    plt.legend(prop={'size': 15})
    plt.show()
```

```python
# Set default plotting style.
sns.set()

# Plot logistic function including its derivative.
plot_leakyReLU_with_derivative()
```


```python
from IPython.display import HTML

HTML('''
<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code for QISKit exercises."></form>''')
```

<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code for QISKit exercises."></form>

# 9. Quantum cryptography

The advent of quantum computation, which introduces the possibility of using quantum mechanics for information processing, gave rise to the following question: can quantum information be shared more securely than classical information?

In 1982, a very interesting property of quantum states was discovered [1,2]. This is the so-called "no-cloning theorem", which proved how the laws of quantum mechanics prohibit the copy of an unknown quantum state. Therefore, the no-cloning theorem assures us that qubits can hide the quantum information better than classical bits.

This has important implication for example for secure communications, where it allows for the sharing of private keys which cannot be eavesdropped by a third party. We consider the first protocol, the BB84 protocol, which exploits the quantum mechanical properties of qubits for secure exchange of a secret key between two parties.

## 9.1 No-cloning theorem

Let us prove the no-cloning theorem, the fact that an unknown quantum state cannot be copied.
First let us clearly state our problem:

We have a qubit in an unknown quantum state $\lvert \psi \rangle$ and we wish to copy his state on another qubit initilized to the state $\lvert s \rangle$. Therefore, we want to implement the following quantum gate:

\begin{equation}
U\lvert \psi \rangle \lvert s \rangle =\lvert \psi \rangle \lvert \psi \rangle
\end{equation}

Let us take the unknown quantum state to be

\begin{equation}
\lvert \psi \rangle  =\alpha \lvert 0 \rangle + \beta \lvert 1 \rangle
\end{equation}

where the amplitudes $\alpha$ and $\beta$ are unknown.
Therefore we have:

\begin{equation}
U\lvert \psi \rangle \lvert s \rangle =\lvert \psi \rangle \lvert \psi \rangle = (\alpha \lvert 0\rangle +\beta \lvert 1\rangle) (\alpha \lvert 0\rangle +\beta \lvert 1\rangle) = (\alpha^2 \lvert 0\rangle \lvert 0\rangle + \alpha \beta \lvert 0\rangle \lvert 1\rangle +   \beta \alpha \lvert 1 \rangle \lvert 0\rangle + \beta^2 \lvert 1\rangle \lvert 1\rangle)
\tag{1}
\end{equation}

Because of the linearity of operators, we can equivalently write:

\begin{equation}
U\lvert \psi \rangle \lvert s \rangle = U(\alpha \lvert 0\rangle + \beta \lvert 1\rangle )\lvert s\rangle = U(\alpha \lvert 0\rangle \lvert s\rangle + \beta \lvert 1\rangle \lvert s\rangle )=\alpha \lvert 00\rangle +\beta \lvert 11\rangle
\tag{2}
\end{equation}

Comparing Eqs. (1) and (2), one can see that we come to a contraddiction! Thus, the operation $U$ which copies an unknown quantum state of a qubit onto another qubit is not possible.

## 9.2 BB84 protocol

$$\text{1. BB84 protocol overview.}$$

In Ref. [3], the first protocol for the distribution of a secret quantum key between two parties is described.

First, let us assume that Alice and Bob may exchange qubits and classical information. Also, Alice can prepare a qubit in the $\lvert 0 \rangle$, $\lvert 1 \rangle$, $\lvert + \rangle = \frac{1}{\sqrt{2}} \left( \lvert 0 \rangle + \lvert 1 \rangle\right)$ and $\lvert - \rangle = \frac{1}{\sqrt{2}} \left( \lvert 0 \rangle - \lvert 1 \rangle\right)$ state, and Bob can measure in the standard (Z) $\left\{ \lvert 0 \rangle, \lvert 1 \rangle \right\}$ basis and in the Hadamard (H) $\left\{ \lvert + \rangle, \lvert - \rangle \right\} $ basis. Note that the two bases are non-orthogonal with respect to each other. Measuring in the $\left\{ \lvert + \rangle, \lvert - \rangle \right\} $ basis means that before the standard measurement in the $\left\{ \lvert 0 \rangle, \lvert 1 \rangle \right\} $ basis, Bob applies the Hadamard gate to the qubit. Thus

\begin{equation}
\lvert + \rangle =\frac{1}{\sqrt{2}}(\lvert 0\rangle +\lvert 1\rangle )
\end{equation}

gives $\lvert 0 \rangle$ when measured in the Hadamard basis, and

\begin{equation}
\lvert - \rangle =\frac{1}{\sqrt{2}}(\lvert 0\rangle -\lvert 1\rangle )
\end{equation}

gives $\lvert 1 \rangle$ when measured in the Hadamard basis.

The protocol then works in the following way. Alice picks the bit that she wants to transmit to Bob, either $0$ or $1$. She then prepares a qubit in the corresponding state $\lvert 0 \rangle$ or $\lvert 1 \rangle$, respectively. After that, she randomly decides whether or not to transform her qubit from the standard (Z) basis to the Hadamard (H) basis by applying or not the Hadamard gate her qubit, thus preparing the state $\lvert + \rangle$ or $\lvert - \rangle$.

Then Alice sends her first qubit to Bob. Bob receives Alice's qubit, selects one of the measurement bases at random and measures it. After that, Alice and Bob tell each other which basis they used through a classical communication channel.

In general, for every qubit Alice sends to Bob there are four possible scenarios:

Both Alice and Bob used the Hadamard basis.

They both used the standard basis.

Alice transformed to the Hadamard basis, and Bob measured in the standard basis.

Alice used the standard basis, and Bob the Hadamard basis.

When Alice and Bob agree on the same basis, they keep the transferred bit. When they disagree, they discard it. Thus, it is possible for Alice and Bob to securely communicate an $n$ bit private key using $2n$ qubits.

#### Example

For example, let us consider the case where Alice wants to send the bit $0$. She prepares her qubit in the $\lvert 0 \rangle$ state and then randomly selects whether or not she applies the Hadamard gate to it. Let's say she does apply the Hadamard gate to her qubit, obtaining the $\lvert + \rangle$ state.

Then, consider, the cased where Bob measures the qubit in the standard basis. After Bob's measurement, Alice and Bob communicate through the classical channel. Alice tells Bob that she  applied the Hadamard gate to her qubit and Bob tells Alice that he measured it in the standard basis. So, they abandon the first bit.

$$\text{2. Example of one application of the BB84 protocol. In this case, Alice and Bob will discard this bit.}$$

Next, Alice picks a second bit, $1$, encodes it into a qubit and selects at random whether to apply or not the Hadamard gate. Let us now assume that she does not apply the Hadamard gate. Thus, the qubit is in the state $\lvert 1\rangle $. Alice then sends her qubit to Bob. Bob selects at random one of his two measurement bases. Let us consider in this
case that he measures in the standard basis. As the qubit is in the state $\lvert 1\rangle $ the outcome of the measurement will be $1$. Thus, Bob chooses value $1$ for his second classical bit, the same as Alice did. Finally, Alice tells Bob that she did not apply the Hadamard gate, and Bob tells Alice that he measured in the standard basis. So, both Alice and Bob will use the bit with the value $1$ as the first bit in their secret key.

$$\text{3. Example of another application of the BB84 protocol.} \\  \text{In this case, Alice and Bob successfully communicate the value of a bit.}$$

###  QISKit: BB84 protocol

####  1) Show the communication of one bit

```python
from initialize import *
import random

#initialize quantum program
my_alg = initialize(circuit_name = 'bb84', qubit_number=1, bit_number=1, backend = 'local_qasm_simulator', shots = 1)

#add gates to the circuit

# Alice encodes the bit 1 into a qubit
my_alg.q_circuit.x(my_alg.q_reg[0])

# Alice randomly applies the Hadamard gate to go to the Hadamard basis
a = random.randint(0,1)
if a==1:
    my_alg.q_circuit.h(my_alg.q_reg[0])

# Bob randomly applies the Hadamard gate to go to the Hadamard basis
b = random.randint(0,1)
if b==1:
    my_alg.q_circuit.h(my_alg.q_reg[0])

my_alg.q_circuit.measure(my_alg.q_reg[0], my_alg.c_reg[0]) # measures first qubit

# print list of gates in the circuit
print('List of gates:')
for circuit in my_alg.q_circuit:
    print(circuit.name)

#Execute the quantum algorithm
result = my_alg.Q_program.execute(my_alg.circ_name, backend=my_alg.backend, shots= my_alg.shots)

#Show the results obtained from the quantum algorithm
counts = result.get_counts(my_alg.circ_name)

print('\nThe measured outcomes of the circuits are:',counts)

if a == b:
    print('Alice and Bob agree on the basis, thus they keep the bit')
else:
    print("Alice and Bob don't agree the same basis, thus they discard the bit")

```

    List of gates:
    x
    h
    measure

    The measured outcomes of the circuits are: {'0': 1}
    Alice and Bob don't agree the same basis, thus they discard the bit

    /anaconda/lib/python3.6/site-packages/qiskit/backends/local/qasm_simulator_cpp.py:89: DeprecationWarning: The behavior of getting statevector from simulators by setting shots=1 is deprecated and will be removed. Use the local_statevector_simulator instead, or place explicit snapshot instructions.
      DeprecationWarning)

## Problems

Alice wants to send Bob the following private key

\begin{equation}
101011
\end{equation}

She encodes those bits into the correspondig states of qubits and performs the gates H-H-I-I-I-H on each qubit. Bob measures the qubits in the following bases: Z-H-H-Z-H-H

Find the possible outcomes of Bob's measurements

Find the bits of the private key accepted by Alice and Bob

Imagine that a third party, Eve, intercepts Alice's qubits. She measures the intercepted qubit by randomly selecting either the Hadamard or the standard basis and then forwards the qubits to Bob.

Is it possible for Eve to find out the bit that Alice is sending to Bob without being discovered?

What is the probability that Eve successfully finds out the value of a bit?

Write a QISKit program for the transmission of a 1024 bits provate key between Alice and Bob

## References

[1] D. Dieks, Physics Letters A, 92, 271 (1982).

[2] W. K. Wootters and W. H. Zurek, Nature, 299.802 (1982).

[3] C. H. Bennett and G. Brassard, In Proceedings of IEEE International Conference on Computers, Systems and Signal Processing, volume 175, page 8. New York, 1984.


##### Copyright 2020 The TensorFlow Authors.

```python
#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

# Quantum data

    View on TensorFlow.org

    Run in Google Colab

    View source on GitHub

    Download notebook

Building off of the comparisons made in the [MNIST](https://www.tensorflow.org/quantum/tutorials/mnist) tutorial, this tutorial explores the recent work of [Huang et al.](https://arxiv.org/abs/2011.01938) that shows how different datasets affect performance comparisons. In the work, the authors seek to understand how and when classical machine learning models can learn as well as (or better than) quantum models. The work also showcases an empirical performance separation between classical and quantum machine learning model via a carefully crafted dataset. You will:

1.   Prepare a reduced dimension Fashion-MNIST dataset.
2.   Use quantum circuits to re-label the dataset and compute Projected Quantum Kernel features (PQK).
3.   Train a classical neural network on the re-labeled dataset and compare the performance with a model that has access to the PQK features.

## Setup

```python
!pip install tensorflow==2.7.0 tensorflow-quantum
```

    Collecting tensorflow==2.7.0
      Downloading tensorflow-2.7.0-cp37-cp37m-manylinux2010_x86_64.whl (489.6 MB)
         |████████████████████████████████| 489.6 MB 15 kB/s
    Collecting tensorflow-quantum
      Downloading tensorflow_quantum-0.6.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (10.5 MB)
         |████████████████████████████████| 10.5 MB 20.1 MB/s
    Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (3.1.0)
    Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (3.10.0.2)
    Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (1.1.0)
    Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (1.6.3)
    Requirement already satisfied: tensorboard~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (2.8.0)
    Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (1.44.0)
    Requirement already satisfied: keras-preprocessing>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (1.1.2)
    Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (0.2.0)
    Requirement already satisfied: wheel<1.0,>=0.32.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (0.37.1)
    Collecting keras<2.8,>=2.7.0rc0
      Downloading keras-2.7.0-py2.py3-none-any.whl (1.3 MB)
         |████████████████████████████████| 1.3 MB 29.1 MB/s
    Requirement already satisfied: libclang>=9.0.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (13.0.0)
    Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (3.17.3)
    Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (1.15.0)
    Collecting gast<0.5.0,>=0.2.1
      Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
    Requirement already satisfied: absl-py>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (1.0.0)
    Collecting tensorflow-estimator<2.8,~=2.7.0rc0
      Downloading tensorflow_estimator-2.7.0-py2.py3-none-any.whl (463 kB)
         |████████████████████████████████| 463 kB 34.1 MB/s
    Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (3.3.0)
    Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (1.14.0)
    Requirement already satisfied: numpy>=1.14.5 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (1.21.5)
    Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (0.24.0)
    Requirement already satisfied: flatbuffers<3.0,>=1.12 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.7.0) (2.0)
    Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py>=2.9.0->tensorflow==2.7.0) (1.5.2)
    Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow==2.7.0) (1.35.0)
    Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow==2.7.0) (57.4.0)
    Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow==2.7.0) (0.4.6)
    Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow==2.7.0) (2.23.0)
    Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow==2.7.0) (1.0.1)
    Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow==2.7.0) (0.6.1)
    Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow==2.7.0) (3.3.6)
    Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow==2.7.0) (1.8.1)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2.7.0) (0.2.8)
    Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2.7.0) (4.8)
    Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2.7.0) (4.2.4)
    Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow==2.7.0) (1.3.1)
    Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard~=2.6->tensorflow==2.7.0) (4.11.3)
    Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard~=2.6->tensorflow==2.7.0) (3.7.0)
    Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2.7.0) (0.4.8)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.7.0) (2.10)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.7.0) (1.24.3)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.7.0) (2021.10.8)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.7.0) (3.0.4)
    Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow==2.7.0) (3.2.0)
    Collecting cirq-core>=0.13.1
      Downloading cirq_core-0.14.0-py3-none-any.whl (1.8 MB)
         |████████████████████████████████| 1.8 MB 34.0 MB/s
    Collecting googleapis-common-protos==1.52.0
      Downloading googleapis_common_protos-1.52.0-py2.py3-none-any.whl (100 kB)
         |████████████████████████████████| 100 kB 8.2 MB/s
    Collecting sympy==1.8
      Downloading sympy-1.8-py3-none-any.whl (6.1 MB)
         |████████████████████████████████| 6.1 MB 24.1 MB/s
    Collecting cirq-google>=0.13.1
      Downloading cirq_google-0.14.0-py3-none-any.whl (541 kB)
         |████████████████████████████████| 541 kB 47.0 MB/s
    Collecting google-api-core==1.21.0
      Downloading google_api_core-1.21.0-py2.py3-none-any.whl (90 kB)
         |████████████████████████████████| 90 kB 8.3 MB/s
    Collecting google-auth<3,>=1.6.3
      Downloading google_auth-1.18.0-py2.py3-none-any.whl (90 kB)
         |████████████████████████████████| 90 kB 7.8 MB/s
    Requirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from google-api-core==1.21.0->tensorflow-quantum) (2018.9)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy==1.8->tensorflow-quantum) (1.2.1)
    Collecting backports.cached-property~=1.0.1
      Downloading backports.cached_property-1.0.1-py3-none-any.whl (5.7 kB)
    Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from cirq-core>=0.13.1->tensorflow-quantum) (4.63.0)
    Collecting duet~=0.2.0
      Downloading duet-0.2.5-py3-none-any.whl (28 kB)
    Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from cirq-core>=0.13.1->tensorflow-quantum) (1.3.5)
    Requirement already satisfied: sortedcontainers~=2.0 in /usr/local/lib/python3.7/dist-packages (from cirq-core>=0.13.1->tensorflow-quantum) (2.4.0)
    Requirement already satisfied: networkx~=2.4 in /usr/local/lib/python3.7/dist-packages (from cirq-core>=0.13.1->tensorflow-quantum) (2.6.3)
    Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from cirq-core>=0.13.1->tensorflow-quantum) (1.4.1)
    Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.7/dist-packages (from cirq-core>=0.13.1->tensorflow-quantum) (3.2.2)
    Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.14.0 in /usr/local/lib/python3.7/dist-packages (from cirq-google>=0.13.1->tensorflow-quantum) (1.26.3)
    Collecting typing-extensions>=3.6.6
      Downloading typing_extensions-3.10.0.0-py3-none-any.whl (26 kB)
    Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0
      Downloading google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
         |████████████████████████████████| 93 kB 1.0 MB/s
      Downloading google_api_core-1.31.4-py2.py3-none-any.whl (93 kB)
         |████████████████████████████████| 93 kB 1.5 MB/s
      Downloading google_api_core-1.31.3-py2.py3-none-any.whl (93 kB)
         |████████████████████████████████| 93 kB 1.5 MB/s
      Downloading google_api_core-1.31.2-py2.py3-none-any.whl (93 kB)
         |████████████████████████████████| 93 kB 1.4 MB/s
      Downloading google_api_core-1.31.1-py2.py3-none-any.whl (93 kB)
         |████████████████████████████████| 93 kB 1.3 MB/s
      Downloading google_api_core-1.31.0-py2.py3-none-any.whl (93 kB)
         |████████████████████████████████| 93 kB 1.3 MB/s
      Downloading google_api_core-1.30.0-py2.py3-none-any.whl (93 kB)
         |████████████████████████████████| 93 kB 1.4 MB/s
      Downloading google_api_core-1.29.0-py2.py3-none-any.whl (93 kB)
         |████████████████████████████████| 93 kB 1.2 MB/s
      Downloading google_api_core-1.28.0-py2.py3-none-any.whl (92 kB)
         |████████████████████████████████| 92 kB 1.1 MB/s
      Downloading google_api_core-1.27.0-py2.py3-none-any.whl (93 kB)
         |████████████████████████████████| 93 kB 1.4 MB/s
      Downloading google_api_core-1.26.2-py2.py3-none-any.whl (93 kB)
         |████████████████████████████████| 93 kB 1.1 MB/s
      Downloading google_api_core-1.26.1-py2.py3-none-any.whl (92 kB)
         |████████████████████████████████| 92 kB 948 kB/s
      Downloading google_api_core-1.26.0-py2.py3-none-any.whl (92 kB)
         |████████████████████████████████| 92 kB 1.0 MB/s
      Downloading google_api_core-1.25.1-py2.py3-none-any.whl (92 kB)
         |████████████████████████████████| 92 kB 229 kB/s
      Downloading google_api_core-1.25.0-py2.py3-none-any.whl (92 kB)
         |████████████████████████████████| 92 kB 171 kB/s
      Downloading google_api_core-1.24.1-py2.py3-none-any.whl (92 kB)
         |████████████████████████████████| 92 kB 10.3 MB/s
      Downloading google_api_core-1.24.0-py2.py3-none-any.whl (91 kB)
         |████████████████████████████████| 91 kB 9.5 MB/s
      Downloading google_api_core-1.23.0-py2.py3-none-any.whl (91 kB)
         |████████████████████████████████| 91 kB 9.8 MB/s
      Downloading google_api_core-1.22.4-py2.py3-none-any.whl (91 kB)
         |████████████████████████████████| 91 kB 9.0 MB/s
      Downloading google_api_core-1.22.3-py2.py3-none-any.whl (91 kB)
         |████████████████████████████████| 91 kB 9.2 MB/s
      Downloading google_api_core-1.22.2-py2.py3-none-any.whl (91 kB)
         |████████████████████████████████| 91 kB 8.5 MB/s
      Downloading google_api_core-1.22.1-py2.py3-none-any.whl (91 kB)
         |████████████████████████████████| 91 kB 8.5 MB/s
      Downloading google_api_core-1.22.0-py2.py3-none-any.whl (91 kB)
         |████████████████████████████████| 91 kB 9.1 MB/s
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib~=3.0->cirq-core>=0.13.1->tensorflow-quantum) (3.0.7)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib~=3.0->cirq-core>=0.13.1->tensorflow-quantum) (0.11.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib~=3.0->cirq-core>=0.13.1->tensorflow-quantum) (1.4.0)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib~=3.0->cirq-core>=0.13.1->tensorflow-quantum) (2.8.2)
    Installing collected packages: typing-extensions, googleapis-common-protos, google-auth, sympy, google-api-core, duet, backports.cached-property, cirq-core, tensorflow-estimator, keras, gast, cirq-google, tensorflow-quantum, tensorflow
      Attempting uninstall: typing-extensions
        Found existing installation: typing-extensions 3.10.0.2
        Uninstalling typing-extensions-3.10.0.2:
          Successfully uninstalled typing-extensions-3.10.0.2
      Attempting uninstall: googleapis-common-protos
        Found existing installation: googleapis-common-protos 1.56.0
        Uninstalling googleapis-common-protos-1.56.0:
          Successfully uninstalled googleapis-common-protos-1.56.0
      Attempting uninstall: google-auth
        Found existing installation: google-auth 1.35.0
        Uninstalling google-auth-1.35.0:
          Successfully uninstalled google-auth-1.35.0
      Attempting uninstall: sympy
        Found existing installation: sympy 1.7.1
        Uninstalling sympy-1.7.1:
          Successfully uninstalled sympy-1.7.1
      Attempting uninstall: google-api-core
        Found existing installation: google-api-core 1.26.3
        Uninstalling google-api-core-1.26.3:
          Successfully uninstalled google-api-core-1.26.3
      Attempting uninstall: tensorflow-estimator
        Found existing installation: tensorflow-estimator 2.8.0
        Uninstalling tensorflow-estimator-2.8.0:
          Successfully uninstalled tensorflow-estimator-2.8.0
      Attempting uninstall: keras
        Found existing installation: keras 2.8.0
        Uninstalling keras-2.8.0:
          Successfully uninstalled keras-2.8.0
      Attempting uninstall: gast
        Found existing installation: gast 0.5.3
        Uninstalling gast-0.5.3:
          Successfully uninstalled gast-0.5.3
      Attempting uninstall: tensorflow
        Found existing installation: tensorflow 2.8.0
        Uninstalling tensorflow-2.8.0:
          Successfully uninstalled tensorflow-2.8.0
    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
    pydata-google-auth 1.4.0 requires google-auth<3.0dev,>=1.25.0; python_version >= "3.6", but you have google-auth 1.18.0 which is incompatible.
    Successfully installed backports.cached-property-1.0.1 cirq-core-0.14.0 cirq-google-0.14.0 duet-0.2.5 gast-0.4.0 google-api-core-1.21.0 google-auth-1.18.0 googleapis-common-protos-1.52.0 keras-2.7.0 sympy-1.8 tensorflow-2.7.0 tensorflow-estimator-2.7.0 tensorflow-quantum-0.6.1 typing-extensions-3.10.0.0

```python
# Update package resources to account for version changes.
import importlib, pkg_resources
importlib.reload(pkg_resources)
```

    <module 'pkg_resources' from '/usr/local/lib/python3.7/dist-packages/pkg_resources/__init__.py'>

```python
import cirq
import sympy
import numpy as np
import tensorflow as tf
import tensorflow_quantum as tfq

# visualization tools
%matplotlib inline
import matplotlib.pyplot as plt
from cirq.contrib.svg import SVGCircuit
np.random.seed(1234)
```

## 1. Data preparation

You will begin by preparing the fashion-MNIST dataset for running on a quantum computer.

### 1.1 Download fashion-MNIST

The first step is to get the traditional fashion-mnist dataset. This can be done using the `tf.keras.datasets` module.

```python
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.fashion_mnist.load_data()

# Rescale the images from [0,255] to the [0.0,1.0] range.
x_train, x_test = x_train/255.0, x_test/255.0

print("Number of original training examples:", len(x_train))
print("Number of original test examples:", len(x_test))
```

    Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz
    32768/29515 [=================================] - 0s 0us/step
    40960/29515 [=========================================] - 0s 0us/step
    Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz
    26427392/26421880 [==============================] - 0s 0us/step
    26435584/26421880 [==============================] - 0s 0us/step
    Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz
    16384/5148 [===============================================================================================] - 0s 0us/step
    Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz
    4423680/4422102 [==============================] - 0s 0us/step
    4431872/4422102 [==============================] - 0s 0us/step
    Number of original training examples: 60000
    Number of original test examples: 10000

Filter the dataset to keep just the T-shirts/tops and dresses, remove the other classes. At the same time convert the label, `y`, to boolean: True for 0 and False for 3.

```python
def filter_03(x, y):
    keep = (y == 0) | (y == 3)
    x, y = x[keep], y[keep]
    y = y == 0
    return x,y
```

```python
x_train, y_train = filter_03(x_train, y_train)
x_test, y_test = filter_03(x_test, y_test)

print("Number of filtered training examples:", len(x_train))
print("Number of filtered test examples:", len(x_test))
```

    Number of filtered training examples: 12000
    Number of filtered test examples: 2000

```python
print(y_train[0])

plt.imshow(x_train[0, :, :])
plt.colorbar()
```

### 1.2 Downscale the images

Just like the MNIST example, you will need to downscale these images in order to be within the boundaries for current quantum computers. This time however you will use a PCA transformation to reduce the dimensions instead of a `tf.image.resize` operation.

```python
def truncate_x(x_train, x_test, n_components=10):
  """Perform PCA on image dataset keeping the top `n_components` components."""
  n_points_train = tf.gather(tf.shape(x_train), 0)
  n_points_test = tf.gather(tf.shape(x_test), 0)

  # Flatten to 1D
  x_train = tf.reshape(x_train, [n_points_train, -1])
  x_test = tf.reshape(x_test, [n_points_test, -1])

  # Normalize.
  feature_mean = tf.reduce_mean(x_train, axis=0)
  x_train_normalized = x_train - feature_mean
  x_test_normalized = x_test - feature_mean

  # Truncate.
  e_values, e_vectors = tf.linalg.eigh(
      tf.einsum('ji,jk->ik', x_train_normalized, x_train_normalized))
  return tf.einsum('ij,jk->ik', x_train_normalized, e_vectors[:,-n_components:]), \
    tf.einsum('ij,jk->ik', x_test_normalized, e_vectors[:, -n_components:])
```

```python
DATASET_DIM = 10
x_train, x_test = truncate_x(x_train, x_test, n_components=DATASET_DIM)
print(f'New datapoint dimension:', len(x_train[0]))
```

    New datapoint dimension: 10

The last step is to reduce the size of the dataset to just 1000 training datapoints and 200 testing datapoints.

```python
N_TRAIN = 1000
N_TEST = 200
x_train, x_test = x_train[:N_TRAIN], x_test[:N_TEST]
y_train, y_test = y_train[:N_TRAIN], y_test[:N_TEST]
```

```python
print("New number of training examples:", len(x_train))
print("New number of test examples:", len(x_test))
```

    New number of training examples: 1000
    New number of test examples: 200

## 2. Relabeling and computing PQK features

You will now prepare a "stilted" quantum dataset by incorporating quantum components and re-labeling the truncated fashion-MNIST dataset you've created above. In order to get the most seperation between quantum and classical methods, you will first prepare the PQK features and then relabel outputs based on their values.

### 2.1 Quantum encoding and PQK features
You will create a new set of features, based on `x_train`, `y_train`, `x_test` and `y_test` that is defined to be the 1-RDM on all qubits of:

$V(x_{\text{train}} / n_{\text{trotter}}) ^ {n_{\text{trotter}}} U_{\text{1qb}} | 0 \rangle$

Where $U_\text{1qb}$ is a wall of single qubit rotations and $V(\hat{\theta}) = e^{-i\sum_i \hat{\theta_i} (X_i X_{i+1} + Y_i Y_{i+1} + Z_i Z_{i+1})}$

First, you can generate the wall of single qubit rotations:

```python
def single_qubit_wall(qubits, rotations):
  """Prepare a single qubit X,Y,Z rotation wall on `qubits`."""
  wall_circuit = cirq.Circuit()
  for i, qubit in enumerate(qubits):
    for j, gate in enumerate([cirq.X, cirq.Y, cirq.Z]):
      wall_circuit.append(gate(qubit) ** rotations[i][j])

  return wall_circuit
```

You can quickly verify this works by looking at the circuit:

```python
SVGCircuit(single_qubit_wall(
    cirq.GridQubit.rect(1,4), np.random.uniform(size=(4, 3))))
```

    findfont: Font family ['Arial'] not found. Falling back to DejaVu Sans.

Next you can prepare $V(\hat{\theta})$ with the help of `tfq.util.exponential` which can exponentiate any commuting `cirq.PauliSum` objects:

```python
def v_theta(qubits):
  """Prepares a circuit that generates V(\theta)."""
  ref_paulis = [
      cirq.X(q0) * cirq.X(q1) + \
      cirq.Y(q0) * cirq.Y(q1) + \
      cirq.Z(q0) * cirq.Z(q1) for q0, q1 in zip(qubits, qubits[1:])
  ]
  exp_symbols = list(sympy.symbols('ref_0:'+str(len(ref_paulis))))
  return tfq.util.exponential(ref_paulis, exp_symbols), exp_symbols
```

This circuit might be a little bit harder to verify by looking at, but you can still examine a two qubit case to see what is happening:

```python
test_circuit, test_symbols = v_theta(cirq.GridQubit.rect(1, 2))
print(f'Symbols found in circuit:{test_symbols}')
SVGCircuit(test_circuit)
```

    Symbols found in circuit:[ref_0]

Now you have all the building blocks you need to put your full encoding circuits together:

```python
def prepare_pqk_circuits(qubits, classical_source, n_trotter=10):
  """Prepare the pqk feature circuits around a dataset."""
  n_qubits = len(qubits)
  n_points = len(classical_source)

  # Prepare random single qubit rotation wall.
  random_rots = np.random.uniform(-2, 2, size=(n_qubits, 3))
  initial_U = single_qubit_wall(qubits, random_rots)

  # Prepare parametrized V
  V_circuit, symbols = v_theta(qubits)
  exp_circuit = cirq.Circuit(V_circuit for t in range(n_trotter))

  # Convert to `tf.Tensor`
  initial_U_tensor = tfq.convert_to_tensor([initial_U])
  initial_U_splat = tf.tile(initial_U_tensor, [n_points])

  full_circuits = tfq.layers.AddCircuit()(
      initial_U_splat, append=exp_circuit)
  # Replace placeholders in circuits with values from `classical_source`.
  return tfq.resolve_parameters(
      full_circuits, tf.convert_to_tensor([str(x) for x in symbols]),
      tf.convert_to_tensor(classical_source*(n_qubits/3)/n_trotter))
```

Choose some qubits and prepare the data encoding circuits:

```python
qubits = cirq.GridQubit.rect(1, DATASET_DIM + 1)
q_x_train_circuits = prepare_pqk_circuits(qubits, x_train)
q_x_test_circuits = prepare_pqk_circuits(qubits, x_test)
```

Next, compute the PQK features based on the 1-RDM of the dataset circuits above and store the results in `rdm`, a `tf.Tensor` with shape `[n_points, n_qubits, 3]`. The entries in `rdm[i][j][k]` = $\langle \psi_i | OP^k_j | \psi_i \rangle$ where `i` indexes over datapoints, `j` indexes over qubits and `k` indexes over $\lbrace \hat{X}, \hat{Y}, \hat{Z} \rbrace$ .

```python
def get_pqk_features(qubits, data_batch):
  """Get PQK features based on above construction."""
  ops = [[cirq.X(q), cirq.Y(q), cirq.Z(q)] for q in qubits]
  ops_tensor = tf.expand_dims(tf.reshape(tfq.convert_to_tensor(ops), -1), 0)
  batch_dim = tf.gather(tf.shape(data_batch), 0)
  ops_splat = tf.tile(ops_tensor, [batch_dim, 1])
  exp_vals = tfq.layers.Expectation()(data_batch, operators=ops_splat)
  rdm = tf.reshape(exp_vals, [batch_dim, len(qubits), -1])
  return rdm
```

```python
x_train_pqk = get_pqk_features(qubits, q_x_train_circuits)
x_test_pqk = get_pqk_features(qubits, q_x_test_circuits)
print('New PQK training dataset has shape:', x_train_pqk.shape)
print('New PQK testing dataset has shape:', x_test_pqk.shape)
```

    New PQK training dataset has shape: (1000, 11, 3)
    New PQK testing dataset has shape: (200, 11, 3)

### 2.2 Re-labeling based on PQK features
Now that you have these quantum generated features in `x_train_pqk` and `x_test_pqk`, it is time to re-label the dataset. To achieve maximum seperation between quantum and classical performance you can re-label the dataset based on the spectrum information found in `x_train_pqk` and `x_test_pqk`.

Note: This preparation of your dataset to explicitly maximize the seperation in performance between the classical and quantum models might feel like cheating, but it provides a **very** important proof of existance for datasets that are hard for classical computers and easy for quantum computers to model. There would be no point in searching for quantum advantage in QML if you couldn't first create something like this to demonstrate advantage.

```python
def compute_kernel_matrix(vecs, gamma):
  """Computes d[i][j] = e^ -gamma * (vecs[i] - vecs[j]) ** 2 """
  scaled_gamma = gamma / (
      tf.cast(tf.gather(tf.shape(vecs), 1), tf.float32) * tf.math.reduce_std(vecs))
  return scaled_gamma * tf.einsum('ijk->ij',(vecs[:,None,:] - vecs) ** 2)

def get_spectrum(datapoints, gamma=1.0):
  """Compute the eigenvalues and eigenvectors of the kernel of datapoints."""
  KC_qs = compute_kernel_matrix(datapoints, gamma)
  S, V = tf.linalg.eigh(KC_qs)
  S = tf.math.abs(S)
  return S, V
```

```python
S_pqk, V_pqk = get_spectrum(
    tf.reshape(tf.concat([x_train_pqk, x_test_pqk], 0), [-1, len(qubits) * 3]))

S_original, V_original = get_spectrum(
    tf.cast(tf.concat([x_train, x_test], 0), tf.float32), gamma=0.005)

print('Eigenvectors of pqk kernel matrix:', V_pqk)
print('Eigenvectors of original kernel matrix:', V_original)
```

    Eigenvectors of pqk kernel matrix: tf.Tensor(
    [[ 0.02095697  0.01059745  0.02166322 ...  0.09526508  0.00300356
       0.02826785]
     [ 0.02293038  0.04663572  0.00791177 ...  0.00220983 -0.6957587
       0.02859015]
     [ 0.01778554 -0.0030075  -0.02552235 ...  0.02335721  0.00414519
       0.02690097]
     ...
     [-0.0605794   0.01324826  0.02695336 ...  0.00716051  0.03977184
       0.03853431]
     [-0.06333087 -0.00304116  0.00977427 ... -0.03250755  0.02224028
       0.03674842]
     [-0.05860277  0.00584422  0.00264832 ... -0.04459745 -0.01932838
       0.03299437]], shape=(1200, 1200), dtype=float32)
    Eigenvectors of original kernel matrix: tf.Tensor(
    [[ 3.8356818e-02  2.8347293e-02 -1.1697864e-02 ... -4.0755421e-02
       2.0624822e-02  3.2069720e-02]
     [-4.0181600e-02  8.8809701e-03 -1.3882567e-02 ... -7.6112538e-03
       7.1638334e-01  2.8819481e-02]
     [-1.6671857e-02  1.3503703e-02 -3.6638588e-02 ...  4.2131193e-02
      -3.7604037e-03  2.1954076e-02]
     ...
     [-3.0156480e-02 -1.6716314e-02 -1.6033923e-02 ...  2.1481956e-03
      -5.8309413e-03  2.3656894e-02]
     [ 3.9776899e-03 -4.9988784e-02 -5.2833343e-03 ... -2.2350436e-02
      -4.1845851e-02  2.7820019e-02]
     [-1.6657291e-02 -8.1861708e-03 -4.3234091e-02 ... -3.2867838e-04
       9.1463570e-03  1.8750878e-02]], shape=(1200, 1200), dtype=float32)

Now you have everything you need to re-label the dataset! Now you can consult with the flowchart to better understand how to maximize performance seperation when re-labeling the dataset:

In order to maximize the seperation between quantum and classical models, you will attempt to maximize the geometric difference between the original dataset and the PQK features kernel matrices $g(K_1 || K_2) = \sqrt{ || \sqrt{K_2} K_1^{-1} \sqrt{K_2} || _\infty}$ using `S_pqk, V_pqk` and `S_original, V_original`. A large value of $g$ ensures that you initially move to the right in the flowchart down towards a prediction advantage in the quantum case.

Note: Computing quantities for $s$ and $d$ are also very useful when looking to better understand performance seperations. In this case ensuring a large $g$ value is enough to see performance seperation.

```python
def get_stilted_dataset(S, V, S_2, V_2, lambdav=1.1):
  """Prepare new labels that maximize geometric distance between kernels."""
  S_diag = tf.linalg.diag(S ** 0.5)
  S_2_diag = tf.linalg.diag(S_2 / (S_2 + lambdav) ** 2)
  scaling = S_diag @ tf.transpose(V) @ \
            V_2 @ S_2_diag @ tf.transpose(V_2) @ \
            V @ S_diag

  # Generate new lables using the largest eigenvector.
  _, vecs = tf.linalg.eig(scaling)
  new_labels = tf.math.real(
      tf.einsum('ij,j->i', tf.cast(V @ S_diag, tf.complex64), vecs[-1])).numpy()
  # Create new labels and add some small amount of noise.
  final_y = new_labels > np.median(new_labels)
  noisy_y = (final_y ^ (np.random.uniform(size=final_y.shape) > 0.95))
  return noisy_y
```

```python
y_relabel = get_stilted_dataset(S_pqk, V_pqk, S_original, V_original)
y_train_new, y_test_new = y_relabel[:N_TRAIN], y_relabel[N_TRAIN:]
```

## 3. Comparing models
Now that you have prepared your dataset it is time to compare model performance. You will create two small feedforward neural networks and compare performance when they are given access to the PQK features found in `x_train_pqk`.

### 3.1 Create PQK enhanced model
Using standard `tf.keras` library features you can now create and a train a model on the `x_train_pqk` and `y_train_new` datapoints:

```python
#docs_infra: no_execute
def create_pqk_model():
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Dense(32, activation='sigmoid', input_shape=[len(qubits) * 3,]))
    model.add(tf.keras.layers.Dense(16, activation='sigmoid'))
    model.add(tf.keras.layers.Dense(1))
    return model

pqk_model = create_pqk_model()
pqk_model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              optimizer=tf.keras.optimizers.Adam(learning_rate=0.003),
              metrics=['accuracy'])

pqk_model.summary()
```

    Model: "sequential"
    _________________________________________________________________
     Layer (type)                Output Shape              Param #
    =================================================================
     dense (Dense)               (None, 32)                1088

     dense_1 (Dense)             (None, 16)                528

     dense_2 (Dense)             (None, 1)                 17

    =================================================================
    Total params: 1,633
    Trainable params: 1,633
    Non-trainable params: 0
    _________________________________________________________________

```python
#docs_infra: no_execute
pqk_history = pqk_model.fit(tf.reshape(x_train_pqk, [N_TRAIN, -1]),
          y_train_new,
          batch_size=32,
          epochs=1000,
          verbose=0,
          validation_data=(tf.reshape(x_test_pqk, [N_TEST, -1]), y_test_new))
```

### 3.2 Create a classical model
Similar to the code above you can now also create a classical model that doesn't have access to the PQK features in your stilted dataset. This model can be trained using `x_train` and `y_label_new`.

```python
#docs_infra: no_execute
def create_fair_classical_model():
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Dense(32, activation='sigmoid', input_shape=[DATASET_DIM,]))
    model.add(tf.keras.layers.Dense(16, activation='sigmoid'))
    model.add(tf.keras.layers.Dense(1))
    return model

model = create_fair_classical_model()
model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              optimizer=tf.keras.optimizers.Adam(learning_rate=0.03),
              metrics=['accuracy'])

model.summary()
```

    Model: "sequential_1"
    _________________________________________________________________
     Layer (type)                Output Shape              Param #
    =================================================================
     dense_3 (Dense)             (None, 32)                352

     dense_4 (Dense)             (None, 16)                528

     dense_5 (Dense)             (None, 1)                 17

    =================================================================
    Total params: 897
    Trainable params: 897
    Non-trainable params: 0
    _________________________________________________________________

```python
#docs_infra: no_execute
classical_history = model.fit(x_train,
          y_train_new,
          batch_size=32,
          epochs=1000,
          verbose=0,
          validation_data=(x_test, y_test_new))
```

### 3.3 Compare performance
Now that you have trained the two models you can quickly plot the performance gaps in the validation data between the two. Typically both models will achieve > 0.9 accuaracy on the training data. However on the validation data it becomes clear that only the information found in the PQK features is enough to make the model generalize well to unseen instances.

```python
#docs_infra: no_execute
plt.figure(figsize=(10,5))
plt.plot(classical_history.history['accuracy'], label='accuracy_classical')
plt.plot(classical_history.history['val_accuracy'], label='val_accuracy_classical')
plt.plot(pqk_history.history['accuracy'], label='accuracy_quantum')
plt.plot(pqk_history.history['val_accuracy'], label='val_accuracy_quantum')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend()
```

Success: You have engineered a stilted quantum dataset that can intentionally defeat classical models in a fair (but contrived) setting. Try comparing results using other types of classical models. The next step is to try and see if you can find new and interesting datasets that can defeat classical models without needing to engineer them yourself!

## 4. Important conclusions

There are several important conclusions you can draw from this and the [MNIST](https://www.tensorflow.org/quantum/tutorials/mnist) experiments:

1. It's very unlikely that the quantum models of today will beat classical model performance on classical data. Especially on today's classical datasets that can have upwards of a million datapoints.

2. Just because the data might come from a hard to classically simulate quantum circuit, doesn't necessarily make the data hard to learn for a classical model.

3. Datasets (ultimately quantum in nature) that are easy for quantum models to learn and hard for classical models to learn do exist, regardless of model architecture or training algorithms used.


```python
import numpy as np
import pandas as pd
import linearsolve as ls
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Homework 7

**Instructions:** Complete the notebook below. Download the completed notebook in HTML format. Upload assignment using Canvas.

**Due:** Feb. 23 at **2pm.**

## Exercise: The Labor-Leisure Tradeoff

\begin{align}
\frac{\varphi}{1-L_t} & = \frac{(1-\alpha)A_tK_t^{\alpha}L_t^{-\alpha}}{C_t} \tag{1}
\end{align}

**Questions**

1. Explain words why the left-hand side of equation (1) represents the marginal cost to the household of working. A complete answer will make use of the term *marginal utility* .
2. Explain words why the right-hand side of equation (1) represents the marginal benefit to the household of supplying labor (i.e., working). A complete answer will make use of the terms *marginal utility* and *marginal product*.
3. Holding everything else constant, according to equation (1), what effect will an increase in TFP have on equilibrium labor? Explain the economic intuition behind your answer.
4. Holding everything else constant, according to equation (1), what effect will an increase in household consumption have on equilibrium labor? Explain the economic intuition behind your answer.

**Answers**

1.

2.

3.

4.

## Exercise: The Euler Equation

\begin{align}
\frac{1}{C_t} & = \beta \left[\frac{\alpha A_{t+1}K_{t+1}^{\alpha-1}L_{t+1}^{1-\alpha} +1-\delta }{C_{t+1}}\right]\tag{2}
\end{align}

**Questions**

1. Explain words why the left-hand side of equation (2) represents the marginal cost to the household of saving (i.e., building new capital). A complete answer will make use of the term *marginal utility* .
2. Explain words why the right-hand side of equation (2) represents the marginal benefit to the household of saving. A complete answer will make use of the terms *marginal utility* and *marginal product*.
3. Holding everything else constant, according to equation (2), what effect will an increase in TFP in period $t+1$ have on the household's choice for capital in period $t+1$? Explain the economic intuition behind your answer.
3. Holding everything else constant, according to equation (2), what effect will an increase in consumption in period $t$ have on the household's choice for capital in period $t+1$? Explain the economic intuition behind your answer.
3. Holding everything else constant, according to equation (2), what effect will an increase in consumption in period $t+1$ have on the household's choice for capital in period $t+1$? Explain the economic intuition behind your answer.

**Answers**

1.

2.

3.

3.

4.


```python
name = "Ricardo Hideki Hangai Kojo"  # write YOUR NAME

honorPledge = "I affirm that I have not given or received any unauthorized " \
              "help on this assignment, and that this work is my own.\n"

print("\nName: ", name)
print("\nHonor pledge: ", honorPledge)
```

    Name:  Ricardo Hideki Hangai Kojo

    Honor pledge:  I affirm that I have not given or received any unauthorized help on this assignment, and that this work is my own.

# MAC0460 / MAC5832 (2021)

# EP2: Linear regression - analytic solution

### Objectives:

- to implement and test the analytic solution for the linear regression task (see, for instance, Slides of Lecture 03 and Lecture 03 of *Learning from Data*)
- to understand the core idea (*optimization of a loss or cost function*) for parameter adjustment in machine learning

# Linear regression

Given a dataset $\{(\mathbf{x}^{(1)}, y^{(1)}), \dots ,(\mathbf{x}^{(N)}, y^{(N)})\}$ with $\mathbf{x}^{(i)} \in \mathbb{R}^{d}$ and $y^{(i)} \in \mathbb{R}$, we would like to approximate the unknown function $f:\mathbb{R}^{d} \rightarrow \mathbb{R}$ (recall that $y^{(i)} =f(\mathbf{x}^{(i)})$) by means of a linear model $h$:
$$
h(\mathbf{x}^{(i)}; \mathbf{w}, b) = \mathbf{w}^\top  \mathbf{x}^{(i)} + b
$$

Note that $h(\mathbf{x}^{(i)}; \mathbf{w}, b)$ is, in fact, an  [affine transformation](https://en.wikipedia.org/wiki/Affine_transformation) of  $\mathbf{x}^{(i)}$. As commonly done, we will use the term "linear" to refer to an affine transformation.

The output of $h$ is a linear transformation of $\mathbf{x}^{(i)}$. We use the notation $h(\mathbf{x}^{(i)}; \mathbf{w}, b)$ to make clear that $h$ is a parametric model, i.e., the transformation $h$ is defined  by the parameters $\mathbf{w}$ and $b$. We can view vector $\mathbf{w}$ as a *weight* vector that controls the effect of each *feature* in the prediction.

By adding one component with value equal to 1 to the observations $\mathbf{x}$ (an artificial coordinate), we have:

$$\tilde{\mathbf{x}} = (1, x_1, \ldots, x_d) \in \mathbb{R}^{1+d}$$

and then we can simplify the notation:
$$
h(\mathbf{x}^{(i)}; \mathbf{w}) = \hat{y}^{(i)} = \mathbf{w}^\top  \tilde{\mathbf{x}}^{(i)}
$$

We would like to determine the optimal parameters $\mathbf{w}$ such that prediction $\hat{y}^{(i)}$ is as closest as possible to $y^{(i)}$ according to some error metric. Adopting the *mean square error* as such metric we have the following cost function:

\begin{equation}
J(\mathbf{w}) = \frac{1}{N}\sum_{i=1}^{N}\big(\hat{y}^{(i)} - y^{(i)}\big)^{2}
\end{equation}

Thus, the task of determining a function $h$ that is closest to $f$ is reduced to the task of finding the values $\mathbf{w}$ that minimize $J(\mathbf{w})$.

**Now we will explore this model, starting with a simple dataset.**

### Auxiliary functions

```python
# some imports
import numpy as np
import time
import matplotlib.pyplot as plt

%matplotlib inline
```

```python
# An auxiliary function
def get_housing_prices_data(N, verbose=True):
    """
    Generates artificial linear data,
    where x = square meter, y = house price

    :param N: data set size
    :type N: int

    :param verbose: param to control print
    :type verbose: bool
    :return: design matrix, regression targets
    :rtype: np.array, np.array
    """
    cond = False
    while not cond:
        x = np.linspace(90, 1200, N)
        gamma = np.random.normal(30, 10, x.size)
        y = 50 * x + gamma * 400
        x = x.astype("float32")
        x = x.reshape((x.shape[0], 1))
        y = y.astype("float32")
        y = y.reshape((y.shape[0], 1))
        cond = min(y) > 0

    xmean, xsdt, xmax, xmin = np.mean(x), np.std(x), np.max(x), np.min(x)
    ymean, ysdt, ymax, ymin = np.mean(y), np.std(y), np.max(y), np.min(y)
    if verbose:
        print("\nX shape = {}".format(x.shape))
        print("y shape = {}\n".format(y.shape))
        print("X: mean {}, sdt {:.2f}, max {:.2f}, min {:.2f}".format(xmean,
                                                               xsdt,
                                                               xmax,
                                                               xmin))
        print("y: mean {:.2f}, sdt {:.2f}, max {:.2f}, min {:.2f}".format(ymean,
                                                                 ysdt,
                                                                 ymax,
                                                                 ymin))
    return x, y
```

```python
# Another auxiliary function
def plot_points_regression(x,
                           y,
                           title,
                           xlabel,
                           ylabel,
                           prediction=None,
                           legend=False,
                           r_squared=None,
                           position=(90, 100)):
    """
    Plots the data points and the prediction,
    if there is one.

    :param x: design matrix
    :type x: np.array
    :param y: regression targets
    :type y: np.array
    :param title: plot's title
    :type title: str
    :param xlabel: x axis label
    :type xlabel: str
    :param ylabel: y axis label
    :type ylabel: str
    :param prediction: model's prediction
    :type prediction: np.array
    :param legend: param to control print legends
    :type legend: bool
    :param r_squared: r^2 value
    :type r_squared: float
    :param position: text position
    :type position: tuple
    """
    fig, ax = plt.subplots(1, 1, figsize=(8, 8))
    line1, = ax.plot(x, y, 'bo', label='Real data')
    if prediction is not None:
        line2, = ax.plot(x, prediction, 'r', label='Predicted data')
        if legend:
            plt.legend(handles=[line1, line2], loc=2)
        ax.set_title(title,
                 fontsize=20,
                 fontweight='bold')
    if r_squared is not None:
        bbox_props = dict(boxstyle="square,pad=0.3",
                          fc="white", ec="black", lw=0.2)
        t = ax.text(position[0], position[1], "$R^2 ={:.4f}$".format(r_squared),
                    size=15, bbox=bbox_props)

    ax.set_xlabel(xlabel, fontsize=20)
    ax.set_ylabel(ylabel, fontsize=20)
    plt.show()

```

### The dataset

The first dataset we will use is a toy dataset. We will generate $N=100$ observations with only one *feature* and a real value associated to each of them. We can view these observations as being pairs *(area of a real state in square meters, price of the real state)*. Our task is to construct a model that is able to predict the price of a real state, given its area.

```python
X, y = get_housing_prices_data(N=100)
```

    X shape = (100, 1)
    y shape = (100, 1)

    X: mean 645.0, sdt 323.65, max 1200.00, min 90.00
    y: mean 44221.50, sdt 16674.25, max 79182.43, min 12196.20

### Ploting the data

```python
plot_points_regression(X,
                       y,
                       title='Real estate prices prediction',
                       xlabel="m\u00b2",
                       ylabel='$')
```

### The solution

Given $f:\mathbb{R}^{N\times M} \rightarrow \mathbb{R}$ and $\mathbf{A} \in \mathbb{R}^{N\times M}$, we define the gradient of $f$ with respect to $\mathbf{A}$ as:

\begin{equation*}
\nabla_{\mathbf{A}}f = \frac{\partial f}{\partial \mathbf{A}} =  \begin{bmatrix}
\frac{\partial f}{\partial \mathbf{A}_{1,1}} & \dots & \frac{\partial f}{\partial \mathbf{A}_{1,m}} \\
\vdots &  \ddots & \vdots \\
\frac{\partial f}{\partial \mathbf{A}_{n,1}} &  \dots & \frac{\partial f}{\partial \mathbf{A}_{n,m}}
\end{bmatrix}
\end{equation*}

Let $\mathbf{X} \in \mathbb{R}^{N\times d}$ be a matrix (sometimes also called the *design matrix*) whose rows are the observations of the dataset and let $\mathbf{y} \in \mathbb{R}^{N}$ be the vector consisting of all values $y^{(i)}$ (i.e., $\mathbf{X}^{(i,:)} = \mathbf{x}^{(i)}$ and $\mathbf{y}^{(i)} = y^{(i)}$). It can be verified that:

\begin{equation}
J(\mathbf{w}) = \frac{1}{N}(\mathbf{X}\mathbf{w} - \mathbf{y})^{T}(\mathbf{X}\mathbf{w} - \mathbf{y})
\end{equation}

Using basic matrix derivative concepts we can compute the gradient of $J(\mathbf{w})$ with respect to $\mathbf{w}$:

\begin{equation}
\nabla_{\mathbf{w}}J(\mathbf{w}) = \frac{2}{N} (\mathbf{X}^{T}\mathbf{X}\mathbf{w} -\mathbf{X}^{T}\mathbf{y})
\end{equation}

Thus, when $\nabla_{\mathbf{w}}J(\mathbf{w}) = 0$ we have

\begin{equation}
\mathbf{X}^{T}\mathbf{X}\mathbf{w} = \mathbf{X}^{T}\mathbf{y}
\end{equation}

Hence,

\begin{equation}
\mathbf{w} = (\mathbf{X}^{T}\mathbf{X})^{-1}\mathbf{X}^{T}\mathbf{y}
\end{equation}

Note that this solution has a high computational cost. As the number of variables (*features*) increases, the cost for matrix inversion becomes prohibitive. See  [this text](https://sgfin.github.io/files/notes/CS229_Lecture_Notes.pdf) for more details.

# Exercise 1
Using only **NumPy** (a quick introduction to this library can be found  [here](http://cs231n.github.io/python-numpy-tutorial/)), complete the two functions below. Recall that $\mathbf{X} \in \mathbb{R}^{N\times d}$; thus you will need to add a component of value 1 to each of  the observations in $\mathbf{X}$ before performing the computation described above.

NOTE: Although the dataset above has data of dimension $d=1$, your code must be generic (it should work for $d\geq1$)

## 1.1. Weight computation function

```python
def normal_equation_weights(X, y):
    """
    Calculates the weights of a linear function using the normal equation method.
    You should add into X a new column with 1s.

    :param X: design matrix
    :type X: np.ndarray(shape=(N, d))
    :param y: regression targets
    :type y: np.ndarray(shape=(N, 1))
    :return: weight vector
    :rtype: np.ndarray(shape=(d+1, 1))
    """

    # START OF YOUR CODE:
    N = X.shape[0]
    X_tilde = np.column_stack((np.ones((N, 1)), X))
    X_cross = np.dot(np.linalg.inv(np.dot(X_tilde.T, X_tilde)), X_tilde.T)
    w = np.dot(X_cross, y)

    return w
    # END OF YOUR CODE
```

```python
# test of function normal_equation_weights()

w = 0  # this is not necessary
w = normal_equation_weights(X, y)
print("Estimated w =\n", w)
```

    Estimated w =
     [[12043.47818971]
     [   49.88841379]]

## 1.2. Prediction function

```python
def normal_equation_prediction(X, w):
    """
    Calculates the prediction over a set of observations X using the linear function
    characterized by the weight vector w.
    You should add into X a new column with 1s.

    :param X: design matrix
    :type X: np.ndarray(shape=(N, d))
    :param w: weight vector
    :type w: np.ndarray(shape=(d+1, 1))
    :param y: regression prediction
    :type y: np.ndarray(shape=(N, 1))
    """

    # START OF YOUR CODE:
    N = X.shape[0]
    X_tilde = np.column_stack((np.ones((N, 1)), X))
    y = np.dot(X_tilde, w)

    return y
    # END OF YOUR CODE
```

## 1.3. Coefficient of determination
We can use the [$R^2$](https://pt.wikipedia.org/wiki/R%C2%B2) metric (Coefficient of determination) to evaluate how well the linear model fits the data.

**Which $𝑅^2$ value would you expect to observe ?**

```python
from sklearn.metrics import r2_score

# test of function normal_equation_prediction()
prediction = normal_equation_prediction(X, w)

# compute the R2 score using the r2_score function from sklearn
# Replace 0 with an appropriate call of the function

# START OF YOUR CODE:
r_2 = r2_score(y, prediction)
# END OF YOUR CODE

plot_points_regression(X,
                       y,
                       title='Real estate prices prediction',
                       xlabel="m\u00b2",
                       ylabel='$',
                       prediction=prediction,
                       legend=True,
                       r_squared=r_2)
```

## Additional tests

Let us compute a prediction for $x=650$

```python
# Let us use the prediction function
x = np.asarray([650]).reshape(1,1)
prediction = normal_equation_prediction(x, w)
print("Area = %.2f  Predicted price = %.4f" %(x[0], prediction))
```

    Area = 650.00  Predicted price = 44470.9472

## 1.4. Processing time

Experiment with different nummber of samples $N$ and observe how processing time varies.

Be careful not to use a too large value; it may make jupyter freeze ...

```python
# Add other values for N
# START OF YOUR CODE:
N = [100, 200, 500, 1000, 5000, 10000, 100000, 10000000]
# END OF YOUR CODE

for i in N:
    X, y = get_housing_prices_data(N=i)
    init = time.time()
    w = normal_equation_weights(X, y)
    prediction = normal_equation_prediction(X,w)
    init = time.time() - init

    print("\nExecution time = {:.8f}(s)\n".format(init))
```

    X shape = (100, 1)
    y shape = (100, 1)

    X: mean 645.0, sdt 323.65, max 1200.00, min 90.00
    y: mean 44131.31, sdt 16462.71, max 77546.66, min 10220.15

    Execution time = 0.00058222(s)

    X shape = (200, 1)
    y shape = (200, 1)

    X: mean 645.0, sdt 322.04, max 1200.00, min 90.00
    y: mean 44643.07, sdt 16741.32, max 77416.36, min 11328.75

    Execution time = 0.00007701(s)

    X shape = (500, 1)
    y shape = (500, 1)

    X: mean 645.0, sdt 321.07, max 1200.00, min 90.00
    y: mean 44582.73, sdt 16791.18, max 83074.05, min 9172.27

    Execution time = 0.00033569(s)

    X shape = (1000, 1)
    y shape = (1000, 1)

    X: mean 645.0, sdt 320.75, max 1200.00, min 90.00
    y: mean 44441.20, sdt 16539.07, max 79329.80, min 10420.06

    Execution time = 0.00034523(s)

    X shape = (5000, 1)
    y shape = (5000, 1)

    X: mean 645.0, sdt 320.49, max 1200.00, min 90.00
    y: mean 44266.45, sdt 16549.26, max 80368.79, min 5516.63

    Execution time = 0.00048804(s)

    X shape = (10000, 1)
    y shape = (10000, 1)

    X: mean 645.0000610351562, sdt 320.46, max 1200.00, min 90.00
    y: mean 44271.82, sdt 16555.38, max 85085.10, min 3211.80

    Execution time = 0.00039077(s)

    X shape = (100000, 1)
    y shape = (100000, 1)

    X: mean 645.0000610351562, sdt 320.43, max 1200.00, min 90.00
    y: mean 44233.35, sdt 16525.27, max 85116.39, min 2534.62

    Execution time = 0.00301242(s)

    X shape = (10000000, 1)
    y shape = (10000000, 1)

    X: mean 644.9998779296875, sdt 320.43, max 1200.00, min 90.00
    y: mean 44250.33, sdt 16514.00, max 89948.86, min 199.12

    Execution time = 0.29421997(s)

# Exercise 2

Let us test the code with $𝑑>1$.
We will use the data we have collected in our first class. The [file](https://edisciplinas.usp.br/pluginfile.php/5982803/course/section/6115454/QT1data.csv) can be found on e-disciplinas.

Let us try to predict the weight based on one or more features.

```python
import pandas as pd

# load the dataset
df = pd.read_csv('QT1data.csv')
df.head()
```

      Sex
      Age
      Height
      Weight
      Shoe number
      Trouser number

      0
      Female
      53
      154
      59
      36
      40

      1
      Male
      23
      170
      56
      40
      38

      2
      Female
      23
      167
      63
      37
      40

      3
      Male
      21
      178
      78
      40
      40

      4
      Female
      25
      153
      58
      36
      38

```python
df.describe()
```

      Age
      Height
      Weight
      Shoe number

      count
      130.000000
      130.000000
      130.000000
      130.000000

      mean
      28.238462
      170.684615
      70.238462
      39.507692

      std
      12.387042
      11.568491
      15.534809
      2.973386

      min
      3.000000
      100.000000
      15.000000
      24.000000

      25%
      21.000000
      164.250000
      60.000000
      38.000000

      50%
      23.000000
      172.000000
      69.500000
      40.000000

      75%
      29.000000
      178.000000
      80.000000
      41.000000

      max
      62.000000
      194.000000
      130.000000
      46.000000

```python
# Our target variable is the weight
y = df.pop('Weight').values
y
```

    array([ 59,  56,  63,  78,  58,  89,  68,  83,  70,  56,  65,  66,  78,
            75,  47,  68,  65,  99,  80,  62,  60,  84,  91,  60,  15,  85,
            56,  62,  69,  78,  60,  48,  66,  85, 101,  74,  52,  52,  80,
            72,  75,  78,  61,  74,  70,  90,  66,  79,  80,  65,  90,  69,
            58,  63,  62,  73,  55,  65,  62,  75,  48,  59,  74,  80,  51,
            90,  58, 117,  77,  75,  56,  50,  67,  93,  70,  76,  85,  50,
            86,  96,  63,  56,  90,  95, 130,  70,  83,  70,  64,  57,  54,
            69,  53,  28,  62,  68,  73,  54,  75,  85,  62,  69,  55,  82,
            84,  52,  64,  73,  86,  77,  64,  65,  55,  50,  98,  77,  51,
            66,  83,  61,  80,  81,  76,  78,  70,  75,  72,  80,  90,  53])

## 2.1. One feature ($d=1$)

We will use 'Height' as the input feature and predict the weight

```python
feature_cols = ['Height']
X = df.loc[:, feature_cols]
X.shape
```

    (130, 1)

Write the code for computing the following
- compute the regression weights using $\mathbf{X}$ and $\mathbf{y}$
- compute the prediction
- compute the $R^2$ value
- plot the regression graph (use appropriate values for the parameters of function <tt>plot_points_regression()</tt>)

```python
# START OF YOUR CODE:
w = normal_equation_weights(X, y)
prediction = normal_equation_prediction(X, w)
r_2 = r2_score(y, prediction)
plot_points_regression(X,
                       y,
                       title='Weight based on height prediction',
                       xlabel="Height",
                       ylabel='Weight',
                       prediction=prediction,
                       legend=True,
                       r_squared=r_2)
# END OF YOUR CODE
```

## 2.2 - Two input features ($d=2$)

Now repeat the exercise with using as input the features 'Height' and 'Shoe number'

- compute the regression weights using $\mathbf{X}$ and $\mathbf{y}$
- compute the prediction
- compute and print the $R^2$ value

Note that our plotting function can not be used. There is no need to do plotting here.

```python
# START OF YOUR CODE:
feature_cols = ['Height', 'Shoe number']
X = df.loc[:, feature_cols]

w = normal_equation_weights(X, y)
prediction = normal_equation_prediction(X, w)
r_2 = r2_score(y, prediction)

print(r_2)
# END OF YOUR CODE
```

    0.45381183096658584

## 2.3 - Three input features ($d=3$)

Now try with three features. There is no need to do plotting here.
- compute the regression weights using $\mathbf{X}$ and $\mathbf{y}$
- compute the prediction
- compute and print the $R^2$ value

```python
# START OF YOUR CODE:
feature_cols = ['Height', 'Shoe number', 'Age']
X = df.loc[:, feature_cols]

w = normal_equation_weights(X, y)
prediction = normal_equation_prediction(X, w)
r_2 = r2_score(y, prediction)

print(r_2)
# END OF YOUR CODE
```

    0.4776499498669615

## 2.4 - Your comments

Did you observe anything interesting with varying values of $d$ ? Comment about it.

YOUR COMMENT BELOW:

As you increase $d$, the $R^2$ score increases slightly.