```python
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Philosopher', sans-serif;
    font-weight: 400;
    font-size: 2.2em;
    line-height: 100%;
    color: rgb(0, 80, 120);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Philosopher', serif;
    font-weight: 400;
    font-size: 1.9em;
    line-height: 100%;
    color: rgb(245,179,64);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Philosopher', serif;
    margin-top:12px;
    margin-bottom: 3px;
    font-style: italic;
    color: rgb(94,127,192);
}

.text_cell_render h4 {
    font-family: 'Philosopher', serif;
}

.text_cell_render h5 {
    font-family: 'Alegreya Sans', sans-serif;
    font-weight: 300;
    font-size: 16pt;
    color: grey;
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h6 {
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 10pt;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 100%;
}

</style>

### Library import

```python
from sympy import init_printing, Matrix, symbols, eye, Rational
init_printing()
```

# Elimination

## A system of linear equations

In the previous lesson, we had a brief glimpse at linear systems.  The _linear_ in _linear systems_ refers to the fact that each variable appears on its own (i.e. to the power $1$ and not in the form $x \times y$ or the like) and it is not transcendental.  If a solution exists, it then satisfies all of the equations at once.  We will consider the linear system in (1).

$$ \begin{align} 1x+2y+1z &= 2 \\ 3x + 8y + 1z &= 12 \\ 0x + 4y + 1z &= 2 \end{align} \tag{1} $$

A possible solution for $x,y$, and $z$ is given in (2), where $x=2$, $y=1$, and $z = -2$.

$$ \begin{align} 1\left(2\right)+2\left(1\right)+1\left(-2\right) &= 2 \\ 3\left(2\right)+8\left(1\right)+1\left(-2\right) &= 12 \\ 0\left(2\right)+4\left(1\right)+1\left(-2\right) &= 2 \end{align} \tag{2} $$

Since (1) is a set (three) equations that have a solution ( or possibly solutions) for their variables in common, all left- and all right hand sides can be manipulated in certain ways.

We could simply exchange the order of the equations.  In (3) the second and third equations have been exchanged, called _row exchange_.

$$ \begin{align}1x+2y+1z &= 2 \\ 0x + 4y + 1z &= 2 \\ 3x + 8y + 1z &= 12 \end{align} \tag{3} $$

We could multiply both the left- and right-hand side of one of the equations with a scalar.  In (4) we multiply the first equation by $2$.

$$ \begin{align} 2x+4y+2z &= 4 \\ 3x + 8y + 1z &= 12 \\ 0x + 4y + 1z &= 2 \end{align} \tag{4}$$

Lastly, we can subtract a constant multiple of one equation from another.

These three _manipulations_ serve an excellent purpose, as it allows us to eliminate of one (or more) of the variables (that is, give it a coefficient of $0$).  Remember that we are trying to solve for all three equations and have three unknowns.  We can most definitely struggle by doing this problem algebraically by substitution, but linear algebra makes it much easier.

In (5) we have multiplied the first equation by $3$ (both sides, so that we maintain integrity of the equation) and subtracted the left hand side of this new equation from the left-hand side of the second equation and the new right-hand side of the first equation from the right-hand side of the second equation.  This is quite legitimate, as the left- and right-hand sides are equal (it is an equation after all) and so, when subtracting from the second equation, we are still doing the same thing to the left-hand side as the right-hand side.

$$ \begin{align} 1x+2y+1z &= 2 \\ 0x + 2y - 2z &= 6 \\ 0x + 4y + 1z &= 2 \end{align} \tag{5} $$

This has introduced a nice $0$ in the second equation.  Let's go further and multiply the second equation by $2$ and subtract that from the third equation as seen in (6) below.

$$ \begin{align} 1x+2y+1z &= 2 \\ 0x + 2y - 2z &= 6 \\ 0x + 0y + 5z &= -10 \end{align} \tag{6} $$

Now let last equation is easy to solve for $z$.

$$ z=-2 \tag{7}$$

Knowing this, we can go back up to the second equation and solve for $y$.

$$ \begin{align} 2y+2(-2) &= 6 \\ y &= 1 \end{align} \tag{8} $$

Finally, up to the first equation.

$$ \begin{align} x+2(1)+1(-2) &= 2 \\ x &= 2 \end{align} \tag{9} $$

W have solve the linear system by substitution.  We need to have gone straight for substitution, though.  Indeed, we could have tried to get zeros above all our leading (non-zero) coefficients.  Let's just clean up the third equation by multiplying throughout by $\frac{1}{5}$ as in (10) below.

$$ \begin{align} 1x+2y+1z &= 2 \\ 0x + 2y - 2z &= 6 \\ 0x + 0y + 1z &= -2 \end{align} \tag{10} $$

Now we have to get rid of the $-2z$ in the second equation, which we can do by multiplying the third equation by $-2$ and subtracting from the second equation.

$$ \begin{align}  1x+2y+1z &= 2 \\ 0x + 2y - 0z &= 2 \\ 0x + 0y + 1z &= -2 \end{align} \tag{11}$$

Multiplying the second equation by $\frac{1}{2}$ yields (12).

$$ \begin{align}1x+2y+1z &= 2 \\ 0x + 1y + 0z &= 1 \\ 0x + 0y + 1z &= -2 \end{align} \tag{12} $$

Now we can do the same to get rid of the $1z$ in the first equation (multiply the third equation by $1$ and subtract it from the first equation.

$$ \begin{align} 1x+2y+0z &= 4 \\ 0x + 1y + 0z &= 1 \\ 0x + 0y + 1z &= -2 \end{align} \tag{12}$$

Now tow get rid of the $2y$ in the first equation, which is above our leading $1y$ in the second equation.  Simple enough, we multiply he second equation by $2$ and subtract that from the first equation.

$$ \begin{align} 1x+0y+0z &= 2 \\ 0x + 1y + 0z &= 1 \\ 0x + 0y + 1z &= -2 \end{align} \tag{13} $$

The solution is now clear for $x,y$, and $z$.

We need not rewrite all of the variables all the time.  We can simply write the coefficients.  The augmented matrix of coefficients is in (14).

$$ \begin{bmatrix} 1&2&1&2\\3&8&1&12\\0&4&1&2 \end{bmatrix} \tag{14} $$

A matrix has rows and columns (attached, in position, to our algebraic equation above).  We simply omit the variables.  The left-upper entry is called the pivot.  Our aim is to get everything below it to be a zero (as we did with the algebra).  We do exactly the same as we did above, which is multiply row 1 by 3 and subtract these new values from row 2.

$$ \begin{bmatrix} 1&2&1&2\\0&2&-2&6\\0&4&1&2 \end{bmatrix} \tag{15} $$

Now $2$ times row 2 subtracted from row 3.

$$ \begin{bmatrix} 1&2&1&2\\0&2&-2&6\\0&0&5&-10 \end{bmatrix} \tag{16} $$

Multiply the last row with $\frac{1}{5}$.

$$ \begin{bmatrix} 1&2&1&2\\0&2&-2&6\\0&0&1&-2 \end{bmatrix} \tag{17} $$

This shows $z = -2$ in the last row of (17).

With this small matrix, it's easy to do back substitution as we did algebraically.  The first non-zero number in each row is the pivot (just like the upper-left entry).  The steps we have taken up to this point is called _Gauss elimination_ and the form we end up with is _row-echelon form_.  We could carry on and do the same sort of thing to get rid of all the non-zero entries above each pivot.  This is called _Gauss-Jordan elimination_ and the result is _reduced row-echelon form_ (see the computer code below).

All of these steps are called _elementary row operations_.  The only one we didn't do is _row exchange_.  We reserve this action so as not to have leading (in the pivot position) zeros.

Let's create some code to show-case elementary row operations.

```python
A_augmented = Matrix([[1, 2, 1, 2], [3, 8, 1, 12], [0, 4, 1, 2]])
A_augmented
```

We can ask `sympy` to simply get the augmented matrix in reduced row-echelon form and read off the solutions.  This is done with the `.rref()` method.

```python
A_augmented.rref() # The rref() method returns the reduced row-echelon form
```

## Elimination matrices

Matrices can only be multiplied by each other if in order we have the first column size equal the second row size.  Rows are usually called $m$ and columns $n$ when considering their dimensions.  So, our augmented matrix above will be $m \times n = 3 \times 4$.

Let's look at how matrices are multiplied by looking at two small matrices in (18).

$$ \begin{bmatrix} {a}_{11}&{a}_{12} \\ {a}_{21}&{a}_{22} \end{bmatrix} \\ \\ \begin{bmatrix} {b}_{11}&{b}_{12}\\{b}_{21}&{b}_{22} \end{bmatrix} \tag{18} $$

The subscripts refer to row and column position, i.e. $21$ means row $2$ column $1$>

We see that we have a two $ 2 \times 2 $ matrices.  The *inner* two values are the same ($2$ and $2$), so this multiplication is allowed.  The resultant matrix will have the size equal to the *outer* two values (first row and last columns); here also a $2 \times 2$ matrix.

So let's look at position $11$ (row $1$ and column $1$).  To get this we take the entries in row $1$ of the first matrix and multiply them by the entries in the first column of the second matrix.  We do this element by element and add the multiplication of each set of separate elements to each other.  The python code below shows you exactly how this is done.

```python
a11, a12, a21, a22, b11, b12, b21, b22 = symbols('a11 a12 a21 a22 b11 b12 b21 b22')
```

```python
A = Matrix([[a11, a12], [a21, a22]])
B = Matrix([[b11, b12], [b21, b22]])
A, B
```

```python
A * B
```

Let's constrain ourselves to the matrix of coefficients (this discards the right-hand side from the augmented matrix above).

```python
A = Matrix([[1, 2, 1], [3, 8, 1], [0, 4, 1]]) # I use the same computer variable above, which
# will change its value in the computer memory
A # A 3 by 3 matrix, which we call square
```

The _identity matrix_ is akin to the number $1$, i.e. multiplying by it leaves everything unchanged.  It has ones along what is called the main diagonal and zeros everywhere else.

```python
I = eye(3) # Identity matrices are always square and the argument
# here is 3, so it is a 3 by 3 matrix
I # Note what the main diagonal is
```

Let's multiply $I$ by $A$.

```python
I * A # Nothing will change
```

To get rid of the leading $3$ in the second row(because we want a $0$ under the first pivot in the first row), we multiply the first row by $3$ and subtracted that from the second row.  Interestingly enough, we can do the same to the identity matrix.

```python
E21 = Matrix([[1, 0, 0], [-3, 1, 0], [0, 0, 1]])
E21 # 21 because we are working on row 2, column 1
```

That gives us the required 3 times the first row and the negative shows that we subtract (add the negative).  It's a thing of beauty!

```python
E21 * A
```

Just what we wanted.  $E1$ is called the first elimination matrix.

Let's do something to the identity matrix to get rid of the $4$ in the third row (the second column).  It would require $2$ times the second row subtracted from the third row.  Look carefully at the positions.

```python
E32 = Matrix([[1, 0, 0], [0, 1, 0], [0, -2, 1]])
E32
```

```python
E32 * (E21 * A)
```

Spot on!  We now have nice pivots (leading non-zeros), with nothing under them (along the columns).  As a tip, try not to get fractions involved.  As far as the other two row operations are concerned, we can either exchange rows in the identity matrix or multiply the required row by a scalar constant.

Look at what happens when we multiply $E2$ and $E1$.

```python
L_inv = E32 * E21
L_inv
```

Later we'll call this matrix the inverse of $L$.  It is in triangular form, in this case lower triangular (note all the zeros above the main diagonal).

```python
L_inv * A # Later we'll call this result the matrix U
```

We now have the following, shown in (19).

$$ {L}^{-1}{A}={U} \tag{19} $$

Left-multiplying by $L$ leaves (20).

$$ {L}{L}^{-1}{A}={L}{U} \tag{20} $$

The inverse of a square matrix multiplied by itself gives the identity matrix.

$$ {I}{A}={L}{U} \\ {A}={L}{U} \tag{20} $$

We can construct $L$ from $E32$ and $E21$ above.

$$ {E}_{21}^{-1}{E}_{32}^{-1}{E}_{32}{E}_{21}={E}_{21}^{-1}{E}_{32}^{-1}{U} \\ \therefore {E}_{21}^{-1}{E}_{32}^{-1}={L} \tag{21} $$

```python
E21.inv() # The inverse is easy to understand in words
# We just want to add 3 instead of subtracting 3
```

```python
E32.inv()
```

```python
E21.inv() * E32.inv()
```

This is exactly the inverse of our inverse of $L$ above.

```python
L_inv.inv()
```

This is called _LU-decomposition_ of $A$.  More about this in two chapters from now (I_05_LU_decomposition).

As an aside we can also do elementary column operation, but then we have to multiply on the right of $A$ and not on the left as above.

## Example problems

### Example problem 1

Solve the linear system in (22).

$$ \begin{align} x-y-z+u &= 0 \\ 2x+2z &= 8 \\ -y-2z &= -8 \\ 3x-3y-2z+4u &= 7 \end{align} \tag{22} $$

#### Solution

```python
A_augm = Matrix([[1, -1, -1, 1, 0], [2, 0, 2, 0, 8], [0, -1, -2, 0, -8], [3, -3, -2, 4, 7]])
A_augm
```

```python
A_augm.rref()
```

Whoa! That was easy!  Let's take it a notch down and create some elementary matrices.  First off, we want the matrix of coefficients.

```python
A = Matrix([[1, -1, -1, 1], [2, 0, 2, 0], [0, -1, -2, 0], [3, -3, -2, 4]])
A
```

Now we need to get rid of the $2$ in position row `2`, column `1`.  We start by numbering the elementary matrix by this position and modifying the identity matrix.

```python
E21 = Matrix([[1, 0, 0, 0], [-2, 1, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]])
E21 * A
```

Now for position row `3`, column `2`.  We have to use row `2` to do this.  If we used row `1`, we would introduce a non-zero into position row `3`, column `1`.

```python
E32 = Matrix([[1, 0, 0, 0], [0, 1, 0, 0], [0, Rational(1, 2), 1, 0], [0, 0, 0, 1]])
E32 * (E21 * A)
```

Now for the $3$ in position row `4`, column `1`.

```python
E41 = Matrix([[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [-3, 0, 0, 1]])
E41 * (E32 * E21 * A)
```

Let's exchange rows `3` and `4`.

```python
Ee34 = Matrix([[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0]])
Ee34 * E41 * E32 * E21 * A
```

Let's see where that leaves $\underline{b}$.  After all, what we do to the left, we must do to the right.

$$ {Ee}_{34}\times{E}_{41}\times{E}_{32}\times{E}_{21}{A}{x}={Ee}_{34}\times{E}_{41}\times{E}_{32}\times{E}_{21}{b} \tag{23}$$

```python
b_vect = Matrix([[0], [8], [-8], [7]])
b_vect
```

```python
Ee34 * E41 * E32 * E21 * b_vect
```

Let's print them next to each other on the screen.

```python
Ee34 * E41 * E32 * E21 * A, Ee34 * E41 * E32 * E21 * b_vect
```

So we can simply do back substitution.  We note that $-1u = -4$ and thus $u = 4$.  From here, we work our way back up.

$$ \begin{align} -1(u) = -4 \quad &\therefore \quad u=4 \\ 1(z)+1(4) = 7 \quad &\therefore \quad z=3 \\  2(y) + 4(3) - 2(4) = 8 \quad &\therefore \quad y=2 \\ 1(x)-1(2)-1(3)+1(4)=0 \quad &\therefore \quad x=1 \end{align} \tag{24}$$

```python

```

Mathematics/Linear Algebra/0.0 MIT-18.06 - Jupyter/Lecture_2_Elimination.ipynb


### Checklist for submission

It is extremely important to make sure that:

1. Everything runs as expected (no bugs when running cells);
2. The output from each cell corresponds to its code (don't change any cell's contents without rerunning it afterwards);
3. All outputs are present (don't delete any of the outputs);
4. Fill in all the places that say `# YOUR CODE HERE`, or "**Your answer:** (fill in here)".
5. Never copy/paste any notebook cells. Inserting new cells is allowed, but it should not be necessary.
6. The notebook contains some hidden metadata which is important during our grading process. **Make sure not to corrupt any of this metadata!** The metadata may for example be corrupted if you copy/paste any notebook cells, or if you perform an unsuccessful git merge / git pull. It may also be pruned completely if using Google Colab, so watch out for this. Searching for "nbgrader" when opening the notebook in a text editor should take you to the important metadata entries.
7. Although we will try our very best to avoid this, it may happen that bugs are found after an assignment is released, and that we will push an updated version of the assignment to GitHub. If this happens, it is important that you update to the new version, while making sure the notebook metadata is properly updated as well. The safest way to make sure nothing gets messed up is to start from scratch on a clean updated version of the notebook, copy/pasting your code from the cells of the previous version into the cells of the new version.
8. If you need to have multiple parallel versions of this notebook, make sure not to move them to another directory.
9. Although not forced to work exclusively in the course Docker environment, you need to make sure that the notebook will run in that environment, i.e. that you have not added any additional dependencies.

Failing to meet any of these requirements might lead to either a subtraction of POEs (at best) or a request for resubmission (at worst).

We advise you the following steps before submission for ensuring that requirements 1, 2, and 3 are always met: **Restart the kernel** (in the menubar, select Kernel$\rightarrow$Restart) and then **run all cells** (in the menubar, select Cell$\rightarrow$Run All). This might require a bit of time, so plan ahead for this (and possibly use Google Cloud's GPU in HA1 and HA2 for this step). Finally press the "Save and Checkout" button before handing in, to make sure that all your changes are saved to this .ipynb file.

### Check Python version

```python
from platform import python_version_tuple
assert python_version_tuple()[:2] == ('3','7'), "You are not running Python 3.7. Make sure to run Python through the course Docker environment, or alternatively in the provided Conda environment."
```

### Check that notebook server has access to all required resources, and that notebook has not moved

```python
import os
nb_dirname = os.path.abspath('')
assert nb_dirname != '/notebooks', \
    '[ERROR] The notebook server appears to have been started at the same directory as the assignment. Make sure to start it at least one level above.'
assignment_name = os.path.basename(nb_dirname)
assert assignment_name in ['IHA1', 'IHA2', 'HA1', 'HA2', 'HA3'], \
    '[ERROR] The notebook appears to have been moved from its original directory'
```

### Run the following cells to verify that your notebook is up-to-date and not corrupted in any way

```javascript
%%javascript
IPython.notebook.kernel.execute(`nb_fname = '${IPython.notebook.notebook_name}'`);
```

```python
import sys
sys.path.append('..')
from ha_utils import check_notebook_uptodate_and_not_corrupted
check_notebook_uptodate_and_not_corrupted(nb_dirname, nb_fname)
```

### Fill in group number and member names:

```python
GROUP = ""
NAME1 = ""
NAME2 = ""
```

# IHA1 - Assignment

Welcome to the first individual home assignment!

This assignment consists of two parts:
 * Python and NumPy exercises
 * Build a deep neural network for forward propagation

The focus of this assignment is for you to gain practical knowledge with implementing forward propagation of deep neural networks without using any deep learning framework. You will also gain practical knowledge in two of Python's scientific libraries [NumPy](https://docs.scipy.org/doc/numpy-1.13.0/index.html) and [Matplotlib](https://matplotlib.org/devdocs/index.html).

Skeleton code is provided for most tasks and every part you are expected to implement is marked with **TODO**

We expect you to search and learn by yourself any commands you think are useful for these tasks. Don't limit yourself to only what was taught in CL1. Use the help function, [stackoverflow](https://stackoverflow.com/), google, the [python documentation](https://docs.python.org/3.5/library/index.html) and the [NumPy](https://docs.scipy.org/doc/numpy-1.13.0/index.html) documentation to your advantage.

**IMPORTANT NOTE**: The tests available are not exhaustive, meaning that if you pass a test you have avoided the most common mistakes, but it is still not guaranteed that you solution is 100% correct.

Lets start by importing the necessary libraries below

```python
import numpy as np
import matplotlib.pyplot as plt
from utils.tests.iha1Tests import *
```

## 1. Lists and arrays introduction
First, we will warm up with a Python exercise and few NumPy exercises

### 1.1 List comprehensions
Examine the code snippet provided below

```python
myList = []
for i in range(25):
    if i % 2 == 0:
        myList.append(i**2)

print(myList)
```

This is not a very "[pythonic](http://docs.python-guide.org/en/latest/writing/style/)" way of writing. Lets re-write the code above using a [list comprehension](https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions). The result will be less code, more readable and elegant. Your solution should be able to fit into one line of code.

```python
myList = None # TODO
# YOUR CODE HERE
print(myList)
```

```python
# sample output from cell above for reference
# [0, 4, 16, 36, 64, 100, 144, 196, 256, 324, 400, 484, 576]
```

### 1.2 Numpy array vs numpy vectors
Run the cell below to create a numpy array.

```python
myArr = np.array([1, 9, 25, 49, 81, 121, 169, 225, 289, 361, 441, 529])
print(myArr)
print(myArr.shape)
```

One of the core features of numpy is to efficiently perform linear algebra operations.
There are two types of one-dimensional representations in numpy: arrays of shape (x,) and vectors of shape (x,1)

The the above result indicates that **myArr** is an array of 12 elements with shape (12,).

Numpy's arrays and vectors both have the type of `numpy.ndarray` but have in some cases different characteristics and it is important to separate the two types because it will save a lot of debugging time later on. Read more about numpy shapes [here](https://stackoverflow.com/a/22074424)

Run the code below to see how the transpose operation behaves differently between an array and vector

```python
# print the shape of an array and the shape of a transposed array
print('myArr is an array of shape:')
print(myArr.shape)
print('The transpose of myArr has the shape:')
print(myArr.T.shape)

# print the shape of a vector and the transpose of a vector
myVec = myArr.reshape(12,1)
print('myVec is a vector of shape:')
print(myVec.shape)
print('The transpose of myVec has the shape:')
print(myVec.T.shape)
```

### 1.3 Numpy exercises
Now run the cell below to create the numpy array `numbers` and then complete the exercises sequentially

```python
numbers = np.arange(24)
print(numbers)
```

```python
# TODO: reshape numbers into a 6x4 matrix

# YOUR CODE HERE
print(numbers)

```

```python
# sample output from cell above for reference
#  [[ 0  1  2  3]
#   [ 4  5  6  7]
#   [ 8  9 10 11]
#   [12 13 14 15]
#   [16 17 18 19]
#   [20 21 22 23]]
```

```python
# test case
test_numpy_reshape(numbers)
```

```python
# TODO: set the element of the last row of the last column to zero
# Hint: Try what happends when indices are negative

# YOUR CODE HERE
print(numbers)
```

```python
# sample output from cell above for reference
#  [[ 0  1  2  3]
#   [ 4  5  6  7]
#   [ 8  9 10 11]
#   [12 13 14 15]
#   [16 17 18 19]
#   [20 21 22  0]]
```

```python
# test case
test_numpy_neg_ix(numbers)
```

```python
# TODO: set every element of the 0th row to 0

# YOUR CODE HERE
print(numbers)
```

```python
# sample output from cell above for reference
#  [[ 0  0  0  0]
#   [ 4  5  6  7]
#   [ 8  9 10 11]
#   [12 13 14 15]
#   [16 17 18 19]
#   [20 21 22  0]]
```

```python
# test case
test_numpy_row_ix(numbers)
```

```python
# TODO: append a 1x4 row vector of zeros to `numbers`,
# resulting in a 7x4 matrix where the new row of zeros is the last row
# Hint: A new matrix must be created in the procedure. Numpy arrays are not dynamic.

# YOUR CODE HERE
print(numbers)
print(numbers.shape)
```

```python
# sample output from cell above for reference
#   [[ 0  0  0  0]
#    [ 4  5  6  7]
#    [ 8  9 10 11]
#    [12 13 14 15]
#    [16 17 18 19]
#    [20 21 22  0]
#    [ 0  0  0  0]]
#  (7, 4)
```

```python
# test case
test_numpy_append_row(numbers)
```

```python
# TODO: set all elements above 10 to the value 1

# YOUR CODE HERE
print(numbers)
```

```python
# sample output from cell above for reference
#  [[ 0  0  0  0]
#   [ 4  5  6  7]
#   [ 8  9 10  1]
#   [ 1  1  1  1]
#   [ 1  1  1  1]
#   [ 1  1  1  0]
#   [ 0  0  0  0]]
```

```python
# test case
test_numpy_bool_matrix(numbers)
```

```python
# TODO: compute the sum of every row and replace `numbers` with the answer
# `numbers` will be a (7,) array as a result

# YOUR CODE HERE
print(numbers.shape)
print(numbers)
```

```python
# sample output from cell above for reference
#   (7,)
#   [ 0 22 28  4  4  3  0]
```

```python
# test case
test_numpy_sum(numbers)
```

## 2 Building your deep neural network
It is time to start implementing your first feed-forward neural network. In this lab you will only focus on implementing the forward propagation procedure.

When using a neural network, you can not forward propagate the entire dataset at once. Therefore, you divide the dataset into a number of sets/parts called batches. A batch will make up for the first dimension of every input to a layer and the notation `(BATCH_SIZE, NUM_FEATURES)` simply means the dimension of a batch of samples where every sample has `NUM_FEATURES` features

### 2.1 activation functions
You will start by defining a few activation functions that are later needed by the neural network.

#### 2.1.1 ReLU
The neural network will use the ReLU activation function in every layer except for the last. ReLU does element-wise comparison of the input matrix. For example, if the input is `X`, and `X[i,j] == 2` and `X[k,l] == -1`, then after applying ReLU, `X[i,j] == 2` and `X[k,l] == 0` should be true.

The formula for implementing ReLU for a single neuron $i$ is:
\begin{equation}
relu(z_i) =
    \begin{cases}
      0, & \text{if}\ z_i \leq 0 \\
      z_i, & \text{otherwise}
    \end{cases}
\end{equation}

Now implement `relu` in vectorized form

```python
def relu(z):
    """ Implement the ReLU activation function

    Arguments:
    z - the input of the activation function. Has a type of `numpy.ndarray`

    Returns:
    a - the output of the activation function. Has a type of numpy.ndarray and the same shape as `z`
    """

    a = None # TODO
    # YOUR CODE HERE

    return a
```

```python
# test case
test_relu(relu)
```

#### 2.1.2 Sigmoid
The sigmoid activation function is common for binary classification. This is because it squashes its input to the range [0,1].
Implement the activation function `sigmoid` using the formula:
\begin{equation}
    \sigma(z) = \frac{1}{1 + e^{-z}}
\end{equation}

```python
def sigmoid(z):
    """ Implement the sigmoid activation function

    Arguments:
    z - the input of the activation function. Has a type of `numpy.ndarray`

    Returns:
    a - the output of the activation function. Has a type of `numpy.ndarray` and the same shape as `z`
    """

    a = None # TODO
    # YOUR CODE HERE

    return a
```

```python
# test case
test_sigmoid(sigmoid)
```

#### 2.1.3 Visualization
Make a plot using matplotlib to visualize the activation functions between the input interval [-3,3]. The plot should have the following properties
 * one plot should contain a visualization of both `ReLU` and `sigmoid`
 * x-axis: range of values between [-3,3], **hint**: np.linspace
 * y-axis: the value of the activation functions at a given input `x`
 * a legend explaining which line represents which activation function

```python
# TODO: make a plot of ReLU and sigmoid values in the interval [-3,3]

# YOUR CODE HERE
```

#### 2.1.4 Softmax
You will use the softmax activation function / classifier as the final layer of your neural network later in the assignment. Implement `softmax` according the the formula below. The subtraction of the maximum value is there solely to avoid overflows in a practical implementation.
\begin{equation}
softmax(z_i) = \frac{e^{z_i - max(\mathbf{z})}}{ \sum^j e^{z_j - max(\mathbf{z})}}
\end{equation}

```python
def softmax(z):
    """ Implement the softmax activation function

    Arguments:
    z - the input of the activation function, shape (BATCH_SIZE, FEATURES) and type `numpy.ndarray`

    Returns:
    a - the output of the activation function, shape (BATCH_SIZE, FEATURES) and type umpy.ndarray
    """

    a = None # TODO
    # YOUR CODE HERE

    return a
```

```python
# test case
test_softmax(softmax)
```

### 2.2 Initialize weights
You will implement a helper function that takes the shape of a layer as input, and returns an initialized weight matrix $\mathbf{W}$ and bias vector $\mathbf{b}$ as output. $\mathbf{W}$ should be sampled from a normal distribution of mean 0 and standard deviation 2, and $\mathbf{b}$ should be initialized to all zeros.

```python
def initialize_weights(layer_shape):
    """ Implement initialization of the weight matrix and biases

    Arguments:
    layer_shape - a tuple of length 2, type (int, int), that determines the dimensions of the weight matrix: (input_dim, output_dim)

    Returns:
    w - a weight matrix with dimensions of `layer_shape`, (input_dim, output_dim), that is normally distributed with
        properties mu = 0, stddev = 2. Has a type of `numpy.ndarray`
    b - a vector of initialized biases with shape (1,output_dim), all of value zero. Has a type of `numpy.ndarray`
    """
    w = None # TODO
    b = None # TODO
    # YOUR CODE HERE

    return w, b
```

```python
# test case
test_initialize_weights(initialize_weights)
```

### 2.3 Feed-forward neural network layer module
To build a feed-forward neural network of arbitrary depth you are going to define a neural network layer as a module that can be used to stack upon eachother.

Your task is to complete the `Layer` class by following the descriptions in the comments.

Recall the formula for forward propagation of an arbitrary layer $l$:

\begin{equation}
\mathbf{a}^{[l]} = g(\mathbf{z}^{[l]}) = g(\mathbf{a}^{[l-1]}\mathbf{w}^{[l]} +\mathbf{b}^{[l]})
\end{equation}

$g$ is the activation function given by `activation_fn`, which can be relu, sigmoid or softmax.

```python
class Layer:
    """
    TODO: Build a class called Layer that satisfies the descriptions of the methods
    Make sure to utilize the helper functions you implemented before
    """

    def __init__(self, input_dim, output_dim, activation_fn=relu):
        """
        Arguments:
        input_dim - the number of inputs of the layer. type int
        output_dim - the number of outputs of the layer. type int
        activation_fn - a reference to the activation function to use. Should be `relu` as a default
                        possible values are the `relu`, `sigmoid` and `softmax` functions you implemented earlier.
                        Has the type `function`

        Attributes:
        w - the weight matrix of the layer, should be initialized with `initialize_weights`
            and has the shape (INPUT_FEATURES, OUTPUT_FEATURES) and type `numpy.ndarray`
        b - the bias vector of the layer, should be initialized with `initialize_weights`
            and has the shape (1, OUTPUT_FEATURES) and type `numpy.ndarray`
        activation_fn - a reference to the activation function to use.
                        Has the type `function`
        """
        self.w, self.b = None, None # TODO
        self.activation_fn = None # TODO
        # YOUR CODE HERE

    def forward_prop(self, a_prev):
        """ Implement the forward propagation module of the neural network layer
        Should use whatever activation function that `activation_fn` references to

        Arguments:
        a_prev - the input to the layer, which may be the data `X`, or the output from the previous layer.
            a_prev has the shape of (BATCH_SIZE, INPUT_FEATURES) and the type `numpy.ndarray`

        Returns:
        a - the output of the layer when performing forward propagation. Has the type `numpy.ndarray`
        """

        a = None # TODO
        # YOUR CODE HERE

        return a
```

```python
# test case, be sure that you pass the previous activation function tests before running this test
test_layer(Layer, relu, sigmoid, softmax)
```

### 2.4 Logistic regression
Binary logistic regression is a classifier where classification is performed by applying the sigmoid activation function to a linear combination of input values. You will now try out your neural network layer by utilizing it as a linear combination of input values and apply the sigmoid activation function to classify a simple problem.

The cell below defines a dataset of 5 points of either class `0` or class `1`. Your assignment is to:
1. Create an instance of a `Layer` with sigmoid activation function
2. Manually tune the weights `w` and `b` of your layer

You can use `test_logistic` to visually inspect how your classifier is performing.

```python
# Run this cell to create the dataset
X_s = np.array([[1, 2],
               [5, 3],
               [8, 8],
               [7, 5],
               [3, 6]])
Y_s = np.array([0,0,1,0,1])

test_logistic(X_s, Y_s)
```

```python
# create an instance of layer
l = Layer(2,1,sigmoid)

# TODO: manually tune weights
l.w = None
l.b = None
# YOUR CODE HERE

# testing your choice of weights with this function
test_logistic(X_s,Y_s,l,sigmoid)
```

### 2.5 Feed-forward neural network
Now define the actual neural network class. It is an L-layer neural network, meaning that the number of layers and neurons in each layer is specified as input by the user. Once again, you will only focus on implementing the forward propagation part.

Read the descriptions in the comments and complete the todos

```python
class NeuralNetwork:
    """
    TODO: Implement an L-layer neural network class by utilizing the Layer module defined above
    Each layer should use `relu` activation function, except for the output layer, which should use `softmax`
    """

    def __init__(self, input_n, layer_dims):
        """
        Arguments:
        input_n    - the number of inputs to the network. Should be the same as the length of a data sample
                     Has type int
        layer_dims - a python list or tuple of the number of neurons in each layer. Layer `l` should have a weight matrix
                     with the shape (`layer_dims[l-1]`, `layer_dims[l]`).
                     `layer_dims[-1]` is the dimension of the output layer.
                     Layer 1 should have the dimensions (`input_n`, `layer_dims[0]`).
                     len(layer_dims) is the depth of the neural network
        Attributes:
        input_n - the number of inputs to the network. Has type int
        layers  - a python list of each layer in the network. Each layer should use the `relu` activation function,
                  except for the last layer, which should use `softmax`.
                  Has type `list` containing layers of type `Layer`
        """

        self.input_n = None # TODO
        self.layers = None # TODO
        # YOUR CODE HERE

    def forward_prop(self, x):
        """
        Implement the forward propagation procedure through the entire network, from input to output.
        You will now connect each layer's forward propagation function into a chain of layer-wise forward propagations.

        Arguments:
        x - the input data, which has the shape (BATCH_SIZE, NUM_FEATURES) and type `numpy.ndarray`

        Returns:
        a - the output of the last layer after forward propagating through the every layer in `layers`.
            Should have the dimension (BATCH_SIZE, layers[-1].w.shape[1]) and type `numpy.ndarray`
        """
        a = None # TODO
        # YOUR CODE HERE

        return a
```

```python
# test case
test_neuralnetwork(NeuralNetwork)
```

## 3 Making predictions with a neural network
In practice, its common to load weights to your neural network that has already been trained.
In this section, you will create an instance of your neural network, load trained weights from disk, and perform predictions.

### 3.1 Load weights from disk
Create an instance of `NeuralNetwork` with input size $28 \times 28 = 784$, two hidden layers of size 100 and an output layer of size 10. Thereafter, load the weights contained in `./utils/ann_weights.npz` to your network.

```python
ann = None # TODO: create instance of ann
# YOUR CODE HERE

# load weights
weights = np.load('./utils/ann_weights.npz')
for l in range(len(ann.layers)):
    ann.layers[l].w = weights['w' + str(l)]
    ann.layers[l].b = weights['b' + str(l)]
```

### 3.2 Prediction
Now, implement the function `predict_and_correct` which does the following:
1. Load `./utils/test_data.npz` from disk
2. Extract test data `X` and `Y` from file
2. Perform for every pair of data:
    a. plot the image `x`
    b. make a prediction using your neural network by forward propagating and picking the most probable class
    c. check wether the prediction is correct (compare with the ground truth number `y`)
    d. print the predicted label and wether it was correct or not

```python
def predict_and_correct(ann):
    """ Load test data from file and predict using your neural network.
    Make a prediction for ever data sample and print it along with wether it was a correct prediction or not

    Arguments:
    ann - the neural network to use for prediction. Has type `NeuralNetwork`

    Returns: # for test case purposes
    A `numpy.ndarray` of predicted classes (integers [0-9]) with shape (11,)
    """
    data = np.load('./utils/test_data.npz')
    X, cls = data['X'], data['Y']

    cls_preds = None # TODO: make a predicted number for every image in X
    # YOUR CODE HERE

    for i in range(len(X)):
        plt.imshow(X[i].reshape(28,28), cmap='gray')
        plt.show()
        correct = cls_preds[i] == cls[i]
        print('The prediction was {0}, it was {1}!'.format(cls_preds[i], 'correct' if correct else 'incorrect'))

    return cls_preds

cls_pred = predict_and_correct(ann)
```

```python
# final test case
test_predict_and_correct_answer(cls_pred)
```

## Congratulations!
You have successfully implemented a neural network from scratch using only NumPy!


```python
import sys
sys.path.append('..')
```

# Milestone 2

## Introduction

Automatic differentiation is a tool for calculating derivatives using machine accuracy. It has several advantages over traditional methods of derivative calculations such as symbolic and finite differentiation. Automatic differentiation is useful for calculating complex derivatives where errors are more likely with classical methods. For instance , with finite differentiation, h values that are too small will lead to accuracy errors though floating point roundoff error, while h values that are too large will start making vastly inaccurate approximations.

Automatic differentiation is useful due to its practicality in real world applications that involve thousands of parameters in a complicated function, which would take a long runtime as well as strong possibility for error in calculating the derivatives individually.

Our package allows users to calculate derivatives of complex functions, some with many parameters, allowing machine precision.

## Background

Essentially automatic differentiation works by  breaking down a complicated function and performing a sequence of elementary arithmetic such as addition, subtraction, multiplication, and division as well as elementary functions like exp, log, sin, etc. These operations are then repeated by the chain rule and the derivatives of these sequences are calculated. There are two ways that automatic differentiation can be implemented - forward mode and reverse mode.

### 2.1 The Chain Rule

The chain rule makes up a fundamental component of auto differentiation. The basic idea is:
For univariate function, $$ F(x) = f(g(x))$$

 $$F^{\prime} = (f(g))^{\prime} = f^{\prime}(g(x))g^{\prime}(x)$$

For multivariate function, $$F(x) = f(g(x),h(x))$$

$$ \frac{\partial F}{\partial x}=\frac{\partial f}{\partial g} \frac{\partial g}{\partial x}+\frac{\partial f}{\partial h} \frac{\partial h}{\partial x}$$

For generalized cases, if F is a combination of more sub-functions,  $$F(x) = f(g_{1}(x), g_{2}(x), …, g_{m}(x))$$

$$\frac{\partial F}{\partial x}=\sum_{i=1}^{m}\frac{\partial F}{\partial g_{i}} \frac{\partial g_{i}}{\partial x}$$

For F is a function f(g): f: $R^n$ -> $R^m$ and g: $R^m$ -> $R^k$,

$$\mathbf{J}_{\mathrm{gof}}(\mathbf{x})=\mathbf{J}_{\mathrm{g}}(\mathbf{f}(\mathbf{x})) \mathbf{J}_{\mathrm{f}}(\mathbf{x})$$

where $J(f) =\left[\begin{array}{ccc}{\frac{\partial \mathbf{f}}{\partial x_{1}}} & {\cdots} & {\frac{\partial \mathbf{f}}{\partial x_{n}}}\end{array}\right]=\left[\begin{array}{ccc}{\frac{\partial f_{1}}{\partial x_{1}}} & {\cdots} & {\frac{\partial f_{1}}{\partial x_{n}}} \\ {\vdots} & {\ddots} & {\vdots} \\ {\frac{\partial f_{m}}{\partial x_{1}}} & {\cdots} & {\frac{\partial f_{m}}{\partial x_{n}}}\end{array}\right]$ is the Jacobian Matrix.

### 2.2 Auto Differentiation: Forward Mode

The forward mode automatic differentiation is accomplished by firstly splitting the function process into one-by-one steps, each including only one basic operation. It focuses on calculating two things in each step, the value of scalar or vector x in $R^n$, and the 'seed' vector for the derivatives or Jacobian Matrix. From the first node, the value and derivatives will be calculated based on the values and derivatives of forward nodes. AD exploits the fact that every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations (addition, subtraction, multiplication, division, etc.) and elementary functions (exp, log, sin, cos, etc.). By applying the chain rule repeatedly to these operations, derivatives of arbitrary order can be computed automatically, accurately to working precision, and using at most a small constant factor more arithmetic operations than the original program.

The automatic differentiation is superior to analytic or symbolic differentiation because it could be computed on modern machines. It is also superior to numerical differentiation because numeric method is not precise but AD deals with machine precision problems properly.

An example of computational graph and table for forward mode AD is shown as follows:

\begin{align}
  f\left(x,y\right) =\sin\left(xy\right)
\end{align}
We will be evaluating the function at $f(1, 0)$

Evaluation trace:

| Trace   | Elementary Function      | Current Value           | Elementary Function Derivative       | $\nabla_{x}$ Value  | $\nabla_{y}$ Value  |
| :---: | :-----------------: | :-----------: | :----------------------------: | :-----------------:  | :-----------------: |
| $x_{1}$ | $x_{1}$                  | $1$        | $\dot{x}_{1}$                        | $1$ | $0$ |
| $x_{2}$ | $x_{2}$                  | $0$        | $\dot{x}_{2}$                        | $0$ | $1$ |
| $x_{3}$ | $x_{1}x_{2}$                  | $0$        | $\dot{x}_{2}$                        | $0$ | $1$ |

### 2.3 Reverse Mode

The reverse mode automatic differentiation has a process similar to the forward mode auto differentiation, but has another reverse process. It does not apply the chain rule and only partial derivatives to a node are stored. First, for the forward process, the partial derivatives are stored for each node. For the reverse process, it starts with the differentiation to the last node, and then activations in the forward process are deployed in the differentiation step by step.

### 2.4 Forward Mode v.s. Reverse Mode

Two main aspects can be considered when choosing between Forward and Reverse mode auto differentiation.
* Memory Storage & Time of Computation

The forward mode needs memory storage for values and derivatives for each node, while the reverse mode only needs to store the activations of partial differentiation to each node. The forward mode do the computation at the same time as the variable evaluation, while the reverse mode do the calculation in the backward process.
* Input & Output Dimensionality

If the input dimension is much larger than output dimension, then reverse mode is more attractive. If the output dimension is much larger than the input dimension, the forward mode is much computational cheaper.

## Installation

Our package can be installed from our GitHub Repository at: https://github.com/VoraciousFour/cs207-FinalProject.

After the package is installed, it needs to be imported into their workspace. Doing so, will automatically download any dependencies that are required by our package such as math or numpy. Then, the user can create and activate a virtual envitronment to use the package in.

The user can set up and use our package using their terminal as follows.

1. Clone the VorDiff package from our Github Repository into your directory
        git clone https://github.com/VoraciousFour/cs207-FinalProject.git
2. Create and activate a virtual environment
        '''Installing virtualenv'''
        sudo easy_install virtualenv
        '''Creating the Virtual Environment'''
        virtualenv env
        '''Activating the Virtual Environment'''
        source env/bin/activate
3. Install required dependencies
        pip install -r requirements.txt
4. Importing VorDiff package for use
        import VorDiff

## How to use VorDiff

Our Automatic Differentiation package is called VorDiff. The two main objects you will interact with are `AutoDiff` and `Operator`. In short, the user will first instantiate a scalar variable as an `AutoDiff` object, and then feed those variables to operators specified in the `Operator` object. The `Operator` object allows users to build their own functions for auto-differentiation. Simple operations (e.g. addition, multiplication, power) may be used normally. More complex functions (e.g. log, sin, cos) must use the operations defined in the `Operator` class. Lastly, the user may retrieve the values and first derivatives from the objects defined above by using the `get()` method.

A short example is provided below:

```python
from VorDiff.autodiff import AutoDiff as ad
from VorDiff.operator import Operator as op

# Define variables
x = ad.scalar(3.14159)
y = ad.scalar(0)

# Build functions
fx = op.sin(x) + 3
fy = op.exp(y) + op.log(y+1)

# Get values and derivates
print(fx.get())
print(fy.get())
```

    (3.0000026535897932, -0.9999999999964793)
    (1.0, 2.0)

## Software Organization

### Directory Structure
The package's directory will be structured as follows:
```
VorDiff/
	__init__.py
    nodes/
        __init__.py
	    scalar.py
        reverse_scalar.py
        reverse_vector.py
	    vector.py
	tests/
        __init__.py
        test_autodiff.py
        test_node.py
        test_operator.py
        test_reverse_autodiff.py
        test_reverse_operator.py
        test_reverse_scalar.py
        test_reverse_vector.py
        test_scaler.py
        test_vector.py
    autodiff.py
    operator.py
    reverse_autodiff.py
    reverse_operator.py
    README.md
    ...
demo/
    demo_reverse.py
    demo_scalar.py
    demo_vector.py
docs/
    ...
```
### Modules
-   VorDiff: The VorDiff module contains the operator class to be directly used by users to evaluate functions and calculate their derivatives, and an autodiff class that acts as the central interface for automatic differentiation.

-   Nodes: The Nodes module contains the the scalar and vector classes, which define the basic operations that can be performed on scalar and vector variables for the autodiff class.

-   Test_Vordiff: The Test_Vordiff module contains the test suite for this project. TravisCI and CodeCov are used to test our operator classes, node classes, and auto-differentiator.

-   Demo: The Demo module contains python files demonstrating how to perform automatic differentiation with the implemented functions.

### Testing
In this project we use TravisCI to perform continuous integration testing and CodeCov to check the code coverage of our test suite. The status us TravisCI and CodeCov can be found in README.md, in the top level of our package. Since the test suite is included in the project distribution, users can also install the project package and use pytest and pytest-cov to check the test results locally.

### Distribution:
Our open-source VorDiff package will be uploaded to PyPI by using twine because it uses a verified connection for secure authentication to PyPI over HTTPS. Users will be able to install our project package by using the convential `pip install VorDiff`.

## Implementation

### Scalar
The `Scalar` class represents a single scalar node in the computational graph of a function. It implements the interface for user defined scalar variables. The object contains two hidden attributes, `._val` and `._der`, which can be retrieved with the `get()` method.

### Vector
The `Vector` class represents a single vector variable. Vectors are comprised of `Element` objects, which implement much of the computation necessary for vector automatic differentiation.

```python
import numpy as np

# Documentation Hidden
class Scalar():

    def __init__(self, value, *kwargs):
        self._val = value
        if len(kwargs) == 0:
            self._der = 1
        else:
            self._der = kwargs[0]

    def get(self):
        return self._val, self._der

    def __add__(self, other):
        try:
            return Scalar(self._val+other._val, self._der+other._der)
        except AttributeError:
            return self.__radd__(other)

    def __radd__(self, other):
        return Scalar(self._val+other, self._der)

    def __mul__(self, other):
        try:
            return Scalar(self._val*other._val, self._der*other._val+self._val*other._der)
        except AttributeError:
            return self.__rmul__(other)

    def __rmul__(self, other):
        return Scalar(self._val*other, self._der*other)

    def __sub__(self, other):
        return self + (-other)

    def __rsub__(self, other):
        return -self + other

    def __truediv__(self, other):
        try:
            return Scalar(self._val/other._val, (self._der*other._val-self._val*other._der)/(other._val**2))
        except AttributeError:
            return Scalar(self._val/other, self._der/other)

    def __rtruediv__(self, other):
        return Scalar(other/self._val, other*(-self._der)/(self._val)**2)

    def __pow__(self, other):
        try:
            return Scalar(self._val**other._val, (other._val*self._der/self._val+np.log(self._val)*other._der)*(self._val**other._val))
        except AttributeError:
            return Scalar(self._val**other, other*(self._val**(other-1))*self._der)

    def __rpow__(self, other):
        return Scalar(other**self._val, (other**self._val)*np.log(other)*self._der)

    def __neg__(self):
        return Scalar((-1)*self._val, (-1)*self._der)
```

### Operator
The operator class contains all mathematical operations that users can call to build their functions. Each function returns a `Scalar` object or a numeric constant, depending on the input type. Each function raises an erro if its input falls outside its domain. All functions in the class are static.

In this implementation, we include the following elementary functions. Derivatives are calculated with the chain rule.

```python
import numpy as np
from VorDiff.nodes.scalar import Scalar

# Documentation Hidden
class Operator():

    @staticmethod
    def sin(x):
        try: # If scalar variable
            return Scalar(np.sin(x._val), x._der*np.cos(x._val))

        except AttributeError: # If contant
            return np.sin(x)

    @staticmethod
    def cos(x):
        try: # If scalar variable
            return Scalar(np.cos(x._val), -np.sin(x._val)*x._der)

        except AttributeError: # If contant
            return np.cos(x)

    @staticmethod
    def tan(x):
        try: # If scalar variable
            return Scalar(np.tan(x._val), x._der/np.cos(x._val)**2)

        except AttributeError: # If contant
            return np.tan(x)

    @staticmethod
    def arcsin(x):
        try: # If scalar variable
            if x._val<-1 or x._val>1:
                raise ValueError('out of domain')
            else:
                return Scalar(np.arcsin(x._val), 1/(x._der*(1-x._val**2)**.5))

        except AttributeError: # If contant
            if x<-1 or x>1:
                raise ValueError('out of domain')
            else:
                return np.arcsin(x)

    @staticmethod
    def arccos(x):
        try: # If scalar variable
            if x._val<-1 or x._val>1:
                raise ValueError('out of domain')
            else:
                return Scalar(np.arccos(x._val), -x._der/(1-x._val**2)**.5)

        except AttributeError: # If contant
            if x<-1 or x>1:
                raise ValueError('out of domain')
            else:
                return np.arccos(x)

    @staticmethod
    def arctan(x):
        try: # If scalar variable
            return Scalar(np.arctan(x._val), x._der/(1+x._val**2))

        except: # If contant
            return np.arctan(x)

    @staticmethod
    def log(x):
        try: # If scalar variable
            return Scalar(np.log(x._val), x._der/x._val)

        except AttributeError: # If contant
            return np.log(x)

    @staticmethod
    def exp(x):
        try: # If scalar variable
            return Scalar(np.exp(x._val), x._der*np.exp(x._val))

        except AttributeError: # If contant
            return np.exp(x)

    @staticmethod
    def sinh(x):
        try: # if scalar variable
            return Scalar(np.sinh(x._val), x._der*(np.cosh(x._val)))

        except AttributeError: #if constant
            return np.sinh(x)

    @staticmethod
    def cosh(x):
        try: # if scalar variable
            return Scalar(np.cosh(x._val), x._der*(np.sinh(x._val)))

        except AttributeError: #if constant
            return np.cosh(x)

    @staticmethod
    def tanh(x):
        try: # if scalar variable
            return Scalar(np.tanh(x._val), x._der*(1-np.tanh(x._val)**2))

        except AttributeError: #if constant
            return np.tanh(x)

    @staticmethod
    def arcsinh(x):
        try: # if scalar variable
            return Scalar(np.arcsinh(x._val), x._der*(-np.arcsinh(x._val)*np.arctanh(x._val)))

        except AttributeError: #if constant
            return np.arcsinh(x)

    @staticmethod
    def arccosh(x):
        try: # if scalar variable
            return Scalar(np.arccosh(x._val), x._der*(-np.arccosh(x._val)*np.tanh(x._val)))

        except AttributeError: #if constant
            return np.arccosh(x)

    @staticmethod
    def arctanh(x):
        try: # if scalar variable
            return Scalar(np.arctanh(x._val), x._der*(1-np.arctanh(x._val)**2))

        except AttributeError: #if constant
            return np.arctanh(x)
```

### AutoDiff
The `AutoDiff` class will allow the user to easily create variables and build auto-differentiable functions, without having to interface with the `Node` class. It will make use of the auto-differentiator much more intuitive for the user.

```python
from VorDiff.nodes.scalar import Scalar

# Documentation Hidden
class AutoDiff():

    @staticmethod
    def scalar(val):

        return Scalar(val, 1)
```

## Future Features

### 1. Option for higher-order derivatives

There are plenty of ways we could improve our package. The first is to grant users the option to compute higher-order derivatives like Hessians. We can recursively apply AD first to the target function, i.e., producing the first-order derivative, then moving the operations of the first-order derivatives into a new computational graph then applying AD again. In short, higher order derivatives would be accomplished by repeatedly applying automatic differentiation to function and its derivatives.

### 2. Application using AD library to find the roots of functions

A second way we could extend our work is by writing a separate library to find the roots of given functions. For example, this could include an implementation of Newton’s Method that calculates the exact Hessian matrix of a function using AD to get second-order partial derivatives. We would use Newton's Method to search for the approximations by calculating the exact Hessian matrix of the function using AD to get the second-order partial derivatives.

### 3. Backpropagation in neural networks

We can also extend our implementation of automatic differentiation to the neural networks. Neural networks are able to gradually increase accuracy with every training session through the process of gradient descent. In gradient descent, we aim to minimize the loss (i.e. how inaccurate the model is) through tweaking the weights and biases.

By finding the partial derivative of the loss function, we know how much (and in what direction) we must adjust our weights and biases to decrease loss. In that series, we calculate the derivative mean squared error loss function of a single-neuron neural network.

For computers to calculate the partial derivatives of an expression in neural networks, we can implement the automatic differentiation for both forward pass and back propagation. Then we can calculate the partial derivatives in both scalar and vector modes.

```python

```


**Set up your Google Colab Environment**

Mount your Google Drive

```python
from google.colab import drive
drive.mount('/content/drive')
```

    Mounted at /content/drive

Run this import command to be able to work with the eecs598 folder.

```python
import sys
sys.path.append('/content/drive/My Drive/QCourse511/ResearchProject/working')
```

Run these commands if you have problems.  When you first start Google Colab notebook,a newer version of tensorflow will be installed, but you need tensorflow 2.5.1 (or 2.4.1 depending on the documentation you find).

```python
!pip uninstall tensorflow
```

```python
!pip uninstall tensorflow-quantum
```

```python
!pip install tensorflow==2.5.1
```

```python
!pip install tf-nightly
```

```python
!pip install tensorflow-quantum
```

```python
!pip install tfq-nightly
```

```python
!pip install tensorflow-estimator==2.5.*
```

```python
!pip install keras==2.6.0
```

İmport some useful packages.

```python
import eecs598
import torch
import torchvision
import matplotlib.pyplot as plt
import statistics
import numpy as np

```

```python
import tensorflow as tf
import tensorflow_quantum as tfq

import cirq
import sympy
import numpy as np
import seaborn as sns
import collections

# visualization tools
%matplotlib inline
import matplotlib.pyplot as plt
from cirq.contrib.svg import SVGCircuit
```

# Download, visualize and prepare the CIFAR10 dataset
The CIFAR10 dataset contains 60,000 color images in 10 classes, with 6,000 images in each class. The dataset is divided into 50,000 training images and 10,000 testing images. The classes are mutually exclusive and there is no overlap between them.

The following code calls functions in the eecs598 library that use PyTorch to download the CIFAR dataset, split it into train and test sets and then convert it to torch.Tensors which are a multi-dimensional matrix containing elements of a single data type.

The code in the eecs598 library also normalizes pixel values to be between 0 and 1 by dividing by 255.

```python
x_train, y_train, x_test, y_test = eecs598.data.cifar10()

print('Training set:', )
print('  data shape:', x_train.shape)
print('  labels shape: ', y_train.shape)
print('Test set:')
print('  data shape: ', x_test.shape)
print('  labels shape', y_test.shape)
```

    Training set:
      data shape: torch.Size([50000, 3, 32, 32])
      labels shape:  torch.Size([50000])
    Test set:
      data shape:  torch.Size([10000, 3, 32, 32])
      labels shape torch.Size([10000])

Increase the default figure size.

```python
# Control qrid size for visualization
plt.rcParams['figure.figsize'] = (10.0, 8.0)
plt.rcParams['font.size'] = 16
```

Visualize the dataset

```python
import random
from torchvision.utils import make_grid

classes = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
samples_per_class = 12
samples = []
for y, cls in enumerate(classes):
    plt.text(-4, 34 * y + 18, cls, ha='right')
    idxs, = (y_train == y).nonzero(as_tuple=True)
    for i in range(samples_per_class):
        idx = idxs[random.randrange(idxs.shape[0])].item()
        samples.append(x_train[idx])
img = torchvision.utils.make_grid(samples, nrow=samples_per_class)
plt.imshow(eecs598.tensor_to_image(img))
plt.axis('off')
plt.show()
```

# Label data
The label data is a list of numbers ranging from 0 to 9, which corresponds to each of the 10 classes in CIFAR-10.

0 - airplane,
1 - automobile,
2 - bird,
3 - cat,
4 - deer,
5 - dog,
6 - frog,
7 - horse,
8 - ship,
9 - truck

```python
print(y_train)
```

    tensor([6, 9, 9,  ..., 9, 1, 1])

# 1. Data preparation
You will begin by preparing the CIFAR-10 dataset for running on a quantum computer.

# 1.1 Download the CIFAR-10 dataset
The first step is to get the traditional the CIFAR-10 dataset. This can be done using the `tf.keras.datasets` module.

```python
from tensorflow.keras.datasets import cifar10
(train_images, train_labels), (test_images, test_labels) = cifar10.load_data()

# Rescale the images from [0,255] to the [0.0,1.0] range.
train_images, test_images = train_images/255.0, test_images/255.0
```

```python
y_train=train_labels.flatten()
y_test=test_labels.flatten()
# We have reduced the dimension of the labels
```

It really comes down to math and getting a value between 0-1. Since 255 is the maximum value, dividing by 255 expresses a 0-1 representation. Each channel (Red, Green, and Blue are each channels) is 8 bits, so they are each limited to 256, in this case 255 since 0 is included.

```python
x_train=tf.image.rgb_to_grayscale(train_images)
x_test=tf.image.rgb_to_grayscale(test_images)
#to convert images to grayscale
```

Restrict our dataset to those of only two ground truth labels: cat and frog.  Filter the dataset to keep just the cat  and frog, remove the other classes. At the same time convert the label, y, to boolean: True for 3 (cat) and False for 6 (frog).

```python
def filter_36(x, y):
    keep = (y == 3) | (y == 6)
    x, y = x[keep], y[keep]
    y = y == 3
    return x,y
```

```python
x_train, y_train = filter_36(x_train, y_train)
x_test, y_test = filter_36(x_test, y_test)

print("Number of filtered training examples:", len(x_train))
print("Number of filtered test examples:", len(x_test))

# The lable data is now a mix of True and False values, True for a cat and False for a frog.
print(y_train)
```

    Number of filtered training examples: 10000
    Number of filtered test examples: 2000
    [False  True  True ...  True  True False]

```python
# Let's reduce the size of the data set to 1000 training data points and 200 testing data points.
N_TRAIN = 1000
N_TEST = 200
x_train, x_test = x_train[:N_TRAIN], x_test[:N_TEST]
y_train, y_test = y_train[:N_TRAIN], y_test[:N_TEST]

print("New number of training examples:", len(x_train))
print("New number of test examples:", len(x_test))
```

    New number of training examples: 1000
    New number of test examples: 200

```python
# Review the first four of the images in 32x32 mode
f, axarr = plt.subplots(2,2)
axarr[0,0].imshow(x_train[0, :, :, 0])
axarr[0,1].imshow(x_train[1, :, :, 0])
axarr[1,0].imshow(x_train[2, :, :, 0])
axarr[1,1].imshow(x_train[3, :, :, 0])
```

```python
plt.imshow(x_train[0, :, :, 0])
plt.colorbar()
```

Resize the image 32x32 to down to 2x2 as that is what our FRQI example can work with.  This can be done using bilinear interpolation as another option.

```python
def truncate_x(x_train, x_test, n_components=10):
  """Perform PCA on image dataset keeping the top `n_components` components."""
  n_points_train = tf.gather(tf.shape(x_train), 0)
  n_points_test = tf.gather(tf.shape(x_test), 0)

  # Flatten to 1D
  x_train = tf.reshape(x_train, [n_points_train, -1])
  x_test = tf.reshape(x_test, [n_points_test, -1])

  # Normalize.
  feature_mean = tf.reduce_mean(x_train, axis=0)
  x_train_normalized = x_train - feature_mean
  x_test_normalized = x_test - feature_mean

  # Truncate.
  e_values, e_vectors = tf.linalg.eigh(
      tf.einsum('ji,jk->ik', x_train_normalized, x_train_normalized))
  return tf.einsum('ij,jk->ik', x_train_normalized, e_vectors[:,-n_components:]), \
    tf.einsum('ij,jk->ik', x_test_normalized, e_vectors[:, -n_components:])

#DATASET_DIM = 10
#x_train_s, x_test_s = truncate_x(x_train, x_test, n_components=DATASET_DIM)
#print(f'New datapoint dimension:', len(x_train_s[0]))

image_dimension = 2
x_train_s = tf.image.resize(x_train, (image_dimension,image_dimension)).numpy()
x_test_s = tf.image.resize(x_test, (image_dimension,image_dimension)).numpy()
```

```python
# Review the first four of the images in 32x32 mode
f, axarr = plt.subplots(2,2)
axarr[0,0].imshow(x_train_s[0, :, :, 0])
axarr[0,1].imshow(x_train_s[1, :, :, 0])
axarr[1,0].imshow(x_train_s[2, :, :, 0])
axarr[1,1].imshow(x_train_s[3, :, :, 0])
```

```python
plt.imshow(x_train_s[0, :, :, 0])
# The color bar is getting adjusted for some reason.  May be a bug with tf.image.resize() which has some underlying problems.
# https://hackernoon.com/how-tensorflows-tf-image-resize-stole-60-days-of-my-life-aba5eb093f35
plt.colorbar()
```

## Encode the data as quantum circuits
+ Set the base state based on the pixel
+ run through the algorithm

Transform the images to black and white by thresholding the pixel color.

```python
print(x_train_s[0:4,0:4])

THRESHOLD = 0.5

x_train_bin = np.array(x_train_s > THRESHOLD, dtype=np.float32)
x_test_bin = np.array(x_test_s > THRESHOLD, dtype=np.float32)

print(x_train_bin[0:4,0:4])
```

    [[[[0.36764815]
       [0.28288433]]

      [[0.49960423]
       [0.25495717]]]

     [[[0.35545862]
       [0.6398164 ]]

      [[0.2035408 ]
       [0.08488853]]]

     [[[0.5119202 ]
       [0.21370961]]

      [[0.44399855]
       [0.6437518 ]]]

     [[[0.18349177]
       [0.434492  ]]

      [[0.36693138]
       [0.2093106 ]]]]
    [[[[0.]
       [0.]]

      [[0.]
       [0.]]]

     [[[0.]
       [1.]]

      [[0.]
       [0.]]]

     [[[1.]
       [0.]]

      [[0.]
       [1.]]]

     [[[0.]
       [0.]]

      [[0.]
       [0.]]]]

The qubits at pixel indices with values that exceed a threshold, are rotated through a gate.  This is the part we should replace.

```python
def FRQI(theta):

    circuit = cirq.Circuit()

    theta = theta

    # Or created in a range
    # This will create LineQubit(0), LineQubit(1), LineQubit(2)
    q0, q1, q2 = cirq.LineQubit.range(3)

    circuit.append(cirq.H(q) for q in cirq.LineQubit.range(2))

    qc.barrier()
    #Pixel 1

    qc.cry(theta,0,2)
    qc.cx(0,1)
    qc.cry(-theta,1,2)
    qc.cx(0,1)
    qc.cry(theta,1,2)

    qc.barrier()
    #Pixel 2

    qc.x(1)
    qc.cry(theta,0,2)
    qc.cx(0,1)
    qc.cry(-theta,1,2)
    qc.cx(0,1)
    qc.cry(theta,1,2)

    qc.barrier()

    qc.x(1)
    qc.x(0)
    qc.cry(theta,0,2)
    qc.cx(0,1)
    qc.cry(-theta,1,2)
    qc.cx(0,1)
    qc.cry(theta,1,2)

    qc.barrier()

    qc.x(1)

    qc.cry(theta,0,2)
    qc.cx(0,1)
    qc.cry(-theta,1,2)
    qc.cx(0,1)
    qc.cry(theta,1,2)

    qc.measure_all()

    qc.draw()

FRQI(0)

```

```python
def convert_to_circuit(image):
    """Encode truncated classical image into quantum datapoint."""
    values = np.ndarray.flatten(image)
    qubits = cirq.GridQubit.rect(6, 6)
    circuit = cirq.Circuit()
    for i, value in enumerate(values):
        if value:
            circuit.append(cirq.H(qubits[i]))

    return circuit

x_train_circ = [convert_to_circuit(x) for x in x_train_bin]
x_test_circ = [convert_to_circuit(x) for x in x_test_bin]
```

```python
SVGCircuit(x_train_circ[0])

```

Convert  Cirq circuits to tensors for tfq

```python
x_train_tfcirc = tfq.convert_to_tensor(x_train_circ)
x_test_tfcirc = tfq.convert_to_tensor(x_test_circ)
```

## Quantum neural network

add a layer of these gates to a circuit

```python
class CircuitLayerBuilder():
    def __init__(self, data_qubits, readout):
        self.data_qubits = data_qubits
        self.readout = readout

    def add_layer(self, circuit, gate, prefix):
        for i, qubit in enumerate(self.data_qubits):
            symbol = sympy.Symbol(prefix + '-' + str(i))
            circuit.append(gate(qubit, self.readout)**symbol)

```

```python
demo_builder = CircuitLayerBuilder(data_qubits = cirq.GridQubit.rect(4,1),
                                   readout=cirq.GridQubit(-1,-1))

circuit = cirq.Circuit()
demo_builder.add_layer(circuit, gate = cirq.XX, prefix='xx')
SVGCircuit(circuit)

```

```python
def create_quantum_model():
    """Create a QNN model circuit and readout operation to go along with it."""
    data_qubits = cirq.GridQubit.rect(4, 4)  # a 4x4 grid.
    readout = cirq.GridQubit(-1, -1)         # a single qubit at [-1,-1]
    circuit = cirq.Circuit()

    # Prepare the readout qubit.
    circuit.append(cirq.X(readout))
    circuit.append(cirq.H(readout))

    builder = CircuitLayerBuilder(
        data_qubits = data_qubits,
        readout=readout)

    # Then add layers (experiment by adding more).
    builder.add_layer(circuit, cirq.XX, "xx1")
    builder.add_layer(circuit, cirq.ZZ, "zz1")

    # Finally, prepare the readout qubit.
    circuit.append(cirq.H(readout))

    return circuit, cirq.Z(readout)

```

```python
model_circuit, model_readout = create_quantum_model()

```

 Build the Keras model.

```python
model = tf.keras.Sequential([
    # The input is the data-circuit, encoded as a tf.string
    tf.keras.layers.Input(shape=(), dtype=tf.string),
    # The PQC layer returns the expected value of the readout gate, range [-1,1].
    tfq.layers.PQC(model_circuit, model_readout),
])
```

```python
y_train_hinge = 2.0*y_train-1.0
y_test_hinge = 2.0*y_test-1.0

```

```python
def hinge_accuracy(y_true, y_pred):
    y_true = tf.squeeze(y_true) > 0.0
    y_pred = tf.squeeze(y_pred) > 0.0
    result = tf.cast(y_true == y_pred, tf.float32)

    return tf.reduce_mean(result)
```

```python
model.compile(
    loss=tf.keras.losses.Hinge(),
    optimizer=tf.keras.optimizers.Adam(),
    metrics=[hinge_accuracy])

```

```python
print(model.summary())
```

```python
EPOCHS = 3
BATCH_SIZE = 128

NUM_EXAMPLES = len(x_train_tfcirc)

```

```python
x_train_tfcirc_sub = x_train_tfcirc[:NUM_EXAMPLES]
y_train_hinge_sub = y_train_hinge[:NUM_EXAMPLES]

```

```python
import time
start_time = time.time()
```

```python
qnn_history = model.fit(
      x_train_tfcirc_sub, y_train_hinge_sub,
      batch_size=32,
      epochs=EPOCHS,
      verbose=1,
      validation_data=(x_test_tfcirc, y_test_hinge))

qnn_results = model.evaluate(x_test_tfcirc, y_test)

```

```python
qnn_accuracy = qnn_results[1]
qnn_accuracy

```

```python
model.predict_classes(x_train_tfcirc[0:7])
```

```python

import matplotlib.pyplot as plt
import numpy as np
from matplotlib.gridspec import GridSpec

def format_axes(fig):
    for i, ax in enumerate(fig.axes):
        ax.tick_params(labelbottom=False, labelleft=False)

fig = plt.figure(figsize=(10, 10))

gs = GridSpec(3, 3, figure=fig)
ax1 = fig.add_subplot(gs[0, 0])
# identical to ax1 = plt.subplot(gs.new_subplotspec((0, 0), colspan=3))

for i in range(3):
  for j in range(3):
    ax = fig.add_subplot(gs[i, j])

    ax.imshow(x_train[i+j, :, :, 0])

fig.suptitle("GridSpec")
format_axes(fig)

plt.show()

```

```python
plt.plot(qnn_history.history['hinge_accuracy'], label='QCNN')
#plt.plot(hybrid_history.history['val_custom_accuracy'], label='Hybrid CNN')
#plt.title('Quantum vs Hybrid CNN performance')
plt.xlabel('Epochs')
plt.legend()
plt.ylabel('Validation Accuracy')
plt.show()
```


##### Copyright 2020 The TensorFlow Authors.

```
#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

# Calculate gradients

    View on TensorFlow.org

    Run in Google Colab

    View source on GitHub

    Download notebook

This tutorial explores gradient calculation algorithms for the expectation values of quantum circuits.

Calculating the gradient of the expectation value of a certain observable in a quantum circuit is an involved process. Expectation values of observables do not have the luxury of having analytic gradient formulas that are always easy to write down—unlike traditional machine learning transformations such as matrix multiplication or vector addition that have analytic gradient formulas which are easy to write down. As a result, there are different quantum gradient calculation methods that come in handy for different scenarios. This tutorial compares and contrasts two different differentiation schemes.

## Setup

```
!pip install tensorflow==2.3.1
```

Install TensorFlow Quantum:

```
!pip install tensorflow-quantum
```

Now import TensorFlow and the module dependencies:

```
import tensorflow as tf
import tensorflow_quantum as tfq

import cirq
import sympy
import numpy as np

# visualization tools
%matplotlib inline
import matplotlib.pyplot as plt
from cirq.contrib.svg import SVGCircuit
```

## 1. Preliminary

Let's make the notion of gradient calculation for quantum circuits a little more concrete. Suppose you have a parameterized circuit like this one:

```
qubit = cirq.GridQubit(0, 0)
my_circuit = cirq.Circuit(cirq.Y(qubit)**sympy.Symbol('alpha'))
SVGCircuit(my_circuit)
```

Along with an observable:

```
pauli_x = cirq.X(qubit)
pauli_x
```

Looking at this operator you know that $⟨Y(\alpha)| X | Y(\alpha)⟩ = \sin(\pi \alpha)$

```
def my_expectation(op, alpha):
    """Compute ⟨Y(alpha)| `op` | Y(alpha)⟩"""
    params = {'alpha': alpha}
    sim = cirq.Simulator()
    final_state_vector = sim.simulate(my_circuit, params).final_state_vector
    return op.expectation_from_state_vector(final_state_vector, {qubit: 0}).real

my_alpha = 0.3
print("Expectation=", my_expectation(pauli_x, my_alpha))
print("Sin Formula=", np.sin(np.pi * my_alpha))
```

 and if you define $f_{1}(\alpha) = ⟨Y(\alpha)| X | Y(\alpha)⟩$ then $f_{1}^{'}(\alpha) = \pi \cos(\pi \alpha)$. Let's check this:

```
def my_grad(obs, alpha, eps=0.01):
    grad = 0
    f_x = my_expectation(obs, alpha)
    f_x_prime = my_expectation(obs, alpha + eps)
    return ((f_x_prime - f_x) / eps).real

print('Finite difference:', my_grad(pauli_x, my_alpha))
print('Cosine formula:   ', np.pi * np.cos(np.pi * my_alpha))
```

## 2. The need for a differentiator

With larger circuits, you won't always be so lucky to have a formula that precisely calculates the gradients of a given quantum circuit. In the event that a simple formula isn't enough to calculate the gradient, the `tfq.differentiators.Differentiator` class allows you to define algorithms for computing the gradients of your circuits. For instance you can recreate the above example in TensorFlow Quantum (TFQ) with:

```
expectation_calculation = tfq.layers.Expectation(
    differentiator=tfq.differentiators.ForwardDifference(grid_spacing=0.01))

expectation_calculation(my_circuit,
                        operators=pauli_x,
                        symbol_names=['alpha'],
                        symbol_values=[[my_alpha]])
```

However, if you switch to estimating expectation based on sampling (what would happen on a true device) the values can change a little bit. This means you now have an imperfect estimate:

```
sampled_expectation_calculation = tfq.layers.SampledExpectation(
    differentiator=tfq.differentiators.ForwardDifference(grid_spacing=0.01))

sampled_expectation_calculation(my_circuit,
                                operators=pauli_x,
                                repetitions=500,
                                symbol_names=['alpha'],
                                symbol_values=[[my_alpha]])
```

This can quickly compound into a serious accuracy problem when it comes to gradients:

```
# Make input_points = [batch_size, 1] array.
input_points = np.linspace(0, 5, 200)[:, np.newaxis].astype(np.float32)
exact_outputs = expectation_calculation(my_circuit,
                                        operators=pauli_x,
                                        symbol_names=['alpha'],
                                        symbol_values=input_points)
imperfect_outputs = sampled_expectation_calculation(my_circuit,
                                                    operators=pauli_x,
                                                    repetitions=500,
                                                    symbol_names=['alpha'],
                                                    symbol_values=input_points)
plt.title('Forward Pass Values')
plt.xlabel('$x$')
plt.ylabel('$f(x)$')
plt.plot(input_points, exact_outputs, label='Analytic')
plt.plot(input_points, imperfect_outputs, label='Sampled')
plt.legend()
```

```
# Gradients are a much different story.
values_tensor = tf.convert_to_tensor(input_points)

with tf.GradientTape() as g:
    g.watch(values_tensor)
    exact_outputs = expectation_calculation(my_circuit,
                                            operators=pauli_x,
                                            symbol_names=['alpha'],
                                            symbol_values=values_tensor)
analytic_finite_diff_gradients = g.gradient(exact_outputs, values_tensor)

with tf.GradientTape() as g:
    g.watch(values_tensor)
    imperfect_outputs = sampled_expectation_calculation(
        my_circuit,
        operators=pauli_x,
        repetitions=500,
        symbol_names=['alpha'],
        symbol_values=values_tensor)
sampled_finite_diff_gradients = g.gradient(imperfect_outputs, values_tensor)

plt.title('Gradient Values')
plt.xlabel('$x$')
plt.ylabel('$f^{\'}(x)$')
plt.plot(input_points, analytic_finite_diff_gradients, label='Analytic')
plt.plot(input_points, sampled_finite_diff_gradients, label='Sampled')
plt.legend()
```

Here you can see that although the finite difference formula is fast to compute the gradients themselves in the analytical case, when it came to the sampling based methods it was far too noisy. More careful techniques must be used to ensure a good gradient can be calculated. Next you will look at a much slower technique that wouldn't be as well suited for analytical expectation gradient calculations, but does perform much better in the real-world sample based case:

```
# A smarter differentiation scheme.
gradient_safe_sampled_expectation = tfq.layers.SampledExpectation(
    differentiator=tfq.differentiators.ParameterShift())

with tf.GradientTape() as g:
    g.watch(values_tensor)
    imperfect_outputs = gradient_safe_sampled_expectation(
        my_circuit,
        operators=pauli_x,
        repetitions=500,
        symbol_names=['alpha'],
        symbol_values=values_tensor)

sampled_param_shift_gradients = g.gradient(imperfect_outputs, values_tensor)

plt.title('Gradient Values')
plt.xlabel('$x$')
plt.ylabel('$f^{\'}(x)$')
plt.plot(input_points, analytic_finite_diff_gradients, label='Analytic')
plt.plot(input_points, sampled_param_shift_gradients, label='Sampled')
plt.legend()
```

From the above you can see that certain differentiators are best used for particular research scenarios. In general, the slower sample-based methods that are robust to device noise, etc., are great differentiators when testing or implementing algorithms in a more "real world" setting. Faster methods like finite difference are great for analytical calculations and you want higher throughput, but aren't yet concerned with the device viability of your algorithm.

## 3. Multiple observables

Let's introduce a second observable and see how TensorFlow Quantum supports multiple observables for a single circuit.

```
pauli_z = cirq.Z(qubit)
pauli_z
```

If this observable is used with the same circuit as before, then you have $f_{2}(\alpha) = ⟨Y(\alpha)| Z | Y(\alpha)⟩ = \cos(\pi \alpha)$ and $f_{2}^{'}(\alpha) = -\pi \sin(\pi \alpha)$. Perform a quick check:

```
test_value = 0.

print('Finite difference:', my_grad(pauli_z, test_value))
print('Sin formula:      ', -np.pi * np.sin(np.pi * test_value))
```

It's a match (close enough).

Now if you define $g(\alpha) = f_{1}(\alpha) + f_{2}(\alpha)$ then $g'(\alpha) = f_{1}^{'}(\alpha) + f^{'}_{2}(\alpha)$. Defining more than one observable in TensorFlow Quantum to use along with a circuit is equivalent to adding on more terms to $g$.

This means that the gradient of a particular symbol in a circuit is equal to the sum of the gradients with regards to each observable for that symbol applied to that circuit. This is compatible with TensorFlow gradient taking and backpropagation (where you give the sum of the gradients over all observables as the gradient for a particular symbol).

```
sum_of_outputs = tfq.layers.Expectation(
    differentiator=tfq.differentiators.ForwardDifference(grid_spacing=0.01))

sum_of_outputs(my_circuit,
               operators=[pauli_x, pauli_z],
               symbol_names=['alpha'],
               symbol_values=[[test_value]])
```

Here you see the first entry is the expectation w.r.t Pauli X, and the second is the expectation w.r.t Pauli Z. Now when you take the gradient:

```
test_value_tensor = tf.convert_to_tensor([[test_value]])

with tf.GradientTape() as g:
    g.watch(test_value_tensor)
    outputs = sum_of_outputs(my_circuit,
                             operators=[pauli_x, pauli_z],
                             symbol_names=['alpha'],
                             symbol_values=test_value_tensor)

sum_of_gradients = g.gradient(outputs, test_value_tensor)

print(my_grad(pauli_x, test_value) + my_grad(pauli_z, test_value))
print(sum_of_gradients.numpy())
```

Here you have verified that the sum of the gradients for each observable is indeed the gradient of $\alpha$. This behavior is supported by all TensorFlow Quantum differentiators and plays a crucial role in the compatibility with the rest of TensorFlow.

## 4. Advanced usage
All differentiators that exist inside of TensorFlow Quantum subclass `tfq.differentiators.Differentiator`. To implement a differentiator, a user must implement one of two interfaces.  The standard is to implement `get_gradient_circuits`, which tells the base class which circuits to measure to obtain an estimate of the gradient.  Alternatively, you can overload `differentiate_analytic` and `differentiate_sampled`; the class `tfq.differentiators.Adjoint` takes this route.

The following uses TensorFlow Quantum to implement the gradient of a circuit.  You will use a small example of parameter shifting.

Recall the circuit you defined above, $|\alpha⟩ = Y^{\alpha}|0⟩$.  As before, you can define a function as the expectation value of this circuit against the $X$ observable, $f(\alpha) = ⟨\alpha|X|\alpha⟩$.  Using [parameter shift rules](https://pennylane.ai/qml/glossary/parameter_shift.html), for this circuit, you can find that the derivative is
$$\frac{\partial}{\partial \alpha} f(\alpha) = \frac{\pi}{2} f\left(\alpha + \frac{1}{2}\right) -  \frac{ \pi}{2} f\left(\alpha - \frac{1}{2}\right)$$
The `get_gradient_circuits` function returns the components of this derivative.

```
class MyDifferentiator(tfq.differentiators.Differentiator):
    """A Toy differentiator for <Y^alpha | X |Y^alpha>."""

    def __init__(self):
        pass

    def get_gradient_circuits(self, programs, symbol_names, symbol_values):
        """Return circuits to compute gradients for given forward pass circuits.

        Every gradient on a quantum computer can be computed via measurements
        of transformed quantum circuits.  Here, you implement a custom gradient
        for a specific circuit.  For a real differentiator, you will need to
        implement this function in a more general way.  See the differentiator
        implementations in the TFQ library for examples.
        """

        # The two terms in the derivative are the same circuit...
        batch_programs = tf.stack([programs, programs], axis=1)

        # ... with shifted parameter values.
        shift = tf.constant(1/2)
        forward = symbol_values + shift
        backward = symbol_values - shift
        batch_symbol_values = tf.stack([forward, backward], axis=1)

        # Weights are the coefficients of the terms in the derivative.
        num_program_copies = tf.shape(batch_programs)[0]
        batch_weights = tf.tile(tf.constant([[[np.pi/2, -np.pi/2]]]),
                                [num_program_copies, 1, 1])

        # The index map simply says which weights go with which circuits.
        batch_mapper = tf.tile(
            tf.constant([[[0, 1]]]), [num_program_copies, 1, 1])

        return (batch_programs, symbol_names, batch_symbol_values,
                batch_weights, batch_mapper)
```

The `Differentiator` base class uses the components returned from `get_gradient_circuits` to calculate the derivative, as in the parameter shift formula you saw above.  This new differentiator can now be used with existing `tfq.layer` objects:

```
custom_dif = MyDifferentiator()
custom_grad_expectation = tfq.layers.Expectation(differentiator=custom_dif)

# Now let's get the gradients with finite diff.
with tf.GradientTape() as g:
    g.watch(values_tensor)
    exact_outputs = expectation_calculation(my_circuit,
                                            operators=[pauli_x],
                                            symbol_names=['alpha'],
                                            symbol_values=values_tensor)

analytic_finite_diff_gradients = g.gradient(exact_outputs, values_tensor)

# Now let's get the gradients with custom diff.
with tf.GradientTape() as g:
    g.watch(values_tensor)
    my_outputs = custom_grad_expectation(my_circuit,
                                         operators=[pauli_x],
                                         symbol_names=['alpha'],
                                         symbol_values=values_tensor)

my_gradients = g.gradient(my_outputs, values_tensor)

plt.subplot(1, 2, 1)
plt.title('Exact Gradient')
plt.plot(input_points, analytic_finite_diff_gradients.numpy())
plt.xlabel('x')
plt.ylabel('f(x)')
plt.subplot(1, 2, 2)
plt.title('My Gradient')
plt.plot(input_points, my_gradients.numpy())
plt.xlabel('x')
```

This new differentiator can now be used to generate differentiable ops.

Key Point: A differentiator that has been previously attached to an op must be refreshed before attaching to a new op, because a differentiator may only be attached to one op at a time.

```
# Create a noisy sample based expectation op.
expectation_sampled = tfq.get_sampled_expectation_op(
    cirq.DensityMatrixSimulator(noise=cirq.depolarize(0.01)))

# Make it differentiable with your differentiator:
# Remember to refresh the differentiator before attaching the new op
custom_dif.refresh()
differentiable_op = custom_dif.generate_differentiable_op(
    sampled_op=expectation_sampled)

# Prep op inputs.
circuit_tensor = tfq.convert_to_tensor([my_circuit])
op_tensor = tfq.convert_to_tensor([[pauli_x]])
single_value = tf.convert_to_tensor([[my_alpha]])
num_samples_tensor = tf.convert_to_tensor([[5000]])

with tf.GradientTape() as g:
    g.watch(single_value)
    forward_output = differentiable_op(circuit_tensor, ['alpha'], single_value,
                                       op_tensor, num_samples_tensor)

my_gradients = g.gradient(forward_output, single_value)

print('---TFQ---')
print('Foward:  ', forward_output.numpy())
print('Gradient:', my_gradients.numpy())
print('---Original---')
print('Forward: ', my_expectation(pauli_x, my_alpha))
print('Gradient:', my_grad(pauli_x, my_alpha))
```

Success: Now you can use all the differentiators that TensorFlow Quantum has to offer—and define your own.


```python
# Erasmus+ ICCT project (2018-1-SI01-KA203-047081)

# Toggle cell visibility

from IPython.display import HTML
display(HTML(''))
```

Toggle cell visibility here.

```python
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
import sympy as sym
import scipy.signal as signal
from ipywidgets import widgets, interact
import control as cn
```

## Root locus

Root locus is a plot of the location of closed-loop system poles in relation with a certain parameter (typically amplification). It can be shown that the curves start in the open-loop poles and end up in the open-loop zeros (or infinity). The location of closed-loop system poles not only gives an indication of system stability, but other closed-loop system response properties such as overshoot, rise time and settling time can also be inferred from pole location.

---

### How to use this notebook?
1. Click on *P0*, *P1*, *I0* or *I1* to toggle between the following objects: proportional of the zeroth, first or second order, or an integral one of zeroth or first order. The transfer function of P0 object is $k_p$ (in this example $k_p=2$), of PI object $\frac{k_p}{\tau s+1}$ (in this example $k_p=1$ and $\tau=2$), of IO object $\frac{k_i}{s}$ (in this example $k_i=\frac{1}{10}$) and of I1 object $\frac{k_i}{s(\tau s +1)}$ (in this example $k_i=1$ and $\tau=10$).
2. Click on the *P*, *PI*, *PD* or *PID* button to toogle between proportional, proportional-integral, proportional-derivative or proportional–integral–derivative control algorithm types.
3. Move the sliders to change the values of proportional ($K_p$), integral ($T_i$) and derivative ($T_d$) PID tunning coefficients.
4. Move the slider $t_{max}$ to change the maximum value of the time on x axis.

```python
A = 10
a=0.1
s, P, I, D = sym.symbols('s, P, I, D')

obj = 1/(A*s)
PID = P + P/(I*s) + P*D*s#/(a*D*s+1)
system = obj*PID/(1+obj*PID)
num = [sym.fraction(system.factor())[0].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system.factor())[0], gen=s)))]
den = [sym.fraction(system.factor())[1].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system.factor())[1], gen=s)))]
system_func_open = obj*PID
num_open = [sym.fraction(system_func_open.factor())[0].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func_open.factor())[0], gen=s)))]
den_open = [sym.fraction(system_func_open.factor())[1].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func_open.factor())[1], gen=s)))]

# make figure
fig = plt.figure(figsize=(9.8, 4),num='Root locus')
plt.subplots_adjust(wspace=0.3)

# add axes
ax = fig.add_subplot(121)
ax.grid(which='both', axis='both', color='lightgray')
ax.set_title('Time response')
ax.set_xlabel('t [s]')
ax.set_ylabel('input, output')

rlocus = fig.add_subplot(122)

# plot step function and responses (initalisation)
input_plot, = ax.plot([],[],'C0', lw=1, label='input')
response_plot, = ax.plot([],[], 'C1', lw=2, label='output')
ax.legend()

rlocus_plot, = rlocus.plot([], [], 'r')

plt.show()

system_open = None
system_close = None
def update_plot(KP, TI, TD, Time_span):
    global num, den, num_open, den_open
    global system_open, system_close
    num_temp = [float(i.subs(P,KP).subs(I,TI).subs(D,TD)) for i in num]
    den_temp = [float(i.subs(P,KP).subs(I,TI).subs(D,TD)) for i in den]
    system = signal.TransferFunction(num_temp, den_temp)
    system_close = system
    num_temp_open = [float(i.subs(P,KP).subs(I,TI).subs(D,TD)) for i in num_open]
    den_temp_open = [float(i.subs(P,KP).subs(I,TI).subs(D,TD)) for i in den_open]
    system_open = signal.TransferFunction(num_temp_open, den_temp_open)

    rlocus.clear()
    r, k, xlim, ylim = cn.root_locus_modified(system_open, Plot=False)
#     r, k = cn.root_locus(system_open, Plot=False)
    #rlocus.scatter(r)
    #plot closed loop poles and zeros
    poles = np.roots(system.den)
    rlocus.plot(np.real(poles), np.imag(poles), 'kx')
    zeros = np.roots(system.num)
    if zeros.size > 0:
        rlocus.plot(np.real(zeros), np.imag(zeros), 'ko', alpha=0.5)
    # plot open loop poles and zeros
    poles = np.roots(system_open.den)
    rlocus.plot(np.real(poles), np.imag(poles), 'x', alpha=0.5)
    zeros = np.roots(system_open.num)
    if zeros.size > 0:
        rlocus.plot(np.real(zeros), np.imag(zeros), 'o')
    #plot root locus
    for index, col in enumerate(r.T):
        rlocus.plot(np.real(col), np.imag(col), 'b', alpha=0.5)

    rlocus.set_title('Root locus')
    rlocus.set_xlabel('Re')
    rlocus.set_ylabel('Im')
    rlocus.grid(which='both', axis='both', color='lightgray')

    rlocus.axhline(linewidth=.3, color='g')
    rlocus.axvline(linewidth=.3, color='g')
    rlocus.set_ylim(ylim)
    rlocus.set_xlim(xlim)

    time = np.linspace(0, Time_span, 300)
    u = np.ones_like(time)
    u[0] = 0
    time, response = signal.step(system, T=time)

    response_plot.set_data(time, response)
    input_plot.set_data(time, u)

    ax.set_ylim([min([np.min(u), min(response),-.1]),min(100,max([max(response)*1.05, 1, 1.05*np.max(u)]))])
    ax.set_xlim([-0.1,max(time)])

    plt.show()

controller_ = PID
object_ = obj

def calc_tf():
    global num, den, controller_, object_, num_open, den_open
    system_func = object_*controller_/(1+object_*controller_)

    num = [sym.fraction(system_func.factor())[0].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func.factor())[0], gen=s)))]
    den = [sym.fraction(system_func.factor())[1].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func.factor())[1], gen=s)))]

    system_func_open = object_*controller_
    num_open = [sym.fraction(system_func_open.factor())[0].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func_open.factor())[0], gen=s)))]
    den_open = [sym.fraction(system_func_open.factor())[1].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func_open.factor())[1], gen=s)))]

    update_plot(Kp_widget.value, Ti_widget.value, Td_widget.value, time_span_widget.value)

def transfer_func(controller_type):
    global controller_
    proportional = P
    integral = P/(I*s)
    differential = P*D*s/(a*D*s+1)
    if controller_type =='P':
        controller_func = proportional
        Kp_widget.disabled=False
        Ti_widget.disabled=True
        Td_widget.disabled=True
    elif controller_type =='PI':
        controller_func = proportional+integral
        Kp_widget.disabled=False
        Ti_widget.disabled=False
        Td_widget.disabled=True
    elif controller_type == 'PD':
        controller_func = proportional+differential
        Kp_widget.disabled=False
        Ti_widget.disabled=True
        Td_widget.disabled=False
    else:
        controller_func = proportional+integral+differential
        Kp_widget.disabled=False
        Ti_widget.disabled=False
        Td_widget.disabled=False

    controller_ = controller_func
    calc_tf()

def transfer_func_obj(object_type):
    global object_
    if object_type == 'P0':
        object_ = 2
    elif object_type == 'P1':
        object_ = 1/(2*s+1)
    elif object_type == 'I0':
        object_ = 1/(10*s)
    elif object_type == 'I1':
        object_ = 1/(s*(10*s+1))
    calc_tf()

style = {'description_width': 'initial'}

def buttons_controller_clicked(event):
    controller = buttons_controller.options[buttons_controller.index]
    transfer_func(controller)
buttons_controller = widgets.ToggleButtons(
    options=['P', 'PI', 'PD', 'PID'],
    description='Select control algorithm type:',
    disabled=False,
    style=style)
buttons_controller.observe(buttons_controller_clicked)

def buttons_object_clicked(event):
    object_ = buttons_object.options[buttons_object.index]
    transfer_func_obj(object_)
buttons_object = widgets.ToggleButtons(
    options=['P0', 'P1', 'I0', 'I1'],
    description='Select object:',
    disabled=False,
    style=style)
buttons_object.observe(buttons_object_clicked)

Kp_widget = widgets.FloatLogSlider(value=.5,min=-3,max=2.1,step=.001,description=r'\(K_p\)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.3f')
Ti_widget = widgets.FloatLogSlider(value=1.,min=-3,max=1.8,step=.001,description=r'\(T_{i} \)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.3f')
Td_widget = widgets.FloatLogSlider(value=1.,min=-3,max=1.8,step=.001,description=r'\(T_{d} \)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.3f')

time_span_widget = widgets.FloatSlider(value=10.,min=.5,max=50.,step=0.1,description=r'\(t_{max} \)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.1f')

transfer_func(buttons_controller.options[buttons_controller.index])
transfer_func_obj(buttons_object.options[buttons_object.index])

display(buttons_object)
display(buttons_controller)

interact(update_plot, KP=Kp_widget, TI=Ti_widget, TD=Td_widget, Time_span=time_span_widget);
```

    <IPython.core.display.Javascript object>

    ToggleButtons(description='Select object:', options=('P0', 'P1', 'I0', 'I1'), style=ToggleButtonsStyle(descrip…

    ToggleButtons(description='Select control algorithm type:', options=('P', 'PI', 'PD', 'PID'), style=ToggleButt…

    interactive(children=(FloatLogSlider(value=0.5, description='\\(K_p\\)', max=2.1, min=-3.0, readout_format='.3…