<a href="https://colab.research.google.com/github/ryanleeallred/DS-Unit-1-Sprint-3-Linear-Algebra/blob/master/module1-vectors-and-matrices/LS_DS_131_Vectors_and_Matrices.ipynb" target="_parent"></a>

Lambda School Data Science

*Unit 1, Sprint 3, Module 1*

---



# Vectors and Matrices

- Student can illustrate why we care about linear algebra in the scope of data science.
- Student can graph vectors, identify their dimensionality, calculate their length (norm), and take the dot product of two vectors.
- Student can identify the dimensionality of matrices, multiply them, identify when matrix multiplication is a legal operation, and transpose a matrix.
- Student can identify special types of square matrices including the identity matrix, as well as find the determinant and inverse of a matrix.
- Student can use NumPy to perform basic Linear Algebra operations with Python.

# [Why Linear Algebra?](#why-linear-algebra)

Student can illustrate why we care about linear algebra in the scope of data science.

## Overview

Data Science, Machine Learning, and Artificial intelligence is all about getting computers to do things for us better, cheaper, and faster than we could do them ourselves.

How do we do that? Computers are good at doing small repetitive tasks (like arithmetic). if we tell them what small repetitive tasks to do in the right order then sometimes all of those combined behaviors will result in something that looks like a human's behavior (or at least the decisions/output look like something a human might decide to do/create).



[Le Comte de Belamy](https://obvious-art.com/le-comte-de-belamy.htm)

The set of instructions that we give to a computer to complete certain tasks is called an **algorithm**. The better that we can organize the set of instructions, the faster that computers can do them. The method that we use to organize and store our set of instructions so that the computer can do them super fast is called a **data structure**. The practice of optimizing the organization of our data structures so that they run really fast and efficiently is called **computer science**. (This is why we will have a unit dedicated solely to computer science in a few months). Data Scientists should care how fast computers can process their sets of instructions (algorithms).

## Follow Along

Here's a simple data structure, in Python it's known as a **list**. It's one of the simplest ways that we can store things (data) and maintain their order. When giving instructions to a computer, it's important that the computer knows in what order to execute them.


```
selfDrivingCarInstructions = ["open door", 
          "sit on seat", 
          "put key in ignition", 
          "turn key to the right until it stops", 
          "push brake pedal", 
          "change gear to 'Drive'", 
          "release brake pedal", 
          "push gas pedal",
          '''turn wheel to navigate streets with thousands of small rules and 
          exeptions to rules all while avoiding collision with other 
          objects/humans/cars, obeying traffic laws, not running out of fuel and 
          getting there in a timely manner''',
          "close door"]

# We'll have self-driving cars next week for sure. NBD
```

# Maintaining the order of our sets of ordered instruction-sets
Here's another data structure we can make by putting lists inside of lists, this is called a two-dimensional list. Sometimes it is also known as a two-dimensional array or --if you put some extra methods on it-- a dataframe. As you can see things are starting to get a little bit more complicated.


```
holdMyData = [
    [1,2,3],
    [4,5,6],
    [7,8,9]
]

# Disregard the quality of these bad instructions
```

## Linear Algebra - organize and execute big calculations/operations really fast

So why linear algebra? Because the mathematical principles behinds **vectors** and **matrices** (lists and 2D lists) will help us understand how we can tell computers how to do an insane number of calculations in a very short amount of time.

Remember when we said that computers are really good at doing small and repetitive tasks very quickly?

## I Give You... Matrix Multiplication:



If you mess up any of those multiplications or additions you're up a creek.

## I Give You... Finding the Determinant of a Matrix: (an introductory linear algebra topic)

## 2x2 Matrix


Just use the formula!</center>

## 3x3 Matrix


Just calculate the determinant of 3 different 2x2 matrices and multiply them by 3 other numbers and add it all up.
     
## 4x4 Matrix


Just calculate 3 diferent 3x3 matrix determinants which will require the calculating of 9 different 2x2 matrix determinants, multiply them all by the right numbers and add them all up. And if you mess up any of those multiplications or additions you're up a creek.

## 5x5 Matrix! 


## ...


## ...

Just kidding, any linear algebra professor who assigns the hand calculation of a 5x5 matrix determinant (or larger) is a sadist. This is what computers were invented for! Why risk so much hand calculation in order to do something that computers **never** make a mistake at?

By the way, when was the last time that you worked with a dataframe that was 4 rows x 4 columns or smaller?

Quick, find the determinant of this 42837x42837 dataframe by hand!

### Common Applications of Linear Algebra in Data Science:

### Vectors: Rows, Columns, lists, arrays

### Matrices: tables, spreadsheets, dataframes

### Linear Regression: (You might remember from the intro course)




```
# Linear Regression Example
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

# Read CSV
df = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/Ice_Cream_Sales.csv')

# Create Column of 1s
df['Ones'] = np.ones(11)

# Format X and Y Matrices
X = df[['Ones', 'Farenheit']].values
Y = df['Dollars'].values.reshape(-1, 1)

# Calculate Beta Values
beta = np.matmul(np.linalg.inv(np.matmul(np.transpose(X), X)), np.matmul(np.transpose(X), Y))

print(beta)
```

    [[-596.20648399]
     [  24.68849397]]



```
# Assign Beta Values to Variables
beta_0 = beta[0,0]
beta_1 = beta[1,0]

# Plot points with line of best fit
plt.scatter(df['Farenheit'], df['Dollars'])
axes = plt.gca()
x_vals = np.array(axes.get_xlim())
y_vals = beta_0 + beta_1 * x_vals
plt.plot(x_vals, y_vals, '-', color='b')
plt.title('Ice Cream Sales Regression Line')
plt.xlabel('Farenheit')
plt.ylabel('Dollars')
plt.show()
```

### Dimensionality Reduction Techniques: Principle Component Analysis (PCA) and Singular Value Decomposition (SVD)

Take a giant dataset and distill it down to its important parts. (typically as a pre-processing step for creating visualizations or putting into other models.)



### Deep Learning: Convolutional Neural Networks, (Image Recognition)

"Convolving" is the process of passing a filter/kernel (small matrix) over the pixels of an image, multiplying them together, and using the result to create a new matrix. The resulting matrix will be a new image that has been modified by the filter to emphasize certain qualities of an image. This is entirely a linear algebra-based process. A convolutional neural network learns the filters that help it best identify certain aspects of images and thereby classify immages more accurately. 




```
!pip install imageio
```

    Requirement already satisfied: imageio in /usr/local/lib/python3.6/dist-packages (2.4.1)
    Requirement already satisfied: pillow in /usr/local/lib/python3.6/dist-packages (from imageio) (4.3.0)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from imageio) (1.17.4)
    Requirement already satisfied: olefile in /usr/local/lib/python3.6/dist-packages (from pillow->imageio) (0.46)



```
# Convolution in action
import imageio
import matplotlib.pyplot as plt
import numpy as np
import scipy.ndimage as nd
from skimage.exposure import rescale_intensity
from skimage import color

img = imageio.imread('https://www.dropbox.com/s/dv3vtiqy439pzag/all_the_things.png?raw=1')
plt.axis('off')
plt.imshow(img);
```


```
# Convert I to grayscale, so it will be MxNx1 instead of MxNx4
grayscale = color.rgb2gray(img)
plt.axis('off')
plt.imshow(grayscale, cmap=plt.cm.gray);
```


```
laplacian = np.array([[0,0,1,0,0],
              [0,0,2,0,0],
              [1,2,-16,2,1],
              [0,0,2,0,0],
              [0,0,1,0,0]])


laplacian_image = nd.convolve(grayscale, laplacian)
plt.axis('off')
plt.imshow(laplacian_image, cmap=plt.cm.gray);
```


```
sobel_x = np.array([
    [-1,0,1],
    [-2,0,2],
    [-1,0,1]
])

sobel_x_image = nd.convolve(grayscale, sobel_x)
plt.axis('off')
plt.imshow(sobel_x_image, cmap=plt.cm.gray);
```


```
sobel_y = np.array([
    [1,2,1],
    [0,0,0],
    [-1,-2,-1]
])

sobel_y_image = nd.convolve(grayscale, sobel_y)
plt.axis('off')
plt.imshow(sobel_y_image, cmap=plt.cm.gray);
```

## Are we going to learn to do Linear Algebra by hand?

Let me quote your seventh grade math teacher:

> "ArE yOu GoInG To CaRrY a CaLcUlAtOr ArOuNd wItH yOu EvErYwHeRe YoU gO???"

Of course you're going to carry a calculator around everywhere, so mostly **NO**, we're not going to do a lot of hand calculating. We're going to try and refrain from calculating things by hand unless it is absolutely necessary in order to understand and implement the concepts.

We're not trying to re-invent the wheel.

We're learning how to  **use** the wheel. 

## Challenge

I hope that this teaser helps you be excited about how studying Linear Algebra will help us build the intuition and skills to unlock some of the most exciting and valuable tools and techniques in machine learning. Over the next two modules we are going start building up to the concept of PCA which as you'll see will require a firm understanding of many important linear algebra intuitions.

# [Salars and Vectors](#scalars-and-vectors)

Student can graph vectors, identify their dimensionality, calculate their length (norm), and take the dot product of two vectors.


## Overview

Scalars and vectors are the basic building blocks of linear algebra. Scalars are just numbers stored as a variable, and when vectors or matrices are multiplied by them they will be scaled according to the size of the number. Vectors are a lot like lists of values in Python or like rows/columns in a dataframe many of the principles and intuitions that we learn here can be tied back to our datasets. Understanding vectors is important because matrices can be thought of as a set of vectors just like a dataframe can be thought of as a group of columns or rows. 

## Follow Along

### Scalars

A single number. Variables representing scalars are typically written in lower case.

Scalars can be whole numbers or decimals.

\begin{align}
a = 2
\qquad
b = 4.815162342
\end{align}

They can be positive, negative, 0 or any other real number.

\begin{align}
c = -6.022\mathrm{e}{+23}
\qquad
d = \pi
\end{align}


```
import math
import matplotlib.pyplot as plt
import numpy as np

# Start with a simple vector
blue = [.5, .5]

# Then multiply it by a scalar

green = np.multiply(2, blue)

red = np.multiply(math.pi, blue)

orange = np.multiply(-0.5, blue)

# Plot the Scaled Vectors
plt.arrow(0,0, red[0], red[1],head_width=.05, head_length=0.05, color ='red')
plt.arrow(0,0, green[0], green[1],head_width=.05, head_length=0.05, color ='green')
plt.arrow(0,0, blue[0], blue[1],head_width=.05, head_length=0.05, color ='blue')
plt.arrow(0,0, orange[0], orange[1],head_width=.05, head_length=0.05, color ='orange')
plt.xlim(-1,2)          
plt.ylim(-1,2)
plt.title("Scaled Vectors")
plt.show()
```

### Vectors:

A vector of dimension *n* is an **ordered** collection of *n* elements, which are called **components** (Note, the components of a vector are **not** referred to as "scalars"). Vector notation variables are commonly written as a bold-faced lowercase letters or italicized non-bold-faced lowercase characters with an arrow (→) above the letters:

Written: $\vec{v}$ 

Examples:

\begin{align}
   \vec{a} = 
   \begin{bmatrix}
           1\\
           2
    \end{bmatrix}
    \qquad
    \vec{b} =
    \begin{bmatrix}
          -1\\
           0\\
           2
    \end{bmatrix}
    \qquad
    \vec{c} =
    \begin{bmatrix}
           4.5
    \end{bmatrix}
    \qquad
    \vec{d} =
    \begin{bmatrix}
           Pl\\
           a\\
           b\\
           \frac{2}{3}
    \end{bmatrix}
\end{align}

The above vectors have dimensions 2, 3, 1, and 4 respectively.

Why do the vectors below only have two components?


```
# Vector Examples 
yellow = [.5, .5]
red = [.2, .1]
blue = [.1, .3]

# Coordinate Pairs for where the arrowheads are
# yellow = (.5, .5)
# red = (.2, .1)
# blue = (.1, .3)

plt.arrow(0, 0, .5, .5, head_width=.02, head_length=0.01, color = 'y')
plt.arrow(0, 0, .2, .1, head_width=.02, head_length=0.01, color = 'r')
plt.arrow(0, 0, .1, .3,  head_width=.02, head_length=0.01, color = 'b')
plt.title('Vector Examples')
plt.show()
```

In domains such as physics it is emphasized that vectors have two properties: direction and magnitude. It's rare that we talk about them in that sense in Data Science unless we're specifically in a physics context. We just note that the length of the vector is equal to the number of dimensions of the vector. 

What happens if we add a third component to each of our vectors?


```
from mpl_toolkits.mplot3d import Axes3D
import numpy as np

yellow = [.5, .5, .5]
red = [.2, .1, .0]
blue = [.1, .3, .3]

vectors = np.array([[0, 0, 0, .5, .5, .5], 
                    [0, 0, 0, .2, .1, .0],
                    [0, 0, 0, .1, .3, .3]])

X, Y, Z, U, V, W = zip(*vectors)
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.quiver(X, Y, Z, U, V, W, length=1)
ax.set_xlim([0, 1])
ax.set_ylim([0, 1])
ax.set_zlim([0, 1])
ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_zlabel('Z')
plt.show()
```

### Norm of a Vector (Magnitude or length)

The *Norm* or *Magnitude* of a vector is nothing more than the **length** of the vector. Since a vector is just a line (essentially) if you treat it as the hypotenuse of a triangle you could use the pythagorean theorem to find the equation for the norm of a vector. We're essentially just generalizing the equation for the hypotenuse of a triangle that results from the pythagorean theorem to n dimensional space.

We denote the norm of a vector by wrapping it in double pipes (like double absolute value signs)

\begin{align}
||v|| = 
\sqrt{v_{1}^2 + v_{2}^2 + \ldots + v_{n}^2}
\\
\vec{a} = 
\begin{bmatrix}
3 & 7 & 2 & 4
\end{bmatrix}
\\
||a|| = \sqrt{3^2 + 7^2 + 2^2 + 4^2} \\
||a|| = \sqrt{9 + 49 + 4 + 16} \\
||a|| = \sqrt{78}
\end{align}

The Norm is the square root of the sum of the squared elements of a vector.

Properties of the Norm:

The norm is always positive or zero $||x|| \geq 0$ 

The norm is only equal to zero if all of the elements of the vector are zero.

The Triangle Inequality: $|| x + y ||\leq ||x|| + ||y||$


### The Equation for the norm of a vector is just Pythagorean Theorem extended to more than two dimensions.

$a^2 + b^2 = c^2$

a = one side of a right triangle

b = the other side

c = the "hypotenuse" of a right triangle (We consider the vector that we're looking at to be the hypotenuse of a triangle



$c = \sqrt{a^2 + b^2}$

### Dot Product

The dot product of two vectors $\vec{a}$ and $\vec{b}$ is a scalar quantity that is equal to the sum of pair-wise products of the components of vectors a and b. An example will make this make much more sense:

\begin{align} \vec{a} \cdot \vec{b} = (a_{1} \times b_{1}) + (a_{2} \times b_{2}) + \ldots + ( a_{n} \times b_{n}) \end{align}

Example:

\begin{align}
\vec{a} = 
\begin{bmatrix}
3 & 7 & 2 & 4
\end{bmatrix}
\qquad
\vec{b} = 
\begin{bmatrix}
4 & 1 & 12 & 6
\end{bmatrix}
\end{align}

The dot product of two vectors would be:
\begin{align}
a \cdot b = (3)(4) + (7)(1) + (2)(12) + (4)(6) \\ 
= 12 + 7 + 24 + 24  \\
= 67
\end{align}

The dot product is commutative: $ a \cdot b = b \cdot a$

The dot product is distributive: $a \cdot (b + c) = a \cdot b + a \cdot c$

Two vectors must have the same number of components in order for the dot product to exist. If they lengths differ the dot product is undefined.

## Challenge

Being able to calculate the length of a vector is vital as a vector's length is one of the key attributes that defines it. Likewise taking the dot product of two vectors is a process that you **need** to be comfortable with as it is involved in many more complex linear algebra processes. You will practice both of these concepts as well as graphing vectors on your assignment this afternoon.

# [Matrices](#matrices)

Student can identify the dimensionality of matrices, multiply them, identify when matrix multiplication is a legal operation, and transpose a matrix.

A **matrix** is a rectangular grid of numbers arranged in rows and columns. Variables that represent matrices are typically written as capital letters (boldfaced as well if you want to be super formal).

\begin{align}
A = 
    \begin{bmatrix}
           1 & 2 & 3\\
           4 & 5 & 6\\
           7 & 8 & 9
    \end{bmatrix}
    \qquad
    B = \begin{bmatrix}
           1 & 2 & 3\\
           4 & 5 & 6
    \end{bmatrix}
 \end{align}

You'll notice that we arrange our dataframes in a similar grid-like pattern meaning that anything we learn about matrices also applies to the dataframes that we work with in Pandas

## Overview

A **matrix** is a rectangular grid of numbers arranged in rows and columns. Variables that represent matrices are typically written as capital letters (boldfaced as well if you want to be super formal).

\begin{align}
A = 
    \begin{bmatrix}
           1 & 2 & 3\\
           4 & 5 & 6\\
           7 & 8 & 9
    \end{bmatrix}
    \qquad
    B = \begin{bmatrix}
           1 & 2 & 3\\
           4 & 5 & 6
    \end{bmatrix}
 \end{align}

You'll notice that we arrange our dataframes in a similar grid-like pattern meaning that anything we learn about matrices also applies to the dataframes that we work with in Pandas


```
import pandas as pd

df = pd.DataFrame({'a': [1,2,3], 'b': [4,5,6]})

print(df.shape)
df.head()
```

    (3, 2)





<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>a</th>
      <th>b</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>4</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>5</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>6</td>
    </tr>
  </tbody>
</table>
</div>



## Follow Along

### Dimensionality

The number of rows and columns that a matrix has is called its **dimension**.

When listing the dimension of a matrix we always list rows first and then columns. 

The dimension of matrix A is 3x3. (Note: This is read "Three by Three", the 'x' isn't a multiplication sign.)

What is the Dimension of Matrix B?

### Matrix Equality

In order for two Matrices to be equal the following conditions must be true:

1) They must have the same dimensions.

2) Corresponding elements must be equal.

\begin{align}
    \begin{bmatrix}
           1 & 4\\
           2 & 5\\
           3 & 6 
    \end{bmatrix} 
    \neq
    \begin{bmatrix}
           1 & 2 & 3\\
           4 & 5 & 6
    \end{bmatrix}
\end{align}

### Matrix Multiplication

You can multipy any two matrices where the number of  columns of the first matrix is equal to the number of rows of the second matrix.

The unused dimensions of the factor matrices tell you what the dimensions of the product matrix will be.



There is no commutative property of matrix multiplication (you can't switch the order of the matrices and always get the same result). 

Matrix multiplication is best understood in terms of the dot product. Remember:

\begin{align} \vec{a} \cdot \vec{b} = (a_{1} \times b_{1}) + (a_{2} \times b_{2}) + \ldots + ( a_{n} \times b_{n}) \end{align}

To multiply to matrices together, we will take the dot product of each row of the first matrix with each column of the second matrix. The position of the resulting entries will correspond to the row number and column number of the row and column vector that were used to find that scalar. Lets look at an example to make this more clear.



\begin{align}
\begin{bmatrix}
1 & 2 & 3 \\
4 & 5 & 6
\end{bmatrix} 
\begin{bmatrix}
7 & 8 \\
9 & 10 \\
11 & 12
\end{bmatrix} 
=
\begin{bmatrix}
(1)(7)+(2)(9)+(3)(11) & (1)(8)+(2)(10)+(3)(12)\\
(4)(7)+(5)(9)+(6)(11) & (4)(8)+(5)(10)+(6)(12)
\end{bmatrix}
=
\begin{bmatrix}
(7)+(18)+(33) & (8)+(20)+(36)\\
(28)+(45)+(66) & (32)+(50)+(72)
\end{bmatrix}
=
\begin{bmatrix}
58 & 64\\
139 & 154
\end{bmatrix}
\end{align}

## Transpose

A transposed matrix is one whose rows are the columns of the original and whose columns are the rows of the original.

Common notation for the transpose of a matrix is to have a capital $T$ superscript or a tick mark:

\begin{align}
B^{T}
\qquad
B^{\prime}
\end{align}

The first is read "B transpose" the second is sometimes read as "B prime" but can also be read as "B transpose".

The transpose of any matrix can be found easily by fixing the elements on the main diagonal and flipping the placement of all other elements across that diagonal.


  
\begin{align}
  B = 
\begin{bmatrix}
  1 & 2 & 3 \\
  4 & 5 & 6
\end{bmatrix}
  \qquad
  B^{T} = 
  \begin{bmatrix}
  1 & 4 \\
  2 & 5 \\
  3 & 6
  \end{bmatrix}
\end{align}




```
df
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>a</th>
      <th>b</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>4</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2</td>
      <td>5</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3</td>
      <td>6</td>
    </tr>
  </tbody>
</table>
</div>




```
# Transposing a dataframe switches its rows and columns
df.T
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>0</th>
      <th>1</th>
      <th>2</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>a</th>
      <td>1</td>
      <td>2</td>
      <td>3</td>
    </tr>
    <tr>
      <th>b</th>
      <td>4</td>
      <td>5</td>
      <td>6</td>
    </tr>
  </tbody>
</table>
</div>



## Challenge

Throughout the week we will be learning how the princples of mathematical matrices apply to our datasets. We'll be focusing mostly on math topics and techniques but as the week goes on we will be making more and more connections back to how we think about and work with our datasets.


# [Square Matrices](#square-matrices)

Student can identify special types of square matrices including the identity matrix, as well as find the determinant and inverse of a matrix.

## Overview

In a traditional linear algebra class after the first few weeks you would deal almost exclusively with square matrices. They have very nice properties that their lopsided sisters and brothers just don't possess. 

A square matrix is any matrix that has the same number of rows as columns:

\begin{align}
A =
\begin{bmatrix}
  a_{1,1}
\end{bmatrix}
\qquad
B =
\begin{bmatrix}
b_{1,1} & b_{1,2} \\
b_{2,1} & b_{2,2}
\end{bmatrix}
\qquad
C =
\begin{bmatrix}
c_{1,1} & c_{1,2} & c_{1,3} \\
c_{2,1} & c_{2,2} & c_{2,3} \\
c_{3,1} & c_{3,2} & c_{3,3} 
\end{bmatrix}
\end{align}

## Follow Along

### Special Kinds of Square Matrices

**Diagonal:** Values on the main diagonal, zeroes everywhere else.

\begin{align}
A =
\begin{bmatrix}
a_{1,1} & 0 & 0 \\
0 & a_{2,2} & 0 \\
0 & 0 & a_{3,3} 
\end{bmatrix}
\end{align}

**Upper Triangular:** Values on and above the main diagonal, zeroes everywhere else.

\begin{align}
B =
\begin{bmatrix}
b_{1,1} & b_{1,2} & b_{1,3} \\
0 & b_{2,2} & b_{2,3} \\
0 & 0 & b_{3,3} 
\end{bmatrix}
\end{align}

**Lower Triangular:** Values on and below the main diagonal, zeroes everywhere else.

\begin{align}
C =
\begin{bmatrix}
c_{1,1} & 0 & 0 \\
c_{2,1} & c_{2,2} & 0 \\
c_{3,1} & c_{3,2} & c_{3,3} 
\end{bmatrix}
\end{align}

**Identity Matrix:** A diagonal matrix with ones on the main diagonal and zeroes everywhere else. The product of the any square matrix and the identity matrix is the original square matrix $AI == A$. Also, any matrix multiplied by its inverse will give the identity matrix as its product.  $AA^{-1} = I$

\begin{align}
D =
\begin{bmatrix}
  1
\end{bmatrix}
\qquad
E =
\begin{bmatrix}
1 & 0 \\
0 & 1
\end{bmatrix}
\qquad
F =
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1 
\end{bmatrix}
\end{align}

**Symmetric:** The numbers above the main diagonal are mirrored below/across the main diagonal.

\begin{align}
G =
\begin{bmatrix}
1 & 4 & 5 \\
4 & 2 & 6 \\
5 & 6 & 3 
\end{bmatrix}
\end{align}




### Determinant

The determinant is a property that all square matrices possess and is denoted $det(A)$ or using pipes (absolute value symbols) $|A|$

The equation given for finding the determinant of a 2x2 matrix is as follows:

\begin{align}
A = \begin{bmatrix}
a & b \\
c & d
\end{bmatrix}
\qquad
|A| = ad-bc
\end{align}

The determinant of larger square matrices recursively by finding the determinats of the smaller matrics that make up the large matrix.

For example:

<center></center>

The above equation is **very** similar to the equation that we use to find the cross-product of a 3x3 matrix. 

### Inverse

 There are multiple methods that we could use to find the inverse of a matrix by hand. I would suggest you explore those methods --if this content isn't already overwhelming enough. The inverse is like the reciprocal of the matrix that was used to generate it. Just like $\frac{1}{8}$ is the reciprocal of 8, $A^{-1}$ acts like the reciprocal of $A$.  The equation for finding the determinant of a 2x2 matrix is as follows:
 
 \begin{align}
A = \begin{bmatrix}
a & b \\
c & d
\end{bmatrix}
\qquad
A^{-1} = \frac{1}{ad-bc}\begin{bmatrix}
d & -b\\
-c & a
\end{bmatrix}
\end{align}

 ### What happens if we multiply a matrix by its inverse?

The product of a matrix multiplied by its inverse is the identity matrix of the same dimensions as the original matrix. There is no concept of "matrix division" in linear algebra, but multiplying a matrix by its inverse is very similar since $8\times\frac{1}{8} = 1$. 

\begin{align}
A^{-1}A = I 
\end{align}


### Not all matrices are invertible

Matrices that are not square are not invertible.
 
 A matrix is invertible if and only if its determinant is non-zero. You'll notice that the fraction on the left side of the matrix is $\frac{1}{det(A)}$.
 
 As you know, dividing anything by 0 leads to an undefined quotient. Therefore, if the determinant of a matrix is 0, then the entire inverse becomes undefined. 

### What leads to a 0 determinant?
 
 A square matrix that has a determinant of 0 is known as a "singular" matrix. One thing that can lead to a matrix having a determinant of 0 is if two rows or columns in the matrix are perfectly collinear. Another way of saying this is that the determinant will be zero if the rows or columns of a matrix are not linearly dependent. 
 
One of the most common ways that a matrix can end up having rows that are linearly dependent is if one column a multiple of another column. Lets look at an example:

\begin{align}
C =\begin{bmatrix}
  1 & 5 & 2 \\
  2 & 7 & 4 \\
  3 & 2 & 6
\end{bmatrix}
\end{align}

Look at the columns of the above matrix, column 3 is exactly double column 1. (could be any multiple or fraction) Think about if you had some measure in a dataset of distance in miles, but then you also wanted to convert its units to feet, so you create another column and multiply the mile measure by 5,280 (Thanks Imperial System). But then you forget to drop one of the columns so you end up with two columns that are linearly dependent which causes the determinant of your dataframe to be 0 and will cause certain algorithms to fail. We'll go deeper into this concept next week (this can cause problems with linear regression) so just know that matrices that have columns that are a multiple or fraction of another column will cause the determinant of that matrix to be 0.

## Challenge

Square Matrices have many important properties like the inverse and determinant that we will be using in the future. For more details about the implications of a square matrix having an inverse you can try googling the "Invertible Matrix Theorem" mastering and proving the different partso of this theorem is one of the main focuses of traditional linear algebra education.

# [Doing Linear Algebra with NumPy](#linear-algebra-NumPy)

Student can use NumPy to perform basic Linear Algebra operations with Python.


## Overview

NumPy is at the core of many types of scientific computing and is the main tool that we will use to perform Linear Algebra operations with Python. 

We have talked about how vectors are a lot like Python Lists, however we will mostly use NumPy arrays instead of Python lists as the main data structure. Lets demonstrate some of the main differences betwen Python lists and NumPy Arrays.


```
### What will happen when I add the two Python lists together?

```


```
# Lets create two 1-Dimensional NumPy arrays and add them together?
# What do you think the result will be?

# Array Broadcasting
# Doing some arithmetic operation to vectors "element-wise"

```

What we have just demonstrated is one of the primary benefits of NumPy and it is called "Array Broadcasting" This means that arithmetic operations happen "element-wise"


```

```


```

```


```

```

## Follow Along

Lets calculate a dot product. What do we need to do? We need to multiply two vectors "element-wise" and them sum up all of those products


```
# Find the dot product of vectos a and b

# Dot Product

```

We can also use provided linear algebra functions from the [np.linalg](https://docs.scipy.org/doc/numpy-1.15.1/reference/routines.linalg.html) documentation in order to perform Linear Algebra processes.


```

```

### 1-Dimensional vs 2-Dimensional NumPy Arrays

One dimensional NumPy arrays can be used to represent vectors but cannot be used to differentiate between row and column vectors.

\begin{align}
\text{column vector} = \begin{bmatrix}1 \\ 2 \\ 3\end{bmatrix}
\end{align}

\begin{align}
\text{row vector} = \begin{bmatrix} 1 & 2 & 3\end{bmatrix}
\end{align}

Notice that one dimensional arrays do not have a second element listed for their shape. so that there is no way to tell if they represent column vectors or row vectors


```

```


```
# Notice how only the number of items in the array is listed

```


```
# And if we swap rows and columns the shape does not change

```

### If I explicitly want a row of column vector then I need to use a 2D NumPy Array


```

```

### Only use NumPy Arrans, DO NOT use the NumPy Matrix class.

NumPy has a matrix class which can be used for representing matrices. However, I don't want you to use that datatype because it will be removed from NumPy in the future. However, I want you to be aware that this other datatype exists so that you don't get confused when you look at stack overflow posts or other tutorials that use the matrix datatype. Keep an eye out for resources that use it and just know that they will soon be outdated. Anything that can be done with the matrix class can also be done with simple NumPY arrays.


```
# Do not work with this datatype
# This will be removed from NumPy in the future

```


```

```


```

```


```
# Matrix as a NumPy Array

# ndarrays

```


```

```


```

```


```

```

## Challenge

In your afternoon assignment you will perform many different linear algebra functions using NumPy. Many of which will be done easily by using these [Helpful NumPy Linear Algebra Functions](https://docs.scipy.org/doc/numpy-1.15.1/reference/routines.linalg.html) found in the NumPy documentation.

# Review

Lets good at working with vectors and matrices with NumPy!

Remember that you'll need many functions from [the documentation](https://docs.scipy.org/doc/numpy-1.15.1/reference/routines.linalg.html) in order to complete the assignment

Remember to use 2D NumPy arrays to represent row/column vectors as well as matrices and please share any helpful resources that you find in the cohort channel.

10e636c50c1c9d4c62e921004a4344ba71bea6a8
ipynb
Jupyter Notebook
01_Unit_1/Sprint3-Linear-Algebra/Module1/LS_DS_131_Vectors_and_Matrices.ipynb
mark-morelos/my_DataScience_notes
2e05a3d77949cb8ccd9aac7c04cf7758d7b73d1c
MIT
2020-12-04T22:01:58.000Z
2021-05-28T18:13:44.000Z
01_Unit_1/Sprint3-Linear-Algebra/Module1/LS_DS_131_Vectors_and_Matrices.ipynb
mark-morelos/my_DataScience_notes
2e05a3d77949cb8ccd9aac7c04cf7758d7b73d1c
MIT
01_Unit_1/Sprint3-Linear-Algebra/Module1/LS_DS_131_Vectors_and_Matrices.ipynb
mark-morelos/my_DataScience_notes
2e05a3d77949cb8ccd9aac7c04cf7758d7b73d1c
MIT
Qwen/Qwen-72B
1. YES
2. YES