# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# # Learning the Essence of Linear Algebra
#
# Linear algebra is a branch of mathematics that has applications in almost all scientific domains. But what is it exactly? Linear algebra is the study of linear maps on finite-dimensional vector space, this definition is "impeccably correct but not at all helpful." So what is really linear algebra? 
#
# Let's define the two words first: 
#     - Linear: of the first degree with respect to one or more variables. Something that follows an expected sequence that is graphically represented with a stright line. The word comes from latin and it means "pertaining or resembling a line"
#     - Algebra: a study of mathematical symbols and rules for manipulating these symbols. Originates from the arabic word al-jabir meaning "reunion of broken parts." 
#  
# So linear algebra is the study of objects that represent expected sequences. The objects that we will use are the following: scalars, vectors, matrices and tensors:
#     - Scalars: a quantity without direction but only magnitude (something you can use to scale something else)
#     - Vectors: a quantity that has a magnitude as well as direction, something you can use to determine the position of one point relative to another. An ordered array of numbers, list of values. The python package numpy can be used to create vectors (which are called arrays in numpy)
#     - Matrices: a 2D array of numbers. A table, or list of lists: [ [list1], [list2] ]
#     - Tensors: an array of numbers arranged on a regular grid with a variable number of axes. A table of tables, a list of lists of lists: [ [ [], [] ], [ [], [] ] ]

# +
import numpy as np
import matplotlib.pyplot as plt
from jupyterthemes import jtplot
# !jt -t monokai
# !jt -t monokai -f fira -fs 13 -nf ptsans -nfs 11 -N -kl -cursw 5 -cursc r -cellw 95% -T

# !jtplot.style(theme=’monokai’, context=’notebook’, ticks=True, grid=False)

# +
def plotVectors(vecs, cols=None, alpha=1):
    """
    Plot set of vectors.

    Parameters
    ----------
    vecs : array-like
        Coordinates of the vectors to plot. Each vectors is in an array. For
        instance: [[1, 3], [2, 2]] can be used to plot 2 vectors.
    cols : array-like
        Colors of the vectors. For instance: ['red', 'blue'] will display the
        first vector in red and the second in blue.
    alpha : float
        Opacity of vectors

    Returns:

    fig : instance of matplotlib.figure.Figure
        The figure of the vectors
    """
    plt.figure()
    plt.axvline(x=0, color='#A9A9A9', zorder=0)
    plt.axhline(y=0, color='#A9A9A9', zorder=0)

    for i in range(len(vecs)):
        x = np.concatenate([[0,0],vecs[i]])
        if cols:
            plt.quiver([x[0]],
                       [x[1]],
                       [x[2]],
                       [x[3]],
                       angles='xy', scale_units='xy', scale=1, color=cols[i],
                       alpha=alpha)
        else:
            plt.quiver([x[0]],
                       [x[1]],
                       [x[2]],
                       [x[3]],
                       angles='xy', scale_units='xy', scale=1,
                       alpha=alpha) 
# Source: https://hadrienj.github.io/posts/Deep-Learning-Book-Series-2.11-The-determinant/

orange = '#FF9A13'
blue = '#1190FF'

i = [0, 1]
j = [1, 0]

plotVectors([i, j], [[blue], [orange]])
plt.xlim(-0.5, 3)
plt.ylim(-0.5, 3)
plt.show()
# -

# ## Properties of Linear Objects
#
# Now that we have defined a handful of linear objects, we can start thinking of what sorts of properties susch objects have.
#
# ### Basis Vectors
#
# Suppose you have a two dimensional vector of length 1 represented by  ${{\hat {\imath }}}$  on one direction and  ${{\hat {\jmath }}}$  on the other; thus representing a vector as  ${{\hat {\imath }}}$ +  ${{\hat {\jmath }}}$. These two letters can be thought of as the grids x and y on a 2D coordinate system, thus representing a way to get to a point on the xy grid. 
#
#
# ### Linear Combinations
#
# If you take the basis vectors ${{\hat {\imath }}}$ and ${{\hat {\jmath }}}$ and scale them on each axis, you will be able to land on any point on the grid. This way of scaling basis vectors in order to reach a point is called a linear combination.
#
# ### Span
#
# The span of of a vector is defined as the space the vector is able to reach as a result of scaling. If you have a vector represented by v and w, the span of this vector is then the entire 2D grid since you can reach any point on the grid by linearly combining v and w.
#     - The span of a 1D vector is an infinite line
#     - The span of a 2D vector is an infinite plane
#     - The span of a 3D vector is an infinite 3D space $R^3$
#
# Properties such as this is what allows linear algebra to represent attributes of reality, making it so useful.
#
#     - Formally, the basis of a vector space (1D, 2D, etc.) is a set of linearly independent vectors that span the full space
#     - i and j are the basis of the 2D vector space
#     - i, j and k are the basis for the 3D vector space etc.
#     
#
# ### Linear Independence
#
# If two vectors are linearly inependent, it means that they represent independent directions in the vector space. Here are different ways of thinking about this:
#     - If you have a vector, vector 1, pointing in direction 1, then vector 2 would be independent of vector one if and only if it points in a different direction
#     - Independent columns of of matrix A means the only solution to Av = 0 is v = 0
#     - If I have independent columns in my matrix then the null space of the matrix is just the zero vector
#
# ### Basis for Subspace
#     - Independent vectors that span the space
#     - The number of dimension is the number of basis vectors we need
#     - Dimension: number of basis vectors for the subspace
#         - 1D dimension has exactly 1 basis vector
#         - 2D dimension has exactly 2 basis vectors
#     - Every basis has the same number and that number is the dimension
#     - Take 3D space, and a subspace of a plane, the plan can only have 2 independent vectors
#         - Every basis for this plane contains only and exactly 2 which is why the dimension of the plane is 2
#         
# #### Useful Things to Remember
#     - A set with only one vector is linearly independent if one vector is not the zero vector
#     - A set containing two vectors is linearly independent so long as one vector is not a multiple of the other
#     - A set containing 'n x 1' vectors is linearly dependent if the number of vectors is greather than n
#         - [1, 2] and [3, 5] are linearly inependent but whatever third vector you add will end up being linearly dependent
#     - A set containing the zero vector is always linearly dependent
#     - If the vectors are linearly independent then removing an arbitrary vector, v_i does not affect the linear independence
#     - If the set only contains distinct unit vectors, the set has linear independence
#
#
#
# #### Sources:
# https://www.youtube.com/watch?v=eeMJg4uI7o0
#
# https://hadrienj.github.io/posts/Deep-Learning-Book-Series-2.4-Linear-Dependence-and-Span/
#
# https://www.youtube.com/watch?v=50aWVJBkJNs

# #### How do you know if two vectors are linealy independent?
#
# If two vectors are linearly independent, it means that they are pointing in different directions. In other words, numerically, they can't be multiples (a scaled version) of eachother.
#
# The vectors $\{v_1, v_2, ..., v_n\}$ are linearly independent if the equation involving linear combination involving:<br>
#     $a_1v_1 + a_2v_2 + ... + a_nv_n = 0$<br>
# is true only when the scalars $a_i$ are all equal to zero. Pictorially, this means you are reducing each vector unit by unit from all sides and checking if the vectors are pointing the same direction.

# +
# TODO: write python code that will figure out linearly dependent vectors
# -

# ## The Algebra of Linear Objects
#
# Since linear algebra is the study of mathematical operations on linear objects, we will now investigate such operations and what they mean. These operations are also called linear transformations, since they are able to transform the object from one form to another.

# ### Linear Transformation
#
# <p> Transformation is simply a function that takes in a vector and transforms it into a vector of a different form. Here, only transformations that are linear will be discussed. A linear transformation will keep the lines straight, such that gridlines are kept parallel and evenly spaced. In order to describe linear transformations in a numerical way, we will use basis vectors.<p>
#
# <p> Suppose you have the following equation: $Ax = v$, where A is a matrix and v is a vector. The question is: in what way should $A$ transform $x$ such that it lands on $v$. The matrix $A$ is what will execute the linear transformation. <p>
# $
# \begin{align}
# \begin{bmatrix}
# a & b  \\
# c & d 
# \end{bmatrix}\begin{bmatrix}
# x  \\
# y 
# \end{bmatrix} &= x\begin{bmatrix}
# a  \\
# c
# \end{bmatrix} + y\begin{bmatrix}
# b  \\
# d
# \end{bmatrix}\\
# \begin{bmatrix}
# a & b  \\
# c & d 
# \end{bmatrix}\begin{bmatrix}
# x  \\
# y 
# \end{bmatrix} &= \begin{bmatrix}
# ax & + & by  \\
# cx & + & dy 
# \end{bmatrix}
# \end{align}$
#
# <p> The above equation shows how the vector $[x  y]^T$ is linearly transformed by the matrix on the left. So the vector x is transformed by $[a c]$ and $y$ is transformed by $[b d]$.<p>
#     
# <p> If the columns of the matrix $A$ are linearly dependent, this will mean that the vector will be transformed in such a way that it would end up on a line. <p>
#
#     
# - A matrix is basically a function that has the ability to linearly transform an input vector to an output vector

# +
xvals = np.linspace(-10, 10, 10)
yvals = np.linspace(-3, 3, 10)
xygrid = np.column_stack([[x, y] for x in xvals for y in yvals])


fig, axes = plt.subplots(nrows=1, ncols=5, figsize=(20, 5), facecolor='w')

axes[0].scatter(xygrid[0], xygrid[1], s=36,  edgecolor="none")
axes[0].grid(True)
axes[0].axis("equal")
axes[0].set_title("Original grid in x-y space")

transformers = [
    np.column_stack([[0, -1], [1, 0]]),
    np.column_stack([[1, 0], [0, 1]]),
    np.column_stack([[1000, 0], [0, 1000]]),
    np.column_stack([[0.0001, -1], [-1, 0.0001]])
]

for i in range(len(transformers)):
    result = np.dot(transformers[i], xygrid)
    axes[i+1].scatter(result[0], result[1], s=36, edgecolor="none")
    axes[i+1].grid(True)
    axes[i+1].axis("equal")
    axes[i+1].set_title("Transformed Grid {0}".format(i+1))


# -

# ### Composition
# <p> 
# Multiplying two matricies represents applying one linear transformation followed by another. Numerically this means multiplying two matricies together such that you end up with one matrix that is a composition of the two matricies.
# <p>
#     
# $\begin{bmatrix}
# a & b  \\
# c & d 
# \end{bmatrix}\begin{bmatrix}
# e & f  \\
# g & h 
# \end{bmatrix}  =
# \begin{bmatrix}
# ae + bg & af + bh  \\
# ce + dg & cf + dh 
# \end{bmatrix}$
#     
# <p>Keep in mind that the order in which the matricies compose matters! This makes sense because if you think of a matrix as something that transforms a vector, then the order in which this transformation happens matters because space is not symmetric all the way. Although it might be the case that order does not matter if you have enough transformations to render it symmetric.  <p>
#     
#
# #### In code the numpy dot product can be used:

# +
import numpy as np
matrix_transformer = np.array([ # Bisymmetric matrix
    [ 1, 2, 3 ], 
    [ 2, 5, 2 ], 
    [ 3, 2, 1 ]
])

input_vector = np.array([1, 2, 3])
output1 = matrix_transformer.dot(input_vector)
output2 = input_vector.dot(matrix_transformer)

print("""Output vector is {0} on the left 
And {1} on the right because of symmetry""".format(output1, output2))
# -

# ### The Determinant
# <p>The degree to which a matrix scales an area/volume as a result of transforming a vector is called the determinant. The determinant is a property of a matrix and it $determines$ how much areas/volumes (3D) change as a result of transformation imposed by the matrix. This change in area/volume occurs because the vector represents a certain area within the grid. This means that if the determinant of a matrix is zero, then the transformation imposed by such matrix squizzes the area onto a dot. This is not exactly right because the determinant can take on negative values but this has to do with direction and technicality; the essense of this example maintains validity. Let's just say that the absolute value of the determinant is what represents the area\volume. <p>
#     
# <br>
#     
# <p>What would it mean if the determinant of a matrix is zero? Well, if the determinant is zero it means that the space enclosed by the vectors (area\volume) would converge to a point (or a flat plane in 3D, also a point is possible in 3D). When the determinant is zero, it means that the columns of the matrix are linearly dependent. This is because the matrix transforms given vectors in such a way that the vectors end up either on the same direction or on to a point.<p>
#
# $$det\bigg(\begin{bmatrix}
# a & b  \\
# c & d 
# \end{bmatrix}\bigg) = ad - bc$$
#     
# <br>
#
# <p>If you multiply two matricies and then take their determinant: $det(M_1M_2)$ it would be the same as if you take respective determinants and then multiply: $det(M_1)det(M_2)$. Why is that?<p>
#
# $$det(M_1M_2) = det(M_1)det(M_2)$$
# <br>
# Well when you multiply the two matrices you will end up with a matrix that contains information about how to transform a vector, when you take the determinant of that you will have information about such transformation. The information of the determinant remains the same even if you apply it independently with each matrix given they are in the right order.
#
# <br>
#     
# <p>How much information regarding the matrix is stored in a determinant? The determinant is a function that takes in a square matrix and outputs a number. If two matrices have the same determinant, it doesn't mean that they are the same; it just means the same thing as if two rectangles have the same area.<p>
#
# In code:

# +
# The determinant can be computed with numpy
import numpy as np

M = np.array([[6, 1, 1], [4, -2, 5], [2,8,7]])
det = np.linalg.det(M)
print(det)
# -

# ### Inverse Matrices
#
#
# ### Column Space
#
#
# # Continue Here:

# ### Norms
#
#
# ### Special Matrices and Vectors
#
#
# ### Eigendecomposition
#
#
# ### The Trace Operator



StarcoderdataJupyterScriptsDedupFiltered