# Python imports
import numpy as np

"""
Explanation: Numpy Arrays and Vectorization
Frequently, matrices and vectors are needed for computation and are a convenient way to store and access data. Vectors are more commonly many rows with a single column. A significant amount of work has been done to make computers very fast at doing matrix math, and while the tradeoff is commonly framed as 'more memory for faster calculation', there is typically enough memory in contemporary computation devices to process chunks of matrices.
In Python's NumPy, vectors and matrices are referred to as arrays: a constant-sized collection of elements (of the same type - integer, floating point number, string of characters, etc.).
Underneath, Python arrays use C for greater efficiency.
Note that this is different from the python list - lists are a python datatype, whereas arrays are objects that are made available via the python package numpy. 
Array restrictions:
 - You can't append things to an array (i.e. you can't make it bigger without creating an entirely new array)
 - You can only put things of the same type into an array
The array is the basis of all (fast) scientific computing in Python.
We need to have a solid foundation of what an array is, how to use it, and what it can do.
By the end of this file you should have seen simple examples of:
1. Arrays are faster than lists!
2. Create an array
3. Different types of arrays
4. Creating and accessing (indexing) arrays
5. Building arrays from other arrays (appending)
6. Operations on arrays of different sizes (broadcasting)
7. Arrays as Python objects
Further reading:
https://docs.scipy.org/doc/numpy-dev/user/numpy-for-matlab-users.html
End of explanation
"""


l = 20000
test_list = list(range(l))
test_array = np.arange(l)

print(type(test_list))
print(type(test_array))

print(test_list[:300]) # Print the first 300 elements 
                       # (more on indexing in a bit):

print(test_array)

%timeit [np.sqrt(i) for i in test_list]

%timeit [np.sqrt(test_array)]

"""
Explanation: Arrays versus lists
While both data types hold a series of discrete information, arrays are stored more efficiently in memory and have significantly higher performance than Python lists. They also bring with them a host of properties and syntax that makes them more efficient, especially for numeric operations.
End of explanation
"""


test_array = np.array([[1,2,3,4], [6,7,8,9]])
print(test_array)

"""
Explanation: If statement says "10 loops, best of 3: [time]" it means the fastest of 10 repeated runs was recorded - then the 10 runs were repeated twice more, resulting in an overall fastest time.
Creating and accessing (indexing) arrays
We can create arrays from scratch:
End of explanation
"""


test_array[0,3]

"""
Explanation: Index arrays using square brackets, starting from zero and specifying row, column:
End of explanation
"""


test_array.dtype

"""
Explanation: Arrays are duck typed just like Python variables, that is to say that Python will try to determine what kind of variable it should be based on how it's used.    
Numpy arrays are all the same type of variable. To check the data type (dtype) enter:
End of explanation
"""


test_array = test_array.astype('float64')
print(test_array)

# We can create arrays of boolean values too:
bool_array = np.array([[True, True, False,True],[False,False,True,False]])
print(bool_array)

"""
Explanation: Different variable types use different amounts of memory and can have an effect on performance for very large arrays. 
Changing the type of array is possible via:
End of explanation
"""


test_array[0,3]=99 # Assign value directly
print(test_array)

"""
Explanation: We can replace values in an array:
End of explanation
"""


test_array[0,3] = 'nan'
print(test_array)

"""
Explanation: Deleting values from an array is possible, but due to the way they're stored in memory, it makes sense to keep the array structure. Often, a 'nan' is used (not a number) or some nonsensical value is used, i.e.: 0 or -1.
Keep in mind that 'nan' only works for some types of arrays:
End of explanation
"""


test_array[:,1]     # Use the ':' to index along one dimension fully

test_array[1,1:]    # Adding a colon indexes the rest of the values 
                    #    (includes the numbered index)

test_array[1,1:-1]  # We can index relative to the first and last elements

test_array[1,::2]   # We can specify the indexing order

test_array[1,1::-1] # We can get pretty fancy about it 
                    # Index second row, second from first to second from 
                    #     last in reverse order.

"""
Explanation: Fancy ways of indexing
Slicing Arrays:
Slicing arrays refers to indexing >1 elements in a previous array. Slicing is often used when parallelizing computations using arrays. Indexing is array[row, column].
End of explanation
"""


test_array[bool_array] # Use our bool_array from earlier

"""
Explanation: Logical Indexing
We can specify only the elements we want by using an array of True/False values:
End of explanation
"""


nans = np.isnan(test_array) 
print(nans)

test_array[nans] = 4
print(test_array)

"""
Explanation: Using the isnan function in numpy:
End of explanation
"""


test_array_Vstacked = np.vstack((test_array, [1,2,3,4]))
print(test_array_Vstacked)

test_array_Hstacked = np.hstack((test_array, test_array))
print(test_array_Hstacked)

"""
Explanation: Building arrays from other arrays (appending)
We can build arrays from other array via Python stacking in a horizontal or vertical way:
End of explanation
"""


test_array_Hstacked.flatten()

"""
Explanation: We can bring these dimensions back down to one via flatten:
End of explanation
"""


test_array

print("The broadcasted array is: ", test_array[0,:])
test_array[0,:] * test_array

"""
Explanation: Caution: appending to numpy arrays frequently is memory intensive. Every time this happens, an entirely new chunk of memory needs to be used, so the old array is moved in memory to a new location.
It's faster to 'preallocate' an array with empty values, and simply populate as the computation progresses.
Operations on arrays of different sizes (broadcasting)
Python automatically handles arithmetic operations with arrays of different dimensions. In other words, when arrays have different (but compatible) shapes, the smaller is 'broadcast' across the larger.
End of explanation
"""


print("The broadcasted array is: ", test_array[:,0])
#test_array[:,0] * test_array # Uncomment the line to see that the 
                              #     dimensions don't match

# Make use of the matrix transpose (also can use array.T)
np.transpose( test_array[:,0]*np.transpose(test_array) )

"""
Explanation: However, if the dimensions don't match, it won't work:
End of explanation
"""


print("The original array is: ", test_array)
print("The transposed array is: ", np.transpose(test_array) )

# Alternatively, using test_array as an opject:
print("The transposed array is: ", test_array.transpose() )

"""
Explanation: Arrays as Python objects
Python can be used as an object oriented language, and numpy arrays have lots of properties. There are many functions we can use as numpy.&lt;function&gt;(&lt;array&gt;) and array.&lt;function&gt;
For example, the transpose above:
End of explanation
"""


print("The original array dimensions are: ", test_array.shape)
print("The array transpose dimensions are: ", test_array.transpose().shape)

"""
Explanation: One of the most frequenly used properties of arrays is the dimension:
End of explanation
"""


test_array2 = np.array([1,5,4,0,1])
print("The original array is: ", test_array2)

test_array3 = test_array2.sort() # Run the sort - note that the new variable isn't assigned
print("The reassigned array should be sorted: ", test_array3)
print("test_array2 after sort: ", test_array2)

"""
Explanation: Sorting:
Sorting arrays happens in-place, so once the function is called on an array, the sorting happens to the original array:
End of explanation
"""


