import numpy as np
random = np.random.normal(size=100)
random

"""
Explanation: Session 3: Python Data Analysis World
Python has a very strong community in the data analytics and scientific computing world. There are a lot of great Python packages to support different analyses, but there are a few very key packages:

Workhorses - numpy, pandas, scipy
Spatial tools - shapely, ogr/gdal, geopandas, etc
Environments and Visualisation - Jupiter, matplotlib

You will have access to all of these after installing Anaconda and installing the additional packages described in Session 0. (The additional packages relate to spatial analysis - you can skip them if you don't need them)
Where possible, veneer-py functions will accept and return objects that are directly usable by these packages. In particular, time series and other tabular data structures are returned as pandas DataFrame objects.
This session gives very brief introductions to most of these packages. In most cases, the links in Session 0 are relevant for more information.
numpy
numpy represents multi-dimensional arrays and operations on those arrays. The arrays are typed (eg float, double precision float, integer, etc) and are indexed by integers (one per dimension).
In veneer-py, we use pandas Data Frames more than numpy arrays, but the basics of the array operations in numpy are the foundations on which pandas is built.
You can create an array of random numbers using functions under the np.random namespace. The following example creates 100 random floats using a normal distribution
Note: numpy is typically imported as np.
End of explanation
"""


random.shape

threed = random.reshape(10,5,2)
threed

"""
Explanation: The functions in np.random return one dimensional arrays. You can check this with .shape and change it with .reshape()
End of explanation
"""


a1 = np.array([20.0,12.0,77.0,77.0])
a2 = np.array([25.0,6.0,80.0,80.0])

# You can add:

a1 + a2

# Multiply (element wise):

a1 * a2

# Compute a dot product

a1.dot(a2)

# You can also perform matrix operations
# First tell numpy that your array is a matrix,
# Then transpose to compatible shapes
# Then multiply
np.matrix(a1).transpose() * np.matrix(a2)

"""
Explanation: You can perform basic arithmetic on arrays, using scalars or other arrays.
For example, given the following two arrays
End of explanation
"""


import veneer
v = veneer.Veneer(port=9876)
downstream_flow_vol = v.retrieve_multiple_time_series(criteria={'RecordingVariable':'Downstream Flow Volume'})

"""
Explanation: pandas
Pandas DataFrame objects are one of the key data types used in veneer-py.
A DataFrame is a tabular, two-dimensional data structure, which can be indexed in a range of ways, including a date and date/time index. DataFrames are aranged in named columns, each with a particular type (eg double, string, integer) and in this sense they are more flexible than numpy arrays.
Each column in a DataFrame is a pandas Series, which is useful in its own right.
End of explanation
"""


downstream_flow_vol[0:10] # <-- Look at first 10 rows (timesteps)

downstream_flow_vol[0::3000] # <-- Look at every 3000th timestep

"""
Explanation: Pandas DataFrames have a tabular presentation in the Jupyter notebook.
It's also possible to slice subsets of rows
End of explanation
"""


downstream_flow_vol.mean()

"""
Explanation: You can quickly get stats for each column in a DataFrame
End of explanation
"""


downstream_flow_vol.mean(axis=1)

"""
Explanation: You can get the same stats along rows:
End of explanation
"""


import matplotlib.pyplot as plt
%matplotlib inline

"""
Explanation: Jupyter and visualisation
It is worth spending some time exploring the capabilities of the Jupyter notebook.
In terms of managing your work:

The Edit and Insert menus have useful functions for rearranging cells, creating new cells, etc
The Cell menu has functions for running all cells in a notebook, all cells above a particular point and all cells below a point.
The Kernel menu controls the execution and lifecycle of the Python session. (In this instance, Kernel refers to an instance of an IPython session that is connected to the notebook. The Restart command clears all variables - even though earlier output is still visible in the notebook)

At this stage, most visualisation in Python notebooks is handled by matplotlib.
Matplotlib is powerful, but the learning curve can be steep.
End of explanation
"""


plt.hist(np.random.normal(size=500))

"""
Explanation: Typically, you'll create a single plot from a single cell
End of explanation
"""


methods=[np.random.uniform,np.random.normal,np.random.exponential]

n=len(methods)

# Create n sets of random numbers, where n is the number of methods specified
random_sets = [method(size=1000) for method in methods]

for i in range(n):
    # Arrange subplots 2 rows x 3 columns
    # Access the i'th column on the first row
    ax = plt.subplot(2,3,i+1)
    # Plot the random numbers
    ax.plot(random_sets[i])
    # Access the i'th column on the second row
    ax = plt.subplot(2,3,n+i+1)
    # Plot a histogram of the corresponding numbers
    ax.hist(random_sets[i])

"""
Explanation: ... But the matplotlib subplots functionality allows you to create matrices of plots.
End of explanation
"""


