with open('data/inflammation-01.csv', 'r') as f:
    snippet = f.readlines()[:3]
print(*snippet)

"""
Explanation: First off, an acknowledgement of the usefulness of the Software Carpentry materials, and particularly data, in preparing this session - this course is not, however, affiliated with or endorsed by the Software Carpentry team. Nonetheless, if you would like a more in-depth walkthrough, relevant to this arthritis example, I recommend you explore their resources, particularly the python-novice-inflammation workshop.
A New Treatment for Arthritis
Starting out with some data
In this task, we will examine some data representing trials of an arthritis treatment. Our data is in the form of an ASCII file, where each row represents a patient and each column a consecutive day's measurement of inflammation.
The Scientific Python Trifecta


numpy - handling numbers


matplotlib - plotting


scipy - general scientific methods


numpy has flexible routines for loading raw data...
In the same directory as your notebooks, there is a folder with some data, called data.
End of explanation
"""


import numpy as np
data = np.loadtxt(fname='data/inflammation-01.csv', delimiter=',') # Comma-separated...
print(data)

"""
Explanation: This construct, a with statement, addresses the age-old problem of cleaning up file descriptors. In general, a with context expects the object being used to have some open and close routines that will be called at the entrance and exit of the block, respectively. Note that we don't have scoping here - the snippet variable exists outside the block, making it easy to load data in a short block and handle it later.
We could use more traditional syntax:
python
    f = open('foo.csv', 'r')
    ...
    f.close()
but this succinct approach has the benefit of containing use within a protected block.
It can be used for much more than just simple files, such as RMV.
We also note that a new piece of Python syntax has appeared - a star, indicating that an iterable (here, a list) is used to fill up arguments to a function. This is especially useful with print, and is equivalent to:
python
    print(snippet[0], snippet[1], snippet[2])
However, there are places it really shines. In fact, you can also name arguments to a function - in that case you can additionally provide a dict that fills them out (with two stars to indicate the expansion).
Looking at the printed data, you can see we have a large number of columns (equal for each row), data in the range 0-20 and it is comma-separated. Remember, each row represents a patient and each column, the measurement of inflammation on a given day.
Import the data
End of explanation
"""


type(data)

"""
Explanation: While this has kindly been rendered for us as if it were a list of lists, in reality it is something much more useful...
End of explanation
"""


data.shape

"""
Explanation: This type is numpy's N-dimensional matrix class. This means we get a wide swathe of attributes and methods applicable to matrices, and a whole load of functions inside numpy and other libraries that will happily take it as an argument. One particularly useful property is shape:
End of explanation
"""


", ".join(dir(data))

"""
Explanation: This tells us that there are 60 patients (rows), each with 40 days of inflammation measurements (columns).
We can get an idea of the wide variety of methods available on the data object...
End of explanation
"""


print(data * 2)

"""
Explanation: A lot of the magic methods here, those with double underscores on either side, allow Python to treat this object more like a built-in. For instance, the __neg__ method will get called when we enter -data. If we try data1 &lt; data2, then data1.__lt__ will be called with data2 as an argument, expecting a boolean return. The behaviour when the user writes data.previously_unknown_attr is defined by the __getattr__ method - this need not be an error. In some ways, you can think of this as an expansion of the concept of operator overloading.
The ndarray class uses those methods very effectively, to help us write clear and efficient code.
End of explanation
"""


data[0:3, 0:3]

data[:3,:3]

"""
Explanation: This works because numpy has overridden the __mul__ method.
We can also do our slicing in multiple dimensions... let's get the top-left corner...
End of explanation
"""


# Use this cell!

"""
Explanation: ...or more succinctly...
Try grabbing the bottom left corner (3x3)...
End of explanation
"""


data.mean(), data.max(), data.min()

data[3].max()  # Max inflammation for 4th patient

"""
Explanation: A few more practical uses of nparray:
End of explanation
"""


data[:,1].max() # Max infl for 2nd day

"""
Explanation: This could also be written data[3,:] - colon on its own just signifies all entries in that axis, from first to last
End of explanation
"""


data.max(axis=0)

"""
Explanation: Suppose we want the max for every day...
We could create a new list and write a loop to go through it, but it seems there should be a more transparent, succinct, Pythonic approach
End of explanation
"""


data.max(axis=0).shape

"""
Explanation: As you can see, the second day does indeed have max of 1.0. Notice that it returns an array (this is a superclass of ndarray), so we can do...
An aside: this is the first time you have seen a named argument. In an argument list of a function, you can supply a name for arguments:
python
def some_function(arg1, info=38, axis=None):
   ...
To write a function like this, you must provide a default for the argument. You can then call the function only passing a subset of arguments, as you have just seen.
End of explanation
"""


# Use this cell!

"""
Explanation: This shouldn't be surprising - we take the average over all patients for each of 40 days, so we end up with a 40 element array.
Try using the following cell to get the mean inflammation over all patients for each of the first three days...
(you should use slicing and end up with a 3 element array)
End of explanation
"""


# Switch on the joy
% matplotlib inline

"""
Explanation: RIGHT
Enough text.
Programming isn't about writing reams of code, it's about getting stuff done and having fun while doing it. Right? Right. Right. RIGHT.
In any case, when examining data, it's a whole lot more pleasant and efficient to see visualizations, so lets get started with some plotting.
Plotting
Like we're Brutus the Younger
First, we use a magic method that works in Jupyter (strictly, in its IPython backend)
End of explanation
"""


import matplotlib
pretty_pic = matplotlib.pyplot.imshow(data)
matplotlib.pyplot.show(pretty_pic)

"""
Explanation: Now for inline plotting we can use the matplotlib module, the go-to for Python plotting (but not in the Dijkstra sense)...
End of explanation
"""


# Use this cell!

"""
Explanation: MMMM. Pretty. Red shows greater inflammation, and what we may have guessed from the bits of CSV we saw, inflammation starts low, grows and reduces. Patients along the side, days along the bottom.
Time for a challenge... plot the heat map (as above) for the fourth, fifth and sixth patients, transposed with days up the side. The easiest way is to start with Google... But first, put up your stars so I can see them!
End of explanation
"""


import numpy as np
from matplotlib import pyplot

data = np.loadtxt(fname='data/inflammation-01.csv', delimiter=',')
pyplot.figure(figsize=(5.0, 3.0))

pyplot.xlabel('Day')    
pyplot.ylabel('Inflammation')

pyplot.plot(data.mean(axis=0), label='Average')
pyplot.plot(data.max(axis=0), label='Max')
pyplot.plot(data.min(axis=0), label='Min')
pyplot.legend()

pyplot.show()

exec(In[42])  # Cheeky way to reshow output... forget immediately.

"""
Explanation: Reproducible Plots
If you come from a MATLAB background, or use other plotting tools, you will be used to saving/re-opening a plot...
...in Python you normally have a plotting script that you run each time (and save the image).
Cons

You have no standard GUI for designing plots
If your plot involves lots of processing, it's slow to open

Pros

Changes are trivially easy
Every time you open the plot, it contains current data

Con-workarounds

There are libraries for adding GUI elements to plots
You can split into two scripts
output processed data
plot

Example plotting script
End of explanation
"""


# This is the whole figure, possibly
# with multiple subplots
fig = pyplot.figure(figsize=(5.0, 3.0))

# This is a specific set of axes
axes = fig.add_subplot(1, 1, 1)
axes.set_xlabel('Day')
axes.set_ylabel('Inflammation')

axes.plot(data.mean(axis=0), label='Average')
axes.plot(data.max(axis=0), label='Max')
axes.plot(data.min(axis=0), label='Min')
axes.legend()

pyplot.show()

"""
Explanation: Now if I want to modify some part... I can do so and have the updated plot straight-away
This particular syntax is for fairly basic use, with only one set of axis and so forth, but it's probably quite familiar if you are coming from MATLAB. If we want to expand a bit more, first we use a more classic matplotlib approach, and neater programming approach, by setting properties directly on specific objects, such as axes...
End of explanation
"""


fig = pyplot.figure(figsize=(10.0, 3.0))

axes = []  # Blank list
for i in range(1, 4):
    ax = fig.add_subplot(1, 3, i)
    ax.set_xlabel('Day')
    axes.append(ax)
    
axes[0].set_ylabel('Average')
axes[1].set_ylabel('Max')
axes[2].set_ylabel('Min')

axes[0].plot(data.mean(axis=0))
axes[1].plot(data.max(axis=0))
axes[2].plot(data.min(axis=0))

fig.tight_layout()
pyplot.show(fig)

"""
Explanation: Not particularly more complicated, but more flexible... say we have a series of plots...
End of explanation
"""


import bokeh.plotting as bplot
from bokeh.io import output_notebook
output_notebook()

"""
Explanation: Challenge
Put up your stars!!
Modify the previous cell to give the max line with a dash, the blue line with dots and the red line solid.
Even better, add a title at the top.
When done, swap to the arrow. If you find this very easy and are waiting, leave your arrow up and have a go at plotting all the patients in very light grey behind your three coloured lines. See how long it takes to run for those 63 lines.
RMV: To get you started, Google pyplot line style and pick the pylab_examples example code link...
Bear in mind, and this is particularly the case with matplotlib, that there are multiple similar tools with slightly different configurations and long lists of technical documentation - try not to get lost in them and use Etherpad to flags things up. However, short examples are findable via Google, especially via Stack Overflow, that get you going fast.
But...
Not so interactive
If we are using Jupyter, it is probably because we want to have something interactive. Fixed plots in a web-page make it seem like something is missing.
Bokeh
Dynamic plots
End of explanation
"""


fig = bplot.figure()
days = np.arange(data.shape[1])
fig.line(days, data.mean(axis=0))
fig.xaxis.axis_label = "Day"
bplot.show(fig)

"""
Explanation: If this works, you should see an icon like a rainbow-coloured camera shutter.
End of explanation
"""


# Here's cell for you...

"""
Explanation: Notice the syntax is quite similar to matplotlib but a little different, so make sure you don't get them mixed up. For example, plot has become specifically line and we now explicitly provide the days along the bottom. Note that numpy.arange is just like range but it returns a numpy array.
Challenge II
Reproduce our three-line matplotlib plot in Bokeh
End of explanation
"""


# RMV
# This is the whole figure, possibly
# with multiple subplots
fig = bplot.figure()

days = np.arange(data.shape[1])

# This is a specific set of axes
fig.xaxis.axis_label = 'Day'
fig.yaxis.axis_label = 'Inflammation'

fig.line(days, data.mean(axis=0), legend='Average', color='green')
fig.line(days, data.max(axis=0), legend='Max', color='blue')
fig.line(days, data.min(axis=0), legend='Min', color='red')

bplot.show(fig)

"""
Explanation: Don't forget to put up your star!
Start with the Bokeh documentation and the Bokeh example above...
End of explanation
"""


from bokeh.models import ColumnDataSource
from bokeh.io import push_notebook

# Start out with days vs average
initial_coordinates = {'x': days, 'y': data.mean(axis=0)}
source = ColumnDataSource(initial_coordinates)

# Define a callback to update the plot when we
# pick something else
def update_plot_statistic(statistic):
    if statistic == "Average":
        source.data['y'] = data.mean(axis=0)
    elif statistic == "Max":
        source.data['y'] = data.max(axis=0)
    elif statistic == "Min":
        source.data['y'] = data.min(axis=0)
    push_notebook()

"""
Explanation: Interactive plots
But really, this still isn't very interactive... let's try adding a drop-down to pick which function (average, max, min) that we want to use.
End of explanation
"""


fig = bplot.figure()

days = np.arange(data.shape[1])

fig.xaxis.axis_label = 'Day'
fig.yaxis.axis_label = 'Inflammation'

fig.line(initial_coordinates['x'], initial_coordinates['y'], source=source)
bplot.show(fig)

"""
Explanation: First off, note that we do not have to tell the function to import the global variable, we can just use it. Now, examine the function - this is callback that we will use to update the plot when we change a drop-down menu choice of function.
We create a source object, an instance of the ColumnDataSource class, and when this function is called, it updates the 'y' item in source.data. Inside the function, we check what has been picked (this is the argument to the function), and update the source "y" array on that basis. Finally, we push this change to the notebook.
End of explanation
"""


from ipywidgets import interact

interact(update_plot_statistic, statistic=["Average", "Max", "Min"])

"""
Explanation: Now we need a widget to use this...
End of explanation
"""


# Use this cell for the plot

# And this for the one line `interact` call afterwards

"""
Explanation: This a very basic approach - the widget tool guesses you want a drop-down because you pass a list. Alternatively, you could pass a tuple with two floats as limits and get a slider back.
It seems you do have to output the image before the widgets, if you define the callback in Python, at least. However, there are a lot of other interactors - you can pass a JS callback (as a string), assign URLs to various objects in your plot, animate it, etc. etc. etc.
Final Challenge
Add a slider for variable N from 1 to 60 that reduces the dataset from all Patients to Patients 1 to N. Unless you're feeling particularly ambitious, you can replace the drop-down from the previous example with this slider.
End of explanation
"""


# RMV
# Start out with days vs average
initial_coordinates = {'x': days, 'y': data.mean(axis=0)}
source = ColumnDataSource(initial_coordinates)

# Define a callback to update the plot when we
# pick something else
def update_plot(N):
    source.data['x'] = np.arange(N)
    push_notebook()
    
fig = bplot.figure()

days = np.arange(data.shape[1])

fig.xaxis.axis_label = 'Day'
fig.yaxis.axis_label = 'Inflammation'

fig.line(initial_coordinates['x'], initial_coordinates['y'], source=source)

bplot.show(fig)

interact(update_plot, N=(1, 60, 1))

"""
Explanation: Don't forget your star!
End of explanation
"""


