# This is a comment


"""
Explanation: Python Basics
Syntax
Python is an object oriented scripting language and does not require a specific first or last line (such as <code>public static void main</code> in Java or <code>return</code> in C).
There are no curly braces {} to define code blocks or semi-colons ; to end a line.  Instead of braces, indentation is rigidly enforced to create a block of code.
End of explanation
"""


if (1 == 1):
    print "We're in "
        print "Deep Trouble:"

"""
Explanation: Arbitrary indentation can be used within a code block, as long as the indentation is consistent.
End of explanation
"""


# Int

# Float

"""
Explanation: Variables and Types
Variables can be given alphanumeric names beginning with an underscore or letter.  Variable types do not have to be declared and are inferred at run time.
End of explanation
"""


# Strings

"""
Explanation: Strings can be declared with either single or double quotes.
End of explanation
"""


# Print

"""
Explanation: Printing values uses the print statement which can be passed arguments in different ways
End of explanation
"""


# First function

"""
Explanation: The scope of variables is local to the function, class, and file in that increasing order of scope.  Global variables can also be declared.
End of explanation
"""


# To use Math, we must import it

"""
Explanation: Modules and Import
Files with a .py extension are known as Modules in Python.  Modules are used to store functions, variables, and class definitions.  
Modules that are not part of the standard Python library are included in your program using the <code>import</code> statement.
End of explanation
"""


# whole.part

"""
Explanation: Whoops.  Importing the <code>math</code> module allows us access to all of its functions, but we must call them properly namespaced
End of explanation
"""


# From with pi

"""
Explanation: Alternatively, you can use the <code>from</code> keyword
End of explanation
"""


# From ... *

"""
Explanation: Using the <code>from</code> statement we can import everything from the math module.  
Disclaimer: many Pythonistas discourage doing this for performance reasons.  Just import what you need
End of explanation
"""


mystring = "Go Gators, Come on Gators, Get up and go!"
# Print 1

"""
Explanation: Strings
As you may expect, Python has a powerful, full featured string module.  
Substrings
Python strings can be substringed using bracket syntax
End of explanation
"""


# Print first 4

# Characters before the 9th

# Characters after the 27th

# Ommiting start and end

"""
Explanation: Python is a 0-index based language as you might have guessed. Generally whenever forming a range of values in Python, the first argument is inclusive whereas the second is not, i.e. <code>mystring[11:25]</code> returns characters 11, 12, 13, ..., 23, 24.
End of explanation
"""


# Print almost last 4 chars

"""
Explanation: Using negative values, you can count backwards from the end of the string.
End of explanation
"""


# Find "Gators"

print mystring.find("Gators", 4) # specify an index on which to begin searching

print mystring.find("Seminoles") # no Seminoles here

"""
Explanation: String Functions
Here are some more useful string functions
find
End of explanation
"""


print mystring.lower()
print mystring.upper()

"""
Explanation: Looks like nothing was found. -1 is returned by default. We'll talk about -1 when we talk about truth later.
lower and upper
End of explanation
"""


print mystring.replace("Gators", "Seminoles") # replaces all occurences of Gators with Seminoles

print mystring

print mystring.replace("Gators", "Seminoles", 1) # limit the number of replacements

"""
Explanation: replace
End of explanation
"""


mystring[1] = "O"

"""
Explanation: Notice that replace returned a new string every time.  Nothing was modified in place. Strings (and most basic data types) are immutable in Python.
End of explanation
"""


print mystring.split() # returns a list of strings broken by a space by default

print mystring.split(',') # you can also define the separator

"""
Explanation: split
End of explanation
"""


# Join with spaces

"""
Explanation: join
The <code>join</code> is useful for building strings from lists or other iterables.  Call <code>join</code> on the desired separator
End of explanation
"""


# mylist List of 5

# append 6

mylist.extend([8, 'nine']) # extend the list with the contents of another list
print mylist

mylist.insert(6, 7) # insert the number 7 at index 6
print mylist
# Insert 20 at 20

mylist.remove('five') # removes the first matching occurence 
print mylist

popped = mylist.pop() # by default, the last item in the list is removed and returned
print popped
print mylist

popped2 = mylist.pop(4) # pops at at index
print popped2
print mylist

# len()

mylist.extend(range(-3, 0)) # the range function returns a list from -3 inclusive to 0 non inclusive
print mylist

# default list sorting. When more complex objects are in the list, arguments can be used to customize how to sort
mylist.sort()
print mylist

mylist.reverse() # reverse the list
print mylist

"""
Explanation: Notice that the method is called on the string, not on the list. Lists don't have a join method. This is discussed often on StackOverflow.
For more information on string functions:
https://docs.python.org/2/library/stdtypes.html#string-methods
Data Structures
Lists
The Python standard library does not have traditional C-style fixed-memory fixed-type arrays.  Instead, lists are used and can contain a mix of any type.
Lists are created with square brackets []
Lists are also mutable, we can change them.
End of explanation
"""


# mytuple

print mytuple[1] # access an item

mytuple[1] = "Winston" # results in error

"""
Explanation: For more information on Lists:
https://docs.python.org/2/tutorial/datastructures.html#more-on-lists
Tuples
Python supports n-tuple sequences.  These are non-mutable
End of explanation
"""


schools = ['Florida', 'Florida State', 'Miami', 'Florida']

# make myset


print myset

# in

# more in

badschools = set(['Florida State', 'Miami'])
print myset - badschools # set arithmetic

# set &

print myset | set(['Miami', 'Stetson']) # OR

print myset ^ set(['Miami', 'Stetson']) # XOR

"""
Explanation: Sets
Python includes the set data structure which is an unordered collection with no duplicates
End of explanation
"""


# mydict


# Florida

# Deletes


# Assignment


# Appending

# Keys

"""
Explanation: Dictionaries
Python supports dictionaries which can be thought of as an unordered list of key, value pairs.  Keys can be any immutable type and are typically integers or strings.  Values can be any object, even other dictionaries.
Dictionaries are created with curly braces {}
End of explanation
"""


# if statement, a > b

# if else, b less than or equal to a

# if else elif, b equals a

"""
Explanation: Conditionals
Python supports the standard if-else-if conditional expression
End of explanation
"""


# range

# range start stop

# range start stop step

for a in range(10, 1, -5): # can use a negative step size as well
    print a

for b in range(2, 1, 1): # with a positive step, all values are less than 1. No integers are produced
    print b

for c in range(1, 2, -1): # same goes for a negative step as all values are less than 2
    print c

"""
Explanation: Loops
Python supports for, foreach, and while loops
For (counting)
Traditional counting loops are accomplished in Python with a combination of the <code>for</code> key word and the <code>range</code> function
End of explanation
"""


# for in list

# for key in dict.keys()

"""
Explanation: Foreach
As it turns out, counting loops are just foreach loops in Python.  The <code>range</code> function returns a list of integers over which <code>for in</code> iterates.  This can be extended to any other iterable type
End of explanation
"""


# while condition

"""
Explanation: While
Python supports standard <code>while</code> loops
End of explanation
"""


# while true, do until break

"""
Explanation: Python does not have a construct for a do-while loop, though it can be accomplished using the <code>break</code> statement
End of explanation
"""


# define function

# define player, name, number

"""
Explanation: Functions
Functions in Python do not have a distinction between those that do and do not return a value.  If a value is returned, the type is not declared.
Functions can be declared in any module without any distinction between static and non-static.  Functions can even be declared within other functions
The syntax is the following
End of explanation
"""


def player(name, number, team = 'Florida'): # optional team argument
    print "#" + str(number), name, team
    
player("Kasey Hill", 0) # no team argument supplied

player("Aaron Harrison", 2, "Kentucky") # supplying all three arguments

"""
Explanation: Functions can have optional arguments if a default value is provided in the function signature
End of explanation
"""


player(number = 23, name = 'Chris Walker')

"""
Explanation: Python functions can be called using named arguments, instead of positional
End of explanation
"""


# calling with args

"""
Explanation: *args and **kwargs
In Python, there is a special deferencing scheme that allows for defining and calling functions with argument lists or dictionaries.
*args
End of explanation
"""


# define foo *args, print *args

"""
Explanation: Argument lists can also be used in defining a function as such
End of explanation
"""


# calling with kwargs

"""
Explanation: **kwargs
Similarly, we can define a dictionary of named parameters
End of explanation
"""


def foo(**kwargs):
    for key in kwargs.keys():
        print key, kwargs[key]
        
foo(**kwargs)

"""
Explanation: Just as before, we can define a function taking an arbitrary dictionary
End of explanation
"""


# def sum, return a + b

def sum_and_product(x,y):
    return x + y, x * y # return two values

mysum, myproduct = sum_and_product(1,2)
print mysum, myproduct

"""
Explanation: return
In Python functions, an arbitrary number of values can be returned
End of explanation
"""


def download_file(url, local_filename):
    import requests
    
    # stream = True allows downloading of large files; prevents loading entire file into memory
    r = requests.get(url, stream = True)
    with open(local_filename, 'wb') as f:
        for chunk in r.iter_content(chunk_size=1024): 
            if chunk: # filter out keep-alive new chunks
                f.write(chunk)
                f.flush()

"""
Explanation: Data Science Tutorial
Now that we've covered some Python basics, we will begin a tutorial going through many tasks a data scientist may perform.  We will obtain real world data and go through the process of auditing, analyzing, visualing, and building classifiers from the data.
We will use a database of breast cancer data obtained from the University of Wisconsin Hospitals, Madison from Dr. William H. Wolberg.  The data is a collection of samples from Dr. Wolberg's clinical cases with attributes pertaining to tumors and a class labeling the sample as benign or malignant.
| Attribute                      | Domain                          |
|--------------------------------|---------------------------------|
| 1. Sample code number          | id number                       |
| 2. Clump Thickness             | 1 - 10                          |
| 3. Uniformity of Cell Size     | 1 - 10                          |
| 4. Uniformity of Cell Shape    | 1 - 10                          |
| 5. Marginal Adhesion           | 1 - 10                          |
| 6. Single Epithelial Cell Size | 1 - 10                          |
| 7. Bare Nuclei                 | 1 - 10                          |
| 8. Bland Chromatin             | 1 - 10                          |
| 9. Normal Nucleoli             | 1 - 10                          |
| 10. Mitoses                    | 1 - 10                          |
| 11. Class                      | (2 for benign, 4 for malignant) |
For more information on this data set:
https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29
Obtaining the Data
Lets begin by programmatically obtaining the data.  Here I'll define a function we can use to make HTTP requests and download the data
End of explanation
"""


url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/breast-cancer-wisconsin.data'
filename = 'breast-cancer-wisconsin.csv'

"""
Explanation: Now we'll specify the url of the file and the file name we will save to
End of explanation
"""


download_file(url, filename)

"""
Explanation: And make a call to <code>download_file</code>
End of explanation
"""


import pandas as pd # import the module and alias it as pd

cancer_data = pd.read_csv('breast-cancer-wisconsin.csv')
cancer_data.head() # show the first few rows of the data

"""
Explanation: Now this might seem like overkill for downloading a single, small csv file, but we can use this same function to access countless APIs available on the World Wide Web by building an API request in the url.
Wrangling the Data
Now that we have some data, lets get it into a useful form.  For this task we will use a package called pandas. pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for Python.  The most fundamental data structure in pandas is the dataframe, which is similar to the data.frame data structure found in the R statistical programming language.
For more information: http://pandas.pydata.org
pandas dataframes are a 2-dimensional labeled data structures with columns of potentially different types.  Dataframes can be thought of as similar to a spreadsheet or SQL table.
There are numerous ways to build a dataframe with pandas.  Since we have already attained a csv file, we can use a parser built into pandas called <code>read_csv</code> which will read the contents of a csv file directly into a data frame.
For more information: http://pandas.pydata.org/pandas-docs/dev/generated/pandas.io.parsers.read_csv.html
End of explanation
"""


# \ allows multi line wrapping
cancer_header = [ \
                 'sample_code_number', \
                 'clump_thickness', \
                 'uniformity_cell_size', \
                 'uniformity_cell_shape', \
                 'marginal_adhesion', \
                 'single_epithelial_cell_size', \
                 'bare_nuclei', \
                 'bland_chromatin', \
                 'normal_nucleoli', \
                 'mitoses', \
                 'class']

"""
Explanation: Whoops, looks like our csv file did not contain a header row.  <code>read_csv</code> assumes the first row of the csv is the header by default.
Lets check out the file located here: https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/breast-cancer-wisconsin.names
This contains information about the data set including the names of the attributes.
Lets create a list of these attribute names to use when reading the csv file
End of explanation
"""


cancer_data = pd.read_csv('breast-cancer-wisconsin.csv', header=None, names=cancer_header)
cancer_data.head()

"""
Explanation: Lets try the import again, this time specifying the names.  When specifying names, the <code>read_csv</code> function requires us to set the <code>header</code> row number to <code>None</code>
End of explanation
"""


cancer_data["clump_thickness"].describe()

"""
Explanation: Lets take a look at some simple statistics for the clump_thickness column
End of explanation
"""


cancer_data["bare_nuclei"].describe()

"""
Explanation: Referring to the documentation link above about the data, the count, range of values (min = 1, max = 10), and data type (dtype = float64) look correct.
Lets take a look at another column, this time bare_nuclei
End of explanation
"""


cancer_data["bare_nuclei"].unique()

"""
Explanation: Well at least the count is correct.  We were expecting no more than 10 unique values and now the data type is an object.  
Whats up with our data?
We have arrived at arguably the most important part of performing data science: dealing with messy data.  One of most important tools in a data scientist's toolbox is the ability to audit, clean, and reshape data.  The real world is full of messy data and your sources may not always have data in the exact format you desire.
In this case we are working with csv data, which is a relatively straightforward format, but this will not always be the case when performing real world data science.  Data comes in all varieties from csv all the way to something as unstructured as a collection of emails or documents.  A data scientist must be versed in a wide variety of technologies and methodologies in order to be successful.
Now, lets do a little bit of digging into why were are not getting a numeric pandas column
End of explanation
"""


cancer_data["bare_nuclei"] = cancer_data["bare_nuclei"].convert_objects(convert_numeric=True)

"""
Explanation: Using <code>unique</code> we can see that '?' is one of the distinct values that appears in this series.  Looking again at the documentation for this data set, we find the following:

Missing attribute values: 16
There are 16 instances in Groups 1 to 6 that contain a single missing 
  (i.e., unavailable) attribute value, now denoted by "?". 

It was so nice of them to tell us to expect these missing values, but as a data scientist that will almost never be the case.  Lets see what we can do with these missing values.
End of explanation
"""


cancer_data["bare_nuclei"].unique()

"""
Explanation: Here we have attempted to convert the bare_nuclei series to a numeric type.  Lets see what the unique values are now.
End of explanation
"""


cancer_data.fillna(cancer_data.mean().round(), inplace=True)
cancer_data["bare_nuclei"].unique()

"""
Explanation: The decimal point after each number means that it is an integer value being represented by a floating point number.  Now instead of our pesky '?' we have <code>nan</code> (not a number).  <code>nan</code> is a construct used by pandas to represent the absence of value.  It is a data type that comes from the package numpy, used internally by pandas, and is not part of the standard Python library.
Now that we have <code>nan</code> values in place of '?', we can use some nice features in pandas to deal with these missing values.
What we are about to do is what is called "imputing" or providing a replacement for missing values so the data set becomes easier to work with.  There are a number of strategies for imputing missing values, all with their own pitfalls.  In general, imputation introduces some degree of bias to the data, so the imputation strategy taken should be in an attempt to minimize that bias.
Here, we will simply use the mean of all of the non-nan values in the series as a replacement.  Since we already know that the data is integer in possible values, we will round the mean to the nearest whole number.
End of explanation
"""


cancer_data.mean().round()

"""
Explanation: <code>fillna</code> is a dataframe function that replaces all nan values with either a scalar value, a series of values with the same indices as found in the dataframe, or a dataframe that is indexed by the columns of the target dataframe.
<code>cancer_data.mean().round()</code> will take the mean of each column (this computation ignores the currently present nan values), then round, and return a dataframe indexed by the columns of the original dataframe:
End of explanation
"""


cancer_data = pd.read_csv('breast-cancer-wisconsin.csv', header=None, names=cancer_header)
cancer_data = cancer_data.convert_objects(convert_numeric=True)
cancer_data.fillna(cancer_data.mean().round(), inplace=True)
cancer_data["bare_nuclei"].describe()

cancer_data["bare_nuclei"].unique()

"""
Explanation: <code>inplace=True</code> allows us to make this modification directly on the dataframe, without having to do any assignment.  
Now that we have figured out how to impute these missing values in a single column, lets start over and quickly apply this technique to the entire dataframe.
End of explanation
"""


cancer_data.describe()

"""
Explanation: Structurally, Pandas dataframes are a collection of Series objects sharing a common index.  In general, the Series object and Dataframe object share a large number of functions with some behavioral differences.  In other words, whatever computation you can do on a single column can generally be applied to the entire dataframe.
Now we can use the dataframe version of <code>describe</code> to get an overview of all of our data
End of explanation
"""


# The following line is NOT Python code, but a special syntax for enabling inline plotting in IPython
%matplotlib inline 

from ggplot import *

import warnings

# ggplot usage of pandas throws a future warning
warnings.filterwarnings('ignore')

"""
Explanation: Visualizing the Data
Another important tool in the data scientist's toolbox is the ability to create visualizations from data.  Visualizing data is often the most logical place to start getting a deeper intuition of the data.  This intuition will shape and drive your analysis.
Even more important than visualizing data for your own personal benefit, it is often the job of the data scientist to use the data to tell a story.  Creating illustrative visuals that succinctly convey an idea are the best way to tell that story, especially to stakeholders with less technical skillsets.
Here we will be using a Python package called ggplot (https://ggplot.yhathq.com).  The ggplot package is an attempt to bring visuals following the guidelines outlayed in the grammar of graphics (http://vita.had.co.nz/papers/layered-grammar.html) to Python.  It is based off of and intended to mimic the features of the ggplot2 library found in R.  Additionally, ggplot is designed to work with Pandas dataframes, making things nice and simple. 
We'll start by doing a bit of setup
End of explanation
"""


plt = ggplot(aes(x = 'clump_thickness'), data = cancer_data) + \
        geom_histogram(binwidth = 1, fill = 'steelblue')

# using print gets the plot to show up here within the notebook.  
# In normal Python environment without using print, the plot appears in a window
print plt

"""
Explanation: So we enabled plotting in IPython and imported everything from the ggplot package.  Now we'll create a plot and then break down the components
End of explanation
"""


plt = ggplot(aes(x = 'clump_thickness'), data = cancer_data) + \
        geom_histogram(binwidth = 1, fill = 'steelblue') + \
        geom_vline(xintercept = [cancer_data['clump_thickness'].mean()], linetype='dashed')
print plt

"""
Explanation: A plot begins with the <code>ggplot</code> function.  Here, we pass in the cancer_data pandas dataframe and a special function called <code>aes</code> (short for aesthetic).  The values provided to <code>aes</code> change depending on which type of plot is being used.  Here we are going to make a histogram from the clump_thickness column in cancer_data, so that column name needs to be passed as the x parameter to <code>aes</code>.
The grammar of graphics is based off of a concept of "geoms" (short for geometric objects).  These geoms provide granular control of the plot and are progressively added to the base call to <code>ggplot</code> with + syntax.
Lets say we wanted to show the mean clump_thickness on this plot.  We could do something like the following
End of explanation
"""


plt = ggplot(aes(x = 'uniformity_cell_shape', y = 'bare_nuclei'), data = cancer_data) + \
        geom_point()
print plt

"""
Explanation: As you can see, each geom has its own set of parameters specific to the appearance of that geom (also called aesthetics).
Lets try a scatter plot to get some multi-variable action
End of explanation
"""


plt = ggplot(aes(x = 'uniformity_cell_shape', y = 'bare_nuclei'), data = cancer_data) + \
        geom_point(position = 'jitter')
print plt

"""
Explanation: Sometimes when working with integer data, or data that takes on a limited range of values, it is easier to visualize the plot with added jitter to the points.  We can do that by adding an aesthetic to <code>geom_point</code>.
End of explanation
"""


plt = ggplot(aes(x = 'uniformity_cell_shape', y = 'bare_nuclei', color = 'class'), data = cancer_data) + \
        geom_point(position = 'jitter')
print plt

"""
Explanation: With a simple aesthetic addition, we can see how these two variables play into our cancer classification
End of explanation
"""


plt = ggplot(aes(x = 'uniformity_cell_shape', y = 'bare_nuclei', color = 'class'), data = cancer_data) + \
        geom_point(position = 'jitter') + \
        ggtitle("The Effect of the Bare Nuclei and Cell Shape Uniformity on Classification") + \
        ylab("Amount of Bare Nuclei") + \
        xlab("Uniformity in Cell shape")
print plt

"""
Explanation: By adding <code>color = 'class'</code> as a parameter to the aes function, we now give a color to each unique value found in that column and automatically get a legend.  Remember, 2 is benign and 4 is malignant. 
We can also do things such as add a title or change the axis labeling with geoms
End of explanation
"""


plt = ggplot(aes(x = 'uniformity_cell_shape', y = 'bare_nuclei'), data = cancer_data) + \
        geom_point(position = 'jitter') + \
        ggtitle("The Effect of the Bare Nuclei and Cell Shape Uniformity on Classification") + \
        facet_grid('class')
print plt

"""
Explanation: There is definitely some patterning going on in that plot.
A slightly different way to convey this idea is to use faceting.  Faceting is the creation of multiple related plots arranged by the values of a given faceted variable
End of explanation
"""


plt = ggplot(aes(x = 'uniformity_cell_shape', y = 'bare_nuclei', color = 'class'), data = cancer_data) + \
        geom_point(position = 'jitter') + \
        ggtitle("The Effect of the Bare Nuclei and Cell Shape Uniformity on Classification") + \
        facet_grid('clump_thickness', 'marginal_adhesion')
print plt

"""
Explanation: Rather than set the color equal to the class, we have created two plots based off of the class.  With a facet, we can get very detailed.  Lets through some more variables into the mix
End of explanation
"""


cancer_features = ['uniformity_cell_shape', 'bare_nuclei']

"""
Explanation: Unfortunately, legends for faceting are not yet implemented in the Python ggplot package.  In this example we faceted on the x-axis with clump_thickness and along the y-axis with marginal_adhesion, then created 100 plots of uniformity_cell_shape vs. bare_nuclei effect on class.
I highly encourage you to check out https://ggplot.yhathq.com/docs/index.html to see all of the available geoms.  The best way to learn is to play with and visualize the data with many different plots and aesthetics.
Machine Learning
So now that we've acquired, audited, cleaned, and visualized our data, we have arrived at machine learning.  By formal definition from Tom Mitchell:

A computer program is set to learn from an experience E with 
respect to some task T and some performance measure P if its performance 
on T as measured by P improves with experience E.

Okay, thats a bit ridiculous.  Essentially machine learning is the science of building algorithms that learn from data in order make predictions about the data.  There are two main classes of machine learning: supervised and unsupervised.
In supervised learning, an algorithm will use the features of the data given to make a prediction about a known label.  For example, we will use supervised learning here to take features such as bare_nuclei and uniformity_cell_shape and predict a tumor class (benign or malignant).  This type of machine learning is called supervised because the class labels (benign or malignant) are a known quantity during learning, so we are supervising the algorithm with the "correct" answer.
In unsupervised learning, an algorithm will use the features of the data to discover what types of labels there could be.  The "correct" answer is not known.
In this session we will be mostly focused on supervised learning as we attempt to predict whether a tumor is benign or malignant.  We will also be focused on doing some practical machine learning, and will glaze over the algorithmic details.
The first thing we have to do is to extract the class labels and features from <code>cancer_data</code> and store them as separate arrays.  In our first classifier we will only choose two features from <code>cancer_data</code> to keep things simple
End of explanation
"""


labels = cancer_data['class'].values
features = cancer_data[cancer_features].values

"""
Explanation: Here we call <code>values</code> on the dataframe to extract the values stored in the dataframe as an array of numpy arrays with the same dimensions as our subsetted dataframe.  Numpy is a powerful, high performance scientific computing package that implements arrays.  It is used internally by pandas.  We will use <code>labels</code> and <code>features</code> later on in our machine learning classifier
End of explanation
"""


from sklearn.cross_validation import train_test_split

features_train, features_test, labels_train, labels_test = train_test_split(features, 
                                                                            labels, 
                                                                            test_size = 0.3, 
                                                                            random_state = 42)

"""
Explanation: An important concept in machine learning is to split the data set into training data and testing data.  The machine learning algorithm will use the subset of training data to build a classifier to predict labels.  We then test the accuracy of this classifier on the subset of testing data.  This is done in order to prevent overfitting the classifier to one given set of data.
Overfitting is a major concern in the design of machine learning algorithms.  Conceptually, overfitting is when a classifier is really good at predicting the data used to build it, but isn't robust or general enough to predict new, yet unseen data all that well.
To perform machine learning, we will use a package called sci-kit learn (sklearn for short).  The sklearn cross_validation module contains a function called <code>train_test_split</code> that will take in features and labels, and randomly select values into the training and testing subsets
End of explanation
"""


from sklearn.tree import DecisionTreeClassifier

"""
Explanation: For this example, we will build a Decision Tree Classifier.  The goal of a decision tree is to create a prediction by outlining a simple tree of decision rules.  These rules are built from the training data by slicing the data on simple boundaries and trying to minimize the prediction error of that boundary.  More details on decision trees can be found here: http://scikit-learn.org/stable/modules/tree.html
The first step is to import the classifier from the <code>sklearn.tree</code> module.
End of explanation
"""


clf = DecisionTreeClassifier()

"""
Explanation: Next, we create a variable to store the classifier
End of explanation
"""


clf.fit(features_train, labels_train)

"""
Explanation: Then we have to fit the classifier to the training data. Both the training features (uniformity_cell_shape and bare_nuclei) and the labels (benign vs. malignant) are passed to the fit function
End of explanation
"""


print "Accuracy score:", clf.score(features_test,labels_test)

"""
Explanation: The classifier is now ready to make some predictions.  We can use the score function to see how accurate the classifier is on the test data.  The score function will take the data in <code>features_test</code>, make a prediction of benign or malignant based on the decision tree that was fit to the training data, and compare that prediction to the true values in <code>labels_test</code>
End of explanation
"""


from class_vis import prettyPicture # helper class

prettyPicture(clf, features_test, labels_test)

"""
Explanation: Nearly all classifiers, decision trees included, will have paremeters that can be tuned to build a more accurate model.  Without any parameter tuning and using just two features we have made a pretty accurate prediction.  Good job!
To get a better idea of what is going on, I have included a helper function to plot our test data along with the decision boundary
End of explanation
"""


