!pip install scikit-learn==0.22.2

"""
Explanation: Decision Trees and Random Forests in Python
Learning Objectives

Explore and analyze data using a Pairplot
Train a single Decision Tree
Predict and evaluate the Decision Tree
Compare the Decision Tree model to a Random Forest

Introduction
In this lab, you explore and analyze data using a Pairplot, train a single Decision Tree, predict and evaluate the Decision Tree, and compare the Decision Tree model to a Random Forest.  Recall that the Decision Tree algorithm belongs to the family of supervised learning algorithms. Unlike other supervised learning algorithms, the decision tree algorithm can be used for solving both regression and classification problems too.  Simply, the goal of using a Decision Tree is to create a training model that can use to predict the class or value of the target variable by learning simple decision rules inferred from prior data(training data).
Each learning objective will correspond to a #TODO in this student lab notebook -- try to complete this notebook first and then review the solution notebook
End of explanation
"""


# Importing necessary tensorflow library and printing the TF version.
import tensorflow as tf

print("TensorFlow version: ",tf.version.VERSION)


import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline

"""
Explanation: Restart the kernel before proceeding further (On the Notebook menu, select Kernel > Restart Kernel > Restart).
Load necessary libraries
We will start by importing the necessary libraries for this lab.
End of explanation
"""


# Reading "kyphosis.csv" file using the read_csv() function included in the pandas library
df = pd.read_csv('../kyphosis.csv')

df.head()

"""
Explanation: Get the Data
End of explanation
"""


# Use the pairplot() function to plot multiple pairwise bivariate distributions in a dataset
# TODO 1
# TODO -- Your code here.


"""
Explanation: Exploratory Data Analysis
Lab Task #1: Check a pairplot for this small dataset.
End of explanation
"""


from sklearn.model_selection import train_test_split

X = df.drop('Kyphosis',axis=1)
y = df['Kyphosis']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30)

"""
Explanation: Train Test Split
Let's split up the data into a training set and a test set!
End of explanation
"""


from sklearn.tree import DecisionTreeClassifier

dtree = DecisionTreeClassifier()

# Train Decision Tree Classifer
# TODO 2
# TODO -- Your code here.


"""
Explanation: Decision Trees
Lab Task #2: Train a single decision tree.
End of explanation
"""


predictions = dtree.predict(X_test)

from sklearn.metrics import classification_report,confusion_matrix

# build a text report showing the main classification metrics
# TODO 3a
# TODO -- Your code here.


# compute confusion matrix to evaluate the accuracy of a classification
# TODO 3b
# TODO -- Your code here.


"""
Explanation: Prediction and Evaluation
Lab Task #3: Evaluate our decision tree.
End of explanation
"""


from IPython.display import Image  
from sklearn.externals.six import StringIO  
from sklearn.tree import export_graphviz
import pydot 

features = list(df.columns[1:])
features

dot_data = StringIO()  
export_graphviz(dtree, out_file=dot_data,feature_names=features,filled=True,rounded=True)

graph = pydot.graph_from_dot_data(dot_data.getvalue())  
Image(graph[0].create_png())  

"""
Explanation: Tree Visualization
Scikit learn actually has some built-in visualization capabilities for decision trees, you won't use this often and it requires you to install the pydot library, but here is an example of what it looks like and the code to execute this:
End of explanation
"""


from sklearn.ensemble import RandomForestClassifier
rfc = RandomForestClassifier(n_estimators=100)
rfc.fit(X_train, y_train)

rfc_pred = rfc.predict(X_test)

# compute confusion matrix to evaluate the accuracy
# TODO 4a
# TODO -- Your code here.


# build a text report showing the main metrics
# TODO 4b
# TODO -- Your code here.


"""
Explanation: Random Forests
Lab Task #4: Compare the decision tree model to a random forest.
End of explanation
"""


