# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# + [markdown] id="view-in-github" colab_type="text"
# <a href="https://colab.research.google.com/github/MavrellousG/Azure_ML_DFE/blob/main/CrossValidation.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>

# + id="MD4QHCipmmwb"
#pip install lime

# + [markdown] id="8Ehrsgnxmmwh"
# ## Learning Objectives
#
# Today we will be covering the full ML pipeline in all of its glory, starting from good clean data (that is a big if) to the final predictions.

# + [markdown] id="lfNsqgtWmmwk"
# ## The full picture
#
# With cross validation we can show you the full picture of model building (after you have done the hard work of data munging). The magic that cross validation unlocks is twofold
#
# 1. It allow you to have more training data and therefore get better performance and more accurate representations of your performance
# 2. It actually simplifies the process. You will no longer need to keep 3 sets of data and you can get by with just two.
#
# Let's get started:

# + colab={"base_uri": "https://localhost:8080/"} id="bEDpd62Ymmwm" outputId="b1590fbd-b9b9-4b1b-9008-909f63aaf3d1"
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split


boston_data = load_boston()

# we make our test set
X_train, X_test, y_train, y_test = train_test_split(boston_data['data'], boston_data['target'], test_size=0.2, random_state=1)

# and we make our validation set
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.2, random_state=1)

# + [markdown] id="n54ReACZmmwn"
# Our next step will be to define the model that we are looking at:

# + id="BCtbh1pZmmwn"
from sklearn.tree import DecisionTreeRegressor

reg = DecisionTreeRegressor()

# + [markdown] id="G89rhvJSmmwo"
# Then we determine which parameters we would like to search over:

# + id="93w6LJL3mmwp"
params = {
    'max_depth': range(2, 20, 2),
    'min_samples_leaf': range(5, 25, 5)
}

# + [markdown] id="Gk4AVMzFmmwr"
# And finally we use GridSearchCV which will search over the parameters doing cross validation to determine their performance:

# + id="errL7RCQmmws"
from sklearn.model_selection import GridSearchCV

gs = GridSearchCV(reg, params, scoring='neg_mean_absolute_error')

# + colab={"base_uri": "https://localhost:8080/"} id="GUjhKdo_mmwt" outputId="9fbfd86e-9d5e-4b39-ec16-d69df8037f3d"
gs.fit(X_train, y_train)

# + [markdown] id="5W3xw-JAmmwv"
# We get a lot of goodies. We can see the best score and estimator:

# + colab={"base_uri": "https://localhost:8080/"} id="dTXFh8vzmmww" outputId="aef08d0c-58e2-4b63-e60b-e167df9e8e01"
gs.best_score_

# + colab={"base_uri": "https://localhost:8080/"} id="3AqFJcOOmmwx" outputId="a5d997b5-ee5e-4980-f21c-934fd761315d"
gs.best_estimator_

# + [markdown] id="NwmsHXQCmmwy"
# And we get to use the grid search object as that estimator as well:

# + colab={"base_uri": "https://localhost:8080/"} id="Nad8IE67mmwz" outputId="8c83fbc4-102e-4fd9-aafe-bf8fe2425040"
gs.predict(X_train[:5])

# + [markdown] id="HnC03GeCmmwz"
# ## A note on hyperparam tuning
#
# Grid search might be becoming a bit old school in the next few years, with advancements like random search, hyperband, bayesian hyperparam search and more we might use a more advanced way to search through available params. That being said it is good to know and still widely used in ML.

# + [markdown] id="1La0GAO5mmw0"
# ### Model Explainer

# + colab={"base_uri": "https://localhost:8080/", "height": 339} id="uDkff6mAmmw0" outputId="beabd30a-b71c-4f59-f033-05f1760b4082"
import lime
import lime.lime_tabular
import numpy as np

# + id="OusDO-yZmmw1"
categorical_features = np.argwhere(np.array([len(set(boston_data.data[:,x])) for x in range(boston_data.data.shape[1])]) <= 10).flatten()

# + id="8JHolL61mmw2"
explainer = lime.lime_tabular.LimeTabularExplainer(X_train, feature_names=boston_data.feature_names, class_names=['price'], categorical_features=categorical_features, verbose=True, mode='regression')

# + id="V7aH35qgmmw2"
i = 15
exp = explainer.explain_instance(X_test[i], gs.predict, num_features=5)

# + id="Thb7-_Cemmw4"
exp.show_in_notebook(show_table=True)

# + id="-vSd7yDOmmw5"
exp.as_list()

# + id="8DhsVujFmmw6"


StarcoderdataJupyterScriptsDedupFiltered