# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# + [markdown] id="view-in-github" colab_type="text"
# <a href="https://colab.research.google.com/github/JayellWolfe/DS-Unit-2-Applied-Modeling/blob/master/JW__DS7_Sprint_Challenge_7.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>

# + [markdown] id="Kz9V5lVFMo86" colab_type="text"
# _Lambda School Data Science, Unit 2_
#
# # Applied Modeling Sprint Challenge: Predict Chicago food inspections 🍔

# + [markdown] id="yWOjtM9iMo87" colab_type="text"
# For this Sprint Challenge, you'll use a dataset with information from inspections of restaurants and other food establishments in Chicago from January 2010 to March 2019. 
#
# [See this PDF](https://data.cityofchicago.org/api/assets/BAD5301B-681A-4202-9D25-51B2CAE672FF) for descriptions of the data elements included in this dataset.
#
# According to [Chicago Department of Public Health — Food Protection Services](https://www.chicago.gov/city/en/depts/cdph/provdrs/healthy_restaurants/svcs/food-protection-services.html), "Chicago is home to 16,000 food establishments like restaurants, grocery stores, bakeries, wholesalers, lunchrooms, mobile food vendors and more. Our business is food safety and sanitation with one goal, to prevent the spread of food-borne disease. We do this by inspecting food businesses, responding to complaints and food recalls." 

# + [markdown] id="VvBYdx2xMo88" colab_type="text"
# #### Your challenge: Predict whether inspections failed
#
# The target is the `Fail` column.
#
# - When the food establishment failed the inspection, the target is `1`.
# - When the establishment passed, the target is `0`.

# + [markdown] id="3YMtu4LaF8Jq" colab_type="text"
# #### Run this cell to install packages in Colab:

# + id="vWyiJKQgF6ax" colab_type="code" outputId="3de05cad-748b-4e18-cd56-94549ec3ba29" colab={"base_uri": "https://localhost:8080/", "height": 1000}
import sys
in_colab = 'google.colab' in sys.modules

if in_colab:
    # Install packages in Colab
    # !pip install category_encoders==2.0.0
    # !pip install eli5==0.10.1
    # !pip install pandas-profiling==2.3.0
    # !pip install pdpbox==0.2.0
    # !pip install plotly==4.1.1
    # !pip install shap==0.30.0

# + id="59wuZnZAU-1x" colab_type="code" colab={}
# Imports 
import numpy as np
import pandas as pd
from sklearn.feature_selection import SelectKBest, f_classif, RFECV
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import mean_absolute_error, accuracy_score, classification_report
from sklearn.model_selection import GridSearchCV, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler, PolynomialFeatures

# + [markdown] id="Na5IvRq1Mo89" colab_type="text"
# #### Run this cell to load the data:

# + id="Yyc7zftTMo89" colab_type="code" colab={}
import pandas as pd

train_url = 'https://drive.google.com/uc?export=download&id=13_tP9JpLcZHSPVpWcua4t2rY44K_s4H5'
test_url  = 'https://drive.google.com/uc?export=download&id=1GkDHjsiGrzOXoF_xcYjdzBTSjOIi3g5a'

train = pd.read_csv(train_url)
test  = pd.read_csv(test_url)

assert train.shape == (51916, 17)
assert test.shape  == (17306, 17)

# + id="1dXsUhpARFZ3" colab_type="code" outputId="c733f409-591d-4123-d21f-f48c573e9e41" colab={"base_uri": "https://localhost:8080/", "height": 102}
train.columns

# + [markdown] id="8EhbzqgfMo9A" colab_type="text"
# ### Part 1: Preprocessing
#
# You may choose which features you want to use, and whether/how you will preprocess them. If you use categorical features, you may use any tools and techniques for encoding.
#
# _To earn a score of 3 for this part, find and explain leakage. The dataset has a feature that will give you an ROC AUC score > 0.90 if you process and use the feature. Find the leakage and explain why the feature shouldn't be used in a real-world model to predict the results of future inspections._
#
# ### Part 2: Modeling
#
# **Fit a model** with the train set. (You may use scikit-learn, xgboost, or any other library.) Use cross-validation or do a three-way split (train/validate/test) and **estimate your ROC AUC** validation score.
#
# Use your model to **predict probabilities** for the test set. **Get an ROC AUC test score >= 0.60.**
#
# _To earn a score of 3 for this part, get an ROC AUC test score >= 0.70 (without using the feature with leakage)._
#
#
# ### Part 3: Visualization
#
# Make visualizations for model interpretation. (You may use any libraries.) Choose two of these types:
#
# - Permutation Importances
# - Partial Dependence Plot, 1 feature isolation
# - Partial Dependence Plot, 2 features interaction
# - Shapley Values
#
# _To earn a score of 3 for this part, make all four of these visualization types._

# + [markdown] id="muEIQ4EPGVH_" colab_type="text"
# ## Part 1: Preprocessing
#
# > You may choose which features you want to use, and whether/how you will preprocess them. If you use categorical features, you may use any tools and techniques for encoding.

# + id="P1jztQYKUhod" colab_type="code" outputId="4aa1f3b6-d67a-45aa-87cc-5fb05856f8fc" colab={"base_uri": "https://localhost:8080/", "height": 119}
# our target value is whether or not a participant made a donation in March (1, 0)
y = train['Fail']
majcl = y.mode()[0]
y_pred = np.full(shape=train.shape[0], fill_value=majcl)

# The majority class with a binary classification is simply the 
# percentage of the mode in relation to the total observations
print('Majority Class Baseline: ', accuracy_score(y, y_pred), '\n')
print('Majority Class v.s. Minority as % of total :')

train['Fail'].value_counts(normalize=True)

# + id="6nyfCak8VPUH" colab_type="code" outputId="13cb2139-71c6-4244-bbe6-b6288b1b51e2" colab={"base_uri": "https://localhost:8080/", "height": 224}
print(classification_report(y, y_pred));

# + id="nJ95ogEpXKpe" colab_type="code" colab={}
target = 'Fail'
y_train = train[target]
y_test = test[target]

# + id="S7p0enKTT-MP" colab_type="code" outputId="3d513b42-fb00-4c3d-ec09-c03f03f5c391" colab={"base_uri": "https://localhost:8080/", "height": 119}
from sklearn.model_selection import train_test_split

X_trainval, X_test, y_trainval, y_test = train_test_split(
    X, y, test_size=20000, stratify=y, random_state=42)

X_train, X_val, y_train, y_val = train_test_split(
    X_trainval, y_trainval, test_size=20000, 
    stratify=y_trainval, random_state=42)

print('X_train shape', X_train.shape)
print('y_train shape', y_train.shape)
print('X_val shape', X_val.shape)
print('y_val shape', y_val.shape)
print('X_test shape', X_test.shape)
print('y_test shape', y_test.shape)

# + [markdown] colab_type="text" id="AUyODVJGV_G2"
# ## Part 2: Modeling
#
# > **Fit a model** with the train set. (You may use scikit-learn, xgboost, or any other library.) Use cross-validation or do a three-way split (train/validate/test) and **estimate your ROC AUC** validation score.
# >
# > Use your model to **predict probabilities** for the test set. **Get an ROC AUC test score >= 0.60.**

# + id="yp02qCi1V_ic" colab_type="code" outputId="b66e27df-e6e3-4ea2-b909-4310747322b2" colab={"base_uri": "https://localhost:8080/", "height": 561}
import category_encoders as ce
from sklearn.impute import SimpleImputer
from sklearn.pipeline import make_pipeline
from xgboost import XGBClassifier

processor = make_pipeline(
    ce.OrdinalEncoder(), 
    SimpleImputer(strategy='median')
)

X_train_processed = processor.fit_transform(X_train)
X_val_processed = processor.transform(X_val)

eval_set = [(X_train_processed, y_train), 
            (X_val_processed, y_val)]

model = XGBClassifier(n_estimators=1000, n_jobs=-1)
model.fit(X_train_processed, y_train, eval_set=eval_set, eval_metric='auc', 
          early_stopping_rounds=10)

# + id="7h85v-w1X6IS" colab_type="code" outputId="47e7a82c-6359-4955-fe56-7814f20d07e2" colab={"base_uri": "https://localhost:8080/", "height": 34}
from sklearn.metrics import roc_auc_score
X_test_processed = processor.transform(X_test)
y_pred_proba = model.predict_proba(X_test_processed)[:, 1]
print('Test ROC AUC:', roc_auc_score(y_test, y_pred_proba))

# + id="Pms5auI0YtWx" colab_type="code" outputId="887135f2-7039-4520-f9a0-4dcec7bd69a4" colab={"base_uri": "https://localhost:8080/", "height": 136}
model.predict_proba(X_test_processed)

# + id="L5yqE-GJYvR6" colab_type="code" outputId="02399ada-9178-48ea-df86-c3b5a8f1d249" colab={"base_uri": "https://localhost:8080/", "height": 51}
model.predict_proba(X_test_processed)[:, 0]

# + id="aPt6uIHnYypb" colab_type="code" outputId="bd75ef4f-3f0c-453f-bc55-1a9aa3ff579f" colab={"base_uri": "https://localhost:8080/", "height": 51}
model.predict_proba(X_test_processed)[:, 1]

# + id="8JOz74_HZDzl" colab_type="code" outputId="99e097b3-b06d-43d7-d92a-b1b96e0242da" colab={"base_uri": "https://localhost:8080/", "height": 102}
train.columns

# + [markdown] id="u98nLGBTMo9s" colab_type="text"
# ## Part 3: Visualization
#
# > Make visualizations for model interpretation. (You may use any libraries.) Choose two of these types:
# >
# > - Permutation Importances
# > - Partial Dependence Plot, 1 feature isolation
# > - Partial Dependence Plot, 2 features interaction
# > - Shapley Values

# + id="KEEwv9iKY-SM" colab_type="code" colab={}
row = X_test.iloc[[6355]]

# + id="_Q379i5CIeKY" colab_type="code" outputId="db8ae7d2-8dd0-4b07-d5d2-74c51b6ff32c" colab={"base_uri": "https://localhost:8080/", "height": 193}
import shap

explainer = shap.TreeExplainer(model)
row_processed = processor.transform(row)
shap_values = explainer.shap_values(row_processed)

shap.initjs()
shap.force_plot(
    base_value=explainer.expected_value, 
    shap_values=shap_values, 
    features=row
)

# + id="Ol9-zxFVaMo2" colab_type="code" outputId="653910fd-1343-43e9-f9c5-473d18d6d67b" colab={"base_uri": "https://localhost:8080/", "height": 1000}
feature_names = row.columns
feature_values = row.values[0]
shaps = pd.Series(shap_values[0], zip(feature_names, feature_values))
shaps.sort_values().plot.barh(color='red', figsize=(15,20));

# + id="4twWk3UoeZZM" colab_type="code" colab={}
import matplotlib.pyplot as plt
plt.rcParams['figure.dpi'] = 72

# + id="T7IiR3VyedaV" colab_type="code" outputId="27aaaba1-9720-4e7d-c494-5a72bbc4c32d" colab={"base_uri": "https://localhost:8080/", "height": 224}
from pdpbox.pdp import pdp_isolate, pdp_plot
# %matplotlib inline
import matplotlib.pyplot as plt

from sklearn.datasets import make_hastie_10_2
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.inspection import plot_partial_dependence

X, y = make_hastie_10_2(random_state=0)
clf = GradientBoostingClassifier(n_estimators=100, learning_rate=1.0,
      max_depth=1, random_state=0).fit(X, y)
features = [0, 1, (0, 1)]
plot_partial_dependence(clf, X, features) 

# + id="GgP402YAiFiK" colab_type="code" outputId="8213fcf3-842b-424b-af9c-cb6609fc5f99" colab={"base_uri": "https://localhost:8080/", "height": 224}
>>> from sklearn.datasets import load_iris
>>> iris = load_iris()
>>> mc_clf = GradientBoostingClassifier(n_estimators=10,
...     max_depth=1).fit(iris.data, iris.target)
>>> features = [3, 2, (3, 2)]
>>> plot_partial_dependence(mc_clf, X, features, target=0) 

StarcoderdataJupyterScriptsDedupFiltered