# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     name: python3
# ---

# + [markdown] id="view-in-github" colab_type="text"
# <a href="https://colab.research.google.com/github/harnalashok/classification/blob/main/Project_GiveMeSomeCredit_ByEkta.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>

# + [markdown] id="ITxNweQ3MjA7"
# Author: <NAME>, Central Bank of India  
# Kaggle [URL](https://www.kaggle.com/c/GiveMeSomeCredit/overview)  
# Date: 20th May, 2021  
# Published by: <NAME>
#

# + [markdown] id="mXo-J_9zGvMg"
# ## <font color='blue'>**Project Objective:**</font></br>
# ##### To improve on the state of the art in credit scoring by predicting the probability that somebody will experience financial distress in the next two years.
#
# ### <font color='green'>**Context:**</font></br>
# ##### Banks play a crucial role in market economies. They decide who can get finance and on what terms and can make or break investment decisions. For markets and society to function, individuals and companies need access to credit. </br>
#
# ##### Credit scoring algorithms, which make a guess at the probability of default, are the method banks use to determine whether or not a loan should be granted. This project requires to improve on the state of the art in credit scoring, by predicting the probability that somebody will experience financial distress in the next two years.
#
# The aim of this project is to build a model that borrowers can use to help make the best financial decisions.
#
# Historical data are provided on 250,000 borrowers

# + [markdown] id="phEyUcGFKN0C"
# ####<font color='blue'>**Calling Data Manupulation, Plotting, OS & Warning Libraries:**</font>
#

# + id="Hc7R1t0HG1hQ"
# For data manupulation
import pandas as pd
import numpy as np
from tabulate import tabulate
from scipy.stats import norm 

# For visual interpretation
import seaborn as sns
import matplotlib.pyplot as plt
from matplotlib import cm

# Call os for setting up navigation and environment
import os

# + id="Bxh9SzyC3O8-"
# Import warnings module
import warnings
# Do not print warnings on screen
warnings.filterwarnings("ignore")

# + id="J_natiEZATVQ"
# For displaying all outputs from a cell--not just the last
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

# + [markdown] id="ta1ELSTFzKC-"
# #### <font color='blue'>**Importing Data To Colab Machine**</font>
#
# ##### <font color='green'>**(1)** </font>Establish a connection to Google Drive. Login to the Google account when prompted and copy a validation token each time we connect to Drive from Colab. After establishing the connection, we will see a new folder appear in the side bar as ‘/gdrive’. </br>
#
# ##### <font color='green'>**(2)** </font>Copy the compressed file from '/gdrive' to the virtual machine and unzip it to access the data. Do not remove the original zip from your Drive after copying. We will need to copy it at the start of each session while working with Colab. </br>
#
# ##### <font color='green'>**(2)** </font> Unzip the data
#
#
#

# + colab={"base_uri": "https://localhost:8080/"} id="AyhR3OT-IQ3E" outputId="6a1129e6-8b08-4430-9764-51669d9862bf"
# Mount google drive
from google.colab import drive
drive.mount('/ashok')

# + id="eCMKaepOH9Q0"
# Complete path to storage location of the .zip file of data
zip_path = '/ashok/MyDrive/Colab_data_files/GiveMeSomeCredit'

# + colab={"base_uri": "https://localhost:8080/", "height": 36} id="wb5Rw4EN3yl_" outputId="3dfdb88f-6d32-4d1a-d530-3aa005a6fb70"
# Check current directory (be sure you're in the directory where Colab operates: '/content')
os.getcwd()
os.chdir(zip_path)

# + [markdown] id="LU9DQFaiil1E"
# ### **<font color='blue'>Reading Data</font>**

# + id="YUeZMmDxK2s_"
df_train = pd.read_csv('cs-training.csv.zip')
df_test = pd.read_csv('cs-test.csv.zip')
df_sample = pd.read_csv('sampleEntry.csv.zip')
df_DataDictionary = pd.read_excel('Data Dictionary.xls', header=0, index_col=0)

# + [markdown] id="NqQtpEPPjlU-"
# ##### <font color='green'>**Dataset Contain Following Features:**</font>

# + colab={"base_uri": "https://localhost:8080/"} id="4-DRXIQmgN2v" outputId="d83f9434-ef92-4e7e-bb34-9b026fbb228c"
#df_DataDictionary.style.set_properties(**{'text-align': 'left'})
print(tabulate(df_DataDictionary, headers="firstrow", tablefmt="fancy_grid"))

# + [markdown] id="9105cmV5kygE"
# #### <font color='blue'>**Exploring Dataset:**</font>

# + colab={"base_uri": "https://localhost:8080/"} id="e1rS893r8JfB" outputId="c551292f-c236-4c4c-baaa-520e5a4d6d49"
print(df_train.shape)
print(df_test.shape)

# + colab={"base_uri": "https://localhost:8080/", "height": 244} id="nZwW7su9IBGc" outputId="ae1e2171-5875-4195-b764-19e892546c01"
df_train.head()

# + colab={"base_uri": "https://localhost:8080/"} id="kk9eQlya8CE7" outputId="e3ee156e-40bd-4584-f5b9-01bd3312cb6f"
df_train.info()

# + colab={"base_uri": "https://localhost:8080/", "height": 338} id="8bDrD0USlOcO" outputId="a3538fad-c733-469a-9b56-878c623460ba"
df_train.describe().round(2)

# + colab={"base_uri": "https://localhost:8080/", "height": 244} id="8XTl_yK3mCI1" outputId="09c75a8c-b9d0-4b2e-eead-55f2afb8571b"
df_test.head()

# + colab={"base_uri": "https://localhost:8080/"} id="KpLYCNCcJPfG" outputId="dc93eb56-01d5-49aa-c357-77409426c6e3"
df_test.info()

# + colab={"base_uri": "https://localhost:8080/", "height": 338} id="jeYfV5CBJQn8" outputId="b759cd41-26d4-40de-9efe-a6621ee58c4b"
df_test.describe().round(2)

# + [markdown] id="_p6I3xZSJ--A"
# ###### <font color='dark orange'>*Values are missing from both Test & Train Datasets*</font>

# + [markdown] id="zgh6gbnOmhvz"
# #### <font color='blue'>**Visual Analysis Of the Missing Values:**</font>

# + colab={"base_uri": "https://localhost:8080/", "height": 577} id="n8O_YIDUwp57" outputId="dcce287d-f562-4f48-e080-7ce541c048ad"
# Heatmap for missing values in training data
plt.figure(figsize=(8,5))
sns.heatmap(df_train.isnull(), cmap="coolwarm", yticklabels=False, cbar=True)
plt.show()

# + [markdown] id="QTEFHTYC8_Ux"
# ###### <font color='dark orange'>_There are missing values in **MonthlyIncome** and **NumberOfDependents**_</font>
#  

# + colab={"base_uri": "https://localhost:8080/"} id="dI7GdNHVOsLj" outputId="e036086f-0ad7-44b1-ebd7-dcf7d8577d7b"
# Percentage of data missing
round(df_train.isnull().mean() * 100,2)

# + [markdown] id="h7YQ7iXVPDDW"
# ###### <font color='dark orange'>_**MonthlyIncome** has 19.8% missing values and **NumberOfDependents** has 2.6% missing values_</font>
#

# + colab={"base_uri": "https://localhost:8080/", "height": 577} id="dDErbglb84Db" outputId="7deaf409-4ec7-4de2-9485-3b5fbd508d76"
# Heatmap for missing values in test data
plt.figure(figsize=(8,5))
sns.heatmap(df_test.isnull(), cmap="coolwarm", yticklabels=False, cbar=True)
plt.show()

# + [markdown] id="DiopIHaBP1V2"
# ###### <font color='dark orange'>*There are missing values in __SeriousDlqin2yrs__, __MonthlyIncome__ and __NumberOfDependents__*</font>

# + colab={"base_uri": "https://localhost:8080/"} id="riSvoMX1PiAm" outputId="de18290d-b092-4e41-8b29-6b6fd17f0df3"
round(df_test.isnull().mean() * 100,2)

# + [markdown] id="aA0M0xImQW_m"
# ###### <font color='dark orange'>_**SeriousDlqin2yrs** has 100%, **MonthlyIncome** has 19.8% and **NumberOfDependents** has 2.59% missing values_</font>

# + [markdown] id="GOcfTKxouLeV"
# #### <font color='blue'>**Fixing Missing Values in the training dataset:**</font>

# + id="jaXybJApmy1E"
# We will impute the MonthlyIncome with mean
df_train['MonthlyIncome'].fillna(df_train['MonthlyIncome'].mean(),inplace=True)


# + id="o1mNAY-l8BNz"
# We will impute the NumberOfDependents with mode
df_train['NumberOfDependents'].fillna(df_train['NumberOfDependents'].mode()[0], inplace=True)

# + [markdown] id="Xo1fUnhNKHo8"
# #### <font color='blue'>**Fixing Missing Values in the test dataset:**</font>

# + id="qoPuLUl_KMlp"
# We will impute the MonthlyIncome with mean
df_test['MonthlyIncome'].fillna(df_test['MonthlyIncome'].mean(),inplace=True)

# + id="XgioQA-UKYVq"
# We will impute the NumberOfDependents with mode
df_test['NumberOfDependents'].fillna(df_test['NumberOfDependents'].mode()[0], inplace=True)

# + [markdown] id="iXaZmT82N-RT"
# #### <font color='blue'>**Checking if imputations are propoer:**</font>

# + colab={"base_uri": "https://localhost:8080/"} id="zcnOkkDhN9Q6" outputId="be967b34-6ec1-45ff-ae06-114d39be3705"
df_train.isnull().sum()

# + colab={"base_uri": "https://localhost:8080/"} id="BVqUyGq1OZfI" outputId="7a0068ed-6535-440b-d8da-23cb20c95f2f"
df_test.isnull().sum()

# + [markdown] id="2PEdpb0PSmxM"
# ###### <font color='dark orange'>_Imputations look good_</font>

# + [markdown] id="-NXHZyjcO7t4"
# #### <font color='blue'>**Visualisation of the data prepared:**</font>

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="93_Qz1fNPBFA" outputId="7a31fd2c-8df6-4413-8790-9a6cc99b0c9d"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(6,6))
sns.countplot(x='SeriousDlqin2yrs',data=df_train, palette="Set1_r")
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="CY28s2vywa4J" outputId="7983a04c-a706-47a8-a63d-079ea239577d"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.histplot(x='RevolvingUtilizationOfUnsecuredLines',data=df_train, stat="density", bins=20, kde=True, hue='SeriousDlqin2yrs')
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="y6zlNE9Gwb1i" outputId="174d8a34-fd31-40e0-dd33-8697411561b7"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.boxplot(x = 'SeriousDlqin2yrs', y = 'RevolvingUtilizationOfUnsecuredLines', data = df_train, showfliers=False) 
# removed the outliers as they were making the box plot very small to read)

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="OBzwFwF0Yskg" outputId="3b7c3f43-5652-4e05-8dd4-a6ce1c6f9717"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.histplot(x='age',data=df_train, stat="density", bins=20, kde=True, hue='SeriousDlqin2yrs')
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="AvJHJ1iXmfQN" outputId="61a5ed50-bdf0-4260-94e1-6d4bd3be8cfb"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.boxplot(x = 'SeriousDlqin2yrs', y = 'age', data = df_train)

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="cKyV6zkAsH8t" outputId="e9368390-942a-41fb-dce8-6cb8fa528aea"

sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.histplot(x='NumberOfTime30-59DaysPastDueNotWorse',data=df_train, stat="density", bins=20, kde=True, hue='SeriousDlqin2yrs')
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="SKocIP78sVRZ" outputId="fc6ddcae-c6b1-450c-a8a2-96f58eb7b739"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.boxplot(x = 'SeriousDlqin2yrs', y = 'NumberOfTime30-59DaysPastDueNotWorse', data = df_train, showfliers=False) 
# removed the outliers as they were making the box plot very small to read

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="-xTxtJW62_s3" outputId="93e70a64-f27b-40b2-a577-ffa7ffe964b3"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.histplot(x='DebtRatio',data=df_train, stat="density", bins=25, kde=True, hue='SeriousDlqin2yrs')
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="4Bpx2tpQ2_X3" outputId="1e6fc045-5a60-4e44-81fa-5e67da3fb76e"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.boxplot(x = 'SeriousDlqin2yrs', y = 'DebtRatio', data = df_train, showfliers=False) 
# removed the outliers as they were making the box plot very small to read

# + colab={"base_uri": "https://localhost:8080/", "height": 423} id="7YbPO8cv7xwS" outputId="d5e26cbb-5119-4902-ccd3-45a9ac663485"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.histplot(x='MonthlyIncome',data=df_train, stat="density", bins=25, kde=True, hue='SeriousDlqin2yrs')
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="27pZkND_7xdL" outputId="b44ddf99-fce4-4d8b-bc33-12822d143dc8"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.boxplot(x = 'SeriousDlqin2yrs', y = 'MonthlyIncome', data = df_train, showfliers=False) 
# removed the outliers as they were making the box plot very small to read

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="ivWwVg717xtN" outputId="d20e3e53-271c-44d0-c520-9d7dce1c406e"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.histplot(x='NumberOfOpenCreditLinesAndLoans',data=df_train, stat="density", bins=25, kde=True, hue='SeriousDlqin2yrs')
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="-oGww3NM7xag" outputId="c3baebdd-29f1-4d4b-e873-b64fc8dae9fa"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.boxplot(x = 'SeriousDlqin2yrs', y = 'NumberOfOpenCreditLinesAndLoans', data = df_train, showfliers=False) 
# removed the outliers as they were making the box plot very small to read

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="Qu2zQGkn7xqH" outputId="3e000257-8629-452b-ca25-5e4b11078d26"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.histplot(x='NumberOfTimes90DaysLate',data=df_train, stat="density", bins=25, kde=True, hue='SeriousDlqin2yrs')
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="-Pr7zPPs7xS2" outputId="5aa53877-80bd-4029-b5ea-988559a4155a"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.boxplot(x = 'SeriousDlqin2yrs', y = 'NumberOfTimes90DaysLate', data = df_train, showfliers=False) 
# removed the outliers as they were making the box plot very small to read

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="x9HLehHV7xj0" outputId="4c7e552a-4298-410a-e2c0-cc109edad9ff"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.histplot(x='NumberRealEstateLoansOrLines',data=df_train, stat="density", bins=25, kde=True, hue='SeriousDlqin2yrs')
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="FT5VY2YZ7xIL" outputId="e18a854f-5fc1-44a0-ad1b-32a805e15400"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.boxplot(x = 'SeriousDlqin2yrs', y = 'NumberRealEstateLoansOrLines', data = df_train, showfliers=False) 
# removed the outliers as they were making the box plot very small to read

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="ugAp_wS67xgP" outputId="8517023d-6228-44ad-b4d4-9986b77cc417"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.histplot(x='NumberOfTime60-89DaysPastDueNotWorse',data=df_train, stat="density", bins=25, kde=True, hue='SeriousDlqin2yrs')
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="R3YUC7rI7w-M" outputId="07635f58-214f-470d-ed7e-48fa51e57b58"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.boxplot(x = 'SeriousDlqin2yrs', y = 'NumberOfTime60-89DaysPastDueNotWorse', data = df_train, showfliers=False) 
# removed the outliers as they were making the box plot very small to read

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="4WctPCWffXyt" outputId="530bd7d8-164d-48c6-f215-af0defbadc34"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.histplot(x='NumberOfDependents',data=df_train, stat="density", bins=25, kde=True, hue='SeriousDlqin2yrs')
plt.show()

# + colab={"base_uri": "https://localhost:8080/", "height": 412} id="WsF0ZnpYqJH9" outputId="933d80ed-f128-4aa2-8eaa-21932fa46311"
sns.set_theme(style="darkgrid")
fig = plt.figure(figsize=(10,6))
sns.set_palette("bright")
sns.boxplot(x = 'SeriousDlqin2yrs', y = 'NumberOfDependents', data = df_train, showfliers=False) 
# removed the outliers as they were making the box plot very small to read

# + colab={"base_uri": "https://localhost:8080/", "height": 849} id="ZE4zk7Ep9wm_" outputId="b2fe088e-570c-41f2-dcbe-9a9698f714de"
# Correlation
fig,ax = plt.subplots(figsize=(10, 10))
sns.heatmap(df_train.corr(), annot=True, cmap='Greens', linewidths=.5, fmt='.2f',ax=ax)
plt.show()

# + [markdown] id="pHe26R2JT5Xa"
# #### <font color='blue'>**Data Preparation For Modeling**</font>

# + id="5Ml1Ora4CG-6"
# Dropping unnamed colum from test dataframe
df_train.drop('Unnamed: 0', axis=1, inplace=True)

# + id="sQF8fe_dC_vJ"
# Dropping unnamed colum from test dataframe
df_test.drop('Unnamed: 0', axis=1, inplace=True)

# + id="MM-AcgAJDo8U"
X = df_train.drop('SeriousDlqin2yrs',axis=1)
y = df_train['SeriousDlqin2yrs']

# + [markdown] id="g50kXcAeEzV0"
# ##### <font color='green'>**Splitting Data**</font>

# + colab={"base_uri": "https://localhost:8080/"} id="toRSDoXxDuXg" outputId="d37d582b-e358-46af-8225-870960321ccb"
from sklearn.model_selection import train_test_split
#splitting data into train and test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=1000)
print(X_train.shape)
print(X_test.shape)
print(y_train.shape)
print(y_test.shape)

# + [markdown] id="bPi3Udad3S3n"
# #### <font color='blue'>**Model 1: Using Random Forest**</font>

# + [markdown] id="vx5wxQn8U93N"
# ##### <font color='green'>**Calling Data Modeling Libraries**</font>

# + id="8zcZWg7zE9hz"
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import BaggingClassifier
from sklearn.metrics import confusion_matrix, classification_report
from sklearn.metrics import accuracy_score, recall_score, precision_score, f1_score,roc_curve,auc
from sklearn.model_selection import GridSearchCV, cross_val_score, RandomizedSearchCV
from sklearn.tree import DecisionTreeClassifier

# + id="4kb9-tGAz8fO"
RFC = RandomForestClassifier()

# + id="h_QS0Q22yh5P"
param_grid = { 
           "n_estimators" : [9, 18, 27, 36, 100, 150],
           "max_depth" : [2,3,5,7,9],
           "min_samples_leaf" : [2, 3]}

# + id="f5KTFugi0ANE"
RFC_random = RandomizedSearchCV(RFC, param_distributions=param_grid, cv=5)

# + colab={"base_uri": "https://localhost:8080/"} id="VJ9VVin00LIe" outputId="080fe8a9-6460-4132-bbbf-5713709dc97e"
RFC_random.fit(X_train, y_train)

# + id="eRDWw9xs0Z4L"
best_est_RFC = RFC_random.best_estimator_

# + colab={"base_uri": "https://localhost:8080/"} id="L7kojaAU0cz0" outputId="a881e00a-7bdf-4e77-a56c-d408a8f46aae"
print('Training Set Classifier Accuracy %: {:0.2f}'.format(RFC_random.score(X_train, y_train) * 100))
print('Test Set Classifier Accuracy %: {:0.2f}'.format(RFC_random.score(X_test, y_test) * 100))

# + id="YWaw2FSH0l8h"
y_pred_RFC = best_est_RFC.predict_proba(X_test)
y_pred_RFC = y_pred_RFC[:,1]

# + colab={"base_uri": "https://localhost:8080/", "height": 573} id="-JMsVoEe1S6o" outputId="8f7ac0ec-ace7-4e45-efda-b8cd4ce06312"
fpr,tpr,_ = roc_curve(y_test, y_pred_RFC)
roc_auc = auc(fpr, tpr)
plt.figure(figsize=(10,8))
plt.title('Receiver Operating Characteristic')
sns.lineplot(fpr, tpr, label = 'AUC = %0.2f' % roc_auc)
plt.legend(loc = 'lower right')
plt.plot([0, 1], [0, 1],'r--')
plt.ylabel('True Positive Rate')
plt.xlabel('False Positive Rate')
plt.show()

# + [markdown] id="mpHv8CO9XPpd"
# #### <font color='blue'>**Model 2: Using XGBOOST**</font>

# + [markdown] id="hei093eHn1Sz"
# ##### <font color='green'>**Calling Data Modeling Libraries**</font>

# + id="J0rMluHXwBpZ"
import xgboost as xgb
from xgboost import XGBClassifier

# + id="Edt13nu2xSFN"
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV

# + id="UeJDa-kjxUQ6"
xgb = XGBClassifier(n_jobs=-1) 
 
# Use a grid over parameters of interest
param_grid = {
                  'n_estimators' :[100,150,200,250,300],
                  "learning_rate" : [0.001,0.01,0.0001,0.05, 0.10 ],
                  "gamma"            : [ 0.0, 0.1, 0.2 , 0.3 ],
                  "colsample_bytree" : [0.5,0.7],
                  'max_depth': [3,4]
              }

# + id="6bRNnHjJxe6F"
xgb_randomgrid = RandomizedSearchCV(xgb, param_distributions=param_grid, cv=5)

# + colab={"base_uri": "https://localhost:8080/"} id="nO5IqQmbxpP2" outputId="8fff5531-ffae-4236-caf0-0b4452df7382"
xgb_randomgrid.fit(X_train,y_train)

# + id="etsFAb15xqiE"
best_est_XGB = xgb_randomgrid.best_estimator_

# + colab={"base_uri": "https://localhost:8080/"} id="bIjN5u8Bxwao" outputId="6d2a91fd-b554-407a-db65-7498d97c621f"
print('Training Set Classifier Accuracy %: {:.2f}'.format(xgb_randomgrid.score(X_train, y_train) * 100))
print('Test Set Classifier Accuracy %: {:.2f}'.format(xgb_randomgrid.score(X_test, y_test) * 100))

# + id="fJbaaJUPx4H4"
y_pred_XGB = best_est_XGB.predict_proba(X_train)
y_pred_XGB = y_pred_XGB[:,1]

# + colab={"base_uri": "https://localhost:8080/", "height": 590} id="6iH7V0fMyVtN" outputId="396c0397-c68c-47c4-f0a7-d7c3f3459fd7"
fpr,tpr,_ = roc_curve(y_train, y_pred_XGB)
roc_auc = auc(fpr, tpr)
plt.figure(figsize=(10,8))
plt.title('Receiver Operating Characteristic')
sns.lineplot(fpr, tpr, label = 'AUC = %0.2f' % roc_auc)
plt.legend(loc = 'lower right')
plt.plot([0, 1], [0, 1],'r--')
plt.ylabel('True Positive Rate')
plt.xlabel('False Positive Rate')
plt.show()

# + [markdown] id="OYvGRKdzA3AS"
# ####<font color='green'>**Conclusion:**</font>
#
# Between the two models we built, we saw that:</br>
# (1) Area Under Curve (AUC) for RandomForest Model is 0.88.</br>
# (2) Area Under Curve (AUC) for XGboost Model is 0.87.</br> 
# Therefore, it is inferred that RandomForest is the suitable model for this dataset

StarcoderdataJupyterScriptsDedupFiltered