"""
# Predicting Credit Card Application Approvals
"""
"""
Banks receive a lot of applications for credit cards. Many of them get rejected for many reasons, like high loan balances, low credit scores or low income levels and etc for example.
The task I wish to achieve from this notebook is to build an automatic credit card approval predictor using Data Analysis and Machine Learning. For this, I have: 
1) Load and read the data

2) Perform data cleaning- deal with missing values, duplicate values

3) Data Preprocessing- converting non-numeric values to numeric, scaling the dataset values to best fit a Machine Learning algorithm and finally split the dataset into train and test data

4) Exploratory data analysis to build an intuition about model needed

5) Build a Machine Learning model that is able to predict if an individual credit card application is approved or reject

The dataset that I have picked is the <a href="http://archive.ics.uci.edu/ml/datasets/credit+approval">Credit Card Approval dataset</a> from the UCI Machine Learning Repository.
"""
import pandas as pd
import numpy as np
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import GridSearchCV
# load dataset
df = pd.read_csv("/kaggle/input/credit-card-applications-dataset/cc_approvals.data", header=None)
df.head(20)
"""
As seen, the column names are anonymized by the contributor since this data is confidential.

<a href="http://rstudio-pubs-static.s3.amazonaws.com/73039_9946de135c0a49daa7a0a9eda4a67a72.html"> This blog</a> gives us a pretty good overview of the probable features. The probable features in a typical credit card application are <i>Gender, Age, Debt, Married, BankCustomer, EducationLevel, Ethnicity, YearsEmployed, PriorDefault, Employed, CreditScore, DriversLicense, Citizen, ZipCode, Income and finally the ApprovalStatus.</i> This gives us a pretty good starting point, and we can map these features with respect to the columns in the output.
"""
# dataframe information
df.info()
df.describe()
# summary statistics
df.describe(include = 'O') 
#notice the '?' there, these need to be removed/replaced
df.tail(20) 
# notice the '?' there, these need to be removed/replaced
df.isnull().sum()
"""
### Data Imputation
"""
# replace the '?'s with NaN
df.replace('?', np.nan, inplace=True)

# inspect the missing values again
df.tail(20)
# impute the missing values with mean imputation
df.fillna(df.mean(), inplace=True)
# count the number of NaNs in the dataset and print the counts to verify
df.isnull().sum()
# use backfill method to fill nan values in object columns
for cname in df:
    if df[cname].dtypes == "object":
        df[cname].fillna(method = 'backfill', inplace = True)
# finally check for any duplicate rows
df.duplicated().sum()
"""
### Data Preprocessing
"""
from sklearn import preprocessing
le = preprocessing.LabelEncoder()

# extract columns having data type as object (i.e non numeric)
for col in df:
    if df[col].dtypes =='object':
        df[col]=le.fit_transform(df[col])  # use LabelEncoder to transform values into numeric
df.head(20)
# all values converted to numeric
df.nunique()
# drop the features 11 and 13 because feature 11 corresponds to DriversLicencse and 13 to ZipCode which are both unimportant for us
df = df.drop([11, 13], axis=1)
# view the df to verify
df.head()
# segregate features and labels into separate variables
X = df.drop([15], axis =1)
y = df[15]
# perform one hot encoding on columns that have less than 5 unique values so that all values are considered of equal weight
X = pd.get_dummies(X, columns=[3,4,8,9,12]) 
# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=1)
# data scaling

scaler = MinMaxScaler(feature_range=(0,1))
rescaledX_train = scaler.fit_transform(X_train)
rescaledX_test = scaler.fit_transform(X_test)
"""
### Model Building

This is a classifcation problem and the possible models I would consider for this are- Logistic Regression, Random Forrest Classifer and KNeighbors Classifer. Lets build these models and check accuraries
"""
# Logistic Regression Model
from sklearn.linear_model import LogisticRegression

lr_model = LogisticRegression(random_state = 15)
lr_model.fit(rescaledX_train, y_train)
y_pred = lr_model.predict(rescaledX_test)
# model evaluation
print("Accuracy of logistic regression classifier: ", lr_model.score(rescaledX_test, y_test))
# Random Forest Classifier
from sklearn.ensemble import RandomForestClassifier

rfc_model = RandomForestClassifier(random_state = 36)
rfc_model.fit(rescaledX_train, y_train)
y_pred = rfc_model.predict(rescaledX_test)
# model evaluation
print("Accuracy of random forest classifier: ", rfc_model.score(rescaledX_test, y_test))
#KNeightboursClassifier Model
from sklearn.neighbors import KNeighborsClassifier

kn_model = KNeighborsClassifier()
kn_model.fit(rescaledX_train, y_train)
y_pred = kn_model.predict(rescaledX_test)
# model evaluation
print("Accuracy of KNeighbors classifier: ", kn_model.score(rescaledX_test, y_test))
"""
The Random Forest and KNeighbors Classifiers have the best accuracy. Lets hypertune parameters for these and check our best model for Credit Card Approval predictions!
"""
"""
### Hypertuning the model
"""
# hyperparameter tuning for RandomForestClassifier
max_depth = [int(x) for x in np.linspace(10, 110, num = 11)]
parameters = {'n_estimators': [20, 50, 60, 80, 90, 100, 120, 150, 200], 'max_features': ["auto", "sqrt", "log2"]}
cls = GridSearchCV(estimator = rfc_model, param_grid = parameters)
cls.fit(rescaledX_train, y_train)

# displaying the best params 
cls.best_params_
rfc_model2 = RandomForestClassifier(random_state = 36, n_estimators = 150, max_features = 'auto')
rfc_model2.fit(rescaledX_train, y_train)
y_pred = rfc_model2.predict(rescaledX_test)
rfc_model2.score(rescaledX_test, y_test)
# hyperparameter tuning for RandomForestClassifier

n_neighbors = range(1, 21, 2)
parameters = {'n_neighbors': n_neighbors, 'weights': ['uniform', 'distance'], 'metric': ['euclidean', 'manhattan', 'minkowski']}
cls = GridSearchCV(estimator = kn_model, param_grid = parameters)
cls.fit(rescaledX_train, y_train)

# displaying the best params 
cls.best_params_
kn_model2 = KNeighborsClassifier(n_neighbors=15, weights = 'distance', metric='euclidean')
kn_model2.fit(rescaledX_train, y_train)
y_pred = kn_model2.predict(rescaledX_test)
kn_model2.score(rescaledX_test, y_test)
"""
The best score is using the Random Forest Classifier model with parameters obtained from GridSearchCV(). Hence, our final model for Credit Card Aprovals is now ready!
"""
model = RandomForestClassifier(random_state = 36, n_estimators = 150, max_features = 'auto')
model.fit(rescaledX_train, y_train)
y_pred = model.predict(rescaledX_test)
model.score(rescaledX_test, y_test)
"""
### <b>The Machine Learning Model is able to predict Credit Card Approvals with an enhanced accuracy of 90.36%</b>
"""