"""
<h1><center><div style="background-color:skyblue;border-radius:10px; padding: 10px;">Introduction</div></center></h1>

🙅🏽‍♀️ Motivation
> Suppose you get a call 📞 from your bank, and the customer care executive informs you that your card is about to expire in a week. Immediately, you check your card details and realise that it will expire in the next eight days. Now, to renew your membership, the executive asks you to verify a few details such as your credit card number, the expiry date and the CVV number. Will you share these details with the executive?
> In such situations, you need to be careful because the details that you might share with them could grant them unhindered access to your credit card account.
 
> Banks 🏦 need to be cautious about their customers’ transactions, as they cannot afford to lose their customers’ money to fraudsters 🦸‍♂️. Every fraud is a loss to the bank, as the bank is responsible for the fraudulent transactions if they are reported within a certain time frame by the customer.

🎯 Goal
> The goal of this notebook is to predict the fraud transactions with high accuracy.

#### Index

- 1. Data preparation
    - Exploring data, and removing unnecessary columns
    - Transforming data
    - Handling class imbalance
    - Creating training and testing data
- 2. Model building
    - Building models
    - Hyperparameter tuning
- 3. Model evaluation
    - Evaluating model using AUC ROC curve
"""
"""
<h2><center> <div style="background-color:skyblue;border-radius:10px; padding: 10px;">Data Prepration</div></center></h2>

> The data contains only numerical input variables which are the result of a PCA transformation. Unfortunately, due to confidentiality issues, we cannot provide the original features and more background information about the data. Features V1, V2, … V28 are the principal components obtained with PCA, the only features which have not been transformed with PCA are `Time` and `Amount`. Feature `Time` contains the seconds elapsed between each transaction and the first transaction in the dataset. The feature `Amount` is the transaction Amount, this feature can be used for example-dependant cost-sensitive learning. Feature `Class` is the response variable and it takes value 1 in case of fraud and 0 otherwise.
"""
# Importing the libraries
import pandas as pd
import numpy as np

import matplotlib.pyplot as plt
import seaborn as sns
# Reading the file
df = pd.read_csv('../input/creditcardfraud/creditcard.csv')
df.head()
"""
> `Class` 0 : Fair transaction 👍🏽 | 1 : Fraud transaction 🦸‍♂️

> `Time` column won't of much importance while indentifying a fraud, as it only contains the seconds elapsed between each transaction and the first transaction in the dataset, so I will remove it.
"""
# Removing the 'Time' column
df = df.drop(['Time'], axis = 1)
# Checking for null values
df.isnull().sum().sum()
"""
<h3><center> <div style="background-color:lightpink;border-radius:10px; padding: 10px;">Data Exploration</div></center></h3>

> The data contains PCA transformed features, so the columns are meaningless except the `Amount` and `Class`. I will only perform the EDA which is necessary for building model.
"""
# Understanding amount in fraud transactions
plt.style.use('seaborn')
fig, ax = plt.subplots(1 ,1, figsize = (10, 6), constrained_layout = True)

# I used median instead of mean to avoid outliers
ax = sns.barplot(x = 'Class', y = 'Amount', data = df, estimator = np.median, ax = ax)

plt.title("Average Amount in Fraud Transactions", size = 16)

ax.set_xticklabels(['Fair', 'Fraud'], fontsize = 14)

plt.xlabel(None)
plt.ylabel('Amount of Transaction', fontsize = 14);
"""
> On an average, the amount in fraud transactions is lower than fair transactions. The reason I think can be, the fraud's are smart 🧠, they know if they do a fraud of big amount they may get caught 🚔, so they do multiple frauds of smaller amount.
"""
# EDA for the features V1, V2,..., V28
v_feaures = df.iloc[:,:-2].columns.to_list()

plt.style.use('seaborn')
fig, ax = plt.subplots(7,4, figsize = (14, 24), constrained_layout = True)

for col, axis in zip(v_feaures, ax.ravel()): # ax.ravel() kind of flattens the 2d to 1d for iteration
    axis = sns.kdeplot(x = col, data = df, fill = True, alpha = 0.6, linewidth = 1.5, ax=axis)
    axis.set_title(col, fontsize = 14)
    axis.set_xlabel(None)
    axis.set_ylabel(None)
    
fig.suptitle('Distribution of the features V1, V2,..., V28', fontsize = 16, y = 1.01);
"""
> Many features have normal distribution. Each of the PCs given by PCA is a linear combination of the original features and the original features were normally distributed , so every linear combination of them i.e. PCs are also normally distributed.

> There are also some features which are skew distributed, so I will use quantile transform for this purprose. I will be considering (-0.5,0.5) range for skewness, any feature having value beyound this range will get transformed.

❕ I have tried log transform by adding some constant to negetive values and also inverse hyperbolic tangent transform by scaling features in (-1, 1) range, but none of them reduced the skewness, so I am using quantile transform.
"""
"""
<h3><center> <div style="background-color:lightpink;border-radius:10px; padding: 10px;">🥚 Data Transformation 🐣</div></center></h3>
"""
# Checking the skewness values
plt.style.use('seaborn')
fig, ax = plt.subplots(1,1, figsize = (10, 6), constrained_layout = True)
ax = df.skew(axis = 0).plot(kind = 'bar')

for i in ax.patches:
    ax.text(x = i.get_x() + i.get_width()/2, y = i.get_height()+0.5, 
            s = f"{np.round(i.get_height(), 1)}", 
            ha = 'center', size = 14, rotation = 0, color = 'black')
    
ax.set_ylabel('Skewness', fontsize = 14)
ax.set_title('Skewness of the features', fontsize = 16);
"""
<h4><center> <div style="background-color:violet;border-radius:10px; padding: 10px;">Quantile Transform</div></center></h4>
"""
from sklearn.preprocessing import quantile_transform

v_features = df.iloc[:, :-2].columns
df[v_features] = pd.DataFrame(quantile_transform(df[v_features], n_quantiles=500), columns = v_features)

print('Skewness after Quantile Transform')
df[v_features].skew(axis = 0)
"""
> The skewness has been reduced close to 0! Now let's check for the class imbalance.
"""
"""
<h3><center> <div style="background-color:lightpink;border-radius:10px; padding: 10px;">Class Imbalance ⚖</div></center></h3>

> Imbalanced classifications pose a challenge for predictive modeling as most of the machine learning algorithms used for classification were designed around the assumption of an equal number of examples for each class. This results in models that have poor predictive performance, specifically for the minority class.
"""
# Checking for class imbalance 
plt.style.use('seaborn')
fig = plt.figure(figsize = (10, 6))

ax = sns.countplot(x = 'Class', data = df)

for i in ax.patches:
    ax.text(x = i.get_x() + i.get_width()/2, y = i.get_height()/2, 
            s = f"{np.round(i.get_height()/len(df)*100, 3)}%", 
            ha = 'center', size = 20, weight = 'bold', rotation = 0, color = 'black')

plt.title("Credit Card Fraud Count", size = 16)

ax.set_xticklabels(['Fair', 'Fraud'], fontsize = 14)

plt.xlabel(None)
plt.ylabel('Number of Transactions', fontsize = 14);
"""
> Only 0.173% tractions are fraud, which is very much less than other class. If we train our model on this unbalanced class data, then the model won't learn much from Fraud class and will try to give more importance to Fair class, which we don't want. Let's see that in action, I will train a simple logistic regression model and see its performance.
"""
# Splitting data into train and test
from sklearn.model_selection import train_test_split

X = df.drop(['Class'], axis = 1)
y = df['Class']

X_train, X_test, y_train, y_test = train_test_split(X, y, train_size = 0.7, stratify = df['Class'], random_state = 99)
# Training model on imbalanced class data 
from sklearn.linear_model import LogisticRegression
from sklearn import metrics

model = LogisticRegression(solver='liblinear').fit(X_train, y_train)

matrix = metrics.confusion_matrix(y_test, model.predict(X_test))
plt.style.use('seaborn')
fig, axis = plt.subplots(1,1, figsize=(10, 6), constrained_layout = True)
axis = sns.heatmap(matrix, annot=True, fmt = '.0f', cbar=False, cmap='Blues',
                        linewidths=3, square=True, ax = axis, annot_kws={"fontsize":20})

axis.set_xlabel('Predicted', fontsize=14)
axis.set_ylabel('Actual', fontsize=14)
axis.set_xticklabels(['Fair','Fraud'], fontsize=12)
axis.set_yticklabels(['Fair','Fraud'], fontsize=12, rotation=0);
plt.title('Model Evluation using Confusion Matrix (Before Class Balancing)', fontsize = 16);
"""
> There are 35 False Negetives out of 148. It means that, out of 148 frauds, our model couldn't identify 35 (24%), this is mainly due to class imbalance. We don't want to let the fraud escape this easily, so let's balance the classes.

> I am using SMOTE and ADASYN methods. They both are oversampling methods, i.e. they create synthetic data for manority class so that it becomes similar in count with majority class.

<center><img src = https://miro.medium.com/max/1200/1*0jwntVGaj7qQkr-MeueQcQ.jpeg></center>

SMOTE (Synthetic Minority Over-sampling Technique) creates the synthetic data for minority class, using linear method. It takes two close minority points and creates a new point on a line joining the two points.

ADASYN (Adaptive Synthetic) uses a weighted distribution for different minority class examples according to their level of difficulty in learning, where more synthetic data is generated for minority class examples that are harder to learn compared to those minority examples that are easier to learn. It creates the synthetic data near to the minority points having less density. 
"""
from imblearn.over_sampling import SMOTE, ADASYN
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

X_train_new, y_train_new = SMOTE().fit_resample(X_train, y_train)

print('Before resmapling : ', y_train.value_counts())
print('After resmapling : ', y_train_new.value_counts())

pipeline = Pipeline([('model', LogisticRegression(solver='liblinear'))])
pipeline.fit(X_train_new, y_train_new)
matrix_new = metrics.confusion_matrix(y_test, pipeline.predict(X_test))
plt.style.use('seaborn')
fig, axis = plt.subplots(1,1, figsize=(10, 6), constrained_layout = True)
axis = sns.heatmap(matrix_new, annot=True, fmt = '.0f', cbar=False, cmap='Blues',
                        linewidths=3, square=True, ax = axis, annot_kws={"fontsize":20})

axis.set_xlabel('Predicted', fontsize=14)
axis.set_ylabel('Actual', fontsize=14)
axis.set_xticklabels(['Fair','Fraud'], fontsize=12)
axis.set_yticklabels(['Fair','Fraud'], fontsize=12, rotation=0);
plt.title('Model Evluation using Confusion Matrix (After Class Balancing)', fontsize = 16);
"""
> There are only 9 False Negetives out of 148. It means that, out of 148 frauds, our model couldn't identify only 9 (6%) frauds. Previously, before class balancing, False Negetives were 35 (24%) and now they are 9, 4 times reduction!

> Let's try with ADASYN.
"""
# Oversampling minority class using ADASYN
X_train_ada, y_train_ada = ADASYN().fit_resample(X_train, y_train)

print('Before resmapling : ', y_train.value_counts())
print('After resmapling : ', y_train_ada.value_counts())

pipeline = Pipeline([('model', LogisticRegression(solver='liblinear'))])
pipeline.fit(X_train_ada, y_train_ada)
matrix_new = metrics.confusion_matrix(y_test, pipeline.predict(X_test))
plt.style.use('seaborn')
fig, axis = plt.subplots(1,1, figsize=(10, 6), constrained_layout = True)
axis = sns.heatmap(matrix_new, annot=True, fmt = '.0f', cbar=False, cmap='Blues',
                        linewidths=3, square=True, ax = axis, annot_kws={"fontsize":20})

axis.set_xlabel('Predicted', fontsize=14)
axis.set_ylabel('Actual', fontsize=14)
axis.set_xticklabels(['Fair','Fraud'], fontsize=12)
axis.set_yticklabels(['Fair','Fraud'], fontsize=12, rotation=0);
plt.title('Model Evluation using Confusion Matrix (After Class Balancing)', fontsize = 16);
"""
> The recall is slightly improved but on the expense of precision.
"""
"""
<h2><center> <div style="background-color:skyblue;border-radius:10px; padding: 10px;">Model Building 🤖</div></center></h2>

> I am focusing on recall as, it a good measure of false negetives. If a fair transaction is predicted as fraud, then manual step can mitigate this problem but if fraud is predicted as fair, then it can cause loss to the customer as well as bank.

> But if only recall the prime importance then the threshold will be set at 0 so that we get recall as 100% but very high false positives, so I am giving 50% weightage to accuracy and 50% to recall.
"""
"""
<h4><center> <div style="background-color:violet;border-radius:10px; padding: 10px;">Logistic Regression</div></center></h4>
"""
from sklearn.linear_model import LogisticRegressionCV
pipeline = Pipeline([('model', LogisticRegressionCV(solver='liblinear', cv = 5))])
pipeline.fit(X_train_ada, y_train_ada)
"""
> I will be using AUC ROC curve for model evaluation as it is independent of the threshold. Sklearn considers `0.5` as threshold for calculating Accuracy, Precision, Recall, etc. but the value of threshold varies with models, and so I will be using AUC ROC curve to find out the best threshold and use that to predict.
"""
# Plotting the ROC Curve and finding optimal threshold
# Function for plotting ROC curve, confurion matrix and finding optimal threshold
def my_roc_curve(model, accuracy_weight, title, X_train, y_train):
    from sklearn import metrics
    # Plotting AUC ROC curve
    y_scores = model.predict(X_train)
    fpr, tpr, thresholds = metrics.roc_curve(y_train, y_scores)
    
    fig, (ax1,ax2) = plt.subplots(1,2, figsize = (12, 6), constrained_layout = True)
    ax1.plot(fpr, tpr, color='skyblue', label='ROC')
    ax1.plot([0, 1], [0, 1], color='pink', linestyle='--')
    ax1.text(x = 0.8, y = 0.3,
            s = f"AUC : {round(metrics.roc_auc_score(y_train, y_scores),2)}",
            ha = 'center', size = 12, rotation = 0, color = 'black',
            bbox=dict(boxstyle="round,pad=0.5", fc='skyblue', ec="skyblue", lw=2));

    ax1.set_xlabel('False Positive Rate', fontsize = 12)
    ax1.set_ylabel('True Positive Rate', fontsize = 12)
    ax1.set_title(f'ROC curve',  fontsize=16, y=1.05)

    # Plotting optimal Confusion Matrix
    from sklearn import metrics
    probability = model.predict_proba(X_train)
    matrix = pd.DataFrame()
    
    base_accuracy = metrics.accuracy_score(y_train, model.predict(X_train))
    base_recall = metrics.recall_score(y_train, model.predict(X_train))
    best_score = 0.6*base_accuracy + 0.4*base_recall
    best_thresold = 0.5
    
    # Finding optimul thresold to maximize > accuracy_weight*threshold_accuracy + (1-accuracy_weight)*threshold_recall
    for threshold in np.linspace(0, 1, 100):
        y_predict = (probability>=threshold).astype(int)[:,1]
        threshold_accuracy = metrics.accuracy_score(y_train, y_predict)
        threshold_recall = metrics.recall_score(y_train, y_predict)
        weighted_score = accuracy_weight*threshold_accuracy + (1-accuracy_weight)*threshold_recall
        
        if weighted_score>best_score:
            best_thresold = threshold
            best_score = weighted_score
    
    y_predict = (probability>=best_thresold).astype(int)[:,1]
    matrix = metrics.confusion_matrix(y_train, y_predict)

    ax2 = sns.heatmap(matrix, annot=True, fmt = '.0f', cbar=False, cmap='Blues',
                        linewidths=3, square=True, ax = ax2, annot_kws={"fontsize":20})
    ax2.set_title(f"Confusion Matrics | Threshold : {round(best_thresold, 2)}", fontsize=16, y=1.05);
    ax2.set_xlabel('Predicted', fontsize=12)
    ax2.set_ylabel('Actual', fontsize=12)
    ax2.set_xticklabels([0,1], fontsize=12 )
    ax2.set_yticklabels([0,1], fontsize=12, rotation=0)
    
    print('Optimal Threshold for Accuracy and Recall is : ', round(best_thresold, 2))
    print(f"Train Accuracy for  {title}: ", round(metrics.accuracy_score(y_train, y_predict),2), 
          f"| Train Recall for {title}: ", round(metrics.recall_score(y_train, y_predict),2))
    
    plt.suptitle(f'{title}', fontsize=19, y=1.05)
    
    # For test data
    probability = model.predict_proba(X_test)
    y_predict = (probability>=best_thresold).astype(int)[:,1]
    print(f"Test Accuracy for {title}: ", round(metrics.accuracy_score(y_test, y_predict),2),
         f"| Test Recall for {title}: ", round(metrics.recall_score(y_test, y_predict),2))
my_roc_curve(pipeline, 0.5, 'Logistic Regression', X_train_ada, y_train_ada)
"""
<h4><center> <div style="background-color:violet;border-radius:10px; padding: 10px;">Decision Tree 🌳</div></center></h4>
"""
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import GridSearchCV

pipeline = Pipeline([('tree', DecisionTreeClassifier(random_state = 99))])
parms = {'tree__max_depth': [3, 7, 11],
        'tree__min_samples_leaf':[50, 100, 200]}

tree_model = GridSearchCV(pipeline, parms, cv = 5)
tree_model.fit(X_train_ada, y_train_ada)
# Best Estimator
tree_model.best_estimator_
my_roc_curve(tree_model, 0.5, 'Decision Tree', X_train_ada, y_train_ada)
"""
#### Overfitting!
"""
"""
<h4><center> <div style="background-color:violet;border-radius:10px; padding: 10px;">CatBoost 🌳➡🌳➡🌳➡🌳</div></center></h4>
"""
from catboost import CatBoostClassifier

cat_model = CatBoostClassifier(task_type = 'GPU', od_type = 'Iter')
# Creating evaluation set for catboost
X_train_cat, X_eval, y_train_cat, y_eval = train_test_split(X_train_ada, y_train_ada, stratify = y_train_ada,
                                                            random_state = 99)
cat_model.fit(X_train_cat, y_train_cat, use_best_model = True,
              eval_set = (X_eval, y_eval), verbose = 100, early_stopping_rounds = 50)
my_roc_curve(cat_model, 0.5, 'CatBoost', X_train_cat, y_train_cat)
probability = cat_model.predict_proba(X_test)
y_predict = (probability>=0.7).astype(int)[:,1]
matrix = metrics.confusion_matrix(y_test, y_predict)
matrix
plt.style.use('seaborn')
fig, axis = plt.subplots(1,1, figsize=(10, 6), constrained_layout = True)
axis = sns.heatmap(matrix, annot=True, fmt = '.0f', cbar=False, cmap='Blues',
                        linewidths=3, square=True, ax = axis, annot_kws={"fontsize":20})

axis.set_xlabel('Predicted', fontsize=14)
axis.set_ylabel('Actual', fontsize=14)
axis.set_xticklabels(['Fair','Fraud'], fontsize=12)
axis.set_yticklabels(['Fair','Fraud'], fontsize=12, rotation=0);
plt.title('CatBoost Confusion Matrix (Optimal Threshold)', fontsize = 16);
"""
> That looks good! We got a recall of 84% and 97% accuracy!
"""
"""
<h4><center> <div style="background-color:pink;border-radius:10px; padding: 10px;">If you like it, don't forget to upvote!</div></center></h4>
"""