"""
# Import all critical libraries
"""
#For working with dataframes and numbers

import pandas as pd
import numpy as np


#For Exploratory data analysis
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings('ignore')
import seaborn as sns
%matplotlib inline

#Modelling and eveluation
import sklearn.model_selection as ms
#from xgboost import XGBClassifier
from sklearn.ensemble import RandomForestClassifier
import sklearn.metrics as sklm
from sklearn.metrics import accuracy_score
from sklearn.metrics import mean_squared_error
from sklearn.metrics import roc_curve,auc
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import RandomForestRegressor


#import datasets
train = pd.read_csv("../input/train_technidus_clf.csv")
test = pd.read_csv("../input/test_technidus_clf.csv")
"""
# Exploratory data analysis
"""
"""
It is an approach of analysing data to summarise main characteristics often with visual methods.
"""
#Conduct mini data check/analysis

test.shape
train.head()
test.shape
#Check data for missing values
#train.isnull().sum()
train.isnull().sum()
#Check category split
#train.BikeBuyer.value_counts()
train.BikeBuyer.value_counts()
"""
#Conduct full EDA

Univariate analysis

At this stage, we explore variables one by one. Method to perform uni-variate analysis 
will depend on whether the variable type is categorical or continuous

Continuous Variables:- In case of continuous variables, we need to understand the central tendency 
and spread of the variable. These are measured using various statistical metrics visualization methods as shown below

Categorical Variables:- For categorical variables, we’ll use frequency table to understand distribution 
of each category. We can also read as percentage of values under each category. 
It can be be measured using two metrics, Count and Count% against each category. 
Bar chart can be used as visualization.
"""
"""
Bivariate analysis

Bi-variate Analysis finds out the relationship between two variables. Here, we look for association and disassociation between variables at a pre-defined significance level. We can perform bi-variate analysis for any combination of categorical and continuous variables. The combination can be: Categorical & Categorical, Categorical & Continuous and Continuous & Continuous. Different methods are used to tackle these combinations during analysis process.

Continuous & Continuous: While doing bi-variate analysis between two continuous variables, we should look at scatter plot. It is a nifty way to find out the relationship between two variables. The pattern of scatter plot indicates the relationship between variables. The relationship can be linear or non-linear.

Categorical & Categorical: To find the relationship between two categorical variables, we can use following methods:

Two-way table: We can start analyzing the relationship by creating a two-way table of count and count%. The rows represents the category of one variable and the columns represent the categories of the other variable. We show count or count% of observations available in each combination of row and column categories.
"""
#Univariate continuous- Histogram
train["AveMonthSpend"].plot.hist(color='blue',bins=50)
plt.show()
#Univariate categorical- Countplot(Barplot)
f, ax = plt.subplots(figsize=(8, 4))
sns.countplot("Occupation", data=train)
#Bivariate analysis 
#Continuous and categorical (Box plot)
sns.boxplot('BikeBuyer','AveMonthSpend', data=train)
#Bivariate analysis 
#Continuous and continuous
plt.scatter('AveMonthSpend',"YearlyIncome", data = train)
"""
# Feature Engineering
"""
"""
#Feature Engineering

Feature engineering is the science (and art) of extracting more information from existing data. 
You are not adding any new data here, but you are actually making the data you already have more useful.

Feature engineering itself can be divided in 2 steps:
* Variable transformation.
* Variable / Feature creation.

"""
#Example is generating Age column from Birth date variable
train.BirthDate.head(2)
train['Birthdate_int'] = train.BirthDate.str[-4:]
train  = train.dropna(subset=['Birthdate_int'])
train['Birthdate_int'] = train['Birthdate_int'].astype(int)
train['Birthdate_int'].head()
train['today_date'] = 1998
train['Age'] = train['today_date']-train['Birthdate_int']
train['Age'].head()
"""
# Outlier treatment
"""
"""
#Outlier treatment

Outlier is a commonly used terminology by analysts and data scientists as it needs close attention else 
it can result in wildly wrong estimations. Simply speaking, Outlier is an observation that appears far away and 
diverges from an overall pattern in a sample.


Most commonly used method to detect outliers is visualization. We use various visualization methods, 
like Box-plot, Histogram, Scatter Plot 

Deleting observations: We delete outlier values if it is due to data entry error, data processing error or 
outlier observations are very small in numbers. We can also use trimming at both ends to remove outliers.

Transforming and binning values: Transforming variables can also eliminate outliers.
Natural log of a value reduces the variation caused by extreme values. Binning is also a form of variable transformation. 

Imputing: Like imputation of missing values, we can also impute outliers. 
We can use mean, median, mode imputation methods. Before imputing values, we should analyse if it is natural outlier 
or artificial. 

Treat separately: If there are significant number of outliers, we should treat them separately in the statistical model. 
"""
#Deletion
#For example delete age greater than 90 years
train['Age'] = train['Age']<90
#Example of Binning
bins = [0,25,45,55,100]
labels = ["0-24","25-44", "45-55","56-100"]
train["Agebin"] = pd.cut(train.Age,bins = bins,labels=labels)
#train.Agebin.value_counts()
train[['Age','Agebin']].head()
"""
# Missing value treatment
"""
"""
#Missing value treatment

Deletion: Deletion methods are used when the nature of missing data is “Missing completely at random” else 
non random missing values can bias the model output. 

Mean/ Mode/ Median Imputation: Imputation is a method to fill in the missing values with estimated ones. 
Similar case Imputation

Prediction Model:  Prediction model is one of the sophisticated method for handling missing data. 

KNN Imputation: In this method of imputation, the missing values of an attribute are imputed using the given number of attributes that are most similar to the attribute whose values are missing. 
"""
#Deletion
#Pairwise
#train = train.dropna(axis=1,how='any')
#listwise
#train = train.dropna(axis=1,how='all')

#Mean/mode imputation
train['Age'].fillna(train['Age'].mean(), inplace=True) 
train['Age'].fillna(train['Age'].mode()[0], inplace=True) 
"""
# Feature Selection
"""
"""
#Feature Selection

Filter methods are generally used as a preprocessing step. 
The selection of features is independent of any machine learning algorithms.
Instead, features are selected on the basis of their scores in various statistical tests 
for their correlation with the outcome variable. The correlation is a subjective term here

Forward Selection: Forward selection is an iterative method in which we start with having no feature in the model. 
In each iteration, we keep adding the feature which best improves our model till 
an addition of a new variable does not improve the performance of the model.


"""
corr= train.corr()
#corr
f, ax = plt.subplots(figsize=(10, 5))
sns.heatmap(corr,cmap='coolwarm',linewidths=2.0, annot=True)
"""
# Modelling
"""
#Convert selected variables to array which scikit learn recognises
X = train[['NumberCarsOwned',
          'NumberChildrenAtHome',
          'YearlyIncome','CountryRegionName']]
y = train['BikeBuyer']
Xb=test[['NumberCarsOwned',
          'NumberChildrenAtHome',
          'YearlyIncome','CountryRegionName']]
#Convert categorical variables to numerical through one-hotencoding
X=pd.get_dummies(X)
Xb=pd.get_dummies(Xb)
"""
# Validation
"""
"""
#Holdout method

To avoid the resubstitution error, the data is split into two different datasets labeled as a training and a testing dataset. 
This can be a 60/40 or 70/30 or 80/20 split. This technique is called the hold-out validation technique. 
In this case, there is a likelihood that uneven distribution of different classes of data is found in training and test dataset. To fix this, the training and test dataset is created with equal distribution of different classes of data. This process is called stratification.

"""
#Holdout 30%
from sklearn.model_selection import train_test_split
x_train, x_cv, y_train, y_cv = train_test_split(X,y, test_size =0.3)
#Build various models
#Logistic regression
from sklearn.linear_model import LogisticRegression
model=LogisticRegression()
#model = LogisticRegression()
model.fit(x_train, y_train)

#Evaluate logistic regression
pred_cv = model.predict(x_cv)
score = accuracy_score(y_cv,pred_cv)
print('accuracy_score',score)
#Build Random forest
from sklearn.ensemble import RandomForestClassifier
model1 = RandomForestClassifier(random_state=1, max_depth=110,n_estimators= 1400,class_weight="balanced")
model1.fit(X,y)
#Evaluate RF
pred_cv = model.predict(x_cv)
score = accuracy_score(y_cv,pred_cv)
print('accuracy_score',score)
#Save for submission
test['BikeBuyer']=model1.predict(Xb)
test['CustomerID']= test['CustomerID']
test['BikeBuyer'] = test['BikeBuyer'].astype(int)
test[['CustomerID','BikeBuyer']].head(10)
test[['CustomerID','BikeBuyer']].to_csv('perfrect_score2.csv',index= False)