"""

# About the Notebook

Hi Everyone!!, This notebook  is easily understandable to the beginner. This verbosity tries to explain everything I could possibly know. Once you get through the kernel, you can find this useful and straightforward. I attempted to explain things as simple as possible.

In this notebook, I extensively use Feature Engineering on Numerical Variable which can be further classified into discrete and continuous variables. 

Keep Learning,

Santosh

"""
"""
Numerical variables

The values of a numerical variable are numbers. They can be further classified into discrete and continuous variables. Discrete numerical variable

A variable which values are whole numbers (counts) is called discrete. For example, the number of items bought by a customer in a supermarket is discrete. The customer can buy 1, 25, or 50 items, but not 3.7 items. It is always in the form of integer number not to float number. The following are examples of discrete variables:

Number of active bank accounts of a borrower (1, 4, 7, ...)
Number of pets in the family
Number of children in the family
number of cars 

Continuous numerical variable

A variable that may contain any value within some range is called continuous. For example, the total amount paid by a customer in a supermarket is continuous. The customer can pay, GBP 20.5, GBP 13.10, GBP 83.20 and so on. Other examples of continuous variables are:

House price (in principle, it can take any value) (GBP 350000, 57000, 1000000, ...)
Time spent surfing a website (3.4 seconds, 5.10 seconds, ...)
Total debt as percentage of total income in the last month (0.2, 0.001, 0, 0.75, ...)


"""
import pandas as pd
import numpy as np

import matplotlib.pyplot as plt
%matplotlib inline

import os
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

# Any results you write to the current directory are saved as output.
# let's load the dataset with just a few columns and a few rows
# to speed up  things

use_cols = [
    'loan_amnt', 'int_rate', 'annual_inc', 'open_acc', 'loan_status',
    'open_il_12m'
]

data = pd.read_csv(
    '../input/lending-club-loan-data/loan.csv', usecols=use_cols).sample(
        10000, random_state=44)  # set a seed for reproducibility

data.head()
"""
Continuous Variables¶
"""
# let's look at the values of the variable loan_amnt
# this is the amount of money requested by the borrower
# in US dollars

data.loan_amnt.unique()
 # let's make an histogram to get familiar with the
# distribution of the variable

fig = data.loan_amnt.hist(bins=50)
fig.set_title('Loan Amount Requested')
fig.set_xlabel('Loan Amount')
fig.set_ylabel('Number of Loans')
"""


The values of the variable vary across the entire range of the variable. This is characteristic of continuous variables.

The taller bars correspond to loan sizes of 10000, 15000, 20000, and 35000. There are more loans disbursed for those loan amount values. This indicates that most people tend to ask for these loan amounts. Likely, these particular loan amounts are pre-determined and offered as such in the Lending Club website.

Less frequent loan values, like 23,000 or 33,000 could be requested by people who require a specific amount of money for a definite purpose.

"""
# let's do the same exercise for the variable interest rate,
# which is charged by lending club to the borrowers

data.int_rate.unique()
# let's make an histogram to get familiar with the
# distribution of the variable

fig = data.int_rate.hist(bins=30)
fig.set_title('Interest Rate')
fig.set_xlabel('Interest Rate')
fig.set_ylabel('Number of Loans')
"""
Here , we saw that the values of the variable vary continuously across the variable range.
"""
# and now,let's explore the income declared by the customers,
# that is, how much they earn yearly.

fig = data.annual_inc.hist(bins=100)
fig.set_xlim(0, 400000)
fig.set_title("Customer's Annual Income")
fig.set_xlabel('Annual Income')
fig.set_ylabel('Number of Customers')
"""
The majority of salaries are concentrated towards values in the range 30-70 k, with only a few customers earning higher salaries. Again, the values of the variable, vary continuosly across the variable range.


**Discrete Variables**

Let's explore the variable "Number of open credit lines in the borrower's credit file" (open_acc in the dataset). This is, the total number of credit items (for example, credit cards, car loans, mortgages, etc) that is known for that borrower. By definition it is a discrete variable, because a borrower can have 1 credit card, but not 3.5 credit cards.

"""
# let's inspect the values of the variable

data.open_acc.dropna().unique()
# let's make an histogram to get familiar with the
# distribution of the variable

fig = data.open_acc.hist(bins=100)
fig.set_xlim(0, 30)
fig.set_title('Number of open accounts')
fig.set_xlabel('Number of open accounts')
fig.set_ylabel('Number of Customers')
"""


Histograms of discrete variables have this typical broken shape, as not all the values within the variable range are present in the variable. As I said, the customer can have 3 credit cards, but not 3,5 credit cards.

Let's look at another example of a discrete variable in this dataset: Number of installment accounts opened in past 12 months (open_il_12m in the dataset). Installment accounts are those that at the moment of acquiring them, there is a set period and amount of repayments agreed between the lender and borrower. An example of this is a car loan, or a student loan. The borrower knows that they are going to pay a certain, fixed amount over for example 36 months.

"""
# let's inspect the variable values

data.open_il_12m.unique()
# let's make an histogram to get familiar with the
# distribution of the variable

fig = data.open_il_12m.hist(bins=50)
fig.set_title('Number of installment accounts opened in past 12 months')
fig.set_xlabel('Number of installment accounts opened in past 12 months')
fig.set_ylabel('Number of Borrowers')
"""
The majority of the borrowers have none or 1 installment account, with only a few borrowers having more than 2.
"""
"""

**A variation of discrete variables: the binary variable******************

Binary variables, are discrete variables, that can take only 2 values, therefore binary.

In the next cells I will create an additional variable, called defaulted, to capture the number of loans that have defaulted. A defaulted loan is a loan that a customer has failed to re-pay and the money is lost.

The variable takes the values 0 where the loans are ok and being re-paid regularly, or 1, when the borrower has confirmed that will not be able to re-pay the borrowed amount.

"""
# let's inspect the values of the variable loan status

data.loan_status.unique()
# let's create one additional variable called defaulted.
# This variable indicates if the loan has defaulted, which means,
# if the borrower failed to re-pay the loan, and the money
# is deemed lost.

data['defaulted'] = np.where(data.loan_status.isin(['Default']), 1, 0)
data.defaulted.mean()
# the new variable takes the value of 0
# if the loan is not defaulted

data.head(10)
# the new variable takes the value 1 for loans that
# are defaulted

data[data.loan_status.isin(['Default'])].head()
# A binary variable, can take 2 values. For example,
# the variable defaulted that we just created:
# either the loan is defaulted (1) or not (0)

data.defaulted.unique()
# let's make a histogram, although histograms for
# binary variables do not make a lot of sense

fig = data.defaulted.hist()
fig.set_xlim(0, 2)
fig.set_title('Defaulted accounts')
fig.set_xlabel('Defaulted')
fig.set_ylabel('Number of Loans')
"""


As we can see, the variable shows only 2 values, 0 and 1, and the majority of the loans are ok.

That is all for this demonstration. I hope you enjoyed the notebook, and see you in the next one.

"""