# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     name: python3
# ---

# + [markdown] id="view-in-github" colab_type="text"
# <a href="https://colab.research.google.com/github/abhi-gm/Machine-Learning-Workshop/blob/Classical-Machine-Learning/1_Linear_regression.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>

# + [markdown] id="QQg3OSN80oFU"
# # **Linear regression**
# ## 1. **Abstract**
#
#
# The insurance risk are important point of estimation for the insurance company. The aim of the notebook is to find the factors affecting the this and creating a regression model to predict this.
#
# ## Linear Models
#
# Linear regression predicts the response variable $y$ assuming it has a linear relationship with predictor variable(s) $x$ or $x_1, x_2, ,,, x_n$.
#
# $$y = \beta_0 + \beta_1 x + \varepsilon .$$
#
# *Simple* regression use only one predictor variable $x$. *Mulitple* regression uses a set of predictor variables $x_1, x_2, ,,, x_n$.
#
# The *response variable* $y$ is also called the regressand, forecast, dependent or explained variable. The *predictor variable* $x$ is also called the regressor, independent or explanatory variable.
#
# The parameters $\beta_0$ and $\beta_1$ determine the intercept and the slope of the line respectively. The intercept $\beta_0$ represents the predicted value of $y$ when $x=0$. The slope $\beta_1$ represents the predicted increase in $Y$ resulting from a one unit increase in $x$.
#
# Note that the regression equation is just our famliar equation for a line with an error term.
#
# The equation for a line:  
# $$ Y = bX + a $$
#
# $$y = \beta_0 + \beta_1 x $$
#
# The equation for a line with an error term:  
#
# $$ Y = bX + a + \varepsilon $$
#
# $$y = \beta_0 + \beta_1 x + \varepsilon .$$
#
# - $b$ = $\beta_1$ = slope
# - $a$ = $\beta_0$ = $Y$ intercept
# - $\varepsilon$ = error term
#
#
# We can think of each observation $y_i$ consisting of the systematic or explained part of the model, $\beta_0+\beta_1x_i$, and the random *error*, $\varepsilon_i$.
#
#
#

# + [markdown] id="JWwRbuTo06M8"
# ## **2. Importing necessary libraries**

# + id="J4h5qR_10XK0"
import numpy as np
from matplotlib import pyplot
import pandas as pd
import seaborn as sns
from sklearn.ensemble import RandomForestRegressor
import matplotlib.pylab as plt

# + [markdown] id="ATNhqw5wGN_f"
# ## **3. Data Loading and Preprocessing**
#
# Data is stored in the our github

# + id="QiPf7PTW1BAF"
import pandas as pd
url = 'https://raw.githubusercontent.com/abhi-gm/Machine-Learning-Workshop/main/Datasets/insurance.csv'
data = pd.read_csv(url, error_bad_lines=False)

# + colab={"base_uri": "https://localhost:8080/", "height": 204} id="x2As02pj4_s1" outputId="8eff3e67-84b9-4185-d39d-a9f245a6de76"
data.head()

# + [markdown] id="KWIj1rKDGeQQ"
# ## **3.1. Encoding the sex and smoker**
#
# ### **Sex**
#
# Male = 0
#
# Female = 1
#
# ### **Smoker**
#
# Yes = 1
#
# No = 0

# + colab={"base_uri": "https://localhost:8080/", "height": 676} id="ZF_kZsKBGeby" outputId="ff96e022-a5a2-4bc5-aa24-4983e1b38699"
data['sex'] = data.sex.map({'male':0, 'female':1})

data['smoker'] = data.smoker.map({'no':0, 'yes':1})

data.head(20)

# + [markdown] id="DNiqQes-LRwZ"
# Identfying the Unique values in the Region column

# + colab={"base_uri": "https://localhost:8080/"} id="LbiN02IuHKMh" outputId="0936c6b8-3fea-4a3d-a651-d63098273e1c"
data.region.unique()

# + id="76bLnvIfHNuH"
import numpy as np
from sklearn.preprocessing import OneHotEncoder

# + colab={"base_uri": "https://localhost:8080/", "height": 206} id="Zk0lKxG8HUl2" outputId="2dce35d4-8781-4643-8719-5685c04f014c"
# creating instance of one-hot-encoder
enc = OneHotEncoder()
# passing bridge-types-cat column (label encoded values of bridge_types)
enc_df = pd.DataFrame(enc.fit_transform(data[['region']]).toarray())
enc_df
enc_df.columns = ['northeast','northwest','southeast','southwest'] 

enc_df.apply(np.int64)
data =data.join(enc_df)
data=data.drop(['region'],axis=1)
data.head()

# + [markdown] id="lP2CdM5vHnd9"
# ## **3.3. Normalizing the data**

# + id="Txma206iHipV"
from sklearn import preprocessing

# Create x, where x the 'scores' column's values as floats
x = data[['charges','bmi','age']].values.astype(float)

# Create a minimum and maximum processor object
min_max_scaler = preprocessing.MinMaxScaler()

# Create an object to transform the data to fit minmax processor
x_scaled = min_max_scaler.fit_transform(x)

# Run the normalizer on the dataframe
data[['charges','bmi','age']] = pd.DataFrame(x_scaled)

# + colab={"base_uri": "https://localhost:8080/", "height": 676} id="gb-E4DcYHudK" outputId="14c028d7-a3e0-483d-cd6c-f161e93a3dd0"
#looking at data head after adding dummy variables and nromalizing
data.head(20)

# + colab={"base_uri": "https://localhost:8080/", "height": 362} id="lE-DLZjFH1vu" outputId="0e4e3799-e185-481f-9ff7-ef7401123184"
#finding the correlation between all the features in the data
data.corr()

# + colab={"base_uri": "https://localhost:8080/", "height": 449} id="bzK3BurtH5mV" outputId="09740940-d867-42a5-c5f9-afd88bb83970"
#plotting the heat map of the correlation
plt.figure(figsize=(20,7))
sns.heatmap(data.corr(), annot=True)

# + [markdown] id="O5HSlrAWH_Vw"
# ## **4. Ordinary Least Squares**
# Ordinary Least Squares (OLS) is the most common estimation method for linear models

# + colab={"base_uri": "https://localhost:8080/", "height": 655} id="3vw1-9S1H8nF" outputId="98a575fa-35fa-4dfa-c29b-6df8e8acc735"
#Using OLS for finding the p value and t statistics 
import statsmodels.api as sm

model = sm.OLS(data['charges'], data[['age', 'sex', 'bmi', 'children', 'smoker','northeast',
       'northwest', 'southeast', 'southwest']]).fit()

# Print out the statistics
model.summary()

# + [markdown] id="7JV4kcKGxMDC"
# ### OLS after removing insignificant feature "sex"

# + colab={"base_uri": "https://localhost:8080/", "height": 579} id="SzoZiR0CDPoQ" outputId="20253f7a-2dac-4ec6-dfc2-5973f56873ae"
model = sm.OLS(data['charges'], data[['age','bmi', 'children', 'smoker','northeast',
       'northwest', 'southeast', 'southwest']]).fit()

# Print out the statistics
model.summary()

# + [markdown] id="mXAhK24KIKeW"
# ## **5. Pair Plot**

# + colab={"base_uri": "https://localhost:8080/", "height": 1000} id="P8Liot0SIG_Z" outputId="27205f41-3172-40ea-e1a0-df8153a46604"
#pair plot
plt.figure(figsize=(12,10))
sns.pairplot(data)

# + [markdown] id="21STMwKzIWNb"
# ## **6. Train ,Validation and Test split**
#
# Data is split into 3 parts
#
# Taining data set = 80.75%
#
# Validation data set = 14.25%
#
# Test data set = 5%

# + id="9AlAFk-ZIZRr"
from sklearn.model_selection import  train_test_split

X = data[ ['age', 'bmi', 'children', 'smoker','northeast','northwest', 'southeast', 'southwest']]

y = data['charges']

X_t, X_test, y_t, y_test = train_test_split(X, y, test_size=0.05, random_state=1)

X_train, X_val, y_train, y_val = train_test_split(X_t, y_t, test_size=0.15, random_state=1)

# + [markdown] id="V73zPAlwIeii"
# ## **7. Linear Regression**

# + id="OqLbVlc_Ib62"
from sklearn.model_selection import  train_test_split
from sklearn.metrics import r2_score 
from sklearn.linear_model import LinearRegression
from sklearn import datasets, linear_model
from sklearn.metrics import mean_squared_error, r2_score

# + id="Bfis7mzhIlDu"
# Create linear regression object
regr = linear_model.LinearRegression()

# Train the model using the training sets
model = regr.fit(X_train,y_train)



# + colab={"base_uri": "https://localhost:8080/"} id="4EAWRWveJF2T" outputId="4726705f-8759-469d-b2bf-e1fc78d4ed64"
#Validation set used for tunning the hyperparmeter


# Make predictions using the Validation sets
y_pred = regr.predict(X_val)


# The mean squared error
print('Mean squared error: %.2f'% mean_squared_error(y_val, y_pred))
# The coefficient of determination: 1 is perfect prediction
print('Coefficient of determination: %.2f'% r2_score(y_val, y_pred))

# + colab={"base_uri": "https://localhost:8080/"} id="sh71aKv6IrRm" outputId="c3ade4ac-28af-46ff-e4e5-dff47a81eb66"
# Make predictions using the testing set
y_pred = regr.predict(X_test)


# The mean squared error
print('Mean squared error on the Test dataset : %.2f'% mean_squared_error(y_test, y_pred))
# The coefficient of determination: 1 is perfect prediction
print('R2 score for Test dataset : %.2f'% r2_score(y_test, y_pred))

# + [markdown] id="YIyWs3quI63t"
# ## **7. Conclusion**
# The linear regression model is trained and interpreted based on p-value and those features with a p-value above 0.05 dropped as they do not affect.

# + [markdown] id="ccJkMOp-I-F6"
# ## **8. Refrence**
# https://www.statsmodels.org/stable/regression.html
#
# Copyright 2020 <NAME>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

StarcoderdataJupyterScriptsDedupFiltered