# Importing Pandas, a data processing and CSV file I/O libraries
import os 
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns # Seaborn is a Python data visualization library based on matplotlib. 
%matplotlib inline   

"""
Explanation: Introduction to Linear Regression
Learning Objectives

Analyze a Pandas Dataframe.
Create Seaborn plots for Exploratory Data Analysis.
Train a Linear Regression Model using Scikit-Learn.

Introduction
This lab is an introduction to linear regression using Python and Scikit-Learn.  This lab serves as a foundation for more complex algorithms and machine learning models that you will encounter in the course. We will train a linear regression model to predict housing price.
Each learning objective will correspond to a #TODO in the student lab notebook -- try to complete that notebook first before reviewing this solution notebook.
Import Libraries
End of explanation
"""


# Next, we read the dataset into a Pandas dataframe.
df_USAhousing = pd.read_csv('../USA_Housing_toy.csv')

# Show the first five row.

df_USAhousing.head()

"""
Explanation: Load the Dataset
We will use the USA housing prices dataset found on Kaggle.  The data contains the following columns:

'Avg. Area Income': Avg. Income of residents of the city house is located in.
'Avg. Area House Age': Avg Age of Houses in same city
'Avg. Area Number of Rooms': Avg Number of Rooms for Houses in same city
'Avg. Area Number of Bedrooms': Avg Number of Bedrooms for Houses in same city
'Area Population': Population of city house is located in
'Price': Price that the house sold at
'Address': Address for the house
End of explanation
"""


# The isnull() method is used to check and manage NULL values in a data frame.
df_USAhousing.isnull().sum()

# Pandas describe() is used to view some basic statistical details of a data frame or a series of numeric values.
df_USAhousing.describe()

# Pandas info() function is used to get a concise summary of the dataframe.
df_USAhousing.info()

"""
Explanation: Let's check for any null values.
End of explanation
"""


print(df_USAhousing,5) # TODO 1

"""
Explanation: Let's take a peek at the first and last five rows of the data for all columns.
End of explanation
"""


# Plot pairwise relationships in a dataset. By default, this function will create a grid of Axes such that each numeric variable in data will be
# shared across the y-axes across a single row and the x-axes across a single column.
sns.pairplot(df_USAhousing)

# It is used basically for univariant set of observations and visualizes it through a histogram i.e. only one observation
# and hence we choose one particular column of the dataset.
sns.displot(df_USAhousing['Price'])

# The heatmap is a way of representing the data in a 2-dimensional form. The data values are represented as colors in the graph.
# The goal of the heatmap is to provide a colored visual summary of information.
sns.heatmap(df_USAhousing.corr()) # TODO 2

"""
Explanation: Exploratory Data Analysis (EDA)
Let's create some simple plots to check out the data!
End of explanation
"""


X = df_USAhousing[['Avg. Area Income', 'Avg. Area House Age', 'Avg. Area Number of Rooms',
               'Avg. Area Number of Bedrooms', 'Area Population']]
y = df_USAhousing['Price']

"""
Explanation: Training a Linear Regression Model
Regression is a supervised machine learning process.  It is similar to classification, but rather than predicting a label, we try to predict a continuous value.   Linear regression defines the relationship between a target variable (y) and a set of predictive features (x).  Simply stated, If you need to predict a number, then use regression. 
Let's now begin to train our regression model! We will need to first split up our data into an X array that contains the features to train on, and a y array with the target variable, in this case the Price column. We will toss out the Address column because it only has text info that the linear regression model can't use.
X and y arrays
Next, let's define the features and label.  Briefly, feature is input; label is output. This applies to both classification and regression problems.
End of explanation
"""


# Import train_test_split function from sklearn.model_selection
from sklearn.model_selection import train_test_split

# Split up the data into a training set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=101)

"""
Explanation: Train - Test - Split
Now let's split the data into a training set and a testing set. We will train out model on the training set and then use the test set to evaluate the model.  Note that we are using 40% of the data for testing.  
What is Random State?
If an integer for random state is not specified in the code, then every time the code is executed, a new random value is generated and the train and test datasets will have different values each time.  However, if a fixed value is assigned -- like random_state = 0 or 1 or 101 or any other integer, then no matter how many times you execute your code the result would be the same, e.g. the same values will be in the train and test datasets.  Thus, the random state that you provide is used as a seed to the random number generator. This ensures that the random numbers are generated in the same order.
End of explanation
"""


# Import LinearRegression function from sklearn.model_selection
from sklearn.linear_model import LinearRegression

# LinearRegression fits a linear model with coefficients w = (w1, …, wp) to minimize the residual sum of squares between the observed targets
# in the dataset, and the targets predicted by the linear approximation.
lm = LinearRegression()

# Train the Linear Regression Classifer
lm.fit(X_train,y_train) # TODO 3

"""
Explanation: Creating and Training the Model
End of explanation
"""


# print the intercept
print(lm.intercept_)

# Pandas DataFrame is two-dimensional size-mutable, potentially heterogeneous tabular data structure with labeled axes (rows and columns).
coeff_df = pd.DataFrame(lm.coef_,X.columns,columns=['Coefficient'])
coeff_df

"""
Explanation: Model Evaluation
Let's evaluate the model by checking out it's coefficients and how we can interpret them.
End of explanation
"""


# Predict values based on linear model object.
predictions = lm.predict(X_test)

# Scatter plots are widely used to represent relation among variables and how change in one affects the other.
plt.scatter(y_test,predictions)

"""
Explanation: Interpreting the coefficients:

Holding all other features fixed, a 1 unit increase in Avg. Area Income is associated with an increase of \$21.52 .
Holding all other features fixed, a 1 unit increase in Avg. Area House Age is associated with an increase of \$164883.28 .
Holding all other features fixed, a 1 unit increase in Avg. Area Number of Rooms is associated with an increase of \$122368.67 .
Holding all other features fixed, a 1 unit increase in Avg. Area Number of Bedrooms is associated with an increase of \$2233.80 .
Holding all other features fixed, a 1 unit increase in Area Population is associated with an increase of \$15.15 .

Predictions from our Model
Let's grab predictions off our test set and see how well it did!
End of explanation
"""


# It is used basically for univariant set of observations and visualizes it through a histogram i.e. only one observation
# and hence we choose one particular column of the dataset.
sns.displot((y_test-predictions),bins=50);

"""
Explanation: Residual Histogram
End of explanation
"""


# Importing metrics from sklearn
from sklearn import metrics

# Show the values of MAE, MSE, RMSE
print('MAE:', metrics.mean_absolute_error(y_test, predictions))
print('MSE:', metrics.mean_squared_error(y_test, predictions))
print('RMSE:', np.sqrt(metrics.mean_squared_error(y_test, predictions)))

"""
Explanation: Regression Evaluation Metrics
Here are three common evaluation metrics for regression problems:
Mean Absolute Error (MAE) is the mean of the absolute value of the errors:
$$\frac 1n\sum_{i=1}^n|y_i-\hat{y}_i|$$
Mean Squared Error (MSE) is the mean of the squared errors:
$$\frac 1n\sum_{i=1}^n(y_i-\hat{y}_i)^2$$
Root Mean Squared Error (RMSE) is the square root of the mean of the squared errors:
$$\sqrt{\frac 1n\sum_{i=1}^n(y_i-\hat{y}_i)^2}$$
Comparing these metrics:

MAE is the easiest to understand, because it's the average error.
MSE is more popular than MAE, because MSE "punishes" larger errors, which tends to be useful in the real world.
RMSE is even more popular than MSE, because RMSE is interpretable in the "y" units.

All of these are loss functions, because we want to minimize them.
End of explanation
"""


