# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# + [markdown] id="view-in-github" colab_type="text"
# <a href="https://colab.research.google.com/github/brucebra000/DS-Unit-2-Linear-Models/blob/master/U2S1A4_regression_classification_4_assignment.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>

# + [markdown] id="YrR2OL0e6QoR" colab_type="text"
# Lambda School Data Science
#
# *Unit 2, Sprint 1, Module 4*
#
# ---

# + [markdown] colab_type="text" id="7IXUfiQ2UKj6"
# # Logistic Regression
#
#
# ## Assignment
#
# - [ ] Watch Aaron's [video #1](https://www.youtube.com/watch?v=pREaWFli-5I) (12 minutes) & [video #2](https://www.youtube.com/watch?v=bDQgVt4hFgY) (9 minutes) to learn about the mathematics of Logistic Regression.
# - [ ] [Sign up for a Kaggle account](https://www.kaggle.com/), if you don’t already have one. Go to our Kaggle InClass competition website. You will be given the URL in Slack. Go to the Rules page. Accept the rules of the competition.
# - [ ] Do train/validate/test split with the Tanzania Waterpumps data.
# - [ ] Begin with baselines for classification.
# - [ ] Use scikit-learn for logistic regression.
# - [ ] Get your validation accuracy score.
# - [ ] Submit your predictions to our Kaggle competition. (Go to our Kaggle InClass competition webpage. Use the blue **Submit Predictions** button to upload your CSV file. Or you can use the Kaggle API to submit your predictions.)
# - [ ] Commit your notebook to your fork of the GitHub repo.
#
# ---
#
#
# ## Stretch Goals
#
# - [ ] Add your own stretch goal(s) !
# - [ ] Clean the data. For ideas, refer to [The Quartz guide to bad data](https://github.com/Quartz/bad-data-guide),  a "reference to problems seen in real-world data along with suggestions on how to resolve them." One of the issues is ["Zeros replace missing values."](https://github.com/Quartz/bad-data-guide#zeros-replace-missing-values)
# - [ ] Make exploratory visualizations.
# - [ ] Do one-hot encoding. For example, you could try `quantity`, `basin`, `extraction_type_class`, and more. (But remember it may not work with high cardinality categoricals.)
# - [ ] Do [feature scaling](https://scikit-learn.org/stable/modules/preprocessing.html).
# - [ ] Get and plot your coefficients.
# - [ ] Try [scikit-learn pipelines](https://scikit-learn.org/stable/modules/compose.html).
#
# ---
#
# ## Data Dictionary 
#
# ### Features
#
# Your goal is to predict the operating condition of a waterpoint for each record in the dataset. You are provided the following set of information about the waterpoints:
#
# - `amount_tsh` : Total static head (amount water available to waterpoint)
# - `date_recorded` : The date the row was entered
# - `funder` : Who funded the well
# - `gps_height` : Altitude of the well
# - `installer` : Organization that installed the well
# - `longitude` : GPS coordinate
# - `latitude` : GPS coordinate
# - `wpt_name` : Name of the waterpoint if there is one
# - `num_private` :  
# - `basin` : Geographic water basin
# - `subvillage` : Geographic location
# - `region` : Geographic location
# - `region_code` : Geographic location (coded)
# - `district_code` : Geographic location (coded)
# - `lga` : Geographic location
# - `ward` : Geographic location
# - `population` : Population around the well
# - `public_meeting` : True/False
# - `recorded_by` : Group entering this row of data
# - `scheme_management` : Who operates the waterpoint
# - `scheme_name` : Who operates the waterpoint
# - `permit` : If the waterpoint is permitted
# - `construction_year` : Year the waterpoint was constructed
# - `extraction_type` : The kind of extraction the waterpoint uses
# - `extraction_type_group` : The kind of extraction the waterpoint uses
# - `extraction_type_class` : The kind of extraction the waterpoint uses
# - `management` : How the waterpoint is managed
# - `management_group` : How the waterpoint is managed
# - `payment` : What the water costs
# - `payment_type` : What the water costs
# - `water_quality` : The quality of the water
# - `quality_group` : The quality of the water
# - `quantity` : The quantity of water
# - `quantity_group` : The quantity of water
# - `source` : The source of the water
# - `source_type` : The source of the water
# - `source_class` : The source of the water
# - `waterpoint_type` : The kind of waterpoint
# - `waterpoint_type_group` : The kind of waterpoint
#
# ### Labels
#
# There are three possible values:
#
# - `functional` : the waterpoint is operational and there are no repairs needed
# - `functional needs repair` : the waterpoint is operational, but needs repairs
# - `non functional` : the waterpoint is not operational
#
# --- 
#
# ## Generate a submission
#
# Your code to generate a submission file may look like this:
#
# ```python
# # estimator is your model or pipeline, which you've fit on X_train
#
# # X_test is your pandas dataframe or numpy array, 
# # with the same number of rows, in the same order, as test_features.csv, 
# # and the same number of columns, in the same order, as X_train
#
# y_pred = estimator.predict(X_test)
#
#
# # Makes a dataframe with two columns, id and status_group, 
# # and writes to a csv file, without the index
#
# sample_submission = pd.read_csv('sample_submission.csv')
# submission = sample_submission.copy()
# submission['status_group'] = y_pred
# submission.to_csv('your-submission-filename.csv', index=False)
# ```
#
# If you're working locally, the csv file is saved in the same directory as your notebook.
#
# If you're using Google Colab, you can use this code to download your submission csv file.
#
# ```python
# from google.colab import files
# files.download('your-submission-filename.csv')
# ```
#
# ---

# + id="S0pqQbAz_Dxi" colab_type="code" colab={}
import sys
import warnings
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from sklearn.linear_model import LinearRegression
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression

# + colab_type="code" id="o9eSnDYhUGD7" colab={}
# %%capture
# If you're on Colab:
if 'google.colab' in sys.modules:
    DATA_PATH = 'https://raw.githubusercontent.com/LambdaSchool/DS-Unit-2-Applied-Modeling/master/data/'
    # !pip install category_encoders==2.*

# If you're working locally:
else:
    DATA_PATH = '../data/'

# + colab_type="code" id="ipBYS77PUwNR" colab={}
# Ignore this Numpy warning when using Plotly Express:
# FutureWarning: Method .ptp is deprecated and will be removed in a future version. Use numpy.ptp instead.
warnings.filterwarnings(action='ignore', category=FutureWarning, module='numpy')

# + colab_type="code" id="QJBD4ruICm1m" colab={}
# Read the Tanzania Waterpumps data
# train_features.csv : the training set features
# train_labels.csv : the training set labels
# test_features.csv : the test set features
# sample_submission.csv : a sample submission file in the correct format
    
train_features = pd.read_csv(DATA_PATH+'waterpumps/train_features.csv')
train_labels = pd.read_csv(DATA_PATH+'waterpumps/train_labels.csv')
test_features = pd.read_csv(DATA_PATH+'waterpumps/test_features.csv')
#sample_submission = pd.read_csv(DATA_PATH+'waterpumps/waterpumps/sample_submission.csv')

assert train_features.shape == (59400, 40)
assert train_labels.shape == (59400, 2)
assert test_features.shape == (14358, 40)
#assert sample_submission.shape == (14358, 2)

# + colab_type="code" id="2Amxyx3xphbb" colab={"base_uri": "https://localhost:8080/", "height": 435} outputId="a57715ad-9b7a-494f-a04a-f0d90a617d99"
#train_features
print(train_features.shape)
train_features.head()

# + id="6G9mpbAOA1RC" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 224} outputId="348a0172-2ea4-472f-9513-8c8a788c6c47"
#train_labels
print(train_labels.shape)
train_labels.head()

# + id="qNoRa3zqA8dg" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 452} outputId="d7475c33-18fa-42e1-aa26-cf92c723eaeb"
#test_features
print(test_features.shape)
test_features.head()

# + id="q42khj8-GAmp" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 435} outputId="88634d14-e9f5-4595-ce75-3366a3809811"
#Putting the 'status_group' column in the training set
train_merge = pd.merge(train_features, train_labels)
print(train_merge.shape)
train_merge.head()

# + id="fdWvHWKAQkxf" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 417} outputId="f10f4d47-cf98-4134-a671-2040086139d6"
#Converting status_group into int
train_merge['status_group'] = train_merge['status_group'].replace({'functional': 0, 'functional needs repair': 1, 'non functional': 2})
train_merge.head()

# + id="cbUFvZlRHnTB" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 767} outputId="c559e0bd-7b03-4c39-d038-ed478133f8e3"
#Checking for null values
train_merge.isnull().sum()

# + id="8qOeNpt9BqQQ" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 35} outputId="48f7702d-ac33-45e3-e3b7-4172b6c9875e"
#Making a train and val set.
train, val = train_test_split(train_merge, random_state = 1)
train.shape, val.shape

# + id="xq7Vg3eyE5lX" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 88} outputId="c3070c2d-8fc1-4626-887f-bcd04dfea3e5"
#Baseline for classification
target = 'status_group'
y_train = train[target]
y_train.value_counts(normalize = True)

# + id="NvGMDPXyL8VV" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 1000} outputId="6662ccf0-cf24-41e9-fed6-234fba429014"
majority = y_train.mode()[0]
y_pred = [majority] * len(y_train)
y_pred

# + id="iJuSCOvpMg-A" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 35} outputId="d30686d7-8286-4da2-d5e8-0024f36c3531"
accuracy_score(y_train, y_pred)

# + id="9QJsKEK1MyC9" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 35} outputId="428c03f8-68c3-4ba1-97e0-dbf16ff7a22a"
y_val = val[target]
y_pred = [majority] * len(y_val)
accuracy_score(y_val, y_pred)

# + id="rdhjNxqNJZFt" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 300} outputId="9c64d0a4-f8af-4517-f6e6-459c35a73d65"
#Linear Regression
train.describe()

# + id="T68QoHhBN1b_" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 53} outputId="b8ca42db-acc0-486a-b894-f0af3c01ace2"
linear_reg = LinearRegression()

features = ['amount_tsh', 'population', 'construction_year']
x_train = train[features]
x_val = val[features]

imputer = SimpleImputer()
x_train_imputed = imputer.fit_transform(x_train)
x_val_imputed = imputer.transform(x_val)

linear_reg.fit(x_train_imputed, y_train)
linear_reg.predict(x_val_imputed)

# + id="cG03J4iGRqq2" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 91} outputId="f84b0b1c-660c-4d25-8fa2-6f88e08be0d0"
#Logistic Regression
log_reg = LogisticRegression(solver = 'lbfgs')
log_reg.fit(x_train_imputed, y_train)
log_reg.predict(x_val_imputed)

# + id="JshW0Y_PJb82" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 35} outputId="8372e1a6-123b-43e2-867a-0c3363b4f922"
#Validation accuracy score
print(log_reg.score(x_val_imputed, y_val))

StarcoderdataJupyterScriptsDedupFiltered