# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     name: python3
# ---

# + [markdown] id="yGUNM2BEsBQ5" colab_type="text"
# # Project objective
# This project is designed to review logistic regression method and its python implementation using Wisconsin breast cancer dataset.
#
# Information about the dataset, some technical details about the used machine learning method(s) and mathematical details of the quantifications approaches are provided in the code. 

# + [markdown] id="VjtJFxdsNh05" colab_type="text"
# # Packages we work with in this notebook
# We are going to use the following libraries and packages:
#
# * **numpy**: NumPy is the fundamental package for scientific computing with Python. (http://www.numpy.org/)
# * **sklearn**: Scikit-learn is a machine learning library for Python programming language. (https://scikit-learn.org/stable/)
#

# + id="57oB2idEgr-g" colab_type="code" colab={}
import numpy as np
import sklearn as sk

# + [markdown] id="Bb1Zm7ARN5D5" colab_type="text"
# # Introduction to the dataset
#
# **Name**: Wisconsin breast cancer dataset
#
# **Summary**: Identifying if there is a malignant tumor or not using features that are computed from a digitized image of a fine needle aspirate (FNA) of a breast mass.
#
# **number of features**: 30 (real, positive) 
#
# **Number of data points (instances)**: 569
#
# **dataset accessibility**: Dataset is available as part of sklearn package.
#
# **Link to the dataset**: http://archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(diagnostic)
#
#
#

# + [markdown] id="QjBnejgpP0Gr" colab_type="text"
# ## Loading the dataset and separating features and labels
# The dataset is available as part of sklearn package. Hence, we do not need to import the data directly from UCI ML repository. 

# + id="RILQWrhjQUtF" colab_type="code" outputId="50035491-ada1-4c46-a99d-48c2aefed3b9" colab={"base_uri": "https://localhost:8080/", "height": 49}
from sklearn.datasets import load_breast_cancer

# Loading breast cancer data
target_dataset = load_breast_cancer()

# separating feature arrays of pixel values (X) and labels (y) 
input_features = target_dataset.data
output_var = target_dataset.target
# printing number of features (pixels) and data points 
n_samples, n_features = input_features.shape
print("number of samples (data points):", n_samples)
print("number of features:", n_features)

# + [markdown] id="qgejl_XWhWqN" colab_type="text"
# ## Splitting data to training and testing sets
#
# We need to split the data to train and test, if we do not have a separate dataset for validation and/or testing, to make sure about gneralizability of the model we train.
#
# **test_size**: Traditionally, 30%-40% of the dataset cna be used for test set. If you split the data to train, validation and test, you can use 60%, 20% and 20% of teh dataset, respectively.
#
# **Note.**: We need the validation and test sets to be big enough for checking genralizability of our model. At the same time we would like to have as much data as possible in the training set to train a better model.
#
# **random_state** as the name suggests, is used for initializing the internal random number generator, which will decide the splitting of data into train and test indices in your case.
#

# + id="V3L9BbkSg2vp" colab_type="code" colab={}
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(input_features, output_var, test_size=0.30, random_state=5)

# + [markdown] id="d0Fi1jpVkbhC" colab_type="text"
# ## Building the supervised learning model
# We want to build a binary classification model as the output variable is categorical with 2 classes. Here we build a simple logistic regression model.
#
# ### Logistic regression
# If we have set of features X1 to Xn, y can be obtained as:
# \begin{equation*} y=b0+b1X1+b2X2+...+bnXn\end{equation*}
#
# where y is the predicted value obtained by weighted sum of the feature values.
#
# Then probability of each class (for example if there is a malignant tumor) can be obtained using the logistic function 
#
# \begin{equation*} p(class=malignant)=\frac{1}{(1+exp(-y))} \end{equation*}
#
# Based on the given class labels and the features given in the trainign data, coefficients b0 to bn can be ontained during the optimization process.
#
# b0 to bn are fixed for all samples while X1 to Xn are feature values specific to each sample. Hence, the logistic function will give us probability of each class assigned to each sample. Finally, the model will choose the class with the highest probability for each sample.
#
#
# **Note.** The logistic regression model is parametric and the parameters are the regression coefficiets b0 to bn.

# + id="fj3SSteMkxb2" colab_type="code" colab={}
from sklearn.linear_model import LogisticRegression 

# Create logistic regression object
logreg = LogisticRegression()

# Train the model using the training sets
logreg.fit(X_train, y_train)

# + [markdown] id="zUWtodtDmFAW" colab_type="text"
# ## Prediction of test (or validation) set
# We now have to use the trained model to predict y_test.

# + id="KRQpG2vykzqX" colab_type="code" colab={}
# Make predictions using the testing set
y_pred = logreg.predict(X_test)

# + [markdown] id="xUHXoiaemXai" colab_type="text"
# ## Evaluating performance of the model
# We need to assess performance of the model using the predictions of the test set. We use accuracy and balanced accuracy. Here are their definitions:
#
# * **recall** in this context is also referred to as the true positive rate or sensitivity
#
#
#
#
# $${\displaystyle {\text{recall}}={\frac {tp}{tp+fn}}\,} $$
#
#  
#
# * **specificity** true negative rate
#
#
#
# $${\displaystyle {\text{true negative rate}}={\frac {tn}{tn+fp}}\,}$$
#
# * **accuracy**: This measure gives you a sense of performance for all the classes together as follows:
#
# $$ {\displaystyle {\text{accuracy}}={\frac {tp+tn}{tp+tn+fp+fn}}\,}$$
#
#
# \begin{equation*} accuracy=\frac{number\:of\:correct\:predictions}{(total\:number\:of\:data\:points (samples))} \end{equation*}
#
#
# * **balanced accuracy**: This measure gives you a sense of performance for all the classes together as follows:
#
# $${\displaystyle {\text{balanced accuracy}}={\frac {recall+specificity
# }{2}}\,}$$
#

# + id="cAqcUM5Bmb2d" colab_type="code" outputId="5b7ab04b-a6e0-462d-a102-999782d79a88" colab={"base_uri": "https://localhost:8080/", "height": 49}
from sklearn import metrics

print("accuracy of the predictions:", metrics.accuracy_score(y_test, y_pred))
print("blanced accuracy of the predictions:", metrics.balanced_accuracy_score(y_test, y_pred))

# + [markdown] id="a8d5eJVRm2yO" colab_type="text"
# ## Extracting the coefficient of the model
# The trained logistic regresseion model predicts the class of a datapoint as a fucntion of linear combination of feature values. Hence, each feature has a coefficient in this linear combination for predicting output variable.

# + id="9ELrsRLImqtM" colab_type="code" outputId="9833a850-5c66-4000-f16f-00cc015049ae" colab={"base_uri": "https://localhost:8080/", "height": 99}
print('Coefficients: {}'.format(logreg.coef_))

StarcoderdataJupyterScriptsDedupFiltered