"""
# Heart Disease with Logistic Regression
"""
"""
### Cordivacular diseases, in colloquial speech heart diseases can be big trouble for human kind. In this kernel we are going to try to make analysis about this illness by investigating different parameters.
"""
"""
![heart.jpg](attachment:f8b66335-d09d-4f53-a455-28c08c263d5c.jpg)
"""
# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import matplotlib.pyplot as plt
import seaborn as sns
import plotly as pl


# Input data files are available in the read-only "../input/" directory
# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory

import os
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using "Save & Run All" 
# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session
"""
## Read Data
"""
data = pd.read_csv("/kaggle/input/heart-disease-uci/heart.csv")
data.head()
data.columns
"""
## First of all I want to have a look at the features meanings :
####  * age       --> age
#### * sex       --> gender in binary (1:male, 0:female)
#### * cp        --> chest pain type
#### * trestbps  --> resting blood pressure (in mm Hg on admission to the hospital)
#### * chol      --> serum cholestoral in mg/dl
#### * fbs       --> (fasting blood sugar > 120 mg/dl) (1 = true; 0 = false)
#### * restecg   --> resting electrocardiographic results
#### * thalach   --> maximum heart rate achieved
#### * exang     --> exercise induced angina (1 = yes; 0 = no)
#### * oldpeak   --> ST depression induced by exercise relative to rest
#### * slope     --> the slope of the peak exercise ST segment
#### * ca        --> number of major vessels (0-3) colored by flourosopy
#### * thal      --> 3 = normal; 6 = fixed defect; 7 = reversable defect
#### * target    --> have disease or not (1=yes, 0=no)
"""
data.corr()
"""
#### This is corrolation table. By the helping of this table we can see that if features have corrolation or not (if values is close to 1 there is positive corrolation, if near to -1 negative corrolation, and if it near to 0 there are no corrolation). As you see there are not much corrolated features in our data. The map you will see in below is corrolation map:
"""
f,ax = plt.subplots(figsize=(10, 10))
sns.heatmap(data.corr(), annot=True, linewidths=.5, fmt= '.1f',ax=ax)
plt.show()
"""
#### By this method, you can have a look at what you want to dig in. In this senerio target must be our target :
"""
data.corr()["target"].sort_values()
data.info()
"""
#### This is the data types that what we will work on. 12 integers and 1 float and all of them is filled with non-null. We can double check like that also :
"""
data.isnull().sum()
"""
#### Now its time to getting more familiar with the data :
"""
data.describe()
"""
## Visualization
"""
"""
#### The first thing I wondered is dependecy between illness and age:
"""
"""
#### Initially we need to view a pandas function which is crosstab. Crosstab computes a simple cross tabulation of two (or more) factors. By default computes a frequency table of the factors unless an array of values and an aggregation function are passed. For example: 

"""
pd.crosstab(data.age,data.target)
"""
#### As you see the the information we wanted, they are grouped
"""
crosstabAge = pd.crosstab(data.age,data.target)
crosstabAge.plot(kind="bar",figsize=(20,8), color ="cmyk", alpha = 0.8) #alpha is opacity
plt.title('Heart Disease Frequency Given Ages')
plt.xlabel('Ages')
plt.ylabel('Frequency')
plt.legend(["Have not Disease", "Have Disease"])
plt.show()
"""
#### According to table, 41, 51, 52, 54 are the ages that heart diseases mostly seen.
"""
"""
#### But what about the numbers of the target ?
"""
data["target"].value_counts()
sns.countplot(data["target"], palette="Set2")
plt.xlabel(' 0 = Not Have Disease,  1 = Have Disease')
"""
#### What about the difference between man and woman ?
"""
crosstabSex = pd.crosstab(data["sex"], data["target"])
crosstabSex
crosstabSex.plot(kind="bar", figsize=(15,6), color="cmyk")
plt.title("Heart Disease Frequency Given Sex")
plt.xticks(rotation=0)
plt.xlabel("0 = Female , 1 = Male")
plt.ylabel("Frequency")
plt.legend(["Have not Disease", "Have Disease"])
"""
#### According to table, men have more tendecy to have heart diseases.
"""
"""
#### So whats next ? fbs : Fasting Blood Sugar
"""
crosstabFbs = pd.crosstab(data["fbs"], data["target"])
crosstabFbs
crosstabFbs.plot(kind="bar", figsize=(10, 8), color="cmyk")
plt.xticks(rotation=0)
plt.xlabel("Fasting Blood Sugar < 120 : 0 | Fasting Blood Sugar > 120 : 1")
plt.ylabel("Frequency")
plt.legend(["Have not Disease", "Have Disease"])
"""
#### This result actually suprised me because i expected more datas on fasting blood sugar > 1 but there are not much.
"""
data.fbs.value_counts()
crosstabExang = pd.crosstab(data["exang"], data["target"])
crosstabExang
crosstabExang.plot(kind="bar", figsize=(10, 8), color="cmyk")
plt.xticks(rotation=0)
plt.xlabel("Exercise Induced Angina 0 : No | 1 : Yes")
plt.ylabel("Frequency")
plt.legend(["Have not Disease", "Have Disease"])
"""
#### Another interesting result. Exercise induced angina is the problem that chest pain, relaxation and pressure hiss caused by ischemia or corner spasm in the heart muscle of the day. 
"""
"""
## Logistic Regression
"""
"""
#### In this tutorial, I am going to write logistic regression code by myself basicaly. After that with sklearn.
"""
"""
#### Lets define x and y values
"""
y = data.target.values #values convert values onto numpy array
x_data = data.drop(["target"], axis=1) #except for target the other columns is our x data
print(y)
"""
#### First of all, we need to make normalization on data because in this data there are some values like 140, 250 (trestbs, chol) and there are binary values like 0 and 1. This may cause overtower between datas on features. To prevent this, we are doing normalization.
"""
"""
### X_normalized = (x - x minimum)/(x maximum - x minimum)
"""
x = (x_data - np.min(x_data))/(np.max(x_data) - np.min(x_data)).values
x
"""
#### Alright its time to split to train and test
"""
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x,y, test_size=0.2, random_state=0)
#transpose matrices
x_train = x_train.T
y_train = y_train.T
x_test = x_test.T
y_test = y_test.T
"""
### So, How Its Work ?
"""
"""
![Inkedweights and bias_LI.jpg](attachment:318d4528-bf75-490e-8596-b03256ed4367.jpg)
"""
"""
#### Well, what are those things ?
* Inputs (x1, x2, x3 ... xn) are our specific values on each column
* Weights are coefficent 
* Bias is interception
* Activation functions are mathematical functions like unit step function, sigmoid etc. We will use sigmoid
* Basicaly its working based on very familiar expression y = wx + b
* In this scenario , z = b + w1.x1 + w2.x2 + .... + w302.x302
* y_pred is sigmoid(z) (will work on sigmoid a little later)
"""
"""
### Initializing Weights and Bias
"""
def initialize_weights_and_bias(dimension):
    weight = np.full((dimension, 1), 0.01)
    bias = 0.0
    return weight,bias
"""
#### The values must be defined. In initialize_weights_and_bias function I defined the initial values. This values are not very important because after forward and bacward propagations values will be updated.
"""
"""
#### For whom asking what is np.full : 
"""
np.full((10, 1), 0.01)
"""
### Sigmoid Function
"""
"""
![sigmoid.png](attachment:f8a01b1c-2b33-494c-b5d8-ed02046e014d.png)
"""
"""
#### Sigmoid function returns probabilistic values. For example at the point 2, nearly %75 probability this value classified as 1
"""
def sigmoid(z):
    y_head = 1/(1 + np.exp(-z))
    return y_head
"""
## Forward-Backward Propagation and Gradient Descent
"""
"""
#### Basically in neural networks (logistic regression is fundamental form of neural networks), you forward propagate to get the output and compare it with the real value to get the error. Now, to minimize the error, you propagate backwards by finding the derivative of error with respect to each weight and then subtracting this value from the weight value. The purpose of this calculations is optimize the algorithm. This operation can be done with gradient descent method.
"""
"""
![gradient descent.png](attachment:2578a93a-6185-4e3f-a155-996fa0dd2951.png)
"""
"""
### Loss = (Y)(-log(Y_pred)) + (1-Y)(-log(1-Y_pred))
"""
"""
#### This is loss function. Sumation for each value is cost function. If you predict 1 and the result is 1 you will get 0 loss otherwise big amount of lost.
"""
"""
![grad2.png](attachment:47016ac7-1ee3-4dda-bdad-a7e5b90dc3d9.png)
"""
"""
#### This is updating equations. By taking derivative of cost function according to weight and bias. Then multiply it with α learning rate. Step by step model will be updated by this method. Learning rate can be say like learning speed. In other words, Incremental step is stepping by the helping of learning rate.
"""
"""
#### Lets continue to code :
"""
def forward_backward_propagtion(weight, bias, x_train, y_train):
    # forward propagation
    z = np.dot(weight.T,x_train) + bias # z = b + w1.x1 + w2.x2 + .... + w302.x302
    y_head = sigmoid(z)
    loss = -(y_train*np.log(y_head) + (1-y_train)*np.log(1-y_head))
    cost = np.sum(loss) / x_train.shape[1] #x_train.shape[1] for normalization
    
    # backward propagation
    derivative_weight = np.dot(x_train,((y_head-y_train).T))/x_train.shape[1] #simple derivative
    derivative_bias = np.sum(y_head-y_train)/x_train.shape[1]
    gradients = {"Derivative Weight" : derivative_weight, "Derivative Bias" : derivative_bias} #for storage
     
    return cost,gradients

def update(weight, bias, x_train, y_train, learning_rate, iteration) :
    cost_list = []
    index = []
    
    # updating(learning) parameters in number_of_iterarion times
    for i in range(iteration):
        # make forward and backward propagation and find cost and gradients
        cost,gradients = forward_backward_propagtion(weight,bias,x_train,y_train)
        # update
        weight = weight - learning_rate * gradients["Derivative Weight"]
        bias = bias - learning_rate * gradients["Derivative Bias"]
        
        cost_list.append(cost)
        index.append(i)
        print ("Cost after iteration %i: %f" %(i, cost))
        
    parameters = {"weight": weight,"bias": bias}
    
    print("iteration:",iteration)
    print("cost:",cost)

    plt.plot(index,cost_list)
    plt.xlabel("Number of Iteration")
    plt.ylabel("Cost")
    plt.show()

    return parameters, gradients
def predict(weight, bias, x_test):
    # x_test is an input for forward propagation
    z = np.dot(weight.T,x_test) + bias
    y_head = sigmoid(z)

    y_prediction = np.zeros((1,x_test.shape[1]))
    
    # if z is bigger than 0.5, our prediction is sign one (y_head=1),
    # if z is smaller than 0.5, our prediction is sign zero (y_head=0)
    for i in range(z.shape[1]):
        if z[0,i] <= 0.5:
            y_prediction[0,i] = 0
        else:
            y_prediction[0,i] = 1
    return y_prediction
def logistic_regression(x_train, y_train, x_test, y_test, learning_rate, iteration):
    # initialize
    dimension = x_train.shape[0]
    weight,bias = initialize_weights_and_bias(dimension)
     
    parameters, gradients = update(weight, bias,x_train, y_train, learning_rate, iteration)

    y_prediction = predict(parameters["weight"],parameters["bias"],x_test)
    
    print("Accuracy of Model : {}%".format((100 - np.mean(np.abs(y_prediction - y_test))*100)))
logistic_regression(x_train,y_train,x_test,y_test,2,200)
"""
## Sklearn
"""
from sklearn.linear_model import LogisticRegression
lr = LogisticRegression()

lr.fit(x_train.T, y_train.T)
print("test accuracy {}".format(lr.score(x_test.T,y_test.T))) #
"""
## Conclusion
"""
"""
#### In this tutorial, first we do exploratory data analysis then we generate our logistic regression model by hand then tried it by sklearn
"""