Logistic Regression Overview and its Error Metrics
Logistic in Short

Overview:

Logistic Regression is the extension of ‘Linear regression’. Target variable in linear regression model will be continuous. Whereas in this algorithm the target variable will be discrete. So if our prediction needs to be discrete, we can use Logistic regression. We need not get carried away from the name as it is holding “regression”. This algorithm is built for classification problems.

Classification problems are also called as binary problems, where the output will be between 2 classes. It is either 0 or 1/Yes or No.

If there are greater than 2 classes to classify it is called as Multi class classification

Classification Problems can be as follows:

1. Email is Spam / Not a Spam

2. Passed / Failed

3. Survived / Not Survived

4. Loans — Approved / Rejected

Function used in this algorithm is “Sigmoid” or “Logistic” which is represented as

Sigmoid Curve

We have a concept called Decision Boundary(db) in Logistic regression. This is a line which splits from one class to other class. By default db will be 0.5 , the process goes like if the resultant data are less than (db)0.5 it can be classified as 0 & if data are greater than (db)0.5 it is 1 and vice versa.

The fundamental application of logistic regression is to determine a decision boundary for a binary classification problem. We need to adjust the decision boundary levels after usual predictions to check on accuracy levels and see how it varies. That is we need to vary the DB value from 0.5(default) to any value between 0 to 1.

Error Metrics:

These are some frequently used metrics in industry for classification problems to measure accuracy percentages and error levels they are as follows:

a. Confusion Matrix, b. Classification Report, c. ROC Curve & d. Accuracy Score

I) Confusion Matrix below is used to find the amount of values which are predicted correctly & wrongly.

Confusion Matrix

Here FP is called as Type I error and FN are called as Type II error. Where both the Type I and II errors are the values which are predicted wrongly. Similarly TP and TN are the values which are predicted correctly

II) Classification Report: This includes 3 parameters which are -

a. Precision: This is defined as Number of positive patterns predicted correctly, by total number of patterns in positive class.

b. Recall: This is defined as fraction of the patterns that are correctly classified

c. F1 Score: This is defined as Harmonic mean between Precision and Recall values

III) Accuracy Score: This is the usual metric which predicts the overall accuracy of the model.

IV) ROC Curve: “Receiver Operating Characteristic Curve” is the score which lies between 0 to 1. 0 stands for Bad and 1 stands for Good. 0.5 is better. If ROC score is 0.78 then it means 78% of predicted values are correct and rest 22% are predicted wrongly.
['Nandini Sekar']
['Logistic Regression', 'Roc Curve', 'Classification Algorithms', 'Classification Report', 'Confusion Matrix']