Is logistic regression actually a regression algorithm?
Logistic regression is regression, first and foremost. It becomes a classifier by adding a decision rule. I will give an example that goes backwards. That is, instead of taking data and fitting a model, I'm going to start with the model in order to show how this is truly a regression problem.

In logistic regression, we are modeling the log odds, or logit, that an event occurs, which is a continuous quantity. If the probability that event $A$ occurs is $P(A)$, the odds are:

$$\frac{P(A)}{1 - P(A)}$$

The log odds, then, are:

$$\log \left( \frac{P(A)}{1 - P(A)}\right)$$

As in linear regression, we model this with a linear combination of coefficients and predictors:

$$\operatorname{logit} = b_0 + b_1x_1 + b_2x_2 + \cdots$$

Imagine we are given a model of whether a person has gray hair. Our model uses age as the only predictor. Here, our event A = a person has gray hair:

log odds of gray hair = -10 + 0.25 * age

...Regression! Here is some Python code and a plot:

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns

x = np.linspace(0, 100, 100)

def log_odds(x):
    return -10 + .25 * x

plt.plot(x, log_odds(x))
plt.xlabel("age")
plt.ylabel("log odds of gray hair")




Now, let's make it a classifier. First, we need to transform the log odds to get out our probability $P(A)$. We can use the sigmoid function:

$$P(A) = \frac1{1 + \exp(-\text{log odds}))}$$

Here's the code:

plt.plot(x, 1 / (1 + np.exp(-log_odds(x))))
plt.xlabel("age")
plt.ylabel("probability of gray hair")




The last thing we need to make this a classifier is to add a decision rule. One very common rule is to classify a success whenever $P(A) &gt; 0.5$. We will adopt that rule, which implies that our classifier will predict gray hair whenever a person is older than 40 and will predict non-gray hair whenever a person is under 40.

Logistic regression works great as a classifier in more realistic examples too, but before it can be a classifier, it must be a regression technique!