Super basic logistic regression example
What is the meaning for the coef and intercept

A (binary) logistic regression algorithm tries to determine whether the data $x$ belongs to class 0 or class 1 by the value
$f(x)=\omega x+b$. If the value $f(x)&gt;0$, the algorithm believe $x$ is more likely to be in class 1; while if $f(x)&lt;0$, then $x$ is more likely to be in class 0.

In your code alg.coef_ is the $\omega$ above, and alg.intercept_ is the $b$ above.

Why everything is predicted to be in class 0

If you run your code, you should see the result alg.coef_=-0.354, alg.intercept_=0.307. Therefore your alg is calculating $f(x)=-0.354x+0.307$. Now if you plug in your data, i.e. $x=$1, 2, 1.5 and 2.5 respectively, you should get the value $f(x)=$-0.047,-0.401,-0.224 and -0.579 respectively. 

As you see, all the four $f(x)$ values are smaller than 0, therefore your alg determines that they all belong be class 0.

But still why? Why the result is not what I expected?

Here is were regularization comes in. In the context of logistic regression, the learning algorithm regularizes that the learned parameter, $w$, should not be too large. To be specific, by default it regularizes the $l_2$ norm norm of $w$. In your case $w$ is a scalar, then $l_2$ norm is its absolute value.

In general, regularization is used to prevent overfitting. But that's another broad topic. 

If you want your algorithm to behave as you expected (classify 1 and 1.5 to class 1, and classify 2 and 2.5 to class 0) , there are 2 ways.


You tell you program to use extremely weak regularization. 
That can be achieved by passing a large value to the C parameter in alg = LogisticRegression() (default value is 1.0, see detail), e.g.


alg = LogisticRegression(C=1000)

In this case your algorithm should return alg.coef_=-9.894, alg.intercept_=17.156, and it can classify your four data points correctly. Notice that the absotely value of alg.coef_ is now much larger than before.


Provide your algorithm with more data.

For example, simply repeat your four data points 100 times:


data   = [x[:-1] for x in points]*100
target = [int(x[-1]) for x in points]*100


Then you can get alg.coef_=-4.438, alg.intercept_=7.594 without setting C. In this case the same default regularization strength is still applied, but the additional data provide stronger evidence for your algorithm to believe that the alg.coef_ should be allowed larger (in absolutely value).