Does the SVM in sklearn support incremental (online) learning?
Question: I am currently in the process of designing a recommender system for text
articles (a binary case of 'interesting' or 'not interesting'). One of my
specifications is that it should continuously update to changing trends.

From what I can tell, the best way to do this is to make use of machine
learning algorithm that supports incremental/[online
learning](http://en.wikipedia.org/wiki/Online%5fmachine%5flearning).

Algorithms like the Perceptron and Winnow support online learning but I am not
completely certain about Support Vector Machines. Does the scikit-learn python
library support online learning and if so, is a support vector machine one of
the algorithms that can make use of it?

I am obviously not completely tied down to using support vector machines, but
they are usually the go to algorithm for binary classification due to their
all round performance. I would be willing to change to whatever fits best in
the end.


Answer: While online algorithms for SVMs do exist, it has become important to specify
if you want kernel or linear SVMs, as many efficient algorithms have been
developed for the special case of linear SVMs.

For the linear case, if you use the [SGD classifier in scikit-
learn](http://scikit-learn.org/stable/modules/sgd.html#sgd) with the hinge
loss and L2 regularization you will get an SVM that can be updated
online/incrementall. You can combine this with [feature transforms that
approximate a kernel](http://scikit-
learn.org/stable/modules/kernel_approximation.html) to get similar to an
online kernel SVM.

> One of my specifications is that it should continuously update to changing
> trends.

This is referred to as _concept drift,_ and will not be handled well by a
simple online SVM. Using the PassiveAggresive classifier will likely give you
better results, as it's learning rate does not decrease over time.

Assuming you get feedback while training / running, you can attempt to detect
decreases in accuracy over time and begin training a new model when the
accuracy starts to decrease (and switch to the new one when you believe that
it has become more accurate). [JSAT](https://code.google.com/p/java-
statistical-analysis-tool/) has 2 drift detection methods (see
[jsat.driftdetectors](https://code.google.com/p/java-statistical-analysis-
tool/source/browse/#svn/trunk/JSAT/src/jsat/driftdetectors)) that can be used
to track accuracy and alert you when it has changed.

It also has more online linear and kernel methods.

(bias note: I'm the author of JSAT).

