Multiclass linear SVM in python that return probability
Question: How can I implement a linear SVM for multi-class which returns the
proabability matrix for the test samples. Train samples: mxn Train labels: mxc
Test labels : mxc, where column has the probability of each class.

The function in sklearn which does "one-vs-the-rest"
[LinearSVC](http://scikit-
learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html#sklearn.svm.LinearSVC)
doesn't return probablity array for each sample like [SVC](http://scikit-
learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC) which
has `predict_proba`

**Edit**

Code:

    
    
            print X_train.shape,y.shape
            svc = LinearSVC()
            clf = CalibratedClassifierCV(svc, cv=10)
            clf.fit(X_train, y)
    

Output:

    
    
    (7112L, 32L) (7112L, 6L)
    Traceback (most recent call last):
      File "SVC_Calibirated_Probability.py", line 171, in <module>
        clf.fit(X_train, y)
      File "C:\Anaconda\lib\site-packages\sklearn\calibration.py", line 110, in fit
        force_all_finite=False)
      File "C:\Anaconda\lib\site-packages\sklearn\utils\validation.py", line 449, in check_X_y
        y = column_or_1d(y, warn=True)
      File "C:\Anaconda\lib\site-packages\sklearn\utils\validation.py", line 485, in column_or_1d
        raise ValueError("bad input shape {0}".format(shape))
    ValueError: bad input shape (7112L, 6L)
    


Answer: `LinearSVC` does not support probability estimates because it is based on
`liblinear` but `liblinear` supports probability estimates for logistic
regression only.

If you just need confidence scores, but these do not have to be probabilities,
you can use `decision_function` instead.

If it it not required to choose penalties and loss functions of linear SVM,
you can also use `SVC` by setting kernel to be `'linear'`, then you can have
`predict_proba`.

### Update #1:

You can use `SVC` with `OneVsRestClassifier` to support one-vs-rest scheme,
for example

    
    
    from sklearn import datasets
    from sklearn.multiclass import OneVsRestClassifier
    from sklearn.svm import SVC
    iris = datasets.load_iris()
    X, y = iris.data, iris.target
    clf = OneVsRestClassifier(SVC(kernel='linear', probability=True, class_weight='auto'))
    clf.fit(X, y)
    proba = clf.predict_proba(X)
    

### Update #2:

There is another way to estimate probabilities with `LinearSVC` as classifier.

    
    
    from sklearn.svm import LinearSVC
    from sklearn.calibration import CalibratedClassifierCV
    from sklearn.datasets import load_iris
    
    iris = load_iris()
    X = iris.data
    Y = iris.target
    svc = LinearSVC()
    clf = CalibratedClassifierCV(svc, cv=10)
    clf.fit(X, Y)
    proba = clf.predict_proba(X)
    

However for the other question ([Making SVM run faster in
python](http://stackoverflow.com/questions/31681373/making-svm-run-faster-in-
python)), this solution is not likely to enhance performance either as it
involves additional cross-validation and does not support parallelization.

### Update #3:

For the second solution, because `LinearSVC` does not support multilabel
classification, so you have to wrap it in `OneVsRestClassifier`, here is an
example:

    
    
    from sklearn.svm import LinearSVC
    from sklearn.calibration import CalibratedClassifierCV
    from sklearn.multiclass import OneVsRestClassifier
    from sklearn.datasets import make_multilabel_classification
    
    X, Y = make_multilabel_classification(n_classes=2, n_labels=1,
                                          allow_unlabeled=True,
                                          return_indicator=True,
                                          random_state=1)
    clf0 = CalibratedClassifierCV(LinearSVC(), cv=10)
    clf = OneVsRestClassifier(clf0)
    clf.fit(X, Y)
    proba = clf.predict_proba(X)
    

