title: Having trouble understanding sklearn's SVM's predict_proba function
body: <p>I am having trouble understanding a function from sklearn and would like some clarification. At first I thought that sklearn's SVM's predict_proba function gave out the level of confidence of the classifier's prediction, but after playing around with it with my emotion recognition program, I am starting to form doubts and feel like I misunderstood the use and how the predict_proba function worked.</p>
  
  <p>For example, I have my code setup something like this:  </p>
  
  <pre><code># Just finished training and now is splitting data (cross validation)
  # and will give an accuracy after testing the accuracy of the test data
  
  features_train, features_test, labels_train, labels_test = cross_validation.train_test_split(main, target, test_size = 0.4)
  
  model = SVC(probability=True)
  model.fit(features_train, labels_train)
  pred = model.predict(features_test)
  
  accuracy = accuracy_score(labels_test, pred)
  print accuracy
  
  # Code that records video of 17 frames and forms matrix know as
  # sub_main with features that would be fed into SVM
  
  # Few lines of code later. . .  
  
  model.predict(sub_main)
  prob = model.predict_proba(sub_main)
  
  prob_s = np.around(prob, decimals=5)
  prob_s = prob_s* 100
  pred = model.predict(sub_main)
  
  print ''
  print 'Prediction: '
  print pred
  print 'Probability: '
  print 'Neutral: ', prob_s[0,0]
  print 'Smiling: ', prob_s[0,1]
  print 'Shocked: ', prob_s[0,2]
  print 'Angry: ', prob_s[0,3]
  print ''
  </code></pre>
  
  <p>And when I test it out, it gives me something like this:</p>
  
  <pre><code>Prediction: 
  ['Neutral']
  Probability: 
  Neutral:  66.084
  Smiling:  17.875
  Shocked:  11.883
  Angry:  4.157
  </code></pre>
  
  <p>It managed to have a 66 percent confidence that the correct classification is "Neutral". 66 was next to "Neutral" which happened to be the highest number. The highest number was labeled with the actual prediction and I was happy about that.</p>
  
  <p>But eventually eventually. . .</p>
  
  <pre><code>Prediction: 
  ['Angry']
  Probability: 
  Neutral:  99.309
  Smiling:  0.16
  Shocked:  0.511
  Angry:  0.02
  </code></pre>
  
  <p>It made the prediction, "Angry" (which is the correct classification btw) and it assigned a confidence level of 99.3 percent next to "Neutral". The highest level of confidence (highest number) was assigned to Neutral despite the prediction being completely different.</p>
  
  <p>Somtimes it also does this:</p>
  
  <pre><code>Prediction: 
  ['Smiling']
  Probability: 
  Neutral:  0.0
  Smiling:  0.011
  Shocked:  0.098
  Angry:  99.891
  
  Prediction: 
  ['Angry']
  Probability: 
  Neutral:  99.982
  Smiling:  0.0
  Shocked:  0.016
  Angry:  0.001
  </code></pre>
  
  <p>I don't think understand how SVM's predict_proba function works and would like some clarification on how it works and what is going on with my code. What is going on in my code?</p>