Why predicted proababilities from this binary classifier does not sum up to 1? I have a C5.0 model that is trained to predict binary class (C1/C2) on a dataset with 20 features. The model is configured to perform boosting (10 trials) and it has a miss-classification cost function (100:1 where 100 is the cost for miss-classifying a Negative sample as Positive and 1 is the cost for miss-classifying Negative as Positive).

Looking at predicted probabilities generated by the model, I can see that it ranges from 0 to 1 for each class. i.e, I have instances where the predicted class (C1) has a probability lower than 0.5 ( for example: predicted class=C1 and predicted probability=0.1 ) - This is where the question arises: if P(C1) &lt; 50% why is it classified as C1 (there are only two classes, C1 and C2)

Based on my understanding of decision trees, the predicted probabilities are often generated based on the percentage of test cases on the leaf node that were labeled in each class divided by the total number of instances hitting that leaf node. This method will dictate the probability for two classes must sum up to 1.

My question is why does the model classify an instance in class C1 if it has only got 0.1 confidence in it. Does a predicted probability of 0.1 on class C1 mean that there is a 0.9 confidence in it belonging to class C2? If so, why does it classify an instance in class C1 is it has less than 0.5 confidence in it?

My own theory is that this might be due to boosting and miss-classification cost and the way they are influencing the predicted class.
You are absolutely right to be skeptical about the results you are getting. I've been using boosted trees (not necessarily C5.0) for over 5 years, and I've never encountered results like what you've described here. Excluding the possibility that you simply had a typo in your code, I can think of two reasons:


Low decision threshold: As is often the case in class imbalance problems (e.g., fraud detection, marketing response, bankruptcy, etc) a low probability could still indicate the minority class if the decision threshold is also low. In your case, if the model might have automatically decided to pick a threshold of 0.05, then 0.1 would be flagged as C1.
Cost function: This is what you mentioned -- if your model has a high cost associated with missing true positives then 0.1 could be classified as C1.