What is the difference in plotting ROC curve with probability scores vs binary decisions As the title reads what is the difference? Plotting the ROC w.r.t probability scores gives the stair cased version.
But in my opinion I find that using binary decision is better because the ROC curve is just a summary of the different confusion matrices w.r.t different thresholds. And the confusion matrix shows the binary decisions.
Is their an advantage or certain situation where plotting the ROC curve with the prob.scores is better than the binary decisions?
How do you go from probabilities to binary decisions? The default choice is to use a threshold of 0.5 but maybe a threshold of 0.3 or 0.7 would have given better results (depending on you metric). The ROC curve gives you more information as it allows to see the results for each probability threshold.
Usually you set some metric to optimize (F1 score for example) and you set the threshold based on this metric. Then you plot the confusion matrix and any other metric that is useful to you