What is the difference in plotting ROC curve with probability scores vs binary decisions
How do you go from probabilities to binary decisions? The default choice is to use a threshold of 0.5 but maybe a threshold of 0.3 or 0.7 would have given better results (depending on you metric). The ROC curve gives you more information as it allows to see the results for each probability threshold.
Usually you set some metric to optimize (F1 score for example) and you set the threshold based on this metric. Then you plot the confusion matrix and any other metric that is useful to you