Two-class model with predicted scores needed - classification or regression approach
This is a classification problem, not a regression problem, so your first instinct was correct. For most models, you can easily get the probability estimate for each prediction. If you did tree based methods or a linear model, you would have probabilities easily. However, because of how an SVM works, it does not automatically output a probability estimate. It makes sense that Caret has to do some extra computations to get your probabilities. 

For the specifics, see: https://stats.stackexchange.com/questions/335527/what-are-the-predicted-probabilities-from-an-svm