Discrete Ordinal Classification with Probabilities
Probit and logit ordinal regression model the cumulative probabilities $P(y \leq i | x)$. So you should be able to get class probabilities by taking the difference $P(y=i |x) = P(y \leq i|x) - P(y\leq i-1 |x)$. Then it just depends on the implementation whether or not it makes the probabilities available to you.

In R, for example, you can get these probabilities from the polr package.

[Even though you lose some information when doing one-vs-rest classification, ordinal regression models make some proportionality assumptions that (if not appropriate for your data) may not necessarily give you better results.]