Latest word on cross validation?
It is not hard to see that without additional stability assumptions one won't be able to get high probability bounds. For example consider predicting unbiased coin using majority label in the sample. With probability $~1/\sqrt{n}$ we get that majority of leave-one-out is exactly the opposite of the excluded point so LOO will give error of 1. Note that the true error is always 1/2.

If one assumes stability then the bounds will depend on the notion of stability being used. Uniform stability will give high probability bounds for both empirical mean and LOO estimator (e.g. https://arxiv.org/abs/1902.10710).
Weaker notions will give you weaker bounds. I guess the latest on variance of the estimator is this paper http://www.satyenkale.com/papers/crossvalidation.pdf