Should I use leave-one-out cross-validation for testing?
Concerning $k$-fold Cross Validation, I like to think of it by considering two extremes you can do: Leave-One-Out Cross-Validation where you leave one sample each time and train your model on the remaining $n-1$, and 2-fold Cross Validation at which you split your dataset in half and train (and validate) two models on two different halves.

The important aspect when choosing $k$ is a bias-variance tradeoff. Note that in LOOCV you train each model using almost as many samples as there are available ($n-1$), so the validation step should give you an unbiased estimate of the real test error. However, each model in LOOCV is trained each time on almost exactly same dataset!. This has important consequences, since the output of each model is highly correlated with each other. Since the mean of highly correlated variables has big variance, LOOCV will suffer from huge variance.

On the other hand, in 2-fold CV the models share no common samples, so they are not correlated and therefore their outputs have low variance. But since we train each model using only a half of available samples, the procedure will have a high bias (the estimated test error will be off from the true test error).

What to do in this scenario? Choose something in the middle. Usually $k=5$ and $k=10$ should be a good choice.