How to avoid over-fitting using early stopping when using R cross validation package caret
You are handling a very small dataset. The only way to prevent overfitting then is to choose a very restrictive model search space. The simpler the better, and you should prefer models involving some regularization. Even tuning hyperparameters will be hard, so avoid families with many hyperparameters. Neural networks are definitely a no-go, IMHO.
Cross-validation is important in this case, a.o. for tuning the regularization hyperparameter. You were right to perform the extra leave-one-out cross-validation. But remember checking the confidence intervals, which will be very broad.
I would suggest to use some prior business knowledge to further reduce the number of candidate predictor variables to 2-4. Then you can use a linear regression or a ridge regression (i.e. linear regression with L2 regularization, amount to be tuned as hyperparameter) using only these variables. If you have reasons to believe that the relation is very non-linear, you'll need to find another family.
Another possibility is to perform a data-driven feature selection (see https://topepo.github.io/caret/feature-selection-overview.html), but it is harder to properly implement and cross-validate. I wouldn't recommend it to a beginner. Remember that this step can also introduce overfit or instability.