Model selection: large mean and variance vs small mean and variance
This is exactly a question I asked Sebastian Raschka (author of the great Python Machine Learning book), here you can find his answer saying &quot;I also recommend the 1-standard error method, which basically means that you select the best model from k-fold based on pure performance, and then you select the simplest model that is within 1 standard error of that model&quot;.
The more extended explanation of his answer can be found on his github link.