Training an ensemble of models and inspecting the variance between predictions of the individual submodels furthermore is a popular method to estimate the uncer-tainty associated with a prediction. Ensemble uncertainties can be used for risk management or active learning, amongst others, and are thus valuable information when judging the reliability of a prediction. However, uncertainties from ensembles only directly represent the true error for variance-dominated systems, i.e. the model uncertainty caused by model bias is not included. To showcase this, the deviation between the uncertainty from the ensemble variance and the true observed error was computed for all systems of Fig. using an ensemble of five submodels.