In addition to model variance, nearly all machine learning models also suffer from model bias, which can stem from fundamental shortcomings of the model, too little data, or ill-chosen features, amongst others. Uncertainty estimates obtained from an ensemble of models only capture variance errors. Since model bias, as introduced due to the aforementioned reasons, can be the dominant contribution, ensemble uncertainties usually underestimate the actual error. We therefore examine whether our approach also holds up for machine learning potentials trained on real data sets.