Given the same error distribution ({ϵ i }) but two different predicted uncertainty distributions, the uncertainties more likely to have resulted in the observed errors will have a lower NLL. However, in the context of chemical ML models, the way of obtaining property predictions (and hence errors) and the way of obtaining uncertainties typically go hand in hand. Therefore, the error distributions are typically not the same and we are not strictly comparing the uncertainties but also the accuracies of the models. Since higher accuracy generally leads to lower NLL, we can easily have two models with the more accurate model having obtained a lower NLL but still having completely random uncertainties. Another issue with the NLL is that it really only makes sense when comparing models; the number in itself is basically meaningless. For a collection of models with similar NLL it is not obvious whether all models perform very good, decent or horrible. Again, to account for some of these drawbacks, we introduce the simulated NLL, where errors are randomly drawn from the predicted uncertainties (assuming Gaussian errors) and the NLL sim is calculated based on the simulated errors. As for ρ sim rank , we do this a number of times (typically 1,000) to obtain an expected mean for the NLL as well as a standard deviation. Discrepancy between simulated and observed NLL hints to a problem with the uncertainties.