Another option for evaluating the UQ estimates is based on the error-based calibration recently proposed by Levi et al. . While application of this metric has seen some adoption within UQ for molecular machine learning , there has been no head-to-head comparison of the error-based calibration metric versus the three above-mentioned popular choices applied to chemical datasets. That is what we do here. Through examples using two different chemical datasets and three different UQ methods (ensemble with random forest (RF), the latent space distance and evidential regression ) we demonstrate the superiority of evaluating uncertainty estimates based on error-based calibration and point to the drawbacks of the three popular UQ evaluation metrics; Spearman's rank correlation coefficient, the miscalibration area and the NLL.