In a recent publication we trained 9 random forest (RF) models on varying training set sizes (from 100 to 150k data points) and used them to show Crippen logP as a useful benchmark for atom attributions within explainable AI (XAI) on regression tasks. Since RF models have easily obtainable uncertainty estimates in the form of the variance of individual tree predictions, we reuse the models here to test the performance of the uncertainty estimates. The data set used to train the models is a 250k molecules subset of the ZINC data base which has been used in several studies . the intercept (b). Ideally these values should be 1, 1, and 0. The corresponding error-based calibration plots are shown in Figure . Three observations are true for all models: the linear fit is decent (R 2 values between 0.81 and 0.91), the slope is too low (between 0.55 and 0.69) and the intercept is too high (intercept decreases from 0.62 for the N train = 100 model to 0.23 for the N train = 150k model). Both R 2 -value, slope and intercept is closest to the ideal value for the model trained on 150k samples but only for the intercept we see a gradual improvement as the training set size is increased. For the two models trained on the least training data (N train = 100 and N train = 500), the RMSE bins generally lie above the expected identically line. This means that the observed errors are generally higher than what would be expected based on the predicted uncertainties and thus the uncertainties are underestimated. For the remaining models we observe a trend of low uncertainties being underestimated (RMSE bins above identity line) and high uncertainties being overestimated (RMSE bins below the identity line).