In agreement with previous studies, we found quite varying performance across validation metric, target property and method . However, while several sets of uncertainty estimates did not perform optimally, all uncertainty estimates studied here possessed valuable information. The important part is then to be aware of the limitations and strengths and for this we found the error-based calibration plots to be an extremely powerful tool for getting detailed information of the uncertainty estimates. In fact most conclusions obtained from the remaining metrics could be drawn directly from analyzing the error-based calibration plot.