Despite the many works on characterizing uncertainty, little advice exists on how to optimize a sub-optimal model once the sources of uncertainty are known. Furthermore, the circumstances under which the epistemic uncertainty modeled by ensembling is actually indicative of the true error are not well researched yet, despite its popularity. We therefore studied the performance of selected deep learning models on chemical regression prediction tasks where we systematically vary noise in the input data, the number of datapoints, the chosen model architecture, molecular representation, and the number of models in an ensemble. To this aim, we rely not only on literature datasets, but also construct a new, noisefree, chemical dataset. In the discussion, we then put forward general guidelines for how to detect and circumvent model errors caused by noise, bias, and variance. We pay particular attention to predictions of physicochemical targets, since we find some of the sources of uncertainty to be specific to chemistry.