The right panel of Fig. depicts the observed test set performance of a noisy test and training set with different levels of noise and different numbers of training points. We can see how the model performance changes as it approaches the aleatoric limit (dashed black line) where the RMSE equals the standard deviation of the noise distribution. With a small number of training points, such as 71 or 711 (indigo and violet curves), the test set error is not governed by noise (but instead dominated by bias and variance errors caused by the tiny number of datapoints), so that the magnitude of added random noise does not influence the observed performance significantly. As the aleatoric limit increases and approaches the performance of the other three dataset sizes, the RMSE of the datasets is deflected upward. As the noise level surpasses the baseline non-noise error for the dataset sizes, model performances converge and becomes indistinguishable as can be seen at the 1 kcal/mol noise level for the two largest dataset sizes. A similar trend with the presence of an aleatoric limit due to controlled addition of noise was also noted by Xie et al. The noise we discussed so far was drawn from a Gaussian distribution. We also tested uniform, hyperbolic, and bimodal noise distributions, where the respective parameters were chosen so that each distribution had a standard deviation of 1 kcal/mol and was centered around 0 kcal/mol. Fig. depicts the respective distributions and their observed model performances. Both the training and test set contained noise. We did not observe any difference in overall model performance between different error distributions, as long as the mean and standard deviation of the noise was the same, respectively. Though noise distributions found in real data may be non-Gaussian, if homoscedastic, they should still follow the same trends of approaching an asymptote due to noise.