More importantly, the choice of C K has a very strong influence on the expected MAE. The goal of this and many other specialty ML models for materials discovery, trained on small-to medium-sized datasets (∼100s-1000s of examples), is to screen properties of structures that represent entirely new compositions, or even chemical systems, that are outside the training data. For this use case, performing a purely random split introduces substantial data leakage which leads to a ∼ 30% underestimation of the expected MAE when, for example, predicting defects in a structure that represents an unseen chemical system in the training data. As an even more extreme example, C K = Elements reveals a ∼2.5 times higher expected MAE than a purely random split, although ensembling can reduce expected MAE by ∼30% relative to a nonensemble prediction.