Recently we developed a defect GNN (dGNN) modeling approach to directly predict relaxed vacancy formation energies, ∆H V , from their respective bulk crystal structures. The accompanying open-source dataset specifically computes neutral cation and oxygen vacancies in ∼200 compounds, to which we added the neutral oxygen vacancy formation energies for ∼50 more structures from the work by Wexler et al 22 in this study. Now, we use MatFold to generate ∼2,700 possible splits and train/test our model performance, as summarized in Figure and Figure , to better understand the modeling approach's generalizability, improvability, and uncertainty.