Validating the dGNN with defect-wise CV is a less challenging task, since train and test sets may contain defects from the same material in similar chemical environments (i.e., just above the symmetry tolerance). Validating with compound-wise CV is more challenging, since the test set contains all the defects from a given material (none of which may appear in the training set). This better reflects the practical performance for materials discovery, because one is usually interested predicting for materials for which no DFT vacancy calculations have been performed and thus not a single defect site could exist in training set. Finally, element-wise, leave-one-out CV proves the hardest challenge, where all defects for a compound containing the held-out element are placed in the Figure shows the results of the element-wise, leave-one-out CV for each of the four different encoding strategies. While this is not a task that one might expect a deep learning approach to succeed at, several elements are well predicted and close to the target error of 0.5 eV, regardless of encoding type. Yet other elements are poorly predicted for all encoding types. Without any a priori knowledge of which held-out elements are well-predicted, it is not possible to assume that predictions on materials with unseen cation types can achieve MAE O K < 0.97 eV (Figure ).