These observations can be explained and understood based on the hardness of the splitting approach (see section Evaluation of the Hardness of Splitting Strategies). We observe that random and scaffold splitting are among the "easiest" splitting approaches (Figures and), which means that they do not create a distinct chemical space for the OOD test set. For these approaches, the relationship between ID and OOD is strongly correlated. In contrast, K-means and max dissimilarity are among the "hardest" approaches, which can create an OOD test set with a distinguishing chemical space compared to the training set. In that case, there is a weaker correlation between ID and OOD ROC-AUC.