Regarding the relationship between ID and OOD performance, we observed that the strength of the correlation substantially depends on the splitting strategy used for generating OOD data. OOD data with the least distinguishing chemical space from the training (measured based on the distance metric) led to a strong correlation between ID-OOD performance. However, as chemical space similarity decreased, this correlation weakened substantially. This observation indicates that ID performance is not always a good proxy and criterion for model selection with the best generalization capacity.