In the first part of this study, we explore what constitutes OOD data in the molecular domain. While some researchers suggest that splitting based on Bemis-Murcko scaffolds significantly impacts model performance, as exemplified in the MoleculeNet benchmark, recent studies have questioned this view. To resolve this ambiguity, we systematically create and evaluate different types of test sets that could potentially be considered OOD, providing a comprehensive analysis of their characteristics and impact on model performance.