The predictive power of a consistent model must remain stable when applied to data that comes from a di↵erent source than the training set. Moreover, possible redundancy in the data must be controlled. Proteins are divided into families, which usually have similarities in sequence or structure. Compounds might be part of the same chemical series. The performance of classification model should be tested when applied to families of proteins or compounds with di↵erent sca↵olds than those used to train it. On the latter, Wallach and Heifets concluded that performance of most of the reported ligand-based classification problems reflect overfitting to training benchmarks rather than good prospective accuracy, mainly because of the redundancy between training and validation sets. This issue becomes more critical when using random cross-validation: in the pharmaceutical field, compounds are usually synthesized serially to enhance molecular properties. This leads to training and validation sets following the same distribution, which is desirable in most machine learning problems, but a poor estimate of reality in drug discovery. Time-split validation is common practice in pharmaceutical environment to overcome this issue. This strategy is well suited to the realistic scenario, where we are interested in prospective performance of the models. However, most public data lack of temporal information, hindering this strategy to be applied. Additionally, time-split data has also shown to be biased because of the high similarity between discovered actives in di↵erent phases. Other techniques have been applied to reduce bias and data redundancy between training and validation sets. Unterthiner et al. clustered compounds using single linkage to avoid having compounds sharing sca↵olds across training and validation sets. Rohrer and Baumann designed the Maximally Unbiased Validation (MUV) benchmark to be challenging for standard virtual screening: actives have been selected to avoid biases of enrichment assessment and inactives have been biologically tested against their target. 32 Xia et al. presented a method to ensure chemical diversity of ligands while keeping the physicochemical similarity between ligands and decoys. Wallach et al. removed analogue bias in active molecules by clustering and selected decoys to match in sets to actives with respect to some 1D physicochemical descriptors while being topologically dissimilar based on 2D fingerprints. However, these unbiasing techniques only focus on redundancy between actives, overlooking the impact of inactive-active or inactive-inactive similarity, which leads to models memorizing the similarity between benchmark inactives and hence, overfitting. Another related issue is that the possible bias across the di↵erent data sources used in some studies has not been properly studied yet. Di↵erent datasets might have di↵erent structure, a↵ecting to the study of the generalization of the model. A related issue is found in the study of Altae-Tran et al, where after the collapse of their transfer learning experiments it is a rmed that one-shot learning methods may struggle to generalize to novel molecular sca↵olds, and that there is a limit to their cross-tasks generalization capability.