This study addresses the challenges posed by OOD data to ML by systematically evaluating the performance of ML models on OOD data in drug discovery, particularly molecular property prediction (Figure ). We examine a comprehensive range of approaches, from classical tabular methods such as random forests (RF) operating on molecular fingerprints to message-passing graph neural networks (GNNs) working directly with molecular graphs. Following recent findings by Gulrajani et al. that demonstrate the effectiveness of Empirical Risk Minimization (ERM) compared to specialized domain generalization methods, we employ ERM as our main training algorithm for this benchmark study.