Researchers have incentives to explore novel chemical space (Fig. ). Moreover, for each publication, reaction yield is biased by the chemist's skills and the way it is measured. This leads to a high heterogeneity between reactions from different publications. We evaluated how ML predicts yields on data from a new publication. A train-test split of the data, where the test set is comprised of all reactions from one publication, and the train set of all other reactions that do not appear in this publication, is used to assess the yield prediction performance of a model on a new publication (Fig. -DOI Hold-out). Our results showed the inability (R 2 of -0.01) of ML to generalize to data from new publications.