The impact of the size of the pre-training reaction data on the downstream task was examined. From the full, cleaned Pistachio set containing a total of 1.68M reactions we randomly sampled 17K, 84K, 421K and 842K examples, corresponding to 1%, 5%, 25% and 50% of the data, respectively. Despite different data set sizes, all models were pre-trained with the same number of steps, to assure a fair comparison. To further increase the robustness of the analysis, each evaluated model was pre-trained three times starting with different initial weights, and then each of those pre-trained models was finetuned three times with different random seeds on the downstream task. This gave a total of nine runs for each of the examined approaches for each data split. Interestingly, for a regression task with the Buchwald-Hartwig and Suzuki sets, a statistically significant difference between pre-training data sizes was observed only for a couple of data splits (Figures and S34-S35, Tables ). Although, the performance of the model pre-trained using 17K of the Pistachio reactions was lower in a small number of cases, increasing the data size to only 84K examples resulted in no noticeable decrease in yield prediction performance when compared to larger sets. Analogous conclusion was drawn for the results of the reaction success classification task (Figures 7, 8 and S36, Tables and). A potential explanation of why the relatively small number of examples in pre-training is sufficient for good model performance on the end-task is that the HTE data sets contain reactants with very simple structures and common functional groups for two popular reaction types. To confirm that a further addition of general domain reaction data to the pre-training stage does not give a performance benefit to the models, a pre-training was performed with a filtered, standardised and deduplicated data set, consisting of 15M combined Reaxys and Pistachio reaction schemes. The evaluation showed that Reaxys-Pistachio set did not improve the metrics on the downstream yield and reaction success predictions over the performance when the Pistachio set is used alone (Figures 7, 8 and S34-36, Tables ). This finding is consistent with the report by Neves et al., who observed no meaningful advantage to pre-training the BEE model on Reactlake (Janssen's internal reaction data base containing combined USPTO, Reaxys, Pistachio and proprietary reaction data) versus pre-training using Pistachio alone, when evaluated on reaction success classification task with Janssen's ELN data. As a sanity check, models were also trained on the four HTE data sets without prior pre-training. Two variants were explored. In the first variant, the models were provided a pre-trained BPE tokenizer with the vocabulary of 500 tokens generated for the full 1.68M Pistachio set, whereas, in the second variant, the models were given no access to a pre-existing token dictionary, but the vocabulary was generated