Large volumes of high-quality reaction data are typically very difficult and expensive to obtain, and thus, the importance of the data set sizes and sources used for the pre-training, in context of the Furthermore, we implemented and examined Transformer-specific adversarial training approach for yield prediction. Neural networks are vulnerable to adversarial examples, i.e. they easily misclassify examples with slight, often imperceptible to humans, non-random perturbations, which otherwise are assigned correct labels. The appropriate perturbations to any given example are found by maximising the prediction error of the network for that example. The work of Goodfellow and colleagues showed that incorporation of adversarial examples into training of neural networks can reduce overfitting, improve performance and in case of the natural language processing also improve the quality of word embeddings. Thus, it is reasonable to assume that application of adversarial training to predictive synthesis tasks would increase the robustness of the networks for predictions involving unseen structures. Our implementation of the adversarial training is analogous to the previous language-based studies on adversarial learning for BERT, where we conduct the adversarial training by perturbing the embeddings calculated for the network inputs. However, in our approach, the embeddings are modified dynamically by a specified fraction of their original value, and not by a predefined constant. We show that employment of the adversarial training during pre-training can have a positive impact on the downstream yield and reaction success predictions without additional effort from the practitioner.