To calculate the final loss (𝐽 ̃) the model is traversed twice. In the first model pass the input is tokenized, embedded, processed by the Encoder with the task specific head and the loss for the predictions (𝐽) is calculated. The gradient for the input embeddings (∇ 𝑥 𝑒𝑚𝑏𝑒𝑑 ) is determined and the element-wise sign of this gradient is found. A product of the sign and the hyperparameter 𝜖 is then multiplied by the embeddings, resulting in embedding perturbation values. These perturbations are summed with the original embeddings, giving new, adversarial embeddings. In the second pass through the model, the network skips the embedding calculations, instead it uses the provided adversarial embeddings directly as the input to the Encoder. The network performs the predictions, and the "adversarial" loss is calculated for the second pass. The total, final loss, used for the model weights update, is obtained by summing the weighted original loss from the first pass, and the "adversarial" loss from the second pass (Figure ). The contribution of the adversarial loss to the final loss is controlled by the 𝛼 hyperparameter (eq. 1). A model pre-trained using the smallest Pistachio set of 17K reactions was selected as the baseline for the evaluation of two variants of the adversarial training. In the first variant, the adversarial approach was used only during pre-training with the Pistachio 17K set, and a standard fine-tuning was employed.