The ChemMLM models were implemented with a Python API, namely Hugging Face, which is a platform to implement multiple NLP transformer variants. The training calculations of the ChemMLM models were powered by Google Colab's GPU cloud service. The model structure details configured for the RoBERTa model are shown below. The dimensionality of the encoder layers is 768. The maximum token length that the model can be used with is 512, which covers all reaction SMILES token length (up to 333). In the encoder, the number of attention heads for each attention layer is 12, and the number of hidden layers is 6. The activation function used in the model is Gaussian Error Linear Units (GELU) function. The dropout probability for all fully connected layers in the embeddings and encoder is 0.1, and the dropout ratio for the attention probabilities is 0.1. The epsilon used by the normalisation layers is 10 -12 . The configuration of such a model creates 83,504,416 parameters.