Considering the overall results generated for the random and out-of-sample splits, it appears that the implemented BERT architecture is not sensitive to the specific way a reaction is represented before tokenization, assuming that the text-based input correctly captures the chemical structures. If the appropriate tokenization method is used, the model does not seem to prefer one molecular representation over another. This finding appears to be reasonable and somewhat intuitive, when considered that the Encoder models only need to convert the information provided by the chemical representation into learned embeddings which are then used for a discriminative task, either regression or classification. With the exception of the MLM pre-training task, during which typically only small fragments of the input are masked and reconstructed, the Encoders do not need to generate new chemical structures, where the differences in the representation grammar and the difficulty of constructing correct representations would be more apparent between different methods. As a simple analogy, reading and understanding a sentence in different foreign languages (encoding) is generally significantly easier than writing one (generating). We thus suggest using the most convenient molecular representation appropriate for the available data. For the purpose of this study, we proceed with SMILES, as this is arguably the most common and easiest to process notation, natively compatible with multiple cheminformatics tools.