1. Transformer sequence-to-sequence models generally struggle to identify the reaction centre and stabilising functional groups within the target molecule. Instead, the model learns to translate one SMILES sequence into another. The masked attention model confines its attention to a select few tokens. Consequently, the task resembles a SMILES sequence correction from the product to the reactants. The Transformer does not appear to acquire an understanding of the underlying chemistry. Without chemical understanding, the Transformer would therefore strongly benefit from a larger database to identify patterns within the data. If the Transformer can discover novel chemistry without chemical understanding remains to be seen. Nevertheless, there is no intent to discourage the use of Transformers for retrosynthesis. To address the lack of interpretability and chemical understanding, researchers could explore the development of an uncertainty measure that provides insights into the model's reliability in its predictions. Such a measure would enable the end-user to place a higher level of trust in the model's predictions.