A key advantage of transformers is their ability to capture long-range interactions through attention mechanisms, which enables non-local electronic and steric effects to be captured -both crucial for TS geometry prediction. A limitation of transformers is their architectural complexity, which requires significant computational resources and large datasets to train effectively. However, transformers have been shown to achieve strong generalisation to new chemical spaces when pretrained on large molecular datasets and finetuned on smaller, task-specific datasets. This approach could be particularly useful for TS prediction, where data is limited and costly to obtain. Moreover, standard transformers lack 3D spatial awareness because their input encodings do not inherently capture spatial distances, which may lead to the loss of critical geometric information in TSs. To effectively process molecular geometries, transformer models often require the integration of geometric models, such as equivariant architectures, which can increase the complexity of the architecture further. For more information on transformers, please refer to Ref. .