In addition to stronger performance on downstream tasks, transfer learning can also significantly speed up training convergence. Figure illustrates the considerable effect pre-training can have on performance and convergence speed for the retrosynthesis task. Firstly, the Chemformer model pre-trained on the combined task is able to outperform the existing SMILES-based state-of-theart, the Augmented Transformer, with 20 epochs of fine-tuning. This corresponds to fewer than 30 minutes of training on one GPU. In addition to this, fine-tuning for 50 epochs provides a better top-1 result than 500 epochs of training from randomly initialised weights -an order of magnitude difference in training time.