The original Chemformer vocabulary was constructed by applying regular expression matching, similar to Schwaller et al. , to the canonical SMILES of the molecules in the ChEMBL 27 . The total 523 tokens include 250 chemical tokens, 200 tokens which are unused during the pre-training stage, but filled during fine-tuning, and 73 tokens meta-tokens, such as masking, padding, beginning-of-sentence and end-of-sentence tokens, or tokens for the originally implemented molecular optimisation task. Tokenization and augmentation of SMILES was performed by the ChemformerTokenizer which extends the routines in PySMILESUtils .