We trained our model for 6 epochs on the SMILES strings of QM7b datasets. We used AdamW for optimization and cosine annealing for learning-rate scheduling. The initial and final learning rates were set to 5e -4 and 5e -8, respectively. We kept the default Adam hyperparameters and optimized the batch size (128) and maximum sequence length (512).