The Chemformer models were firstly trained in a self-supervised manner and the learnable weights were saved. These weights were then loaded separately for each downstream task of interest and the task-specific fine-tuning procedure took place. Figure provides an overview of how the pre-training and downstream fine-tuning tasks are applied to the Chemformer model.