For each high-level task -biological activity prediction and molecular property prediction -Chemformer models were trained simultaneously on all subtasks. The models were then evaluated separately on each subtask, to facilitate easy comparison. Property prediction models were trained on all 20 dataset splits and an average of the evaluation results was taken. Hyperparameters were tuned separately for each Chemformer model, and, to combat overfitting, the size (number of layers, attention heads, model dimension and feed-forward dimension) of the randomly initialised model was also tuned. The full details of the tuned hyperparameters for each Chemformer model can be found in the supplementary information.