Unlike sequence-to-sequence tasks, discriminative tasks only make use of the encoder stack of the model. Firstly, the tokenised SMILES string of a molecule is prefixed with one or more task tokens -gene symbols for biological activity prediction, or molecular properties for QSAR modelling. This sequence of tokens is passed through the model's embedding layer, followed by the model's encoder. The output vector for each task token is then passed through a small multi-layer perceptron (MLP) head to produce either a class distribution vector or a single output number for classification and regression tasks, respectively. Since we only investigated regression tasks in this work, a mean squared error loss function is applied to the MLP output for each task token. We augmented the input SMILES string for discriminative tasks with p aug = 1.0 and, as with sequence-to-sequence tasks, this augmentation was performed on-the-fly during training.