• Bioisosteric substitution. Molecules were fragmented using the 'Breaking of Retrosynthetically Interesting Chemical Substructures' (BRICS) algorithm. The list of possible replacements for each substructure was retrieved from SwissBioisosteres. SwissBioisosteres, along with the possible replacements, also include the frequency of how many a certain bioisosteric replacement was found to occur (based on reported bioactivity). The top five most frequent replacements were chosen as candidates for augmentation. Each molecule was parsed for 'augmentable' fragments, the matching fragments were substituted randomly with one of the candidate fragments, and the molecule was then re-assembled and converted into a SMILES string. • Self-training. After hyperparameter optimization (as described below), the CLMs were trained with all available, non-augmented training SMILES strings (in their non-canonical version), using temperature sampling (T = 0.5, Eq. 1). The trained CLMs were used to generate de novo designs. Valid, novel, and unique SMILES strings were retained and used to augment the training set (with the selected augmentation fold). All procedures were applied to achieve the desired augmentation fold, and with the desired probability of perturbation (p) for token deletion, atom masking and bioisosteric substitution (p = 0.05, 0.15, 0.3).