In low-data scenarios, transfer learning is often utilized rather than training from scratch. Transfer learning allows to 'pretrain' a CLM on a large corpora of molecules, and later to fine-tune it on taskspecific data (e.g., bioactive molecules) to learn the underlying property distribution. To test the potential of the augmentation techniques with transfer learning, we pre-trained a CLM on 1.5 M SMILES strings from ChEMBL. The pre-trained CLM was then fine-tuned on the molecules tested on three targets, separately: (1) Peroxisome Proliferator Activated Receptor d (PPARd), ( ) Serine/threonine-protein kinase (PIM1), and (3) Janus kinase 2 (JAK2). For each target, we created two groups of molecules based on their pairwise substructure similarity (determined via Tanimoto similarity on extended connectivity fingerprints ): (1) 'high-similarity' molecules, having pairwise similarity larger than or equal to 0.8, and (2) 'low-similarity' molecules, whose pairwise similarity was equal to or lower than 0.4. For each of these two similarity scenarios, we created two fine-tuning sets of 10 and 100 molecules. In total 12 datasets were used for model fine-tuning and molecule generation (1000 SMILES strings sampled across four repetitions) with each augmentation strategy. 10-fold augmentation was applied to all approaches, whenever possible. For bioisosteric substitution and functional group masking, if 10 augmented SMILES could not be generated (due to a limited number of functional groups to be replaced), augmentation until saturation was performed (Supporting Table ). Validity, uniqueness, and novelty were monitored for 'sanity check' 39 (Supporting Table ).