To develop a language model of the chemical constitution of biologically active molecules, a training dataset was compiled from ChEMBL24 . Bioactive compounds with annotated bioactivities (IC50, EC50, Kd, Ki) <1 µM were extracted from this chemical database and standardized, resulting in a set of ~365k molecules. Each training molecule was presented to the chemical language model as a one-hot vector encoding, i.e., a computer readable format derived from the respective SMILES string (Fig. ). In the one-hot encoding format, each token of the SMILES string vocabulary has a unique mathematical vector representation of a predefined length (equal to 71 in this study). During model training, the chemical language model learns the conditional probability distribution of a token with respect to all the preceding tokens in the SMILES string (Fig. ). To optimize the applicability of the chemical language model to small training data sets, two different strategies were investigated, namely, data augmentation and temperature sampling.