The language modeling approach for de novo drug design uses the Simplified Molecular Input Line Entry Systems (SMILES) representation of molecules, which encodes molecular structure as a sequence of tokens . Recent prospective applications have experimentally verified the potential of SMILES string-based generative de novo design of small molecules with the desired bioactivity . An essential element of these prospective applications is transfer learning , which is the process of transferring knowledge acquired to solve one task to another related task. In the first step (pretraining), the "chemical language" of bioactive molecules is learned by training a model on a large set of SMILES data with known bioactivities.