This work demonstrates that training a transformer model proceeds from learning to creatively generate more scaffolds of bioactive molecules toat the stage of overtraininggenerating new decorations to known scaffolds. The optimal state of a model is reached at the border between these two regimes, ensuring a maximal chemical diversity of the generated structures. We also show that perplexity can serve as a score of the quality of de novo generated molecules, in agreement with previous work, which differs from our work in the following aspects. In that paper, perplexity was used to score SMILES (not SELFIES) of molecules generated with an LSTM (not transformer) model, and Tanimoto similarities on Morgan and topological pharmacophore fingerprints between generated and true bioactive molecules were used as the chemical scores. This conclusion is of practical importance, because chemical scores typically do not provide a differentiable loss function for ML training, and can realistically be computed only for validation of a trained model. This work has certain limitations and leaves some important questions unanswered. We did not perform an exhaustive hyperparameter search and explored the effect of only two parameters on the quality of generated molecules: the number of training epochs, and the cutoff value N for filtering the training set. Another limitation is that only one output molecule was generated per input molecule, which could be particularly restrictive when only few initial hits are available, as is often the case early in the hit expansion process. Increasing the number of generated molecules could further improve the practical applicability of the method, though causing a need for additional prioritization, a complication that we tried to evade in this work. Also, note that the training sets were formed by pairs of molecules without ordering by bioactivity. Moreover, two molecules in a pair could be active against different targets. As a result, models reported here cannot be expected to bias generation of molecules towards higher bioactivity. They only perform medicinallychemically-meaningful steps in the chemical space, and the goal of improving bioactivity (or optimizing ADMET properties, synthesizability, etc., under given restrictions on bioactivity) can be solved, for example, by postprocessing (triaging/filtering) the output of such generative models, or including one into a larger reinforcement-learning-type ML model. Note, however, that despite these limitations, the most active molecules against COX2, DRD2, and HERG have been successfully recovered or approached to in this work. Another direction of performance improvement of the model would be to run fine-tuning on data referring to the same or similar protein targets, possibly ordering the molecular pairs in the training set by their bioactivity against these targets. Finally, an ability to specify target or binding site information as part of training and inference could further improve the performance of such generative models.