It is also of interest to validate the novel transformer model for retrieving similar compounds in a series of interest to a medicinal chemist. Therefore 200 chemical series from the recent literature were extracted from the ChEMBL database and collected in ChEMBL-series. The 200 series were used to evaluate if the trained transformer model can efficiently retrieve near-neighbours of interest. This was done through retrospectively investigating if known near-neighbours could be identified within the 200 chemical series. All the compounds within the 200 series are different from the training set for the transformer models, having been published after our training set was created. As an illustration, Figure depicts a chemical series in ChEMBL-series consisting of five compounds. Here, the series is represented as a graph where nodes are the compounds in the series and edges depict the NLL for the transformation of the source molecule to the target molecule and the Tanimoto similarity (calculated with ECFP4 count fingerprints) for the compound pairs. Note, that the Tanimoto similarity is symmetric for the molecular pair but the NLL is not, meaning that in general p(t|s) ̸ = p(s|t), where s and t are the source molecule and target molecule, respectively. The transformer used in this experiment was trained with a regularization term and the ECFP4 count fingerprint. As expected, the NLL is strongly correlated with similarity and the NLL is lower for similar compounds and higher for dissimilar compounds. The difference between p(t|s) and p(s|t) is small when the compounds are similar. The lower-left plot in Figure shows the correlation between the molecular similarity and the NLL evaluated for all pairs in all of the 200 extracted chemical series. The correlation coefficient is 0.88, confirming the efficacy of the proposed ranking loss used in the training of the transformer. The similarity between compounds in the chemical series can be as low as 0.2, since the series assignment is solely based on that the compounds were extracted from the same publication. We observed markedly-poorer correlation between the NLL and the similarity for compound pairs with low similarity. The NLL decreases more rapidly for compound pairs with low similarity (deviations from the linear fit in the Figure ). This is expected since our training of the transformer only includes source-target molecular transformations where the similarity is above ≥ 0.50, i.e. the model has not been trained on molecular transformations of pairs with a similarity below < 0.50 and accordingly these source-target molecular transformations have a low probability to be generated. The lower-right plot in Figure depicts the maximum NLL as a function of the beam size. Here, the NLL consistently increases as the beam size increases, allowing an exploration of an increasingly dissimilar chemical space. We repeated the analysis for all of the 200 compound series, for each series we considered in-turn all of the compounds as source molecules. Table shows the results for beam sizes equal to 1,000, 5,000, and 10,000. To understand the results in Table we need to introduce two cutoff thresholds, t nll and t sim , calculated for each series and each source compounds in the series. t nll is defined as the highest NLL associated to a source compound found by beam search, while t sim is defined as the similarity between the source compound and the target compound with the highest NLL found by the beam search. The choice of t sim is reasonable as we report averaged results and therefore potential error would be cancelled out, as confirmed by the low standard deviations reported in Table . We define as true positive (TP) the percentage of target molecules found by beam search, with true negative (TN) the percentage of target molecules not found by beam search where their similarity to the source molecule is lower than t sim , and with false negative (FN) the percentage