Figure : To the top one of the series from ChEMBL-series consisting of five compounds and displayed as a graph, where the nodes depict molecules and the edges depict the NLL of the molecular transformation and the Tanimoto Similarity (Sim in the plot) between the two connected nodes (molecules). Note that the NLL is not symmetric between a compound pair. The arrows on top of NLL denotes the direction of the molecular transformation from the source molecule. If s and t are connected then ---→ N LL denotes -log p(t|s) and ← ---N LL denotes -log p(s|t). The lower-left plot depicts the correlation between Tanimoto similarity (evaluated on fingerprints with counts) and the NLL for the molecular transformation for all the possible pairs contained in the 200 compound series. The red dots denotes all the pairs of the above series. The lower-right plot illustrates the maximum NLL reachable with different beam sizes. The filling color surrounding solid lines represent the variance. of remaining target molecules not found by beam search but should have been found in an ideal scenario. There are two types of FN: either due to beam search being an approximation of an exhaustive search, i.e. target compounds that have Tanimoto similarity greater than t sim and NLL lower than t nll (column FN-B in Table ), or due to precedence i.e. target compounds that have a Tanimoto similarity greater than t sim and NLL greater than t nll (column FN-P in Table ). In the latter case the transformation between the source and target molecule is not well precedented in the training set and accordingly the NLL for the molecular transformation will be high. Table reports mean and median, denoted with x and x, respectively for TP, FN, and TN. The majority of the target compounds are retrieved correctly (TP + TN), while the FN are due to the limitations with beam search (slightly more) and low precedence (slightly less) for the transformation between the source and target molecule. We additionally calculated the rate of FN identified for beam size 1,000 shifting to TP when the beam size increased from 1,000 to 5,000, having a decrease from 4,69% to 1,82% for FN due to the limitations of beam search, and from 3,41% to 3,14% for FN due to low precedence of the molecular transformation. Thus it is possible through increasing the beam size to compensate for that the beam search is only an approximation of exhaustive search. We also computed the same metrics for a beam size of 10,000. A further decline in FN due to the limitations with beam search was observed to 1.55%. FN due to limited precedence of the molecular transformation decreased to 2.94% with 10,000 beam size. The percentages of FN in Table for a beam size of 1,000, 5,000 and 10,000 show a decreasing trend with increased beam size. Thus it is shown that most near neighbours to a compound in a compound series can be retrieved with a transformer regularized with a similarity constraint during training. The analysis of different physico-chemical properties also shows that the generated target molecules follow the same property distributions as the source molecules (see Figure and Figure in Appendix D). Figure highlights some limitations of the proposed method. A compound series with five compounds is used to illustrate the limitations. A compound was selected as source (the left-most in Figure ) and the remaining are the target compounds. Beam search with two different beam sizes, 1,000 and 5,000, was used and it was checked which of the target compounds was retrieved by the beam searches. The maximum NLL reachable by beam search was 11.90 and 12.63 for beam sizes of 1,000, and 5,000, respectively. This means that the compounds (represented to the right of the vertical red line in Figure ) for which the NLL is above these numbers cannot be retrieved as a larger beam size would be required. There are two reasons why a specific target compound is not found. The first is due to the Table : The table gives an overview of how well near neighbours in a chemical series are retrieved with a transformer. True positive (TP -the higher the better), false negative (FN -the lower the better), and true negative (TN) for beam sizes (BS) equal to 1,000, 5,000, and 10,000 are shown. Columns FN-B and FN-P, represents the two types of false negatives. Either due to beam search being an approximation of exhaustive search or low precedence due the molecular transformation is not well represented in the training set, respectively. tnll and tsim represent the average t nll and t sim with standard deviations, respectively. x and x represent mean and median expressed in percentages, respectively, over the 200 chemical series. beam search algorithm being a trade-off between a greedy and an exhaustive search controlled by the beam size. We notice that for beam size equals to 1,000 the second compound in the series is not found even though its NLL is lower than 11.90. This happens because there is no guarantee that beam search will always find all the compounds below the maximum reachable NLL. The second reason is due to precedence as our method does not provide a perfect correlation between NLL and similarity. A high NLL i.e. low precedence can occur for similar molecules if the molecular transformation from the source molecule to the target molecule is not well represented in the training set. This might occur for instance for molecules containing unusual functional groups. Figure shows that the right-most compounds has higher similarity than other two but its NLL is higher, meaning that has lower precedence. The low precedence for the right-most compound might be due to the azo substructure. Unfortunately, determining which tokens are responsible for an increased NLL in relation to a target compound is a complex task. This complexity arises from the fact that the NLL is influenced by both the source molecule and the generated target tokens. It is possible that the NLL is spread across all the target tokens, or alternatively, it may remain low until a specific point and then sharply drop with the introduction of the next token in the target compound. This is a general limitation of transformer based architectures.