A transformer architecture trained on a SMILES (simplified molecularinput line-entry system) string representation of a molecule was used. The purpose of the study is to compare a novel framework for training a molecular transformer with a regularized loss function with existing unconstrained training framework . However, our framework can in general be applied to any transformer model based on molecular pairs constructed with a similarity metric. Formally, we denote with X the chemical space, and with P = {(s, t) | s, t ∈ X × X } the set of molecular pairs constructed from X , s and t are the source and target molecules, respectively. Each element of sources and targets is a token (symbol) which takes values from a vocabulary (ordered set) V . Furthermore, we denote with f θ : X × X → [0, 1] |V | the transformer with θ being the set of its parameters. To simplify, we denote with the same symbol a molecule and its SMILES string representation. The transformer, f θ , consists of an encoder and a decoder which are simultaneously fed by source molecules and target molecules, respectively. Source molecules and target molecules are first tokenized and then converted into sequences of dense vector embeddings and passed over several stacked encoder and decoder layers, respectively. The final encoder output is merged into the decoder through a multihead attention mechanism which captures the transformation from a source molecule to a target molecule. Figure shows the general architecture of our transformer model, more details about the transformer architecture can be found in . f θ models the probability p of the ℓ-th token of a target t iℓ given all the previous t i,1:ℓ-1 = t iℓ-1 , . . . , t i1 target tokens and source s i compound, i.e. f θ (t i,1:ℓ-1 , s i )[t iℓ ] = p(t iℓ |t i,1:ℓ-1 , s i ). The transformer's parameters θ are then trained on P by minimizing the negative log-likelihood (NLL) of the entire SMILES string, p(t i |s i ), (s i , t i ) ∈ P for all i = 1, . . . , |P|, that is