3) Output: Decoding encoder's sequence representations into probability distributions for the subsequent tokens. Can be a linear projection followed by softmax, gumbel-softmax , sparsemax , or analogous probability transformations. Note that using learnable embedding allows parameter sharing (or tying weights) with the output block as it can similarly be viewed as the output token embedding. It reduces the model size and creates an effect of optimizing the regularized objective taking into account the similarity between the target token and other tokens from the vocabulary. SMILESLM predicts the probabilities of the next token given the previous context. To train the model, we need an objective that maximizes predicted probabilities to the target tokens. It is equivalent to minimizing the cross-entropy loss between one-hot encoded target distributions and predicted probabilities of tokens: for a target minibatch Y ∈ V ×τ and output probabilities P ∈ [0, 1] ×τ ×|V| ,