But before we begin, there are three special tokens that we should take into consideration during both training and generation processes. First, we need a special beginning-of-SMILES token {, from which a model learns how to sample the very first token. Of course the model will be able to capture any subsequence as the starting fragment of a sequence being generated, but we might want to commence generation arbitrarily. Therefore, every SMILES string will be prepended by { prior to training. Next, our model should somehow decide when to terminate sequence generation. One option is to define the maximum number of tokens hyperparameter. However, this will force us to guess a "good number" for every novel sequence. Instead, we will append a special endof-SMILES token } to every SMILES string prior to training so that the model will also learn and sample } indicating the termination of the process. The maximum number of tokens will be a supplementary option if we wish to force the creation of medium-length sequences. Finally, some SMILES strings may be shorter than the chosen time span, so we will have to lengthen them with a special padding token _ that will be ignored during training and not likely be sampled during generation. Now, let us introduce minibatch sampling strategies. Let be a minibatch size and τ time span. Let S = s 1 , . . . , s L be a training corpus with SMILES strings s i , 1 ≤ i ≤ L. Suppose we have already tokenized the data and obtained a vocabulary V, which also includes tokens {, }, and _. Our goal is to produce an input subsequence X ∈ V ×τ and target/output subsequence Y ∈ V ×τ (the input shifted by one token).