In this work, a gated recurrent unit (GRU-based) neural network was used as a SMILES-based chemical language model (CLM). These well-detailed models learn to predict the next token (𝑥 𝑖+1 ) in a sequence of tokens (𝑋) by training on a corpus of example sequences while using teacher forcing to ensure that the predicted token at each index 𝑦 𝑖 is calculated with the ground truth previous token 𝑥 𝑖 as input. The model is then trained to maximize the likelihood assigned to the correct token at each index, conditional upon all previous tokens observed, which is then summed over the whole sequence as formulated in Equation . In this work, SMILES was used as the language-based chemical grammar and the model consisted of an embedding layer of size 256 followed by 3 layers of GRUs with a hidden dimension size of 512. The model was trained on the pre-training dataset described earlier for 5 epochs using a batch size of 128 and Adam optimizer with a learning rate of 0.001.