We performed an extensive benchmark of RNN models trained with SMILES obtained from GDB-13 whilst exploring an array of architectural changes. First and foremost, models were trained with three different variants of the SMILES notation. One of them is the commonly used canonical SMILES, another one are randomized SMILES (also known as enumerated SMILES), which have been used as a data amplification technique and are shown to generate more diversity in some model architectures . The third one is DeepSMILES , a recently published modification of the canonical SMILES syntax. Secondly, models were trained with decreasing training set sizes (1,000,000, 10,000 and 1,000 molecules) to explore the data amplification capabilities of randomizes SMILES. Thirdly, the two most used recurrent cell architectures were compared (i.e., Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) ). GRU cells are widely used as a drop-in replacement of LSTM cells with an ostensible speed improvement, but it has been shown that in some tasks they perform worse . Fourthly, regularization techniques such as dropout in conjunction with different batch sizes were also tested and their impact on the generated chemical space assessed. All of the benchmarks were supported by a set of metrics that evaluate the uniformity, completeness and closedness of the generated chemical space. Finally, to demonstrate how the same methodology can be used to train models that generate realworld drug-like compounds, models were trained with a subset of the ChEMBL database.