Several CLM architectures have been proposed for de novo design , the most popular of which are long short-term memory (LSTM) models. LSTMs are trained to produce molecular strings element-by-element and have fast generation capabilities. However, the iterative structure forces those models to compress the sequence into an information bottleneck and challenges the learning of global sequence properties . Transformers , a more recent architecture, overcome this bottleneck by processing the entire input molecular string at once and can learn certain molecular properties better than LSTMs . While this holistic look at the input is promising, it makes Transformers computationally intensive for string generation, thereby potentially limiting their chemical space exploration capabilities. These aspects make it necessary to stretch the boundaries of current CLM approaches further, to chart the chemical space more effectively in search for bioactive molecules .