To create a comprehensive generative language model, we need a large and diverse set of SMILES data. ChEMBL database comprises millions of molecular compounds with their measured biological activity. One can freely download the database and collect a desirable set of SMILES strings of synthesized molecules. Note that for more convenient modeling, canonicalization (determining one special SMILES representation among all valid ones) and subsequent removal of duplicates may be required. Also, lengths of SMILES strings can be much longer than average suggesting that large outlier molecules can reasonably be filtered out. These and other preprocessing techniques, including charge neutralization and removal of salts and molecules with prohibited subcompounds, were performed by Brown et al. , which led to a dataset of almost 1.3 million SMILES strings from ChEMBL 24. We will train and evaluate our language framework on this dataset.