In this work, we attempt to provide a comprehensive outline of SMILES-based neural molecule generation problem: formulation of a language model, preprocessing of sequence data, generation strategies, and evaluation of models and generated data. We first formulate a general framework of estimating the joint probability of a sequence, and given a SMILES data corpus, discuss data preparation methods suited for this framework. Next, we present a neural network architecture for language modeling, focusing particularly on recurrent neural networks. Using trained neural networks, we describe how to generate novel data and assess their validity, diversity, and other molecular properties. Finally, we test the presented SMILES language framework on a large corpus and discuss the obtained results as well as future perspectives regarding new language models. In our repository MOLECULEGEN-ML a , we introduce a Python package for experimenting with the presented methods and reproducing the benchmarking results. Neural networks in the package are implemented using MXNET backend and follow its GLUON API ; evaluation methods use RDKIT package .