Our SMILES-tokenizer was pretrained from SMILES-GPT. This transformer decoder replicates GPT-2 except that during tokenization, it used the character-level byte-pair encoding instead of byte-level. We reserved 72 characters from the SMILES alphabet as an initial vocabulary and supplemented the vocabulary with up to 1000 most frequent merges. The model uses parameterized token and position embeddings, 8 attention heads, and 4 attention blocks. With the embedding/hidden dimension of 512, it has 13.4M parameters.