Nevertheless, the advent of generative AI models marks the beginning of a paradigm shift in the discovery and design of new materials. Among the most promising developments are transformer-based models that have been successfully applied to various molecular design tasks. For example, C5T5-type models have been used in the design of antiviral drugs. More recently, Matsukiyo et al. generated candidate molecules by exploring the latent space of a Transformer-based VAE, identifying inhibitors for the design of target proteins in therapeutic applications. Additionally, MolGPT, a lightweight generative pre-trained transformer model based on a masked-attention transformer-decoder architecture, demonstrated molecular generation according to desired scaffolds while controlling multiple properties. Furthermore, Tysinger et al. showed that transformers can be trained to make meaningful molecular modifications for hit expansion in bioactive molecular drug design. These examples highlight the powerful utility of transformers across multiple molecular design problems.