The transformer architecture was invented by Vaswani et al. in their paper "Attention is All You Need", which was published in 2017. The paper introduced the transformer model as an alternative to recurrent neural networks (RNNs) for sequence-to-sequence learning tasks, such as machine translation. The transformer architecture relies on self-attention mechanisms, which allow it to capture long-range dependencies between elements in a sequence without the need for recurrent connections. The transformer architecture has since become a popular choice for many natural language processing (NLP) tasks, and has been widely adopted in various applications, including language translation, question answering, and text generation.
spam fails_task lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor helpfulness creativity violence