(Prototype) BetterTransformer: Fastpath execution for Transformer Encoder Inference
PyTorch now supports CPU and GPU fastpath implementations (“BetterTransformer”) for several Transformer Encoder modules including TransformerEncoder, TransformerEncoderLayer, and MultiHeadAttention (MHA). The BetterTransformer fastpath architecture Better Transformer is consistently faster –  2x for many common execution scenarios, depending on model and input characteristics. The new BetterTransformer-enabled modules are API compatible with previous releases of the PyTorch Transformer API and will accelerate existing models if they meet fastpath execution requirements, as well as read models trained with previous versions of PyTorch. PyTorch 1.12 includes: 
- BetterTransformer integration for Torchtext’s pretrained RoBERTa and XLM-R models
- Torchtext which builds on the PyTorch Transformer API