nn.Transformer
In PyTorch 1.2, we now include a standard nn.Transformer module, based on the paper “Attention is All You Need”. The nn.Transformer module relies entirely on an attention mechanism to draw global dependencies between input and output.  The individual components of the nn.Transformer module are designed so they can be adopted independently. For example, the nn.TransformerEncoder can be used by itself, without the larger nn.Transformer. The new APIs include:

nn.Transformer
nn.TransformerEncoder and nn.TransformerEncoderLayer
nn.TransformerDecoder and nn.TransformerDecoderLayer

