class_docstr: This is an implementation of transformer language model.
  
  The architecture is based on the paper "Attention Is All You Need": https://arxiv.org/pdf/1706.03762.pdf
  
  Arguments
  ----------
  d_model : int
      The number of expected features in the encoder/decoder inputs (default=512).
  nhead : int
      The number of heads in the multiheadattention models (default=8).
  num_encoder_layers : int
      The number of sub-encoder-layers in the encoder (default=6).
  num_decoder_layers : int
      The number of sub-decoder-layers in the decoder (default=6).
  dim_ffn : int
      The dimension of the feedforward network model (default=2048).
  dropout : int
      The dropout value (default=0.1).
  activation: torch class
      The activation function of encoder/decoder intermediate layer, relu or gelu (default=relu).
  decoder_use_memory: bool
      whether to use the hidden state in the decoder
  
  Example
  -------
  >>> src = torch.randint(0, 720, [8, 120])
  >>> net = TransformerLM(720, 512, 8, 1, 0, 1024, activation=torch.nn.GELU)
  >>> enc_out = net.forward(src)
  >>> print(enc_out.shape)
  torch.Size([8, 120, 720])
docstr: Arguments
  ---------
  src : tensor
      The sequence to the encoder (required).
code: def forward(self, src, hx=None):
      """
      Arguments
      ---------
      src : tensor
          The sequence to the encoder (required).
      """
      src_mask, src_key_padding_mask = self.make_masks(src)
  
      src = self.custom_src_module(src)
      if self.embedding_proj is not None:
          src = self.embedding_proj(src)
      src = src + self.positional_encoding(src)
      if self.num_encoder_layers > 0:
          encoder_out, _ = self.encoder(
              src=src,
              src_mask=src_mask,
              src_key_padding_mask=src_key_padding_mask,
          )
  
      if self.num_decoder_layers > 0:
          if self.decoder_use_memory:
              encoder_out, _, _ = self.decoder(
                  tgt=src,
                  memory=encoder_out,
                  tgt_mask=src_mask,
                  tgt_key_padding_mask=src_key_padding_mask,
              )
          else:
              encoder_out, _ = self.decoder(
                  src=src,
                  tgt=src,
                  tgt_mask=src_mask,
                  tgt_key_padding_mask=src_key_padding_mask,
              )
  
      pred = self.output_proj(encoder_out)
      return pred
  
file_path: speechbrain/speechbrain/lobes/models/transformer/TransformerLM.py
imports: import torch
  from torch import nn
  from speechbrain.nnet.linear import Linear
  from speechbrain.nnet.normalization import LayerNorm
  from speechbrain.nnet.containers import ModuleList
  from speechbrain.lobes.models.transformer.Transformer import TransformerInterface, get_lookahead_mask, get_key_padding_mask, NormalizedEmbedding
label_desc: Use image node_id 2 for calling the TransformerLM obj's underlying member method code with example usage: obj.forward(src, hx) and returns: pred