class_docstr: Decoder layer block.
  In the original paper each operation (multi-head attention, encoder
  attention or FFN) is postprocessed with: `dropout -> add residual ->
  layernorm`. In the tensor2tensor code they suggest that learning is more
  robust when preprocessing each layer with layernorm and postprocessing with:
  `dropout -> add residual`. We default to the approach in the paper, but the
  tensor2tensor approach can be enabled by setting
  *args.decoder_normalize_before* to ``True``.
  Args:
      args (argparse.Namespace): parsed command-line arguments
      no_encoder_attn (bool, optional): whether to attend to encoder outputs.
          Default: ``False``
class_docstr_tok: ["Decoder","layer","block",".","In","the","original","paper","each","operation","(","multi-head","attention",",","encoder","attention","or","FFN",")","is","postprocessed","with",":","`","dropout","-",">","add","residual","-",">","layernorm","`",".","In","the","tensor2tensor","code","they","suggest","that","learning","is","more","robust","when","preprocessing","each","layer","with","layernorm","and","postprocessing","with",":","`","dropout","-",">","add","residual","`",".","We","default","to","the","approach","in","the","paper",",","but","the","tensor2tensor","approach","can","be","enabled","by","setting","*","args.decoder_normalize_before","*","to","``","True","``",".","Args",":","args","(","argparse.Namespace",")",":","parsed","command-line","arguments","no_encoder_attn","(","bool",",","optional",")",":","whether","to","attend","to","encoder","outputs",".","Default",":","``","False","``"]
code: def __init__(self, args, no_encoder_attn=False):
      super().__init__()
      self.embed_dim = args.decoder_embed_dim
      self.dropout = args.dropout
      self.relu_dropout = args.relu_dropout
      self.more_dropouts = args.decoder_aan_more_dropouts
      if args.decoder_attn_window_size <= 0:
          self.avg_attn = AverageAttention(
              self.embed_dim, dropout=args.attention_dropout
          )
      else:
          self.avg_attn = AverageWindowAttention(
              self.embed_dim,
              dropout=args.attention_dropout,
              window_size=args.decoder_attn_window_size,
          )
      # self.activation = getattr(args, "decoder_ffn_activation", "relu")
      self.aan_layer_norm = LayerNorm(self.embed_dim)
      if args.no_decoder_aan_ffn:
          self.aan_ffn = None
      else:
          aan_ffn_hidden_dim = (
              args.decoder_ffn_embed_dim
              if args.decoder_aan_ffn_use_embed_dim
              else args.decoder_ffn_embed_dim
          )
          self.aan_ffn = FeedForwardNetwork(
              self.embed_dim,
              aan_ffn_hidden_dim,
              self.embed_dim,
              num_layers=2,
              dropout=args.relu_dropout,
          )
  
      if args.no_decoder_aan_gating:
          self.aan_gating_fc = None
      else:
          self.aan_gating_fc = Linear(
              self.embed_dim * 2, self.embed_dim * 2
          )
      self.normalize_before = args.decoder_normalize_before
  
      if no_encoder_attn:
          self.encoder_attn = None
          self.encoder_attn_layer_norm = None
      else:
          self.encoder_attn = MultiheadAttention(
              self.embed_dim,
              args.decoder_attention_heads,
              kdim=args.encoder_embed_dim,
              vdim=args.encoder_embed_dim,
              dropout=args.attention_dropout,
              encoder_decoder_attention=True,
          )
          self.encoder_attn_layer_norm = LayerNorm(self.embed_dim)
  
      self.ffn = FeedForwardNetwork(
          self.embed_dim,
          args.decoder_ffn_embed_dim,
          self.embed_dim,
          num_layers=2,
          dropout=args.relu_dropout,
      )
  
      self.final_layer_norm = LayerNorm(self.embed_dim)
      self.need_attn = True
  
      self.onnx_trace = False
  
imports: import math
  import torch
  import torch.nn
  import torch.nn.functional
  from fairseq import options, utils
  from fairseq.models import FairseqEncoderDecoderModel, FairseqIncrementalDecoder, register_model, register_model_architecture, transformer
  from fairseq.modules import AdaptiveSoftmax, LearnedPositionalEmbedding, MultiheadAttention, SinusoidalPositionalEmbedding
  from pytorch_translate import transformer, vocab_reduction
  from pytorch_translate.average_attention import AverageAttention, AverageWindowAttention
  from pytorch_translate.utils import torch_find
label_desc: Use image node_id 1 to create a new TransformerAANDecoderLayer object from inherited base classes: nn with example: obj = TransformerAANDecoderLayer(args, no_encoder_attn)