class_docstr: Decoder layer block.
  In the original paper each operation (multi-head attention, encoder
  attention or FFN) is postprocessed with: `dropout -> add residual ->
  layernorm`. In the tensor2tensor code they suggest that learning is more
  robust when preprocessing each layer with layernorm and postprocessing with:
  `dropout -> add residual`. We default to the approach in the paper, but the
  tensor2tensor approach can be enabled by setting
  *args.decoder_normalize_before* to ``True``.
  Args:
      args (argparse.Namespace): parsed command-line arguments
      no_encoder_attn (bool, optional): whether to attend to encoder outputs.
          Default: ``False``
class_docstr_tok: ["Decoder","layer","block",".","In","the","original","paper","each","operation","(","multi-head","attention",",","encoder","attention","or","FFN",")","is","postprocessed","with",":","`","dropout","-",">","add","residual","-",">","layernorm","`",".","In","the","tensor2tensor","code","they","suggest","that","learning","is","more","robust","when","preprocessing","each","layer","with","layernorm","and","postprocessing","with",":","`","dropout","-",">","add","residual","`",".","We","default","to","the","approach","in","the","paper",",","but","the","tensor2tensor","approach","can","be","enabled","by","setting","*","args.decoder_normalize_before","*","to","``","True","``",".","Args",":","args","(","argparse.Namespace",")",":","parsed","command-line","arguments","no_encoder_attn","(","bool",",","optional",")",":","whether","to","attend","to","encoder","outputs",".","Default",":","``","False","``"]
docstr: Args:
      x (Tensor): input to the layer of shape `(seq_len, batch, embed_dim)`
      encoder_padding_mask (ByteTensor): binary ByteTensor of shape
          `(batch, src_len)` where padding elements are indicated by ``1``.
  Returns:
      encoded output of shape `(batch, src_len, embed_dim)`
code: def forward(
      self,
      x,
      encoder_out,
      encoder_padding_mask,
      incremental_state,
      prev_self_attn_state=None,
      prev_attn_state=None,
      self_attn_mask=None,
      self_attn_padding_mask=None,
  ):
      """
      Args:
          x (Tensor): input to the layer of shape `(seq_len, batch, embed_dim)`
          encoder_padding_mask (ByteTensor): binary ByteTensor of shape
              `(batch, src_len)` where padding elements are indicated by ``1``.
      Returns:
          encoded output of shape `(batch, src_len, embed_dim)`
      """
      residual = x
      if "residual" in self.more_dropouts:
          residual = F.dropout(
              residual, p=self.dropout, training=self.training
          )
  
      if prev_self_attn_state is not None:
          if incremental_state is None:
              incremental_state = {}
          prev_key, prev_value = prev_self_attn_state
          saved_state = {"prev_key": prev_key, "prev_value": prev_value}
          self.avg_attn._set_input_buffer(
              incremental_state, saved_state
          )
      x, _ = self.avg_attn(
          value=x,
          mask_future_timesteps=True,
          incremental_state=incremental_state,
          mask_trick=self.training,
      )
      if "after_avg" in self.more_dropouts:
          x = F.dropout(x, p=self.dropout, training=self.training)
  
      if self.aan_layer_norm is not None:
          x = self.maybe_layer_norm(self.aan_layer_norm, x, before=True)
  
      if self.aan_ffn is not None:
          x = self.aan_ffn(x)
          if "after_ffn" in self.more_dropouts:
              x = F.dropout(x, p=self.dropout, training=self.training)
  
      if self.aan_gating_fc is not None:
          i, f = self.aan_gating_fc(
              torch.cat([residual, x], dim=-1)
          ).chunk(2, dim=-1)
          x = torch.sigmoid(f) * residual + torch.sigmoid(i) * x
          if "after_gating" in self.more_dropouts:
              x = F.dropout(x, p=self.dropout, training=self.training)
  
      x = F.dropout(x, p=self.dropout, training=self.training)
      x = residual + x
  
      if self.aan_layer_norm is not None:
          x = self.maybe_layer_norm(self.aan_layer_norm, x, after=True)
  
      attn = None
      if self.encoder_attn is not None:
          residual = x
          x = self.maybe_layer_norm(
              self.encoder_attn_layer_norm, x, before=True
          )
          if prev_attn_state is not None:
              if incremental_state is None:
                  incremental_state = {}
              prev_key, prev_value = prev_attn_state
              saved_state = {
                  "prev_key": prev_key,
                  "prev_value": prev_value,
              }
              self.encoder_attn._set_input_buffer(
                  incremental_state, saved_state
              )
          x, attn = self.encoder_attn(
              query=x,
              key=encoder_out,
              value=encoder_out,
              key_padding_mask=encoder_padding_mask,
              incremental_state=incremental_state,
              static_kv=True,
              need_weights=(not self.training and self.need_attn),
          )
          x = F.dropout(x, p=self.dropout, training=self.training)
          x = residual + x
          x = self.maybe_layer_norm(
              self.encoder_attn_layer_norm, x, after=True
          )
  
      residual = x
      x = self.maybe_layer_norm(self.final_layer_norm, x, before=True)
      x = self.ffn(x)
      x = F.dropout(x, p=self.dropout, training=self.training)
      x = residual + x
      x = self.maybe_layer_norm(self.final_layer_norm, x, after=True)
  
      return x, attn
  
imports: import math
  import torch
  import torch.nn
  import torch.nn.functional
  from fairseq import options, utils
  from fairseq.models import FairseqEncoderDecoderModel, FairseqIncrementalDecoder, register_model, register_model_architecture, transformer
  from fairseq.modules import AdaptiveSoftmax, LearnedPositionalEmbedding, MultiheadAttention, SinusoidalPositionalEmbedding
  from pytorch_translate import transformer, vocab_reduction
  from pytorch_translate.average_attention import AverageAttention, AverageWindowAttention
  from pytorch_translate.utils import torch_find
label_desc: Use image node_id 3 for calling the TransformerAANDecoderLayer obj's underlying member method code with example usage: obj.forward(x, encoder_out, encoder_padding_mask, incremental_state, prev__attn_state, prev_attn_state, _attn_mask, _attn_padding_mask) and returns: x,  attn