class_docstr: Transformer decoder consisting of *args.decoder_layers* layers. Each layer
  is a :class:`TransformerAANDecoderLayer`.
class_docstr_tok: ["Transformer","decoder","consisting","of","*","args.decoder_layers","*","layers",".","Each","layer","is","a",":","class",":","`","TransformerAANDecoderLayer","`","."]
docstr: For average attention, prev_sum is initialized to zero.
  For encoder-decoder attention, key and value are computed once from
  the encoder outputs and stay the same throughout decoding.
code: def _init_prev_states(self, encoder_out):
      """
      For average attention, prev_sum is initialized to zero.
      For encoder-decoder attention, key and value are computed once from
      the encoder outputs and stay the same throughout decoding.
      """
      encoder_x, src_tokens, encoder_padding_mask = encoder_out
      states = []
      for layer in self.layers:
          # (bsz, channel)
          prev_sum = torch.zeros([1, layer.avg_attn.embed_dim])
          states.append(prev_sum)
  
          # (key, value) for encoder-decoder attention computed from encoder
          # output and remain the same throughout decoding
          key = layer.encoder_attn.k_proj(encoder_x)
          value = layer.encoder_attn.v_proj(encoder_x)
  
          # (key, value) kept in shape (bsz, num_heads, seq_len, head_dim)
          # to avoid repeated transpose operations
          seq_len, batch_size_int, _ = encoder_x.shape
          num_heads = layer.encoder_attn.num_heads
          head_dim = layer.encoder_attn.head_dim
          key = (
              key.view(seq_len, batch_size_int * num_heads, head_dim)
              .transpose(0, 1)
              .view(batch_size_int, num_heads, seq_len, head_dim)
          )
          value = (
              value.view(seq_len, batch_size_int * num_heads, head_dim)
              .transpose(0, 1)
              .view(batch_size_int, num_heads, seq_len, head_dim)
          )
  
          states.extend([key, value])
  
      return states
  
imports: import math
  import torch
  import torch.nn
  import torch.nn.functional
  from fairseq import options, utils
  from fairseq.models import FairseqEncoderDecoderModel, FairseqIncrementalDecoder, register_model, register_model_architecture, transformer
  from fairseq.modules import AdaptiveSoftmax, LearnedPositionalEmbedding, MultiheadAttention, SinusoidalPositionalEmbedding
  from pytorch_translate import transformer, vocab_reduction
  from pytorch_translate.average_attention import AverageAttention, AverageWindowAttention
  from pytorch_translate.utils import torch_find
label_desc: Use image node_id 7 for calling the TransformerAANDecoder obj's underlying member method code with example usage: obj._init_prev_states(encoder_out) and returns: states