class_docstr: Average Attention module from
  "Accelerating Neural Transformer via an Average Attention Network"
  :cite:`DBLP:journals/corr/abs-1805-00631`.
  
  Args:
     model_dim (int): the dimension of keys/values/queries,
         must be divisible by head_count
     dropout (float): dropout parameter
     pos_ffn_activation_fn (ActivationFunction):
         activation function choice for PositionwiseFeedForward layer
code: def __init__(
      self,
      model_dim,
      dropout=0.1,
      aan_useffn=False,
      pos_ffn_activation_fn=ActivationFunction.relu,
  ):
      self.model_dim = model_dim
      self.aan_useffn = aan_useffn
      super(AverageAttention, self).__init__()
      if aan_useffn:
          self.average_layer = PositionwiseFeedForward(
              model_dim, model_dim, dropout, pos_ffn_activation_fn
          )
      self.gating_layer = nn.Linear(model_dim * 2, model_dim * 2)
      self.layer_cache = False, {"prev_g": torch.tensor([])}
  
file_path: OpenNMT-py/onmt/modules/average_attn.py
imports: import torch
  import torch.nn
  from torch import Tensor
  from typing import Optional
  from onmt.modules.position_ffn import PositionwiseFeedForward
  from onmt.modules.position_ffn import ActivationFunction
label_desc: Use image node_id 1 to create a new AverageAttention object from inherited base classes: nn with example: obj = AverageAttention(model_dim, dropout, aan_useffn, pos_ffn_activation_fn)