class_docstr: Sequential self-attention layer.
  Each token will attend to its previous fixed number of steps.
  Note that attention doesn't include the current step itself.
class_docstr_tok: ["Sequential","self-attention","layer",".","Each","token","will","attend","to","its","previous","fixed","number","of","steps",".","Note","that","attention","does","n't","include","the","current","step","itself","."]
code: def __init__(
      self,
      d_model,
      n_head,
      attn_span,
      dropout,
      adapt_span_layer,
      **kargs,
  ):
      nn.Module.__init__(self)
      self.dropout = nn.Dropout(dropout)
      self.d_model = d_model  # size of a single head
      self.attn_span = attn_span
      self.adaptive_span = AdaptiveSpan(
          attn_span=attn_span,
          n_head=n_head,
          adapt_span_layer=adapt_span_layer,
          **kargs,
      )
  
imports: import math
  import torch
  import torch.nn
  import torch.nn.functional
  from fairseq.modules.layer_norm import LayerNorm
  from .adaptive_span_attention import AdaptiveSpan
label_desc: Use image node_id 1 to create a new SeqAttention object from inherited base classes: nn with example: obj = SeqAttention(d_model, n_head, attn_span, dropout, adapt_span_layer)