class_docstr: Adaptive attention span for Transformerself.
  This module learns an attention span length from data for each
  self-attention head.
  Args:
      attn_span: maximum attention span
      adapt_span_loss: loss coefficient for the span length
      adapt_span_ramp: length of the masking ramp
      adapt_span_init: initial size ratio
      adapt_span_cache: adapt cache size to reduce memory usage
class_docstr_tok: ["Adaptive","attention","span","for","Transformerself",".","This","module","learns","an","attention","span","length","from","data","for","each","self-attention","head",".","Args",":","attn_span",":","maximum","attention","span","adapt_span_loss",":","loss","coefficient","for","the","span","length","adapt_span_ramp",":","length","of","the","masking","ramp","adapt_span_init",":","initial","size","ratio","adapt_span_cache",":","adapt","cache","size","to","reduce","memory","usage"]
code: def get_current_avg_span(self):
      return self._mask.get_current_avg_size()
  
imports: import math
  import torch
  import torch.nn
  import torch.nn.functional
label_desc: Use image node_id 8 for calling the AdaptiveSpan obj's underlying member method code with example usage: obj.get_current_avg_span() and returns: self