class_docstr: MultiHeadDampedEMA module definition.
  
  Args:
      size: Module size.
      num_heads: Number of attention heads.
      activation: Activation function type.
      truncation_length: Maximum length for truncation.
docstr: Construct an MultiHeadDampedEMA object.
code: def __init__(
      self,
      size: int,
      num_heads: int = 4,
      activation: torch.nn.Module = torch.nn.ReLU(),
      truncation_length: Optional[int] = None,
  ) -> None:
      """Construct an MultiHeadDampedEMA object."""
      super().__init__()
  
      self.damping_factor = torch.nn.Parameter(
          torch.Tensor(size, num_heads, 1)
      )
      self.decay_factor = torch.nn.Parameter(
          torch.Tensor(size, num_heads, 1)
      )
  
      self.ema_expansion_matrix = torch.nn.Parameter(
          torch.Tensor(size, num_heads, 1)
      )
      self.kernel_projection_matrix = torch.nn.Parameter(
          torch.Tensor(size, num_heads)
      )
  
      self.residual_weight = torch.nn.Parameter(torch.Tensor(size))
  
      self.scaling = math.sqrt(1.0 / num_heads)
      self.truncation_length = truncation_length
  
      self.activation = activation
  
      self._kernel = None
      self._coeffs = None
  
      self.num_heads = num_heads
  
      self.reset_parameters()
  
file_path: espnet/espnet2/asr_transducer/decoder/modules/mega/multi_head_damped_ema.py
imports: import math
  from typing import Dict, Optional, Tuple, Union
  import torch
label_desc: Use image node_id 1 to create a new MultiHeadDampedEMA object from inherited base classes: torch.nn with example: obj = MultiHeadDampedEMA(size, num_heads, activation, truncation_length)