class_docstr: Fast self-attention used in Fastformer.
code: def __init__(
      self,
      size,
      attention_heads,
      dropout_rate,
  ):
      super().__init__()
      if size % attention_heads != 0:
          raise ValueError(
              f"Hidden size ({size}) is not an integer multiple "
              f"of attention heads ({attention_heads})"
          )
      self.attention_head_size = size // attention_heads
      self.num_attention_heads = attention_heads
  
      self.query = torch.nn.Linear(size, size)
      self.query_att = torch.nn.Linear(size, attention_heads)
      self.key = torch.nn.Linear(size, size)
      self.key_att = torch.nn.Linear(size, attention_heads)
      self.transform = torch.nn.Linear(size, size)
      self.dropout = torch.nn.Dropout(dropout_rate)
  
file_path: espnet/espnet2/asr/layers/fastformer.py
imports: import numpy
  import torch
label_desc: Use image node_id 1 to create a new FastSelfAttention object from inherited base classes: torch.nn with example: obj = FastSelfAttention(size, attention_heads, dropout_rate)