class_docstr: Self-attention module with q,k from the same input tensor.
  The input tensor is the concatenation of `n_groups` of slightly different feature maps.
  Thus the projections are 1x1 group convolutions.
  *NOTE*: d_qk, d_hidden, n_head must be divisible by n_groups
class_docstr_tok: ["Self-attention","module","with","q",",","k","from","the","same","input","tensor",".","The","input","tensor","is","the","concatenation","of","`","n_groups","`","of","slightly","different","feature","maps",".","Thus","the","projections","are","1x1","group","convolutions",".","*","NOTE","*",":","d_qk",",","d_hidden",",","n_head","must","be","divisible","by","n_groups"]
code: def __init__(
      self,
      d_qk: int,
      d_value: int,
      n_head: int = 1,
      n_groups: int = 1,
      bias: bool = True,
      bidirectional: bool = False,
      apply_rel_dist: Optional[str] = None,
      share_values: bool = False,
      dropout: float = 0.0,
      temperature: float = 1.0,
  ):
      try:
          assert d_qk % n_groups == 0
          assert d_value % n_groups == 0
          assert n_head % n_groups == 0
      except AssertionError:
          raise ValueError(
              "d_qk, d_hidden, n_head must be divisible by n_groups"
          )
      if d_qk // n_groups > d_value:
          raise ValueError(
              "dimension of each group should not exceed d_value"
          )
      self.d_qk = d_qk
      self.d_value = d_value
      self.n_groups = n_groups
  
      super(GroupSelfAttention, self).__init__(
          d_hidden=d_value,
          n_head=n_head,
          bias=bias,
          bidirectional=bidirectional,
          apply_rel_dist=apply_rel_dist,
          share_values=share_values,
          dropout=dropout,
          temperature=temperature,
      )
  
imports: from typing import Tuple, Optional
  import math
  import torch
  from torch import Tensor, BoolTensor
  from torch import nn
  from torch.nn import init
  from torch.nn.functional import linear, conv1d
  from .base import Attention
  from .posemb import SinusoidalPositionalEmbedding
label_desc: Use image node_id 1 to create a new GroupSelfAttention object from inherited base classes: SelfAttention with example: obj = GroupSelfAttention(d_qk, d_value, n_head, n_groups, bias, bidirectional, apply_rel_dist, share_values, dropout, temperature)