class_docstr: Window based multi-head self attention (W-MSA) module with relative position bias.
  It supports both of shifted and non-shifted window.
  ref1: https://arxiv.org/pdf/2010.11929.pdf
  ref2: https://arxiv.org/pdf/2106.13230.pdf
  Args:
      dim (`int`):
          Number of input channels, see ref2.
      window_size (`tuple[int]`):
          The temporal length, height and width of the window, see ref2.
      num_heads (`int`):
          Number of attention heads of MSA, see ref1.
      qkv_bias (`bool`):
          If True, add a learnable bias to query, key, value. Default: True
      qk_scale (`float`):
          Override default qk scale of head_dim ** -0.5 if set
      attn_drop (`float`):
          Dropout ratio of attention weight. Default: 0.0
      proj_drop (`float`):
          Dropout ratio of output. Default: 0.0
code: def __init__(
      self,
      dim,
      window_size,
      num_heads,
      qkv_bias=False,
      qk_scale=None,
      attn_drop=0.0,
      proj_drop=0.0,
  ):
      super().__init__()
      self.dim = dim
      self.window_size = window_size  # Wd, Wh, Ww
      self.num_heads = num_heads
      head_dim = dim // num_heads
      self.scale = qk_scale or head_dim**-0.5
  
      # define a parameter table of relative position bias
      # 2*Wd-1 * 2*Wh-1 * 2*Ww-1, nH
      self.relative_position_bias_table = nn.Parameter(
          torch.zeros(
              (2 * window_size[0] - 1)
              * (2 * window_size[1] - 1)
              * (2 * window_size[2] - 1),
              num_heads,
          )
      )
  
      # get pair-wise relative position index for each token inside the window
      coords_d = torch.arange(self.window_size[0])
      coords_h = torch.arange(self.window_size[1])
      coords_w = torch.arange(self.window_size[2])
      coords = torch.stack(
          torch.meshgrid(coords_d, coords_h, coords_w)
      )  # 3, Wd, Wh, Ww
      coords_flatten = torch.flatten(coords, 1)  # 3, Wd*Wh*Ww
      relative_coords = (
          coords_flatten[:, :, None] - coords_flatten[:, None, :]
      )  # 3, Wd*Wh*Ww, Wd*Wh*Ww
      relative_coords = relative_coords.permute(
          1, 2, 0
      ).contiguous()  # Wd*Wh*Ww, Wd*Wh*Ww, 3
      relative_coords[:, :, 0] += (
          self.window_size[0] - 1
      )  # shift to start from 0
      relative_coords[:, :, 1] += self.window_size[1] - 1
      relative_coords[:, :, 2] += self.window_size[2] - 1
  
      relative_coords[:, :, 0] *= (2 * self.window_size[1] - 1) * (
          2 * self.window_size[2] - 1
      )
      relative_coords[:, :, 1] *= 2 * self.window_size[2] - 1
      relative_position_index = relative_coords.sum(
          -1
      )  # Wd*Wh*Ww, Wd*Wh*Ww
      self.register_buffer(
          "relative_position_index", relative_position_index
      )
  
      self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias)
      self.attn_drop = nn.Dropout(attn_drop)
      self.proj = nn.Linear(dim, dim)
      self.proj_drop = nn.Dropout(proj_drop)
  
      trunc_normal_(self.relative_position_bias_table, std=0.02)
      self.softmax = nn.Softmax(dim=-1)
  
imports: from torch import nn
  import torch
  from towhee.models.utils.weight_init import trunc_normal_
label_desc: Use image node_id 1 to create a new WindowAttention3D object from inherited base classes: nn with example: obj = WindowAttention3D(dim, window_size, num_heads, qkv_bias, qk_scale, attn_drop, proj_drop)