class_docstr: An attention layer that allows for downscaling the size of the embedding after projection to queries, keys,
  and values.
class_docstr_tok: ["An","attention","layer","that","allows","for","downscaling","the","size","of","the","embedding","after","projection","to","queries",",","keys",",","and","values","."]
code: def __init__(
      self, embedding_dim: int, num_heads: int, downsample_rate: int = 1
  ) -> None:
      super().__init__()
      self.embedding_dim = embedding_dim
      self.internal_dim = embedding_dim // downsample_rate
      self.num_heads = num_heads
      KORNIA_CHECK(
          self.internal_dim % num_heads == 0,
          "num_heads must divide embedding_dim.",
      )
  
      self.q_proj = nn.Linear(embedding_dim, self.internal_dim)
      self.k_proj = nn.Linear(embedding_dim, self.internal_dim)
      self.v_proj = nn.Linear(embedding_dim, self.internal_dim)
      self.out_proj = nn.Linear(self.internal_dim, embedding_dim)
  
imports: from __future__ import annotations
  import math
  from torch import nn
  from kornia.contrib.models.sam.architecture.common import MLPBlock
  from kornia.core import Module, Tensor
  from kornia.core.check import KORNIA_CHECK
label_desc: Use image node_id 1 to create a new Attention object from inherited base classes: Module with example: obj = Attention(embedding_dim, num_heads, downsample_rate)