class_docstr: Cross-Covariance Attention (XCA).
  
  This is a redistirbution of cross-covariance implementation originally
  released at https://github.com/facebookresearch/xcit.
  
  Cross-covariance attention was introduced in the following paper:
  
  XCiT: Cross-Covariance Image Transformers
  A El-Nouby, H Touvron, M Caron, P Bojanowski, M Douze, A Joulin, I Laptev,
  N Neverova, G Synnaeve, J Verbeek, H Jegou
  
  Operation where the channels are updated using a weighted sum. The weights
  are obtained from the (softmax normalized) Cross-covariance matrix
  (Q^T \cdot K \in d_h \times d_h)/
  
  Args:
      dim: Dimension of input.
      num_heads: Number of attention heads.
      qkv_bias: Whether to use bias in qkv term.
      attn_drop: Attention dropout probability.
      proj_drop: Projection dropout probability.
code: def no_weight_decay(self):
      return {"temperature"}
  
file_path: NeuralCompression/neuralcompression/layers/_xca.py
imports: import torch
  import torch.nn
  import torch.nn.functional
  from torch import Tensor
label_desc: Use image node_id 3 for calling the XCA obj's underlying member method code with example usage: obj.no_weight_decay() and returns: unknown