class_docstr: The rotary position embeddings from RoFormer_ (Su et. al).
  A crucial insight from the method is that the query and keys are
  transformed by rotation matrices which depend on the relative positions.
  
  Other implementations are available in the Rotary Transformer repo_ and in
  GPT-NeoX_, GPT-NeoX was an inspiration
  
  .. _RoFormer: https://arxiv.org/abs/2104.09864
  .. _repo: https://github.com/ZhuiyiTechnology/roformer
  .. _GPT-NeoX: https://github.com/EleutherAI/gpt-neox
  
  
  .. warning: Please note that this embedding is not registered on purpose, as it is transformative
      (it does not create the embedding dimension) and will likely be picked up (imported) on a ad-hoc basis
code: def forward(
      self, q: torch.Tensor, k: torch.Tensor
  ) -> Tuple[torch.Tensor, torch.Tensor]:
      self._cos_cached, self._sin_cached = self._update_cos_sin_tables(
          k, seq_dimension=-2
      )
  
      return (
          apply_rotary_pos_emb(q, self._cos_cached, self._sin_cached),
          apply_rotary_pos_emb(k, self._cos_cached, self._sin_cached),
      )
  
file_path: xformers/xformers/components/positional_embedding/rotary.py
imports: from typing import Tuple
  import torch
label_desc: Use image node_id 3 for calling the RotaryEmbedding obj's underlying member method code with example usage: obj.forward(q, k) and returns: apply_rotary_pos_emb,  apply_rotary_pos_emb