class_docstr: The rotary position embeddings from RoFormer_ (Su et. al).
  A crucial insight from the method is that the query and keys are
  transformed by rotation matrices which depend on the relative positions.
  Other implementations are available in the Rotary Transformer repo_ and in
  GPT-NeoX_, GPT-NeoX was an inspiration
  .. _RoFormer: https://arxiv.org/abs/2104.09864
  .. _repo: https://github.com/ZhuiyiTechnology/roformer
  .. _GPT-NeoX: https://github.com/EleutherAI/gpt-neox
class_docstr_tok: ["The","rotary","position","embeddings","from","RoFormer_","(","Su","et",".","al",")",".","A","crucial","insight","from","the","method","is","that","the","query","and","keys","are","transformed","by","rotation","matrices","which","depend","on","the","relative","positions",".","Other","implementations","are","available","in","the","Rotary","Transformer","repo_","and","in","GPT-NeoX_",",","GPT-NeoX","was","an","inspiration","..","_RoFormer",":","https",":","\/\/arxiv.org\/abs\/2104.09864","..","_repo",":","https",":","\/\/github.com\/ZhuiyiTechnology\/roformer","..","_GPT-NeoX",":","https",":","\/\/github.com\/EleutherAI\/gpt-neox"]
code: def apply_rotary_pos_emb(
      x: torch.Tensor,
      cos: torch.Tensor,
      sin: torch.Tensor,
      seq_dimension: int = -2,
  ):
      # NOTE: This could probably be moved to Triton
  
      # Handle a possible sequence length mismatch in between q and k
      cos = cos[: x.shape[seq_dimension], :]
      sin = sin[: x.shape[seq_dimension], :]
      if seq_dimension == -3:
          cos = cos[:, None, :]
          sin = sin[:, None, :]
      return (x * cos) + (rotate_half(x) * sin)
  
imports: import torch
  import math
  from typing import Tuple
  from einops import repeat
label_desc: Use image node_id 5 for calling the RotaryEleutherAI obj's underlying member method code with example usage: obj.apply_rotary_pos_emb(x, cos, sin, seq_dimension) and returns: unknown