class_docstr: Rotary positional embedding (RoPE) from [Su et al 2022](https://arxiv.org/abs/2104.09864).
  
  Args:
      dim (int): Embedding dimension (twice the number of frequencies).
      max_period (float): Maximum period of the rotation frequencies.
      xpos (bool): Use xPos, applies an exponential decay to rotation matrix.
      scale (float): Scale of positional embedding, set to 0 to deactivate.
      device (torch.device, optional): Device on which to initialize the module.
      dtype (torch.dtype): dtype to use to generate the embedding.
class_docstr_tok: ["Rotary","positional","embedding","(","RoPE",")","from","[","Su","et","al","2022","]","(","https",":","\/\/arxiv.org\/abs\/2104.09864",")",".","Args",":","dim","(","int",")",":","Embedding","dimension","(","twice","the","number","of","frequencies",")",".","max_period","(","float",")",":","Maximum","period","of","the","rotation","frequencies",".","xpos","(","bool",")",":","Use","xPos",",","applies","an","exponential","decay","to","rotation","matrix",".","scale","(","float",")",":","Scale","of","positional","embedding",",","set","to","0","to","deactivate",".","device","(","torch.device",",","optional",")",":","Device","on","which","to","initialize","the","module",".","dtype","(","torch.dtype",")",":","dtype","to","use","to","generate","the","embedding","."]
docstr: Apply rope rotation to both query and key tensors.
  Supports streaming mode, in which query and key are not expected to have the same shape.
  In streaming mode, key will be of length [P + C] with P the cached past timesteps, but
  query will be [C] (typically C == 1).
  
  Args:
      query (torch.Tensor): Query to rotate.
      key (torch.Tensor): Key to rotate.
      start (int): Start index of the sequence for time offset.
      time_dim (int): which dimension represent the time steps.
docstr_tok: ["Apply","rope","rotation","to","both","query","and","key","tensors",".","Supports","streaming","mode",",","in","which","query","and","key","are","not","expected","to","have","the","same","shape",".","In","streaming","mode",",","key","will","be","of","length","[","P","+","C","]","with","P","the","cached","past","timesteps",",","but","query","will","be","[","C","]","(","typically","C","==","1",")",".","Args",":","query","(","torch.Tensor",")",":","Query","to","rotate",".","key","(","torch.Tensor",")",":","Key","to","rotate",".","start","(","int",")",":","Start","index","of","the","sequence","for","time","offset",".","time_dim","(","int",")",":","which","dimension","represent","the","time","steps","."]
code: def rotate_qk(
      self,
      query: torch.Tensor,
      key: torch.Tensor,
      start: int = 0,
      time_dim: int = 1,
  ):
      """Apply rope rotation to both query and key tensors.
      Supports streaming mode, in which query and key are not expected to have the same shape.
      In streaming mode, key will be of length [P + C] with P the cached past timesteps, but
      query will be [C] (typically C == 1).
  
      Args:
          query (torch.Tensor): Query to rotate.
          key (torch.Tensor): Key to rotate.
          start (int): Start index of the sequence for time offset.
          time_dim (int): which dimension represent the time steps.
      """
      query_timesteps = query.shape[time_dim]
      key_timesteps = key.shape[time_dim]
      streaming_offset = key_timesteps - query_timesteps
  
      query_out = self.rotate(query, start + streaming_offset, time_dim)
      key_out = self.rotate(key, start, time_dim, invert_decay=True)
  
      return query_out, key_out
  
file_path: audiocraft/audiocraft/modules/rope.py
imports: import typing
  from torch import nn
  import torch
label_desc: Use image node_id 4 for calling the RotaryEmbedding obj's underlying member method code with example usage: obj.rotate_qk(query, key, start, time_dim) and returns: query_out,  key_out