class_docstr: Facebook implementation of rotary embeddings.
class_docstr_tok: ["Facebook","implementation","of","rotary","embeddings","."]
code: def __init__(
      self,
      hidden_per_head,
      base=10000,
      max_seq_length=512,
      seq_dim: int = 0,
  ):
      super().__init__()
      self.seq_dim: int = seq_dim
      freqs_cis = self.precompute_freqs_cis(
          dim=hidden_per_head, end=max_seq_length * 2, theta=base
      )
      self.register_buffer("freqs_cis", freqs_cis)
  
imports: import torch
  import math
  from typing import Tuple
  from einops import repeat
label_desc: Use image node_id 1 to create a new RotaryLLAMA object from inherited base classes: torch.nn with example: obj = RotaryLLAMA(hidden_per_head, base, max_seq_length, seq_dim)