class_docstr: A cache that grows dynamically as more tokens are generated. This is the default for generative models.
  
  It stores the Key and Value states as a list of tensors, one for each layer. The expected shape for each tensor is
  `[batch_size, num_heads, seq_len, head_dim]`.
class_docstr_tok: ["A","cache","that","grows","dynamically","as","more","tokens","are","generated",".","This","is","the","default","for","generative","models",".","It","stores","the","Key","and","Value","states","as","a","list","of","tensors",",","one","for","each","layer",".","The","expected","shape","for","each","tensor","is","`","[","batch_size",",","num_heads",",","seq_len",",","head_dim","]","`","."]
docstr: Support for backwards-compatible `past_key_value` indexing, e.g. `past_key_value[0][0].shape[2]` to get the
  sequence length.
docstr_tok: ["Support","for","backwards-compatible","`","past_key_value","`","indexing",",","e.g",".","`","past_key_value","[","0","]","[","0","]",".shape","[","2","]","`","to","get","the","sequence","length","."]
code: def __getitem__(self, layer_idx: int) -> List[Tuple[torch.Tensor]]:
      """
      Support for backwards-compatible `past_key_value` indexing, e.g. `past_key_value[0][0].shape[2]` to get the
      sequence length.
      """
      if layer_idx < len(self):
          return (
              self.key_cache[layer_idx],
              self.value_cache[layer_idx],
          )
      else:
          raise KeyError(
              f"Cache only has {len(self)} layers, attempted to access layer with index {layer_idx}"
          )
  
imports: from typing import Any, Dict, List, Optional, Tuple
  import torch
label_desc: Use image node_id 2 for calling the DynamicCache obj's underlying member method code with example usage: obj.__getitem__(layer_idx) and returns: self,  self