class_docstr: A cache that grows dynamically as more tokens are generated. This is the default for generative models.
  
  It stores the Key and Value states as a list of tensors, one for each layer. The expected shape for each tensor is
  `[batch_size, num_heads, seq_len, head_dim]`.
class_docstr_tok: ["A","cache","that","grows","dynamically","as","more","tokens","are","generated",".","This","is","the","default","for","generative","models",".","It","stores","the","Key","and","Value","states","as","a","list","of","tensors",",","one","for","each","layer",".","The","expected","shape","for","each","tensor","is","`","[","batch_size",",","num_heads",",","seq_len",",","head_dim","]","`","."]
docstr: Updates the cache with the new `key_states` and `value_states` for the layer `layer_idx`.
  
  Parameters:
      key_states (`torch.Tensor`):
          The new key states to cache.
      value_states (`torch.Tensor`):
          The new value states to cache.
      layer_idx (`int`):
          The index of the layer to cache the states for.
      cache_kwargs (`Dict[str, Any]`, `optional`):
          Additional arguments for the cache subclass. No additional arguments are used in `DynamicCache`.
  
  Return:
      A tuple containing the updated key and value states.
docstr_tok: ["Updates","the","cache","with","the","new","`","key_states","`","and","`","value_states","`","for","the","layer","`","layer_idx","`",".","Parameters",":","key_states","(","`","torch.Tensor","`",")",":","The","new","key","states","to","cache",".","value_states","(","`","torch.Tensor","`",")",":","The","new","value","states","to","cache",".","layer_idx","(","`","int","`",")",":","The","index","of","the","layer","to","cache","the","states","for",".","cache_kwargs","(","`","Dict","[","str",",","Any","]","`",",","`","optional","`",")",":","Additional","arguments","for","the","cache","subclass",".","No","additional","arguments","are","used","in","`","DynamicCache","`",".","Return",":","A","tuple","containing","the","updated","key","and","value","states","."]
code: def update(
      self,
      key_states: torch.Tensor,
      value_states: torch.Tensor,
      layer_idx: int,
      cache_kwargs: Optional[Dict[str, Any]] = None,
  ) -> Tuple[torch.Tensor, torch.Tensor]:
      """
      Updates the cache with the new `key_states` and `value_states` for the layer `layer_idx`.
  
      Parameters:
          key_states (`torch.Tensor`):
              The new key states to cache.
          value_states (`torch.Tensor`):
              The new value states to cache.
          layer_idx (`int`):
              The index of the layer to cache the states for.
          cache_kwargs (`Dict[str, Any]`, `optional`):
              Additional arguments for the cache subclass. No additional arguments are used in `DynamicCache`.
  
      Return:
          A tuple containing the updated key and value states.
      """
      # Update the number of seen tokens
      if layer_idx == 0:
          self.seen_tokens += key_states.shape[-2]
  
      # Update the cache
      if len(self.key_cache) <= layer_idx:
          self.key_cache.append(key_states)
          self.value_cache.append(value_states)
      else:
          self.key_cache[layer_idx] = torch.cat(
              [self.key_cache[layer_idx], key_states], dim=-2
          )
          self.value_cache[layer_idx] = torch.cat(
              [self.value_cache[layer_idx], value_states], dim=-2
          )
  
      return self.key_cache[layer_idx], self.value_cache[layer_idx]
  
imports: from typing import Any, Dict, List, Optional, Tuple
  import torch
label_desc: Use image node_id 5 for calling the DynamicCache obj's underlying member method code with example usage: obj.update(key_states, value_states, layer_idx, cache_kwargs) and returns: self,  self