class_docstr: Base, abstract class for all caches. The actual data structure is specific to each subclass.
class_docstr_tok: ["Base",",","abstract","class","for","all","caches",".","The","actual","data","structure","is","specific","to","each","subclass","."]
docstr: Given the sequence length of the new inputs, returns the usable length of the cache.
docstr_tok: ["Given","the","sequence","length","of","the","new","inputs",",","returns","the","usable","length","of","the","cache","."]
code: def get_usable_length(
      self, new_seq_length: int, layer_idx: Optional[int] = 0
  ) -> int:
      """Given the sequence length of the new inputs, returns the usable length of the cache."""
      # Cache without size limit -> all cache is usable
      # Cache with size limit -> if the length cache plus the length of the new inputs is larger the maximum cache
      #   length, we will need to evict part of the cache (and thus not all cache is usable)
      max_length = self.get_max_length()
      previous_seq_length = self.get_seq_length(layer_idx)
      if (
          max_length is not None
          and previous_seq_length + new_seq_length > max_length
      ):
          return max_length - new_seq_length
      return previous_seq_length
  
imports: from typing import Any, Dict, List, Optional, Tuple
  import torch
label_desc: Use image node_id 4 for calling the Cache obj's underlying member method code with example usage: obj.get_usable_length(new_seq_length, layer_idx) and returns: previous_seq_length, unknown