class_docstr: Invertible `TextEncoder` using word pieces with a byte-level fallback.
  
  Encoding is fully invertible because all out-of-vocab wordpieces are
  byte-encoded.
  
  The vocabulary is "trained" on a corpus and all wordpieces are stored in a
  vocabulary file. To generate a vocabulary from a corpus, use
  `tfds.deprecated.text.SubwordTextEncoder.build_from_corpus`.
  
  Typical usage:
  
  ```
  # Build
  encoder = tfds.deprecated.text.SubwordTextEncoder.build_from_corpus(
      corpus_generator, target_vocab_size=2**15)
  encoder.save_to_file(vocab_fname)
  
  # Load
  encoder = tfds.deprecated.text.SubwordTextEncoder.load_from_file(vocab_fname)
  ids = encoder.encode("hello world")
  text = encoder.decode([1, 2, 3, 4])
  ```
docstr: Convert a single token to a list of integer ids.
docstr_tok: ["Convert","a","single","token","to","a","list","of","integer","ids","."]
code: def _token_to_ids(self, token):
      """Convert a single token to a list of integer ids."""
      # Check cache
      cache_location = hash(token) % self._cache_size
      cache_key, cache_value = self._token_to_ids_cache[cache_location]
      if cache_key == token:
          return cache_value
  
      subwords = self._token_to_subwords(token)
      ids = []
      for subword in subwords:
          if subword == _UNDERSCORE_REPLACEMENT:
              ids.append(len(self._subwords) + ord("_"))
              continue
          subword_id = self._subword_to_id.get(subword)
          if subword_id is None:
              # Byte-encode
              ids.extend(self._byte_encode(subword))
          else:
              ids.append(subword_id)
  
      # Update cache
      self._token_to_ids_cache[cache_location] = (token, ids)
  
      return ids
  
imports: from __future__ import unicode_literals
  import collections
  from absl import logging
  from tensorflow_datasets.core.deprecated.text import text_encoder
  from tensorflow_datasets.core.utils.lazy_imports_utils import tensorflow
label_desc: Use image node_id 6 for calling the SubwordTextEncoder obj's underlying member method code with example usage: obj._token_to_ids(token) and returns: ids, cache_value