class_docstr: Invertible `TextEncoder` using word pieces with a byte-level fallback.
  
  Encoding is fully invertible because all out-of-vocab wordpieces are
  byte-encoded.
  
  The vocabulary is "trained" on a corpus and all wordpieces are stored in a
  vocabulary file. To generate a vocabulary from a corpus, use
  `tfds.deprecated.text.SubwordTextEncoder.build_from_corpus`.
  
  Typical usage:
  
  ```
  # Build
  encoder = tfds.deprecated.text.SubwordTextEncoder.build_from_corpus(
      corpus_generator, target_vocab_size=2**15)
  encoder.save_to_file(vocab_fname)
  
  # Load
  encoder = tfds.deprecated.text.SubwordTextEncoder.load_from_file(vocab_fname)
  ids = encoder.encode("hello world")
  text = encoder.decode([1, 2, 3, 4])
  ```
docstr: Greedily split token into subwords.
docstr_tok: ["Greedily","split","token","into","subwords","."]
code: def _token_to_subwords(self, token):
      """Greedily split token into subwords."""
      subwords = []
  
      start = 0
      while start < len(token):
          subword = None
          for end in range(
              min(len(token), start + self._max_subword_len), start, -1
          ):
              candidate = token[start:end]
              if (
                  candidate in self._subword_to_id
                  or candidate == _UNDERSCORE_REPLACEMENT
              ):
                  subword = candidate
                  subwords.append(subword)
                  start = end
                  break
          # No subword match found. Consume a single (unicode) character.
          if subword is None:
              subwords.append(token[start])
              start += 1
  
      return subwords
  
imports: from __future__ import unicode_literals
  import collections
  from absl import logging
  from tensorflow_datasets.core.deprecated.text import text_encoder
  from tensorflow_datasets.core.utils.lazy_imports_utils import tensorflow
label_desc: Use image node_id 9 for calling the SubwordTextEncoder obj's underlying member method code with example usage: obj._token_to_subwords(token) and returns: subwords