class_docstr: Runs end-to-end tokenziation.
class_docstr_tok: ["Runs","end-to-end","tokenziation","."]
docstr: Tokenizes a piece of text.
docstr_tok: ["Tokenizes","a","piece","of","text","."]
code: def tokenize(self, text):
      """Tokenizes a piece of text."""
      text = preprocess_text(text, lower=self.do_lower_case)
      return encode_pieces(self.spm_model, text, return_unicode=True)
  
imports: import collections
  import sentencepiece
  import unicodedata
  from .args import str2bool
label_desc: Use image node_id 3 for calling the SentencePieceTokenizer obj's underlying member method code with example usage: obj.tokenize(text) and returns: encode_pieces