class_docstr: Runs Wordseg tokenziation.
class_docstr_tok: ["Runs","Wordseg","tokenziation","."]
docstr: Tokenizes a piece of text into its word pieces.
  
  Returns:
      A list of wordpiece tokens.
code: def tokenize(self, text):
      """Tokenizes a piece of text into its word pieces.
  
      Returns:
          A list of wordpiece tokens.
      """
      text = text.lower() if self.do_lower_case else text
      text = convert_to_unicode(text)
  
      output_tokens = []
      for token in text.split(self.split_token):
          if token in self.vocab:
              output_tokens.append(token)
          else:
              sp_tokens = self.tokenizer.EncodeAsPieces(token)
              for sp_token in sp_tokens:
                  if sp_token in self.vocab:
                      output_tokens.append(sp_token)
      return output_tokens
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import unicodedata
  import six
  import sentencepiece
label_desc: Use image node_id 2 for calling the WordsegTokenizer obj's underlying member method code with example usage: obj.tokenize(text) and returns: output_tokens