class_docstr: Runs SentencePiece tokenziation.
class_docstr_tok: ["Runs","SentencePiece","tokenziation","."]
code: def __init__(self, vocab_file, do_lower_case=True, unk_token="[UNK]"):
      self.vocab = load_vocab(vocab_file)
      self.inv_vocab = {v: k for k, v in self.vocab.items()}
      self.do_lower_case = do_lower_case
      self.tokenizer = sp.SentencePieceProcessor()
      self.tokenizer.Load(vocab_file + ".model")
      self.sp_unk_token = "<unk>"
      self.unk_token = unk_token
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import unicodedata
  import six
  import sentencepiece
label_desc: Use image node_id 1 to create a new SentencepieceTokenizer object from inherited base classes: object with example: obj = SentencepieceTokenizer(vocab_file, do_lower_case, unk_token)