class_docstr: Base class inherited by all tokenizer classes. This class
  handles downloading and loading pre-trained tokenizer and adding tokens to
  the vocabulary.
  
  Derived class can set up a few special tokens to be used in common scripts
  and internals: :attr:`bos_token`, :attr:`eos_token`, :attr:`unk_token`,
  :attr:`sep_token`, :attr:`pad_token`, :attr:`cls_token`,
  :attr:`mask_token`, and :attr:`additional_special_tokens`.
  
  We defined an :attr:`added_tokens_encoder` to add new tokens to the
  vocabulary without having to handle the specific vocabulary augmentation
  methods of the various underlying dictionary structures (`BPE`,
  `sentencepiece` ...).
class_docstr_tok: ["Base","class","inherited","by","all","tokenizer","classes",".","This","class","handles","downloading","and","loading","pre-trained","tokenizer","and","adding","tokens","to","the","vocabulary",".","Derived","class","can","set","up","a","few","special","tokens","to","be","used","in","common","scripts","and","internals",":",":","attr",":","`","bos_token","`",",",":","attr",":","`","eos_token","`",",",":","attr",":","`","unk_token","`",",",":","attr",":","`","sep_token","`",",",":","attr",":","`","pad_token","`",",",":","attr",":","`","cls_token","`",",",":","attr",":","`","mask_token","`",",","and",":","attr",":","`","additional_special_tokens","`",".","We","defined","an",":","attr",":","`","added_tokens_encoder","`","to","add","new","tokens","to","the","vocabulary","without","having","to","handle","the","specific","vocabulary","augmentation","methods","of","the","various","underlying","dictionary","structures","(","`","BPE","`",",","`","sentencepiece","`","...",")","."]
docstr: Save the tokenizer vocabulary to a directory. This method does not
  save added tokens, special token mappings, and the configuration file.
  
  Please use :meth:`~save` to save the full tokenizer state so
  that it can be reloaded using :meth:`~load`.
docstr_tok: ["Save","the","tokenizer","vocabulary","to","a","directory",".","This","method","does","not","save","added","tokens",",","special","token","mappings",",","and","the","configuration","file",".","Please","use",":","meth",":","`","~save","`","to","save","the","full","tokenizer","state","so","that","it","can","be","reloaded","using",":","meth",":","`","~load","`","."]
code: def save_vocab(self, save_dir):
      r"""Save the tokenizer vocabulary to a directory. This method does not
      save added tokens, special token mappings, and the configuration file.
  
      Please use :meth:`~save` to save the full tokenizer state so
      that it can be reloaded using :meth:`~load`.
      """
      raise NotImplementedError
  
imports: from typing import Any, Dict, List, Optional, Tuple, overload
  import json
  import os
  import warnings
  from texar.tf.module_base import ModuleBase
label_desc: Use image node_id 4 for calling the TokenizerBase obj's underlying member method code with example usage: obj.save_vocab(save_dir) without return types