class_docstr: Construct a "Fast" BPE tokenizer for HerBERT (backed by HuggingFace's *tokenizers* library).
  
  Peculiarities:
  
  - uses BERT's pre-tokenizer: BertPreTokenizer splits tokens on spaces, and also on punctuation. Each occurrence of
    a punctuation character will be treated separately.
  
  This tokenizer inherits from [`PreTrainedTokenizer`] which contains most of the methods. Users should refer to the
  superclass for more information regarding methods.
  
  Args:
      vocab_file (`str`):
          Path to the vocabulary file.
      merges_file (`str`):
          Path to the merges file.
class_docstr_tok: ["Construct","a","``","Fast","''","BPE","tokenizer","for","HerBERT","(","backed","by","HuggingFace","'s","*","tokenizers","*","library",")",".","Peculiarities",":","-","uses","BERT","'s","pre-tokenizer",":","BertPreTokenizer","splits","tokens","on","spaces",",","and","also","on","punctuation",".","Each","occurrence","of","a","punctuation","character","will","be","treated","separately",".","This","tokenizer","inherits","from","[","`","PreTrainedTokenizer","`","]","which","contains","most","of","the","methods",".","Users","should","refer","to","the","superclass","for","more","information","regarding","methods",".","Args",":","vocab_file","(","`","str","`",")",":","Path","to","the","vocabulary","file",".","merges_file","(","`","str","`",")",":","Path","to","the","merges","file","."]
code: def __init__(
      self,
      vocab_file=None,
      merges_file=None,
      tokenizer_file=None,
      cls_token="<s>",
      unk_token="<unk>",
      pad_token="<pad>",
      mask_token="<mask>",
      sep_token="</s>",
      **kwargs,
  ):
      super().__init__(
          vocab_file,
          merges_file,
          tokenizer_file=tokenizer_file,
          cls_token=cls_token,
          unk_token=unk_token,
          pad_token=pad_token,
          mask_token=mask_token,
          sep_token=sep_token,
          **kwargs,
      )
  
file_path: H2O/h2o_flexgen/benchmark/third_party/transformers/src/transformers/models/herbert/tokenization_herbert_fast.py
imports: from typing import List, Optional, Tuple
  from ...tokenization_utils_fast import PreTrainedTokenizerFast
  from ...utils import logging
  from .tokenization_herbert import HerbertTokenizer
label_desc: Use image node_id 1 to create a new HerbertTokenizerFast object from inherited base classes: PreTrainedTokenizerFast with example: obj = HerbertTokenizerFast(vocab_file, merges_file, tokenizer_file, cls_token, unk_token, pad_token, mask_token, sep_token)