class_docstr: Construct a BPE tokenizer for HerBERT.
  
  Peculiarities:
  
  - uses BERT's pre-tokenizer: BaseTokenizer splits tokens on spaces, and also on punctuation. Each occurrence of a
    punctuation character will be treated separately.
  
  - Such pretokenized input is BPE subtokenized
  
  This tokenizer inherits from [`XLMTokenizer`] which contains most of the methods. Users should refer to the
  superclass for more information regarding methods.
code: def moses_pipeline(self, text, lang):
      text = replace_unicode_punct(text)
      text = self.moses_punct_norm(text, lang)
      text = remove_non_printing_char(text)
      return text
  
imports: import json
  import os
  import re
  import unicodedata
  from typing import List, Optional, Tuple
  from ...tokenization_utils import PreTrainedTokenizer, _is_control, _is_punctuation, _is_whitespace
  from ...utils import logging
label_desc: Use image node_id 5 for calling the HerbertTokenizer obj's underlying member method code with example usage: obj.moses_pipeline(text, lang) and returns: text