class_docstr: Construct a SeamlessM4T tokenizer.
  
  Adapted from [`RobertaTokenizer`] and [`XLNetTokenizer`]. Based on
  [SentencePiece](https://github.com/google/sentencepiece).
  
  The tokenization method is `<language code> <tokens> <eos>` for source language documents, and `<eos> <language
  code> <tokens> <eos>` for target language documents.
  
  Examples:
  
  ```python
  >>> from transformers import SeamlessM4TTokenizer
  
  >>> tokenizer = SeamlessM4TTokenizer.from_pretrained(
  ...     "facebook/hf-seamless-m4t-medium", src_lang="eng", tgt_lang="fra"
  ... )
  >>> example_english_phrase = " UN Chief Says There Is No Military Solution in Syria"
  >>> expected_translation_french = "Le chef de l'ONU affirme qu'il n'y a pas de solution militaire en Syrie."
  >>> inputs = tokenizer(example_english_phrase, text_target=expected_translation_french, return_tensors="pt")
  ```
  
  Args:
      vocab_file (`str`):
          Path to the vocabulary file.
      bos_token (`str`, *optional*, defaults to `"<s>"`):
          The beginning of sequence token that was used during pretraining. Can be used a sequence classifier token.
  
          <Tip>
  
          When building a sequence using special tokens, this is not the token that is used for the beginning of
          sequence. The token used is the `cls_token`.
  
          </Tip>
  
      eos_token (`str`, *optional*, defaults to `"</s>"`):
          The end of sequence token.
  
          <Tip>
  
          When building a sequence using special tokens, this is not the token that is used for the end of sequence.
          The token used is the `sep_token`.
  
          </Tip>
  
      sep_token (`str`, *optional*, defaults to `"</s>"`):
          The separator token, which is used when building a sequence from multiple sequences, e.g. two sequences for
          sequence classification or for a text and a question for question answering. It is also used as the last
          token of a sequence built with special tokens.
      cls_token (`str`, *optional*, defaults to `"<s>"`):
          The classifier token which is used when doing sequence classification (classification of the whole sequence
          instead of per-token classification). It is the first token of the sequence when built with special tokens.
      unk_token (`str`, *optional*, defaults to `"<unk>"`):
          The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
          token instead.
      pad_token (`str`, *optional*, defaults to `"<pad>"`):
          The token used for padding, for example when batching sequences of different lengths.
      tokenizer_file (`str`, *optional*):
          The path to a tokenizer file to use instead of the vocab file.
      src_lang (`str`, *optional*, defaults to `"eng"`):
          The language to use as source language for translation.
      tgt_lang (`str`, *optional*, defaults to `"fra"`):
          The language to use as target language for translation.
      sp_model_kwargs (`Dict[str, Any]`, *optional*):
          Additional keyword arguments to pass to the model initialization.
      additional_special_tokens (tuple or list of `str` or `tokenizers.AddedToken`, *optional*):
          A tuple or a list of additional special tokens. Can be used to specify the list of languages that will be
          supported by the tokenizer.
code: def __init__(
      self,
      vocab_file,
      bos_token="<s>",
      eos_token="</s>",
      sep_token="</s>",
      cls_token="<s>",
      unk_token="<unk>",
      pad_token="<pad>",
      tokenizer_file=None,
      src_lang="eng",
      tgt_lang="fra",
      sp_model_kwargs: Optional[Dict[str, Any]] = None,
      additional_special_tokens=None,
      **kwargs,
  ):
      self.sp_model_kwargs = (
          {} if sp_model_kwargs is None else sp_model_kwargs
      )
      # Add this unused argument to keep some important Copied from statements
      self.legacy = False
      self.vocab_file = vocab_file
  
      self.sp_model = self.get_spm_processor(
          kwargs.pop("from_slow", False)
      )
  
      # Vocab    |    0    |    1    |   2    |    3    |  4   |  5   |  6   |   7  |   8  |  9
      # -------- | ------- | ------- | ------ | ------- | ---- | ---- | ---- | ---- | ---- | ----
      # spm  | '<unk>'   | '<s>' | '</s>' | 'an' | 'en' | '_d' | 'er' | 'in' | '_s' | '_a'
      # fairseq  | '<pad>'   | '<unk>' | '<s>' | '</s>' | 'an' | 'en' | '▁d' | 'er' | 'in' | '▁s'
  
      # Mimic fairseq token-to-id alignment for the first 4 token
      self._added_tokens_decoder = {
          0: AddedToken(pad_token, special=True)
          if isinstance(pad_token, str)
          else pad_token,
          1: AddedToken(unk_token, special=True)
          if isinstance(unk_token, str)
          else unk_token,
          2: AddedToken(bos_token, special=True)
          if isinstance(bos_token, str)
          else bos_token,
          3: AddedToken(eos_token, special=True)
          if isinstance(eos_token, str)
          else eos_token,
      }
  
      # The first "real" token "an" has position 4 in the original fairseq vocab and position 3 in the spm vocab
      self.fairseq_offset = 1
  
      self.sp_model_size = len(self.sp_model)
  
      self._src_lang = (
          f"__{src_lang}__" if "__" not in src_lang else src_lang
      )
      self._tgt_lang = (
          f"__{tgt_lang}__" if "__" not in tgt_lang else tgt_lang
      )
  
      super().__init__(
          bos_token=bos_token,
          eos_token=eos_token,
          unk_token=unk_token,
          sep_token=sep_token,
          cls_token=cls_token,
          pad_token=pad_token,
          tokenizer_file=tokenizer_file,
          src_lang=src_lang,
          tgt_lang=tgt_lang,
          additional_special_tokens=additional_special_tokens,
          sp_model_kwargs=self.sp_model_kwargs,
          **kwargs,
      )
  
      self.set_src_lang_special_tokens(self._src_lang)
      self.set_tgt_lang_special_tokens(self._tgt_lang)
  
imports: import os
  from shutil import copyfile
  from typing import Any, Dict, List, Optional, Tuple, Union
  import sentencepiece
  from ...convert_slow_tokenizer import import_protobuf
  from ...tokenization_utils import BatchEncoding, PreTokenizedInput, PreTrainedTokenizer, TextInput
  from ...tokenization_utils_base import AddedToken
  from ...utils import PaddingStrategy, logging
label_desc: Use image node_id 1 to create a new SeamlessM4TTokenizer object from inherited base classes: PreTrainedTokenizer with example: obj = SeamlessM4TTokenizer(vocab_file, bos_token, eos_token, sep_token, cls_token, unk_token, pad_token, tokenizer_file, src_lang, tgt_lang, sp_model_kwargs, additional_special_tokens)