class_docstr: Construct a Marian tokenizer. Based on [SentencePiece](https://github.com/google/sentencepiece).
  
  This tokenizer inherits from [`PreTrainedTokenizer`] which contains most of the main methods. Users should refer to
  this superclass for more information regarding those methods.
  
  Args:
      source_spm (`str`):
          [SentencePiece](https://github.com/google/sentencepiece) file (generally has a .spm extension) that
          contains the vocabulary for the source language.
      target_spm (`str`):
          [SentencePiece](https://github.com/google/sentencepiece) file (generally has a .spm extension) that
          contains the vocabulary for the target language.
      source_lang (`str`, *optional*):
          A string representing the source language.
      target_lang (`str`, *optional*):
          A string representing the target language.
      unk_token (`str`, *optional*, defaults to `"<unk>"`):
          The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
          token instead.
      eos_token (`str`, *optional*, defaults to `"</s>"`):
          The end of sequence token.
      pad_token (`str`, *optional*, defaults to `"<pad>"`):
          The token used for padding, for example when batching sequences of different lengths.
      model_max_length (`int`, *optional*, defaults to 512):
          The maximum sentence length the model accepts.
      additional_special_tokens (`List[str]`, *optional*, defaults to `["<eop>", "<eod>"]`):
          Additional special tokens used by the tokenizer.
      sp_model_kwargs (`dict`, *optional*):
          Will be passed to the `SentencePieceProcessor.__init__()` method. The [Python wrapper for
          SentencePiece](https://github.com/google/sentencepiece/tree/master/python) can be used, among other things,
          to set:
  
          - `enable_sampling`: Enable subword regularization.
          - `nbest_size`: Sampling parameters for unigram. Invalid for BPE-Dropout.
  
            - `nbest_size = {0,1}`: No sampling is performed.
            - `nbest_size > 1`: samples from the nbest_size results.
            - `nbest_size < 0`: assuming that nbest_size is infinite and samples from the all hypothesis (lattice)
              using forward-filtering-and-backward-sampling algorithm.
  
          - `alpha`: Smoothing parameter for unigram sampling, and dropout probability of merge operations for
            BPE-dropout.
  
  Examples:
  
  ```python
  >>> from transformers import MarianForCausalLM, MarianTokenizer
  
  >>> model = MarianForCausalLM.from_pretrained("Helsinki-NLP/opus-mt-en-de")
  >>> tokenizer = MarianTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-de")
  >>> src_texts = ["I am a small frog.", "Tom asked his teacher for advice."]
  >>> tgt_texts = ["Ich bin ein kleiner Frosch.", "Tom bat seinen Lehrer um Rat."]  # optional
  >>> inputs = tokenizer(src_texts, text_target=tgt_texts, return_tensors="pt", padding=True)
  
  >>> outputs = model(**inputs)  # should work
  ```
code: def vocab_size(self) -> int:
      return len(self.encoder)
  
imports: import json
  import os
  import re
  import warnings
  from pathlib import Path
  from shutil import copyfile
  from typing import Any, Dict, List, Optional, Tuple, Union
  import sentencepiece
  from ...tokenization_utils import PreTrainedTokenizer
  from ...utils import logging
label_desc: Use image node_id 14 for calling the MarianTokenizer obj's underlying member method code with example usage: obj.vocab_size() and returns: len