class_docstr: Construct an PLBART tokenizer.
  
  Adapted from [`RobertaTokenizer`] and [`XLNetTokenizer`]. Based on
  [SentencePiece](https://github.com/google/sentencepiece).
  
  The tokenization method is `<tokens> <eos> <language code>` for source language documents, and `<language code>
  <tokens> <eos>` for target language documents.
  
  Args:
      vocab_file (`str`):
          Path to the vocabulary file.
      src_lang (`str`, *optional*):
          A string representing the source language.
      tgt_lang (`str`, *optional*):
          A string representing the target language.
      bos_token (`str`, *optional*, defaults to `"<s>"`):
          The start of sequence token.
      eos_token (`str`, *optional*, defaults to `"</s>"`):
          The end of sequence token.
      sep_token (`str`, *optional*, defaults to `"</s>"`):
          The separator token, which is used when building a sequence from multiple sequences, e.g. two sequences for
          sequence classification or for a text and a question for question answering. It is also used as the last
          token of a sequence built with special tokens.
      cls_token (`str`, *optional*, defaults to `"<s>"`):
          The cls token, which is a special token used as the first token for all tasks.
      unk_token (`str`, *optional*, defaults to `"<unk>"`):
          The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
          token instead.
      pad_token (`str`, *optional*, defaults to `"<pad>"`):
          The token used for padding, for example when batching sequences of different lengths.
      mask_token(`str`, *optional*, defaults to `"<mask>"`):
          The token used for masking values. This is the token used when training this model with masking tasks. This
          is only used in the `"base"` tokenizer type. For `"multi"` tokenizer, masking is never done for the
          downstream tasks.
      language_codes (`str`, *optional*, defaults to `"base"`):
          What language codes to use. Should be one of `"base"` or `"multi"`.
      sp_model_kwargs (`dict`, *optional*):
          Will be passed to the `SentencePieceProcessor.__init__()` method. The [Python wrapper for
          SentencePiece](https://github.com/google/sentencepiece/tree/master/python) can be used, among other things,
          to set:
          - `enable_sampling`: Enable subword regularization.
          - `nbest_size`: Sampling parameters for unigram. Invalid for BPE-Dropout.
            - `nbest_size = {0,1}`: No sampling is performed.
            - `nbest_size > 1`: samples from the nbest_size results.
            - `nbest_size < 0`: assuming that nbest_size is infinite and samples from the all hypothesis (lattice)
              using forward-filtering-and-backward-sampling algorithm.
          - `alpha`: Smoothing parameter for unigram sampling, and dropout probability of merge operations for
            BPE-dropout.
  
  Examples:
  
  ```python
  >>> from transformers import PLBartTokenizer
  
  >>> tokenizer = PLBartTokenizer.from_pretrained("uclanlp/plbart-python-en_XX", src_lang="python", tgt_lang="en_XX")
  >>> example_python_phrase = "def maximum(a,b,c):NEW_LINE_INDENTreturn max([a,b,c])"
  >>> expected_translation_english = "Returns the maximum value of a b c."
  >>> inputs = tokenizer(example_python_phrase, text_target=expected_translation_english, return_tensors="pt")
  ```
docstr: Retrieve sequence ids from a token list that has no special tokens added. This method is called when adding
  special tokens using the tokenizer `prepare_for_model` method.
  
  Args:
      token_ids_0 (`List[int]`):
          List of IDs.
      token_ids_1 (`List[int]`, *optional*):
          Optional second list of IDs for sequence pairs.
      already_has_special_tokens (`bool`, *optional*, defaults to `False`):
          Whether or not the token list is already formatted with special tokens for the model.
  
  Returns:
      `List[int]`: A list of integers in the range [0, 1]: 1 for a special token, 0 for a sequence token.
docstr_tok: ["Retrieve","sequence","ids","from","a","token","list","that","has","no","special","tokens","added",".","This","method","is","called","when","adding","special","tokens","using","the","tokenizer","`","prepare_for_model","`","method",".","Args",":","token_ids_0","(","`","List","[","int","]","`",")",":","List","of","IDs",".","token_ids_1","(","`","List","[","int","]","`",",","*","optional","*",")",":","Optional","second","list","of","IDs","for","sequence","pairs",".","already_has_special_tokens","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","or","not","the","token","list","is","already","formatted","with","special","tokens","for","the","model",".","Returns",":","`","List","[","int","]","`",":","A","list","of","integers","in","the","range","[","0",",","1","]",":","1","for","a","special","token",",","0","for","a","sequence","token","."]
code: def get_special_tokens_mask(
      self,
      token_ids_0: List[int],
      token_ids_1: Optional[List[int]] = None,
      already_has_special_tokens: bool = False,
  ) -> List[int]:
      """
      Retrieve sequence ids from a token list that has no special tokens added. This method is called when adding
      special tokens using the tokenizer `prepare_for_model` method.
  
      Args:
          token_ids_0 (`List[int]`):
              List of IDs.
          token_ids_1 (`List[int]`, *optional*):
              Optional second list of IDs for sequence pairs.
          already_has_special_tokens (`bool`, *optional*, defaults to `False`):
              Whether or not the token list is already formatted with special tokens for the model.
  
      Returns:
          `List[int]`: A list of integers in the range [0, 1]: 1 for a special token, 0 for a sequence token.
      """
  
      if already_has_special_tokens:
          return super().get_special_tokens_mask(
              token_ids_0=token_ids_0,
              token_ids_1=token_ids_1,
              already_has_special_tokens=True,
          )
  
      prefix_ones = [1] * len(self.prefix_tokens)
      suffix_ones = [1] * len(self.suffix_tokens)
      if token_ids_1 is None:
          return prefix_ones + ([0] * len(token_ids_0)) + suffix_ones
      return (
          prefix_ones
          + ([0] * len(token_ids_0))
          + ([0] * len(token_ids_1))
          + suffix_ones
      )
  
imports: import os
  from shutil import copyfile
  from typing import Any, Dict, List, Optional, Tuple
  import sentencepiece
  from ...tokenization_utils import AddedToken, BatchEncoding, PreTrainedTokenizer
  from ...utils import logging
label_desc: Use image node_id 7 for calling the PLBartTokenizer obj's underlying member method code with example usage: obj.get_special_tokens_mask(token_ids_0, token_ids_1, already_has_special_tokens) and returns: unknown, super, unknown