class_docstr: Construct an GPTSw3 tokenizer. Based on [SentencePiece](https://github.com/google/sentencepiece).
  
  This tokenizer inherits from [`PreTrainedTokenizer`] which contains most of the main methods. Users should refer to
  this superclass for more information regarding those methods.
  
  Example usage:
  ```python
  >>> from transformers import GPTSw3Tokenizer
  
  >>> tokenizer = GPTSw3Tokenizer.from_pretrained("AI-Sweden-Models/gpt-sw3-126m")
  >>> tokenizer("Svenska är kul!")["input_ids"]
  [1814, 377, 3617, 63504]
  ```
  
  Args:
      vocab_file (`str`):
          [SentencePiece](https://github.com/google/sentencepiece) file (generally has a *.spm* extension) that
          contains the vocabulary necessary to instantiate a tokenizer.
      do_lower_case (`bool`, *optional*, defaults to `False`):
          Whether or not to lowercase the input when tokenizing.
      remove_space (`bool`, *optional*, defaults to `False`):
          Whether or not to strip the text when tokenizing (removing excess spaces before and after the string).
      keep_accents (`bool`, *optional*, defaults to `False`):
          Whether or not to keep accents when tokenizing.
      pad_token (`str`, *optional*):
          The token used for padding, for example when batching sequences of different lengths. If not provided, will
          default to '<pad>' or '<unk>' depending on model size.
      unk_token (`str`, *optional*):
          The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
          token instead. If not provided, will default to '<unk>'.
      eos_token (`str`, *optional*):
          The end of sequence token seen during pretraining. If not provided, will default to '<|endoftext|>'
      bos_token (`str`, *optional*):
          The beginning of sequence token that can be used for downstream task, was not seen during pretraining. If
          not provided, will default to '<s>' or '<|endoftext|>', depending on model size.
      sp_model_kwargs (`dict`, *optional*):
          Will be passed to the `SentencePieceProcessor.__init__()` method. The [Python wrapper for
          SentencePiece](https://github.com/google/sentencepiece/tree/master/python) can be used, among other things,
          to set:
  
          - `enable_sampling`: Enable subword regularization.
          - `nbest_size`: Sampling parameters for unigram. Invalid for BPE-Dropout.
  
            - `nbest_size = {0,1}`: No sampling is performed.
            - `nbest_size > 1`: samples from the nbest_size results.
            - `nbest_size < 0`: assuming that nbest_size is infinite and samples from the all hypothesis (lattice)
              using forward-filtering-and-backward-sampling algorithm.
  
          - `alpha`: Smoothing parameter for unigram sampling, and dropout probability of merge operations for
            BPE-dropout.
  
  Attributes:
      sp_model (`SentencePieceProcessor`):
          The *SentencePiece* processor that is used for every conversion (string, tokens and IDs).
      whitespaces (`set`):
          The whitespaces that are replaced in the whitespace normalization in preprocessing.
      non_printing_characters_re (`Pattern`):
          The compiled regular expression to remove non-printing characters in preprocessing.
docstr: Returns the input string, this function is overridden to remove the default clean up.
docstr_tok: ["Returns","the","input","string",",","this","function","is","overridden","to","remove","the","default","clean","up","."]
code: def clean_up_tokenization(out_string: str) -> str:
      """Returns the input string, this function is overridden to remove the default clean up."""
      return out_string
  
imports: import os
  import re
  import unicodedata
  from shutil import copyfile
  from typing import Any, Dict, List, Optional, Tuple, Union
  import sentencepiece
  from ...tokenization_utils import PreTrainedTokenizer
  from ...utils import is_torch_available, logging
label_desc: Use image node_id 9 for calling the GPTSw3Tokenizer obj's underlying member method code with example usage: obj.clean_up_tokenization(out_string) and returns: out_string