class_docstr: A `CharacterTokenizer` splits strings into character tokens.
  
  Registered as a `Tokenizer` with name "character".
  
  # Parameters
  
  byte_encoding : `str`, optional (default=`None`)
      If not `None`, we will use this encoding to encode the string as bytes, and use the byte
      sequence as characters, instead of the unicode characters in the python string.  E.g., the
      character 'á' would be a single token if this option is `None`, but it would be two
      tokens if this option is set to `"utf-8"`.
  
      If this is not `None`, `tokenize` will return a `List[int]` instead of a
      `List[str]`, and we will bypass the vocabulary in the `TokenIndexer`.
  
  lowercase_characters : `bool`, optional (default=`False`)
      If `True`, we will lowercase all of the characters in the text before doing any other
      operation.  You probably do not want to do this, as character vocabularies are generally
      not very large to begin with, but it's an option if you really want it.
  
  start_tokens : `List[str]`, optional
      If given, these tokens will be added to the beginning of every string we tokenize.  If
      using byte encoding, this should actually be a `List[int]`, not a `List[str]`.
  
  end_tokens : `List[str]`, optional
      If given, these tokens will be added to the end of every string we tokenize.  If using byte
      encoding, this should actually be a `List[int]`, not a `List[str]`.
code: def _to_params(self) -> Dict[str, Any]:
      return {
          "type": "character",
          "byte_encoding": self._byte_encoding,
          "lowercase_characters": self._lowercase_characters,
          "start_tokens": self._start_tokens,
          "end_tokens": self._end_tokens,
      }
  
file_path: allennlp/allennlp/data/tokenizers/character_tokenizer.py
imports: from typing import List, Union, Dict, Any
  from allennlp.data.tokenizers.token_class import Token
  from allennlp.data.tokenizers.tokenizer import Tokenizer
label_desc: Use image node_id 4 for calling the CharacterTokenizer obj's underlying member method code with example usage: obj._to_params() and returns: dict