class_docstr: Construct a CANINE tokenizer (i.e. a character splitter). It turns text into a sequence of characters, and then
  converts each character into its Unicode code point.
  
  [`CanineTokenizer`] inherits from [`PreTrainedTokenizer`].
  
  Refer to superclass [`PreTrainedTokenizer`] for usage examples and documentation concerning parameters.
  
  Args:
      model_max_length (`int`, *optional*, defaults to 2048):
              The maximum sentence length the model accepts.
docstr: Converts a token (i.e. a Unicode character) in an id (i.e. its integer Unicode code point value).
docstr_tok: ["Converts","a","token","(","i.e",".","a","Unicode","character",")","in","an","id","(","i.e",".","its","integer","Unicode","code","point","value",")","."]
code: def _convert_token_to_id(self, token: str) -> int:
      """Converts a token (i.e. a Unicode character) in an id (i.e. its integer Unicode code point value)."""
      try:
          return ord(token)
      except TypeError:
          raise ValueError(f"invalid token: '{token}'")
  
file_path: transformers/src/transformers/models/canine/tokenization_canine.py
imports: from typing import Dict, List, Optional
  from ...tokenization_utils import AddedToken, PreTrainedTokenizer
  from ...utils import logging
label_desc: Use image node_id 5 for calling the CanineTokenizer obj's underlying member method code with example usage: obj._convert_token_to_id(token) and returns: ord