class_docstr: A ``TokenIndexer`` determines how string tokens get represented as arrays of indices in a model.
  This class both converts strings into numerical values, with the help of a
  :class:`~allennlp.data.vocabulary.Vocabulary`, and it produces actual arrays.
  
  Tokens can be represented as single IDs (e.g., the word "cat" gets represented by the number
  34), or as lists of character IDs (e.g., "cat" gets represented by the numbers [23, 10, 18]),
  or in some other way that you can come up with (e.g., if you have some structured input you
  want to represent in a special way in your data arrays, you can do that here).
docstr: This method pads a list of tokens to ``desired_num_tokens`` and returns a padded copy of the
  input tokens.  If the input token list is longer than ``desired_num_tokens`` then it will be
  truncated.
  
  ``padding_lengths`` is used to provide supplemental padding parameters which are needed
  in some cases.  For example, it contains the widths to pad characters to when doing
  character-level padding.
code: def pad_token_sequence(
      self, tokens, desired_num_tokens, padding_lengths
  ):
      """
      This method pads a list of tokens to ``desired_num_tokens`` and returns a padded copy of the
      input tokens.  If the input token list is longer than ``desired_num_tokens`` then it will be
      truncated.
  
      ``padding_lengths`` is used to provide supplemental padding parameters which are needed
      in some cases.  For example, it contains the widths to pad characters to when doing
      character-level padding.
      """
      raise NotImplementedError
  
file_path: magnitude/pymagnitude/third_party/allennlp/data/token_indexers/token_indexer.py
imports: from __future__ import absolute_import
  from allennlp.common import Registrable
  from allennlp.data.tokenizers.token import Token
  from allennlp.data.vocabulary import Vocabulary
label_desc: Use image node_id 5 for calling the TokenIndexer obj's underlying member method code with example usage: obj.pad_token_sequence(tokens, desired_num_tokens, padding_lengths) without return types