class_docstr: Builds and/or loads a dictionary.
  
  The dictionary provides access to the frequency of each token, functions to
  translate sentences from tokens to their vectors (list of ints, each int is the
  index of a token in the dictionary) and back from vectors to tokenized text.
class_docstr_tok: ["Builds","and\/or","loads","a","dictionary",".","The","dictionary","provides","access","to","the","frequency","of","each","token",",","functions","to","translate","sentences","from","tokens","to","their","vectors","(","list","of","ints",",","each","int","is","the","index","of","a","token","in","the","dictionary",")","and","back","from","vectors","to","tokenized","text","."]
docstr: Add commandline arguments related to the dictionary.
code: def add_cmdline_args(
      cls, parser: ParlaiParser, partial_opt: Optional[Opt] = None
  ) -> ParlaiParser:
      """
      Add commandline arguments related to the dictionary.
      """
      dictionary = parser.add_argument_group("Dictionary Arguments")
      dictionary.add_argument(
          "-df",
          "--dict-file",
          help="path to dictionary file. defaults to [model_file].dict if "
          "not set and model_file is set.",
          hidden=True,
      )
      dictionary.add_argument(
          "--dict-initpath",
          hidden=True,
          help="path to a saved dictionary to load tokens / counts from to "
          "seed the dictionary with initial tokens and/or frequencies",
      )
      dictionary.add_argument(
          "--dict-language",
          default=DictionaryAgent.default_lang,
          hidden=True,
          help="sets language for the punkt sentence tokenizer",
      )
      dictionary.add_argument(
          "--dict-max-ngram-size",
          type=int,
          hidden=True,
          default=DictionaryAgent.default_maxngram,
          help="looks for ngrams of up to this size. this is ignored when "
          "building the dictionary. note: this takes approximate "
          "runtime of len(sentence)^max_ngram_size",
      )
      dictionary.add_argument(
          "--dict-minfreq",
          default=DictionaryAgent.default_minfreq,
          type=int,
          help="minimum frequency of words to include them in sorted "
          "dict or minimum frequency of bpe codecs",
          hidden=True,
      )
      dictionary.add_argument(
          "--dict-maxtokens",
          default=DictionaryAgent.default_maxtokens,
          type=int,
          help="max number of tokens to include in dictionary or bpe codecs",
          hidden=True,
      )
      dictionary.add_argument(
          "--dict-nulltoken",
          default=DictionaryAgent.default_null,
          hidden=True,
          help="empty token, can be used for padding or just empty values",
      )
      dictionary.add_argument(
          "--dict-starttoken",
          default=DictionaryAgent.default_start,
          hidden=True,
          help="token for starting sentence generation, if needed",
      )
      dictionary.add_argument(
          "--dict-endtoken",
          default=DictionaryAgent.default_end,
          hidden=True,
          help="token for end of sentence markers, if needed",
      )
      dictionary.add_argument(
          "--dict-unktoken",
          default=DictionaryAgent.default_unk,
          hidden=True,
          help="token to return for unavailable words",
      )
      dictionary.add_argument(
          "-tok",
          "--dict-tokenizer",
          default=DictionaryAgent.default_tok,
          help='Which tokenizer to use. Defaults to "split", which splits '
          "on whitespace as well as recognizing basic punctuation. "
          "Other options include nltk, gpt2 and bytelevelbpe.",
          hidden=True,
      )
      dictionary.add_argument(
          "--dict-lower",
          default=DictionaryAgent.default_lower,
          type="bool",
          help="Whether or not to lowercase all text seen.",
          hidden=True,
      )
      dictionary.add_argument(
          "--bpe-debug",
          action="store_true",
          hidden=True,
          help="Leave BPE tokens untouched in output. Useful for debugging.",
      )
      dictionary.add_argument(
          "--dict-textfields",
          default=DictionaryAgent.default_textfields,
          hidden=True,
          help="Observation fields which dictionary learns vocabulary from. "
          "Tasks with additional fields may add to this list to handle "
          "any extra vocabulary.",
      )
      dictionary = BPEHelper.add_cmdline_args(
          dictionary, partial_opt=partial_opt
      )
      return dictionary
  
imports: from typing import Optional
  from parlai.core.params import ParlaiParser
  from parlai.core.opt import Opt
  from parlai.core.build_data import modelzoo_path
  from parlai.utils.bpe import bpe_factory, BPEHelper
  from .agents import Agent
  from .build_data import make_dir
  from collections import defaultdict
  import copy
  import numpy
  import os
  import json
  import re
  import parlai.utils.logging
  from parlai.utils.io import PathManager
  from typing import List
  import enum
label_desc: Use image node_id 1 for calling the DictionaryAgent obj's underlying member method code with example usage: obj.add_cmdline_args(cls, parser, partial_opt) and returns: dictionary