class_docstr: Builds and/or loads a dictionary.
  
  The dictionary provides access to the frequency of each token, functions to
  translate sentences from tokens to their vectors (list of ints, each int is the
  index of a token in the dictionary) and back from vectors to tokenized text.
class_docstr_tok: ["Builds","and\/or","loads","a","dictionary",".","The","dictionary","provides","access","to","the","frequency","of","each","token",",","functions","to","translate","sentences","from","tokens","to","their","vectors","(","list","of","ints",",","each","int","is","the","index","of","a","token","in","the","dictionary",")","and","back","from","vectors","to","tokenized","text","."]
docstr: Initialize DictionaryAgent.
code: def __init__(self, opt: Opt, shared=None):
      """
      Initialize DictionaryAgent.
      """
      self.opt = copy.deepcopy(opt)
      self.minfreq = opt.get(
          "dict_minfreq", DictionaryAgent.default_minfreq
      )
      self.null_token = opt.get(
          "dict_nulltoken", DictionaryAgent.default_null
      )
      self.end_token = opt.get(
          "dict_endtoken", DictionaryAgent.default_end
      )
      self.unk_token = opt.get(
          "dict_unktoken", DictionaryAgent.default_unk
      )
      self.start_token = opt.get(
          "dict_starttoken", DictionaryAgent.default_start
      )
      self.max_ngram_size = opt.get(
          "dict_max_ngram_size", DictionaryAgent.default_maxngram
      )
      self.tokenizer = opt.get(
          "dict_tokenizer", DictionaryAgent.default_tok
      )
      self.lower = opt.get("dict_lower", DictionaryAgent.default_lower)
      self.maxtokens = opt.get(
          "dict_maxtokens", DictionaryAgent.default_maxtokens
      )
      self.textfields = opt.get(
          "dict_textfields", DictionaryAgent.default_textfields
      ).split(",")
  
      # used to signal whether we should use training time tricks, like bpe droput
      self._tokenization_mode = TokenizationMode.TEST_TIME_LABEL
  
      try:
          self.tokenizer_fun = getattr(
              self, self.tokenizer + "_tokenize"
          )
      except AttributeError:
          raise AttributeError(
              "tokenizer type {} not yet supported".format(
                  self.tokenizer
              )
          )
  
      if shared:
          self.freq = shared.get("freq", {})
          self.tok2ind = shared.get("tok2ind", {})
          self.ind2tok = shared.get("ind2tok", {})
      else:
          self.additional_special_tokens: List[str] = []
          self.freq = defaultdict(int)
          self.tok2ind = {}
          self.ind2tok = {}
  
          if self.null_token:
              self.add_token(self.null_token)
  
          if self.start_token:
              # set special start of sentence word token
              self.add_token(self.start_token)
  
          if self.end_token:
              # set special end of sentence word token
              self.add_token(self.end_token)
  
          if self.unk_token:
              # set special unknown word token
              self.add_token(self.unk_token)
  
          loaded = False
          # If data built via pytorch data teacher, we need to load prebuilt dict
          if opt.get("dict_file"):
              opt["dict_file"] = modelzoo_path(
                  opt.get("datapath"), opt["dict_file"]
              )
              if PathManager.exists(opt["dict_file"]):
                  # load pre-existing dictionary
                  self.load(opt["dict_file"])
                  loaded = True
  
          if not loaded and opt.get("dict_initpath"):
              # load seed dictionary
              opt["dict_initpath"] = modelzoo_path(
                  opt.get("datapath"), opt["dict_initpath"]
              )
              # don't check isfile first, should fail if file not found
              self.load(opt["dict_initpath"])
          opt["dict_loaded"] = loaded
  
      # cache unk token for later
      self._unk_token_idx = self.tok2ind.get(self.unk_token)
  
      # initialize tokenizers
      if self.tokenizer == "nltk":
          try:
              import nltk
          except ImportError:
              raise ImportError(
                  "Please install nltk (pip install nltk)"
              )
          # nltk-specific setup
          st_path = "tokenizers/punkt/{0}.pickle".format(
              opt["dict_language"]
          )
          try:
              self.sent_tok = nltk.data.load(st_path)
          except LookupError:
              nltk.download("punkt")
              self.sent_tok = nltk.data.load(st_path)
          self.word_tok = nltk.tokenize.treebank.TreebankWordTokenizer()
      elif self.tokenizer in [
          "bpe",
          "gpt2",
          "bytelevelbpe",
          "slow_bytelevel_bpe",
      ]:
          self.bpe = bpe_factory(opt, shared)
          self.bpe.sync_with_dict(self)
  
      if not shared:
          if self.null_token:
              # fix count for null token to one billion and three
              self.freq[self.null_token] = 1000000003
  
          if self.start_token:
              # fix count for start of sentence token to one billion and two
              self.freq[self.start_token] = 1000000002
  
          if self.end_token:
              # fix count for end of sentence token to one billion and one
              self.freq[self.end_token] = 1000000001
  
          if self.unk_token:
              # fix count for unknown token to one billion
              self.freq[self.unk_token] = 1000000000
  
          if opt.get("dict_file"):
              self.save_path = opt["dict_file"]
  
imports: from typing import Optional
  from parlai.core.params import ParlaiParser
  from parlai.core.opt import Opt
  from parlai.core.build_data import modelzoo_path
  from parlai.utils.bpe import bpe_factory, BPEHelper
  from .agents import Agent
  from .build_data import make_dir
  from collections import defaultdict
  import copy
  import numpy
  import os
  import json
  import re
  import parlai.utils.logging
  from parlai.utils.io import PathManager
  from typing import List
  import enum
label_desc: Use image node_id 2 to create a new DictionaryAgent object from inherited base classes: Agent with example: obj = DictionaryAgent(opt, shared)