class_docstr: Builds and/or loads a dictionary.
  
  The dictionary provides access to the frequency of each token, functions to
  translate sentences from tokens to their vectors (list of ints, each int is the
  index of a token in the dictionary) and back from vectors to tokenized text.
class_docstr_tok: ["Builds","and\/or","loads","a","dictionary",".","The","dictionary","provides","access","to","the","frequency","of","each","token",",","functions","to","translate","sentences","from","tokens","to","their","vectors","(","list","of","ints",",","each","int","is","the","index","of","a","token","in","the","dictionary",")","and","back","from","vectors","to","tokenized","text","."]
docstr: Set the dictionary vocab to the bpe vocab, merging counts.
code: def _remove_non_bpe(self):
      """
      Set the dictionary vocab to the bpe vocab, merging counts.
      """
      to_remove = []
      to_add = []
      for token, freq in self.freq.items():
          tokens = self.bpe_tokenize(token)
          if len(tokens) != 1:
              for t in tokens:
                  to_add.append((t, freq))
              to_remove.append(token)
      for token in to_remove:
          del self.freq[token]
          idx = self.tok2ind.pop(token)
          del self.ind2tok[idx]
      for token, freq in to_add:
          self.add_token(token)
          self.freq[token] += freq
  
imports: from typing import Optional
  from parlai.core.params import ParlaiParser
  from parlai.core.opt import Opt
  from parlai.core.build_data import modelzoo_path
  from parlai.utils.bpe import bpe_factory, BPEHelper
  from .agents import Agent
  from .build_data import make_dir
  from collections import defaultdict
  import copy
  import numpy
  import os
  import json
  import re
  import parlai.utils.logging
  from parlai.utils.io import PathManager
  from typing import List
  import enum
label_desc: Use image node_id 25 for calling the DictionaryAgent obj's underlying member method code with example usage: obj._remove_non_bpe() without return types