class_docstr: AggregateTokenizer, allowing one to combine multiple regular monolongual tokenizers into one tokenizer.
  The intuition is that we can use existing tokenizers "as is", without retraining, and associate each tokenizer with a language id
  during text processing (language id will be used to route the incoming text sample to the right tokenizer)
  as well as a token id range for detokenization (e.g. [0..127] for tokenizer A, [128..255] for tokenizer B) so
  that the orignal text could be reconstructed. Note that we assume that the incoming dict of langs / tokenizers
  is ordered, e.g. the first tokenizer will be assigned a lower interval of token ids
      Args:
      tokenizers: dict of tokenizers, keys are lang ids, values are actual tokenizers
code: def __init__(self, tokenizers: Dict):
      self.tokenizers_dict = tokenizers
      self.vocabulary = []
  
      # the tokenizers should produce non-overlapping, ordered token ids
      # keys are language ids
      self.token_id_offset = {}
  
      # keys are tokenizer numbers
      self.token_id_offset_by_tokenizer_num = {}
      offset = 0
      i = 0
      for lang, tokenizer in self.tokenizers_dict.items():
          self.token_id_offset[lang] = offset
          self.token_id_offset_by_tokenizer_num[i] = offset
          offset += len(tokenizer.vocab)
          i += 1
  
      for tokenizer in self.tokenizers_dict.values():
          self.vocabulary.extend(tokenizer.vocab)
  
      self.vocab_size = len(self.vocabulary)
      logging.info(f"Aggregate vocab size: {self.vocab_size}")
  
      # for compatibility purposes only -- right now only the get_vocab method
      # is supported, returning the joint vocab across all tokenizers
      self.tokenizer = DummyTokenizer(self.vocabulary)
  
      # lookup tables to speed up token to text operations
      # if there are two tokenizers, [0,1], ['en', 'es'], each with 128 tokens, the aggregate tokenizer
      # token range will be [0,255]. The below method provides three look up tables:
      # one, to convert the incoming token id -- e.g. 200 into its real id (200-127 = 73)
      # second, to compute the tokenizer id that should process that token (1)
      # third, the compute the lang id for that token ('es')
      (
          offset_token_ids_by_token_id,
          tokenizers_by_token_id,
          langs_by_token_id,
      ) = self._calculate_offsets()
  
      self.offset_token_ids_by_token_id = offset_token_ids_by_token_id
      self.tokenizers_by_token_id = tokenizers_by_token_id
      self.langs_by_token_id = langs_by_token_id
  
imports: from typing import Dict, List, Union
  import numpy
  from nemo.collections.common.tokenizers.tokenizer_spec import TokenizerSpec
  from nemo.utils import logging
label_desc: Use image node_id 1 to create a new AggregateTokenizer object from inherited base classes: TokenizerSpec with example: obj = AggregateTokenizer(tokenizers)