class_docstr: Stores multiple instances of FairseqDataset together.
  Unless batch_sample=True, requires each instance
  to be the same dataset, as the collate method needs to work on batches with
  samples from each dataset.
  
  Allows specifying a distribution over the datasets to use. Note that unlike
  MultiCorpusSampledDataset, this distribution allows sampling for each item,
  rather than on a batch level. Note that datasets with sampling probabilty
  of 0 will be skipped.
  
  Each time ordered_indices() is called, a new sample is generated with
  the specified distribution.
  
  Args:
      datasets: a OrderedDict of FairseqDataset instances.
      distribution: a List containing the probability of getting an utterance from
                      corresponding dataset
      seed: random seed for sampling the datsets
      sort_indices: if true, will sort the ordered indices by size
      batch_sample: if true, will ensure each batch is from a single dataset
class_docstr_tok: ["Stores","multiple","instances","of","FairseqDataset","together",".","Unless","batch_sample=True",",","requires","each","instance","to","be","the","same","dataset",",","as","the","collate","method","needs","to","work","on","batches","with","samples","from","each","dataset",".","Allows","specifying","a","distribution","over","the","datasets","to","use",".","Note","that","unlike","MultiCorpusSampledDataset",",","this","distribution","allows","sampling","for","each","item",",","rather","than","on","a","batch","level",".","Note","that","datasets","with","sampling","probabilty","of","0","will","be","skipped",".","Each","time","ordered_indices","(",")","is","called",",","a","new","sample","is","generated","with","the","specified","distribution",".","Args",":","datasets",":","a","OrderedDict","of","FairseqDataset","instances",".","distribution",":","a","List","containing","the","probability","of","getting","an","utterance","from","corresponding","dataset","seed",":","random","seed","for","sampling","the","datsets","sort_indices",":","if","true",",","will","sort","the","ordered","indices","by","size","batch_sample",":","if","true",",","will","ensure","each","batch","is","from","a","single","dataset"]
code: def ordered_indices(self):
      start = time.time()
      with data_utils.numpy_seed(self.seed, self.epoch):
          logger.info(
              f"sampling new dataset with seed {self.seed} epoch {self.epoch}"
          )
          sampled_indices = []
          num_selected_instances = 0
  
          # For each dataset i, sample self.distribution[i] * self.total_num_instances
          for i, key in enumerate(self.datasets):
              if self.distribution[i] == 0:
                  # skip dataset if sampling probability is 0
                  continue
  
              if i < len(self.datasets) - 1:
                  num_instances = int(
                      self.distribution[i] * self.total_num_instances
                  )
                  high = self.dataset_offsets[i + 1]
              else:
                  num_instances = (
                      self.total_num_instances - num_selected_instances
                  )
                  high = self.total_num_instances
  
              logger.info(
                  f"sampling {num_instances} from {key} dataset"
              )
              num_selected_instances += num_instances
  
              # First, add k copies of the dataset where k = num_instances // len(dataset).
              # This ensures an equal distribution of the data points as much as possible.
              # For the remaining entries randomly sample them
              dataset_size = len(self.datasets[key])
              num_copies = num_instances // dataset_size
              dataset_indices = (
                  np.random.permutation(high - self.dataset_offsets[i])
                  + self.dataset_offsets[i]
              )[: num_instances - num_copies * dataset_size]
              if num_copies > 0:
                  sampled_indices += list(
                      np.concatenate(
                          (
                              np.repeat(
                                  np.arange(
                                      self.dataset_offsets[i], high
                                  ),
                                  num_copies,
                              ),
                              dataset_indices,
                          )
                      )
                  )
              else:
                  sampled_indices += list(dataset_indices)
  
          assert (
              len(sampled_indices) == self.total_num_instances
          ), f"{len(sampled_indices)} vs {self.total_num_instances}"
  
          np.random.shuffle(sampled_indices)
          if self.sort_indices:
              sampled_indices.sort(key=lambda i: self.num_tokens(i))
  
          logger.info(
              "multi_corpus_dataset ordered_indices took {}s".format(
                  time.time() - start
              )
          )
          return np.array(sampled_indices, dtype=np.int64)
  
imports: import asyncio
  import logging
  import time
  from collections import OrderedDict
  from typing import Dict, List, Optional
  import numpy
  from fairseq.data import data_utils
  from .None import FairseqDataset
label_desc: Use image node_id 2 for calling the MultiCorpusDataset obj's underlying member method code with example usage: obj.ordered_indices() and returns: np