class_docstr: Custom data loader for multiple datasets.
class_docstr_tok: ["Custom","data","loader","for","multiple","datasets","."]
docstr: Custom data loader for loading batches from multiple datasets.
  
  This dataloader iterates in parallel multiple datasets which are used
  to create mini-batches by concatenating their data together.
  The number of examples from each dataset in each mini-batch
  is defined by the `batch_sizes` parameter.
  
  The length of the loader (that is, the number of iterations in an
  epoch) is determined by the `termination_dataset`.
  
  The `oversample_small_datasets` parameter can be used to control what
  to do when smaller datasets are consumed and the epoch is not ended
  yet.
  
  :param datasets: A list of :class:`AvalancheDataset`.
  :param batch_sizes: A list of int, describing the number of examples
      per minibatch to take from each dataset.
  :param termination_dataset: The index of the dataset used to determine
      the number of iterations per epoch. Defaults to -1, which means
      that the highest number of iterations among all datasets will be
      used.
  :param oversample_small_datasets: If `True`, smaller datasets will be
      cycled again until the epoch is ended. If it is `False`, smaller
      datasets will not be cycled again, which means that some datasets
      will not contribute to the minibatch composition near the end of
      the epoch.
  :param distributed_sampling: If True, apply the PyTorch
      :class:`DistributedSampler`. Defaults to True.
      Note: the distributed sampler is not applied if not running
      a distributed training, even when True is passed.
  :param never_ending: If True, this data loader will cycle indefinitely
      by iterating over all datasets again and again and the epoch will
      never end. In this case, the `termination_dataset` and
      `oversample_small_datasets` parameters are ignored. Defaults to
      False.
  :param kwargs: data loader arguments used to instantiate the loader for
      each dataset. See PyTorch :class:`DataLoader`.
docstr_tok: ["Custom","data","loader","for","loading","batches","from","multiple","datasets",".","This","dataloader","iterates","in","parallel","multiple","datasets","which","are","used","to","create","mini-batches","by","concatenating","their","data","together",".","The","number","of","examples","from","each","dataset","in","each","mini-batch","is","defined","by","the","`","batch_sizes","`","parameter",".","The","length","of","the","loader","(","that","is",",","the","number","of","iterations","in","an","epoch",")","is","determined","by","the","`","termination_dataset","`",".","The","`","oversample_small_datasets","`","parameter","can","be","used","to","control","what","to","do","when","smaller","datasets","are","consumed","and","the","epoch","is","not","ended","yet",".",":","param","datasets",":","A","list","of",":","class",":","`","AvalancheDataset","`",".",":","param","batch_sizes",":","A","list","of","int",",","describing","the","number","of","examples","per","minibatch","to","take","from","each","dataset",".",":","param","termination_dataset",":","The","index","of","the","dataset","used","to","determine","the","number","of","iterations","per","epoch",".","Defaults","to","-1",",","which","means","that","the","highest","number","of","iterations","among","all","datasets","will","be","used",".",":","param","oversample_small_datasets",":","If","`","True","`",",","smaller","datasets","will","be","cycled","again","until","the","epoch","is","ended",".","If","it","is","`","False","`",",","smaller","datasets","will","not","be","cycled","again",",","which","means","that","some","datasets","will","not","contribute","to","the","minibatch","composition","near","the","end","of","the","epoch",".",":","param","distributed_sampling",":","If","True",",","apply","the","PyTorch",":","class",":","`","DistributedSampler","`",".","Defaults","to","True",".","Note",":","the","distributed","sampler","is","not","applied","if","not","running","a","distributed","training",",","even","when","True","is","passed",".",":","param","never_ending",":","If","True",",","this","data","loader","will","cycle","indefinitely","by","iterating","over","all","datasets","again","and","again","and","the","epoch","will","never","end",".","In","this","case",",","the","`","termination_dataset","`","and","`","oversample_small_datasets","`","parameters","are","ignored",".","Defaults","to","False",".",":","param","kwargs",":","data","loader","arguments","used","to","instantiate","the","loader","for","each","dataset",".","See","PyTorch",":","class",":","`","DataLoader","`","."]
code: def __init__(
      self,
      datasets: Sequence[AvalancheDataset],
      batch_sizes: Sequence[int],
      termination_dataset: int = -1,
      oversample_small_datasets: bool = False,
      distributed_sampling: bool = True,
      never_ending: bool = False,
      **kwargs,
  ):
      """Custom data loader for loading batches from multiple datasets.
  
      This dataloader iterates in parallel multiple datasets which are used
      to create mini-batches by concatenating their data together.
      The number of examples from each dataset in each mini-batch
      is defined by the `batch_sizes` parameter.
  
      The length of the loader (that is, the number of iterations in an
      epoch) is determined by the `termination_dataset`.
  
      The `oversample_small_datasets` parameter can be used to control what
      to do when smaller datasets are consumed and the epoch is not ended
      yet.
  
      :param datasets: A list of :class:`AvalancheDataset`.
      :param batch_sizes: A list of int, describing the number of examples
          per minibatch to take from each dataset.
      :param termination_dataset: The index of the dataset used to determine
          the number of iterations per epoch. Defaults to -1, which means
          that the highest number of iterations among all datasets will be
          used.
      :param oversample_small_datasets: If `True`, smaller datasets will be
          cycled again until the epoch is ended. If it is `False`, smaller
          datasets will not be cycled again, which means that some datasets
          will not contribute to the minibatch composition near the end of
          the epoch.
      :param distributed_sampling: If True, apply the PyTorch
          :class:`DistributedSampler`. Defaults to True.
          Note: the distributed sampler is not applied if not running
          a distributed training, even when True is passed.
      :param never_ending: If True, this data loader will cycle indefinitely
          by iterating over all datasets again and again and the epoch will
          never end. In this case, the `termination_dataset` and
          `oversample_small_datasets` parameters are ignored. Defaults to
          False.
      :param kwargs: data loader arguments used to instantiate the loader for
          each dataset. See PyTorch :class:`DataLoader`.
      """
      if "collate_mbatches" in kwargs:
          raise ValueError(
              "collate_mbatches is not needed anymore and it has been "
              "deprecated. Data loaders will use the collate function"
              "`data.collate_fn`."
          )
  
      if termination_dataset >= len(datasets):
          raise ValueError(
              "termination_dataset must be the index of the "
              "dataset used to determine the termination of an epoch"
          )
  
      distributed_sampling = (
          distributed_sampling and DistributedHelper.is_distributed
      )
  
      self.datasets = list(datasets)
      self.oversample_small_datasets: bool = oversample_small_datasets
      self.batch_sizes: List[int] = list(batch_sizes)
      self.distributed_sampling: bool = distributed_sampling
      self.loader_kwargs: Dict[str, Any] = kwargs
      self.termination_dataset: int = termination_dataset
      self.never_ending: bool = never_ending
  
      self.loader_kwargs, self.ffcv_args = self._extract_ffcv_args(
          self.loader_kwargs
      )
  
      # Only used if persistent_workers == True in loader kwargs
      self._persistent_loader: Optional[DataLoader] = None
  
      if "collate_fn" not in self.loader_kwargs:
          self.loader_kwargs["collate_fn"] = self.datasets[0].collate_fn
  
      if self.never_ending:
          # Infinite data loader
          self.termination_dataset = -1
          self.n_iterations = 10**10
          self.oversample_small_datasets = True
      else:
          # Estimate number of iterations per epoch
          loaders_len = np.full(
              (
                  len(
                      self.datasets,
                  )
              ),
              -1,
          )
  
          if self.termination_dataset < 0:
              for i, (data_subset, subset_mb_size) in enumerate(
                  zip(self.datasets, self.batch_sizes)
              ):
                  loaders_len[i] = len(
                      _make_data_loader(
                          data_subset,
                          distributed_sampling,
                          self.loader_kwargs,
                          subset_mb_size,
                          force_no_workers=True,
                      )[0]
                  )
          else:
              loaders_len[self.termination_dataset] = len(
                  _make_data_loader(
                      self.datasets[self.termination_dataset],
                      distributed_sampling,
                      self.loader_kwargs,
                      self.batch_sizes[self.termination_dataset],
                      force_no_workers=True,
                  )[0]
              )
  
          self.termination_dataset = loaders_len.argmax().item()
          self.n_iterations = loaders_len.max().item()
  
imports: from typing import Any, Dict, Iterator, List, Optional, Sequence, Sized, Union
  import numpy
  from torch.utils.data import DistributedSampler, Dataset
  from torch.utils.data.dataloader import DataLoader
  from avalanche.benchmarks.utils.data import AvalancheDataset
  from avalanche.benchmarks.utils.data_attribute import DataAttribute
  from avalanche.benchmarks.utils.ffcv_support.ffcv_components import HybridFfcvLoader, has_ffcv_support
  from avalanche.distributed.distributed_helper import DistributedHelper
  from torch.utils.data.sampler import Sampler, BatchSampler
  from torch.utils.data import ConcatDataset
label_desc: Use image node_id 1 to create a new MultiDatasetDataLoader object  with example: obj = MultiDatasetDataLoader(datasets, batch_sizes, termination_dataset, oversample_small_datasets, distributed_sampling, never_ending)