class_docstr: An abstract ``DataIterator`` class. ``DataIterators`` must override ``_create_batches()``.
  
  Parameters
  ----------
  batch_size : ``int``, optional, (default = 32)
      The size of each batch of instances yielded when calling the iterator.
  instances_per_epoch : ``int``, optional, (default = None)
      If specified, each epoch will consist of precisely this many instances.
      If not specified, each epoch will consist of a single pass through the dataset.
  max_instances_in_memory : ``int``, optional, (default = None)
      If specified, the iterator will load this many instances at a time into an
      in-memory list and then produce batches from one such list at a time. This
      could be useful if your instances are read lazily from disk.
  cache_instances : ``bool``, optional, (default = False)
      If true, the iterator will cache the tensorized instances in memory.
      If false, it will do the tensorization anew each iteration.
  track_epoch : ``bool``, optional, (default = False)
      If true, each instance will get a ``MetadataField`` containing the epoch number.
  maximum_samples_per_batch : ``Tuple[str, int]``, (default = None)
      If specified, then is a tuple (padding_key, limit) and we will
      shrink the batch size for very long sequences such that
      batch_size * sequence_length <= limit where sequence_length is given
      by the padding_key.
docstr: Returns a generator that yields batches over the given dataset
  for the given number of epochs. If ``num_epochs`` is not specified,
  it will yield batches forever.
  
  Parameters
  ----------
  instances : ``Iterable[Instance]``
      The instances in the dataset. IMPORTANT: this must be able to be
      iterated over *multiple times*. That is, it must be either a List
      or some other object whose ``__iter__`` method returns a fresh iterator
      each time it's called.
  num_epochs : ``int``, optional (default=``None``)
      How times should we iterate over this dataset?  If ``None``, we will iterate over it
      forever.
  shuffle : ``bool``, optional (default=``True``)
      If ``True``, we will shuffle the instances in ``dataset`` before constructing batches
      and iterating over the data.
  cuda_device : ``int``
      If cuda_device >= 0, GPUs are available and Pytorch was compiled with CUDA support, the
      tensor will be copied to the cuda_device specified.
docstr_tok: ["Returns","a","generator","that","yields","batches","over","the","given","dataset","for","the","given","number","of","epochs",".","If","``","num_epochs","``","is","not","specified",",","it","will","yield","batches","forever",".","Parameters","--","--","--","--","--","instances",":","``","Iterable","[","Instance","]","``","The","instances","in","the","dataset",".","IMPORTANT",":","this","must","be","able","to","be","iterated","over","*","multiple","times","*",".","That","is",",","it","must","be","either","a","List","or","some","other","object","whose","``","__iter__","``","method","returns","a","fresh","iterator","each","time","it","'s","called",".","num_epochs",":","``","int","``",",","optional","(","default=","``","None","``",")","How","times","should","we","iterate","over","this","dataset","?","If","``","None","``",",","we","will","iterate","over","it","forever",".","shuffle",":","``","bool","``",",","optional","(","default=","``","True","``",")","If","``","True","``",",","we","will","shuffle","the","instances","in","``","dataset","``","before","constructing","batches","and","iterating","over","the","data",".","cuda_device",":","``","int","``","If","cuda_device",">","=","0",",","GPUs","are","available","and","Pytorch","was","compiled","with","CUDA","support",",","the","tensor","will","be","copied","to","the","cuda_device","specified","."]
code: def __call__(
      self, instances, num_epochs=None, shuffle=True, cuda_device=-1
  ):
      """
      Returns a generator that yields batches over the given dataset
      for the given number of epochs. If ``num_epochs`` is not specified,
      it will yield batches forever.
  
      Parameters
      ----------
      instances : ``Iterable[Instance]``
          The instances in the dataset. IMPORTANT: this must be able to be
          iterated over *multiple times*. That is, it must be either a List
          or some other object whose ``__iter__`` method returns a fresh iterator
          each time it's called.
      num_epochs : ``int``, optional (default=``None``)
          How times should we iterate over this dataset?  If ``None``, we will iterate over it
          forever.
      shuffle : ``bool``, optional (default=``True``)
          If ``True``, we will shuffle the instances in ``dataset`` before constructing batches
          and iterating over the data.
      cuda_device : ``int``
          If cuda_device >= 0, GPUs are available and Pytorch was compiled with CUDA support, the
          tensor will be copied to the cuda_device specified.
      """
      # Instances is likely to be a list, which cannot be used as a key,
      # so we take the object id instead.
      key = id(instances)
      starting_epoch = self._epochs[key]
  
      if num_epochs is None:
          epochs = itertools.count(starting_epoch)
      else:
          epochs = range(starting_epoch, starting_epoch + num_epochs)
  
      for epoch in epochs:
          self._epochs[key] = epoch
  
          if self._cache_instances and key in self._cache:
              # Serve the results from the cache.
              tensor_dicts = self._cache[key]
  
              if shuffle:
                  random.shuffle(tensor_dicts)
              for tensor_dict in tensor_dicts:
                  if self._track_epoch:
                      # The tensor_dict already has an "epoch_num" tensor,
                      # so just fill it with the right value.
                      epoch_tensor = tensor_dict["epoch_num"]
                      epoch_tensor.fill_(epoch)
                  yield tensor_dict
          else:
              batches = self._create_batches(instances, shuffle)
  
              # Should we add the instances to the cache this epoch?
              add_to_cache = (
                  self._cache_instances and key not in self._cache
              )
  
              for batch in batches:
                  if self._track_epoch:
                      add_epoch_number(batch, epoch)
  
                  if self.vocab is not None:
                      batch.index_instances(self.vocab)
  
                  padding_lengths = batch.get_padding_lengths()
                  logger.debug(
                      "Batch padding lengths: %s",
                      unicode(padding_lengths),
                  )
                  logger.debug("Batch size: %d", len(batch.instances))
                  tensor_dict = batch.as_tensor_dict(
                      padding_lengths, cuda_device=cuda_device
                  )
  
                  if add_to_cache:
                      self._cache[key].append(tensor_dict)
  
                  yield tensor_dict
  
imports: from __future__ import division
  from __future__ import absolute_import
  import logging
  from collections import defaultdict
  import itertools
  import math
  import random
  import torch
  from allennlp.common.registrable import Registrable
  from allennlp.common.util import is_lazy, lazy_groups_of, ensure_list
  from allennlp.data.dataset import Batch
  from allennlp.data.fields import MetadataField
  from allennlp.data.instance import Instance
  from allennlp.data.vocabulary import Vocabulary
label_desc: Use image node_id 2 for calling the DataIterator obj's underlying member method code with example usage: obj.__call__(instances, num_epochs, shuffle, cuda_device) without return types