class_docstr: An abstract ``DataIterator`` class. ``DataIterators`` must override ``_create_batches()``.
  
  Parameters
  ----------
  batch_size : ``int``, optional, (default = 32)
      The size of each batch of instances yielded when calling the iterator.
  instances_per_epoch : ``int``, optional, (default = None)
      If specified, each epoch will consist of precisely this many instances.
      If not specified, each epoch will consist of a single pass through the dataset.
  max_instances_in_memory : ``int``, optional, (default = None)
      If specified, the iterator will load this many instances at a time into an
      in-memory list and then produce batches from one such list at a time. This
      could be useful if your instances are read lazily from disk.
  cache_instances : ``bool``, optional, (default = False)
      If true, the iterator will cache the tensorized instances in memory.
      If false, it will do the tensorization anew each iteration.
  track_epoch : ``bool``, optional, (default = False)
      If true, each instance will get a ``MetadataField`` containing the epoch number.
  maximum_samples_per_batch : ``Tuple[str, int]``, (default = None)
      If specified, then is a tuple (padding_key, limit) and we will
      shrink the batch size for very long sequences such that
      batch_size * sequence_length <= limit where sequence_length is given
      by the padding_key.
docstr: Breaks the dataset into "memory-sized" lists of instances,
  which it yields up one at a time until it gets through a full epoch.
  
  For example, if the dataset is already an in-memory list, and each epoch
  represents one pass through the dataset, it just yields back the dataset.
  Whereas if the dataset is lazily read from disk and we've specified to
  load 1000 instances at a time, then it yields lists of 1000 instances each.
docstr_tok: ["Breaks","the","dataset","into","``","memory-sized","''","lists","of","instances",",","which","it","yields","up","one","at","a","time","until","it","gets","through","a","full","epoch",".","For","example",",","if","the","dataset","is","already","an","in-memory","list",",","and","each","epoch","represents","one","pass","through","the","dataset",",","it","just","yields","back","the","dataset",".","Whereas","if","the","dataset","is","lazily","read","from","disk","and","we","'ve","specified","to","load","1000","instances","at","a","time",",","then","it","yields","lists","of","1000","instances","each","."]
code: def _memory_sized_lists(self, instances):
      """
      Breaks the dataset into "memory-sized" lists of instances,
      which it yields up one at a time until it gets through a full epoch.
  
      For example, if the dataset is already an in-memory list, and each epoch
      represents one pass through the dataset, it just yields back the dataset.
      Whereas if the dataset is lazily read from disk and we've specified to
      load 1000 instances at a time, then it yields lists of 1000 instances each.
      """
      lazy = is_lazy(instances)
  
      # Get an iterator over the next epoch worth of instances.
      iterator = self._take_instances(
          instances, self._instances_per_epoch
      )
  
      # We have four different cases to deal with:
  
      # With lazy instances and no guidance about how many to load into memory,
      # we just load ``batch_size`` instances at a time:
      if lazy and self._max_instances_in_memory is None:
          _i = lazy_groups_of(iterator, self._batch_size)
          while True:
              yield _i.next()
      # If we specified max instances in memory, lazy or not, we just
      # load ``max_instances_in_memory`` instances at a time:
      elif self._max_instances_in_memory is not None:
          _i = lazy_groups_of(iterator, self._max_instances_in_memory)
          while True:
              yield _i.next()
      # If we have non-lazy instances, and we want all instances each epoch,
      # then we just yield back the list of instances:
      elif self._instances_per_epoch is None:
          yield ensure_list(instances)
      # In the final case we have non-lazy instances, we want a specific number
      # of instances each epoch, and we didn't specify how to many instances to load
      # into memory. So we convert the whole iterator to a list:
      else:
          yield list(iterator)
  
imports: from __future__ import division
  from __future__ import absolute_import
  import logging
  from collections import defaultdict
  import itertools
  import math
  import random
  import torch
  from allennlp.common.registrable import Registrable
  from allennlp.common.util import is_lazy, lazy_groups_of, ensure_list
  from allennlp.data.dataset import Batch
  from allennlp.data.fields import MetadataField
  from allennlp.data.instance import Instance
  from allennlp.data.vocabulary import Vocabulary
label_desc: Use image node_id 4 for calling the DataIterator obj's underlying member method code with example usage: obj._memory_sized_lists(instances) without return types