class_docstr: DALI base iterator class. Shouldn't be used directly.
  
  Parameters
  ----------
  pipelines : list of nvidia.dali.Pipeline
              List of pipelines to use
  output_map : list of (str, str)
               List of pairs (output_name, tag) which maps consecutive
               outputs of DALI pipelines to proper field in MXNet's
               DataBatch.
               tag is one of DALIGenericIterator.DATA_TAG
               and DALIGenericIterator.LABEL_TAG mapping given output
               for data or label correspondingly.
               output_names should be distinct.
  size : int, default = -1
              Number of samples in the shard for the wrapped pipeline (if there is more than one
              it is a sum). Providing -1 means that the iterator will work until StopIteration
              is raised from the inside of iter_setup(). The options `last_batch_policy`,
              `last_batch_padded` and `auto_reset` don't work in such case. It works with only
              one pipeline inside the iterator.
              Mutually exclusive with `reader_name` argument
  reader_name : str, default = None
              Name of the reader which will be queried for the shard size, number of shards, and
              all other properties necessary to count properly the number of relevant and padded
              samples that iterator needs to deal with. Sets `last_batch_padded`
              accordingly to the reader's configuration (`pad_last_batch` reader argument)
  auto_reset : string or bool, optional, default = False
              Whether the iterator resets itself for the next epoch or it requires reset() to be
              called explicitly.
  
              It can be one of the following values:
  
              * ``"no"``, ``False`` or ``None`` - at the end of epoch StopIteration is raised and
                reset() needs to be called. Calling ``iter()`` on the iterator would reset
                it as well.
              * ``"yes"`` or ``"True"``- at the end of epoch StopIteration is raised but reset()
                is called internally automatically
  
  fill_last_batch : bool, optional, default = None
              **Deprecated** Please use ``last_batch_policy`` instead
  
              Whether to fill the last batch with data up to 'self.batch_size'.
              The iterator would return the first integer multiple
              of self._num_gpus * self.batch_size entries which exceeds 'size'.
              Setting this flag to False will cause the iterator to return
              exactly 'size' entries.
  last_batch_policy: optional, default = LastBatchPolicy.FILL
              What to do with the last batch when there are not enough samples in the epoch
              to fully fill it. See :meth:`nvidia.dali.plugin.base_iterator.LastBatchPolicy`
  last_batch_padded : bool, optional, default = False
              Whether the last batch provided by DALI is padded with the last sample
              or it just wraps up. In the conjunction with ``last_batch_policy`` it tells
              if the iterator returning last batch with data only partially filled with
              data from the current epoch is dropping padding samples or samples from
              the next epoch. If set to False next
              epoch will end sooner as data from it was consumed but dropped. If set to
              True next epoch would be the same length as the first one. For this to happen,
              the option `pad_last_batch` in the reader needs to be set to True as well.
              It is overwritten when `reader_name` argument is provided
  prepare_first_batch : bool, optional, default = True
              Whether DALI should buffer the first batch right after the creation of the iterator,
              so one batch is already prepared when the iterator is prompted for the data
  
  Example
  -------
  With the data set ``[1,2,3,4,5,6,7]`` and the batch size 2:
  
  last_batch_policy = LastBatchPolicy.PARTIAL, last_batch_padded = True  -> last batch = ``[7]``,
  next iteration will return ``[1, 2]``
  
  last_batch_policy = LastBatchPolicy.PARTIAL, last_batch_padded = False -> last batch = ``[7]``,
  next iteration will return ``[2, 3]``
  
  last_batch_policy = LastBatchPolicy.FILL, last_batch_padded = True   -> last batch = ``[7, 7]``,
   next iteration will return ``[1, 2]``
  
  last_batch_policy = LastBatchPolicy.FILL, last_batch_padded = False  -> last batch = ``[7, 1]``,
  next iteration will return ``[2, 3]``
  
  last_batch_policy = LastBatchPolicy.DROP, last_batch_padded = True   -> last batch = ``[5, 6]``,
  next iteration will return ``[1, 2]``
  
  last_batch_policy = LastBatchPolicy.DROP, last_batch_padded = False  -> last batch = ``[5, 6]``,
  next iteration will return ``[2, 3]``
class_docstr_tok: ["DALI","base","iterator","class",".","Should","n't","be","used","directly",".","Parameters","--","--","--","--","--","pipelines",":","list","of","nvidia.dali.Pipeline","List","of","pipelines","to","use","output_map",":","list","of","(","str",",","str",")","List","of","pairs","(","output_name",",","tag",")","which","maps","consecutive","outputs","of","DALI","pipelines","to","proper","field","in","MXNet's","DataBatch",".","tag","is","one","of","DALIGenericIterator.DATA_TAG","and","DALIGenericIterator.LABEL_TAG","mapping","given","output","for","data","or","label","correspondingly",".","output_names","should","be","distinct",".","size",":","int",",","default","=","-1","Number","of","samples","in","the","shard","for","the","wrapped","pipeline","(","if","there","is","more","than","one","it","is","a","sum",")",".","Providing","-1","means","that","the","iterator","will","work","until","StopIteration","is","raised","from","the","inside","of","iter_setup","(",")",".","The","options","`","last_batch_policy","`",",","`","last_batch_padded","`","and","`","auto_reset","`","do","n't","work","in","such","case",".","It","works","with","only","one","pipeline","inside","the","iterator",".","Mutually","exclusive","with","`","reader_name","`","argument","reader_name",":","str",",","default","=","None","Name","of","the","reader","which","will","be","queried","for","the","shard","size",",","number","of","shards",",","and","all","other","properties","necessary","to","count","properly","the","number","of","relevant","and","padded","samples","that","iterator","needs","to","deal","with",".","Sets","`","last_batch_padded","`","accordingly","to","the","reader","'s","configuration","(","`","pad_last_batch","`","reader","argument",")","auto_reset",":","string","or","bool",",","optional",",","default","=","False","Whether","the","iterator","resets","itself","for","the","next","epoch","or","it","requires","reset","(",")","to","be","called","explicitly",".","It","can","be","one","of","the","following","values",":","*","``","``","no","''","``",",","``","False","``","or","``","None","``","-","at","the","end","of","epoch","StopIteration","is","raised","and","reset","(",")","needs","to","be","called",".","Calling","``","iter","(",")","``","on","the","iterator","would","reset","it","as","well",".","*","``","``","yes","''","``","or","``","``","True","''","``","-","at","the","end","of","epoch","StopIteration","is","raised","but","reset","(",")","is","called","internally","automatically","fill_last_batch",":","bool",",","optional",",","default","=","None","*","*","Deprecated","*","*","Please","use","``","last_batch_policy","``","instead","Whether","to","fill","the","last","batch","with","data","up","to","'self.batch_size","'",".","The","iterator","would","return","the","first","integer","multiple","of","self._num_gpus","*","self.batch_size","entries","which","exceeds","'size","'",".","Setting","this","flag","to","False","will","cause","the","iterator","to","return","exactly","'size","'","entries",".","last_batch_policy",":","optional",",","default","=","LastBatchPolicy.FILL","What","to","do","with","the","last","batch","when","there","are","not","enough","samples","in","the","epoch","to","fully","fill","it",".","See",":","meth",":","`","nvidia.dali.plugin.base_iterator.LastBatchPolicy","`","last_batch_padded",":","bool",",","optional",",","default","=","False","Whether","the","last","batch","provided","by","DALI","is","padded","with","the","last","sample","or","it","just","wraps","up",".","In","the","conjunction","with","``","last_batch_policy","``","it","tells","if","the","iterator","returning","last","batch","with","data","only","partially","filled","with","data","from","the","current","epoch","is","dropping","padding","samples","or","samples","from","the","next","epoch",".","If","set","to","False","next","epoch","will","end","sooner","as","data","from","it","was","consumed","but","dropped",".","If","set","to","True","next","epoch","would","be","the","same","length","as","the","first","one",".","For","this","to","happen",",","the","option","`","pad_last_batch","`","in","the","reader","needs","to","be","set","to","True","as","well",".","It","is","overwritten","when","`","reader_name","`","argument","is","provided","prepare_first_batch",":","bool",",","optional",",","default","=","True","Whether","DALI","should","buffer","the","first","batch","right","after","the","creation","of","the","iterator",",","so","one","batch","is","already","prepared","when","the","iterator","is","prompted","for","the","data","Example","--","--","--","-","With","the","data","set","``","[","1,2,3,4,5,6,7","]","``","and","the","batch","size","2",":","last_batch_policy","=","LastBatchPolicy.PARTIAL",",","last_batch_padded","=","True","-",">","last","batch","=","``","[","7","]","``",",","next","iteration","will","return","``","[","1",",","2","]","``","last_batch_policy","=","LastBatchPolicy.PARTIAL",",","last_batch_padded","=","False","-",">","last","batch","=","``","[","7","]","``",",","next","iteration","will","return","``","[","2",",","3","]","``","last_batch_policy","=","LastBatchPolicy.FILL",",","last_batch_padded","=","True","-",">","last","batch","=","``","[","7",",","7","]","``",",","next","iteration","will","return","``","[","1",",","2","]","``","last_batch_policy","=","LastBatchPolicy.FILL",",","last_batch_padded","=","False","-",">","last","batch","=","``","[","7",",","1","]","``",",","next","iteration","will","return","``","[","2",",","3","]","``","last_batch_policy","=","LastBatchPolicy.DROP",",","last_batch_padded","=","True","-",">","last","batch","=","``","[","5",",","6","]","``",",","next","iteration","will","return","``","[","1",",","2","]","``","last_batch_policy","=","LastBatchPolicy.DROP",",","last_batch_padded","=","False","-",">","last","batch","=","``","[","5",",","6","]","``",",","next","iteration","will","return","``","[","2",",","3","]","``"]
docstr: Resets the iterator after the full epoch.
  DALI iterators do not support resetting before the end of the epoch
  and will ignore such request.
docstr_tok: ["Resets","the","iterator","after","the","full","epoch",".","DALI","iterators","do","not","support","resetting","before","the","end","of","the","epoch","and","will","ignore","such","request","."]
code: def reset(self):
      """
      Resets the iterator after the full epoch.
      DALI iterators do not support resetting before the end of the epoch
      and will ignore such request.
      """
      # in the case of the DROP policy the user who runs DALI, based on the iterator length,
      # can assume there is no more data in the pipeline where there still is the last,
      # incomplete batch, we need to extract from the pipeline and drop before rising
      # StopIteration indicating the pipeline is depleted. Here we first check if that
      # is the case, and if so we run the pipeline and drop the last batch
      if self._last_batch_policy == LastBatchPolicy.DROP:
          should_end = self._advance_and_check_drop_last(
              dry_run=True, end_iteration=False
          )
          already_ended = self._size > 0 and self._counter >= self._size
          if should_end and not already_ended:
              self._get_outputs()
              self._schedule_runs()
              self._advance_and_check_drop_last(end_iteration=False)
  
      if self._counter >= self._size or self._size < 0:
          if (
              self._last_batch_policy == LastBatchPolicy.FILL
              and not self._last_batch_padded
          ):
              if self._reader_name:
                  # accurate way
                  # get the number of samples read in this epoch by each GPU
                  # self._counter had initial value of min(self._counter_per_gpu) so subtract
                  # this to get the actual value
                  self._counter -= min(self._counter_per_gpu)
                  self._counter_per_gpu = (
                      self._counter_per_gpu + self._counter
                  )
                  # check how much each GPU read ahead from next shard, as shards have different
                  # size each epoch GPU may read ahead or not
                  self._counter_per_gpu = (
                      self._counter_per_gpu - self._shard_sizes_per_gpu
                  )
                  # to make sure that in the next epoch we read the whole shard we need
                  # to set start value to the smallest one
                  self._counter = min(self._counter_per_gpu)
              else:
                  # legacy way
                  self._counter = self._counter % self._size
          else:
              self._counter = 0
          # advance to the next shard
          if self._reader_name:
              if not self._is_stick_to_shard:
                  # move shards id for wrapped pipelines
                  self._shards_id = (
                      self._shards_id + 1
                  ) % self._shards_num
              # revaluate _size
              if (
                  self._last_batch_policy == LastBatchPolicy.FILL
                  and not self._last_batch_padded
              ):
                  # move all shards ids GPU ahead
                  if not self._is_stick_to_shard:
                      self._shard_sizes_per_gpu = np.roll(
                          self._shard_sizes_per_gpu, 1
                      )
                  # check how many samples we need to reach from each shard in next epoch
                  # per each GPU taking into account already read
                  read_in_next_epoch = (
                      self._shard_sizes_per_gpu - self._counter_per_gpu
                  )
                  # get the maximum number of samples and round it up to full batch sizes
                  self._size = (
                      math.ceil(
                          max(read_in_next_epoch) / self.batch_size
                      )
                      * self.batch_size
                  )
                  # in case some epoch is skipped because we have read ahead in this epoch so
                  # much that in the next one we done already
                  if self._size == 0:
                      # it means that self._shard_sizes_per_gpu == self._counter_per_gpu,
                      # so we can jump to the next epoch and zero self._counter_per_gpu
                      self._counter_per_gpu = np.zeros(
                          self._shards_num, dtype=np.int64
                      )
                      # self._counter = min(self._counter_per_gpu), but just set 0
                      # to make it simpler
                      self._counter = 0
                      # roll once again
                      self._shard_sizes_per_gpu = np.roll(
                          self._shard_sizes_per_gpu, 1
                      )
                      # as self._counter_per_gpu is 0 we can just use
                      # read_in_next_epoch = self._shard_sizes_per_gpu
                      self._size = (
                          math.ceil(
                              max(self._shard_sizes_per_gpu)
                              / self.batch_size
                          )
                          * self.batch_size
                      )
  
          for p in self._pipes:
              p.reset()
              if p.empty():
                  with p._check_api_type_scope(
                      types.PipelineAPIType.ITERATOR
                  ):
                      p.schedule_run()
      else:
          logging.warning(
              "DALI iterator does not support resetting while epoch is not finished. \
                               Ignoring..."
          )
  
imports: from nvidia.dali import types
  import math
  import logging
  import numpy
  import warnings
  from enum import Enum, unique
  from collections.abc import Iterable
label_desc: Use image node_id 11 for calling the _DaliBaseIterator obj's underlying member method code with example usage: obj.reset() without return types