class_docstr: Wraps a PyTorch `IterableDataset` to generate samples for one of the processes only. Instances of this class will
  always yield a number of samples that is a round multiple of the actual batch size (depending of the value of
  `split_batches`, this is either `batch_size` or `batch_size x num_processes`). Depending on the value of the
  `drop_last` attribute of the batch sampler passed, it will either stop the iteration at the first batch that would
  be too small or loop with indices from the beginning.
  
  Args:
      dataset (`torch.utils.data.dataset.IterableDataset`):
          The batch sampler to split in several shards.
      batch_size (`int`, *optional*, defaults to 1):
          The size of the batches per shard (if `split_batches=False`) or the size of the batches (if
          `split_batches=True`).
      drop_last (`bool`, *optional*, defaults to `False`):
          Whether or not to drop the last incomplete batch or complete the last batches by using the samples from the
          beginning.
      num_processes (`int`, *optional*, defaults to 1):
          The number of processes running concurrently.
      process_index (`int`, *optional*, defaults to 0):
          The index of the current process.
      split_batches (`bool`, *optional*, defaults to `False`):
          Whether the shards should be created by splitting a batch to give a piece of it on each process, or by
          yielding different full batches on each process.
  
          On two processes with an iterable dataset yielding of `[0, 1, 2, 3, 4, 5, 6, 7]`, this will result in:
  
          - the shard on process 0 to yield `[0, 1, 2, 3]` and the shard on process 1 to yield `[4, 5, 6, 7]` if this
            argument is set to `False`.
          - the shard on process 0 to yield `[0, 1, 4, 5]` and the sampler on process 1 to yield `[2, 3, 6, 7]` if
            this argument is set to `True`.
class_docstr_tok: ["Wraps","a","PyTorch","`","IterableDataset","`","to","generate","samples","for","one","of","the","processes","only",".","Instances","of","this","class","will","always","yield","a","number","of","samples","that","is","a","round","multiple","of","the","actual","batch","size","(","depending","of","the","value","of","`","split_batches","`",",","this","is","either","`","batch_size","`","or","`","batch_size","x","num_processes","`",")",".","Depending","on","the","value","of","the","`","drop_last","`","attribute","of","the","batch","sampler","passed",",","it","will","either","stop","the","iteration","at","the","first","batch","that","would","be","too","small","or","loop","with","indices","from","the","beginning",".","Args",":","dataset","(","`","torch.utils.data.dataset.IterableDataset","`",")",":","The","batch","sampler","to","split","in","several","shards",".","batch_size","(","`","int","`",",","*","optional","*",",","defaults","to","1",")",":","The","size","of","the","batches","per","shard","(","if","`","split_batches=False","`",")","or","the","size","of","the","batches","(","if","`","split_batches=True","`",")",".","drop_last","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","or","not","to","drop","the","last","incomplete","batch","or","complete","the","last","batches","by","using","the","samples","from","the","beginning",".","num_processes","(","`","int","`",",","*","optional","*",",","defaults","to","1",")",":","The","number","of","processes","running","concurrently",".","process_index","(","`","int","`",",","*","optional","*",",","defaults","to","0",")",":","The","index","of","the","current","process",".","split_batches","(","`","bool","`",",","*","optional","*",",","defaults","to","`","False","`",")",":","Whether","the","shards","should","be","created","by","splitting","a","batch","to","give","a","piece","of","it","on","each","process",",","or","by","yielding","different","full","batches","on","each","process",".","On","two","processes","with","an","iterable","dataset","yielding","of","`","[","0",",","1",",","2",",","3",",","4",",","5",",","6",",","7","]","`",",","this","will","result","in",":","-","the","shard","on","process","0","to","yield","`","[","0",",","1",",","2",",","3","]","`","and","the","shard","on","process","1","to","yield","`","[","4",",","5",",","6",",","7","]","`","if","this","argument","is","set","to","`","False","`",".","-","the","shard","on","process","0","to","yield","`","[","0",",","1",",","4",",","5","]","`","and","the","sampler","on","process","1","to","yield","`","[","2",",","3",",","6",",","7","]","`","if","this","argument","is","set","to","`","True","`","."]
code: def __len__(self):
      # We will just raise the downstream error if the underlying dataset is not sized
      if self.drop_last:
          return (
              len(self.dataset)
              // (self.batch_size * self.num_processes)
          ) * self.batch_size
      else:
          return (
              math.ceil(
                  len(self.dataset)
                  / (self.batch_size * self.num_processes)
              )
              * self.batch_size
          )
  
imports: import math
  from contextlib import suppress
  from typing import Callable, List, Optional, Union
  import torch
  from torch.utils.data import BatchSampler, DataLoader, IterableDataset, RandomSampler
  from .logging import get_logger
  from .state import AcceleratorState, DistributedType, GradientState, is_tpu_available
  from .utils import RNGType, broadcast, broadcast_object_list, concatenate, find_batch_size, get_data_structure, initialize_tensors, is_torch_version, send_to_device, slice_tensors, synchronize_rng_states
label_desc: Use image node_id 3 for calling the IterableDatasetShard obj's underlying member method code with example usage: obj.__len__() and returns: unknown, unknown