class_docstr: DistributedBatchSampler will iterate through an underlying batch sampler and return batches
  which belong to this shard.
  
  DistributedBatchSampler is different from the PyTorch built-in
  torch.utils.data.distributed.DistributedSampler, because that
  DistributedSampler expects to bbe called before the BatchSampler, and
  additionally the DistributedSampler is meant to be a stand-alone sampler.
  
  DistributedBatchSampler has the potential gotcha that when wrapping a
  non-repeating BatchSampler, if the length of the BatchSampler is not
  divisible by the number of replicas the length of the resulting
  DistributedBatchSampler will differ based on the rank. In that case, the
  divergent paths of multiple workers could cause problems during training.
  PyTorchTrial always uses RepeatBatchSampler during training, PyTorchTrial
  does not require that the workers stay in step during validation, so this
  potential gotcha is not a problem in Determined.
class_docstr_tok: ["DistributedBatchSampler","will","iterate","through","an","underlying","batch","sampler","and","return","batches","which","belong","to","this","shard",".","DistributedBatchSampler","is","different","from","the","PyTorch","built-in","torch.utils.data.distributed.DistributedSampler",",","because","that","DistributedSampler","expects","to","bbe","called","before","the","BatchSampler",",","and","additionally","the","DistributedSampler","is","meant","to","be","a","stand-alone","sampler",".","DistributedBatchSampler","has","the","potential","gotcha","that","when","wrapping","a","non-repeating","BatchSampler",",","if","the","length","of","the","BatchSampler","is","not","divisible","by","the","number","of","replicas","the","length","of","the","resulting","DistributedBatchSampler","will","differ","based","on","the","rank",".","In","that","case",",","the","divergent","paths","of","multiple","workers","could","cause","problems","during","training",".","PyTorchTrial","always","uses","RepeatBatchSampler","during","training",",","PyTorchTrial","does","not","require","that","the","workers","stay","in","step","during","validation",",","so","this","potential","gotcha","is","not","a","problem","in","Determined","."]
code: def __iter__(self) -> Iterator:
      if self.num_workers == 1:
          yield from self.batch_sampler
      else:
          for i, batch in enumerate(self.batch_sampler):
              if i % self.num_workers == self.rank:
                  yield batch
  
imports: from typing import Iterator
  import numpy
  import torch
label_desc: Use image node_id 3 for calling the DistributedBatchSampler obj's underlying member method code with example usage: obj.__iter__() without return types