class_docstr: DistributedBatchSampler will iterate through an underlying batch sampler and return batches
  which belong to this shard.
  
  DistributedBatchSampler is different from the PyTorch built-in
  torch.utils.data.distributed.DistributedSampler, because that
  DistributedSampler expects to bbe called before the BatchSampler, and
  additionally the DistributedSampler is meant to be a stand-alone sampler.
  
  DistributedBatchSampler has the potential gotcha that when wrapping a
  non-repeating BatchSampler, if the length of the BatchSampler is not
  divisible by the number of replicas the length of the resulting
  DistributedBatchSampler will differ based on the rank. In that case, the
  divergent paths of multiple workers could cause problems during training.
  PyTorchTrial always uses RepeatBatchSampler during training, PyTorchTrial
  does not require that the workers stay in step during validation, so this
  potential gotcha is not a problem in Determined.
class_docstr_tok: ["DistributedBatchSampler","will","iterate","through","an","underlying","batch","sampler","and","return","batches","which","belong","to","this","shard",".","DistributedBatchSampler","is","different","from","the","PyTorch","built-in","torch.utils.data.distributed.DistributedSampler",",","because","that","DistributedSampler","expects","to","bbe","called","before","the","BatchSampler",",","and","additionally","the","DistributedSampler","is","meant","to","be","a","stand-alone","sampler",".","DistributedBatchSampler","has","the","potential","gotcha","that","when","wrapping","a","non-repeating","BatchSampler",",","if","the","length","of","the","BatchSampler","is","not","divisible","by","the","number","of","replicas","the","length","of","the","resulting","DistributedBatchSampler","will","differ","based","on","the","rank",".","In","that","case",",","the","divergent","paths","of","multiple","workers","could","cause","problems","during","training",".","PyTorchTrial","always","uses","RepeatBatchSampler","during","training",",","PyTorchTrial","does","not","require","that","the","workers","stay","in","step","during","validation",",","so","this","potential","gotcha","is","not","a","problem","in","Determined","."]
code: def __init__(
      self,
      batch_sampler: torch.utils.data.BatchSampler,
      num_workers: int,
      rank: int,
  ) -> None:
      if rank < 0:
          raise ValueError("rank must be non-negative.")
      if num_workers < 1:
          raise ValueError("num_workers must be greater than zero.")
      if rank >= num_workers:
          raise ValueError("rank must be less than num_workers.")
  
      self.batch_sampler = batch_sampler
      self.num_workers = num_workers
      self.rank = rank
  
imports: from typing import Iterator
  import numpy
  import torch
label_desc: Use image node_id 1 to create a new DistributedBatchSampler object from inherited base classes: torch.utils.data with example: obj = DistributedBatchSampler(batch_sampler, num_workers, rank)