class_docstr: This subclass of `RNGStateTracker` defines the default policy of how RNG states
  should be shared and synchronized among all ranks to respect the semantics of DTensor
  random operators.
class_docstr_tok: ["This","subclass","of","`","RNGStateTracker","`","defines","the","default","policy","of","how","RNG","states","should","be","shared","and","synchronized","among","all","ranks","to","respect","the","semantics","of","DTensor","random","operators","."]
code: def __init__(self, device_type: str = "cuda"):
      super().__init__(device_type)
      # synchronize RNG state using rank 0's current one
      rng_state = self._device_handle.get_rng_state().to(device_type)
      dist.broadcast(rng_state, 0)
      self.rng_states["parallel-rng"] = rng_state.to("cpu")
  
imports: import contextlib
  import warnings
  from typing import Dict, List, Optional
  import torch
  import torch.distributed
  from torch import Tensor
  from torch.distributed._tensor.placement_types import DTensorSpec, Shard
  from torch.distributed.device_mesh import _get_device_handle, DeviceMesh
label_desc: Use image node_id 1 to create a new OffsetBasedRNGTracker object from inherited base classes: RNGStateTracker with example: obj = OffsetBasedRNGTracker(device_type)