class_docstr: A `DdpAccelerator` is a generalization of PyTorch's `DistributedDataParallel` class.
  
  This is primarly used within the :class:`allennlp.training.trainer.GradientDescentTrainer` to allow
  for different DDP implementations, such as FairScale's [`FullyShardedDataParallel`]
  (https://fairscale.readthedocs.io/en/latest/api/nn/fsdp.html).
  
  In a typical AllenNLP configuration file, `local_rank`, `world_size`, and `cuda_device`
  should not be specified.
  
  !!! Warning
      This API is experimental and may change in the future.
code: def __init__(
      self,
      local_rank: Optional[int] = None,
      world_size: Optional[int] = None,
      cuda_device: Union[torch.device, int] = -1,
  ) -> None:
      self.local_rank: int = (
          local_rank if local_rank is not None else dist.get_rank()
      )
      self.world_size: int = (
          world_size
          if world_size is not None
          else dist.get_world_size()
      )
      self.is_primary: bool = local_rank == 0
      self.cuda_device = int_to_device(cuda_device)
  
imports: import os
  from typing import Union, Tuple, OrderedDict, Dict, NamedTuple, List, Optional, Any, Sequence, TYPE_CHECKING
  import torch
  import torch.distributed
  from torch.cuda import amp
  from torch.nn.utils import clip_grad_norm_
  from allennlp.common import Registrable
  from allennlp.common.util import int_to_device
  from allennlp.nn.parallel.sharded_module_mixin import ShardedModuleMixin
label_desc: Use image node_id 1 to create a new DdpAccelerator object from inherited base classes: Registrable with example: obj = DdpAccelerator(local_rank, world_size, cuda_device)