class_docstr: A :class:`allennlp.nn.parallel.ddp_accelerator.DdpAccelerator` for FairScale's [`FullyShardedDataParallel`]
  (https://fairscale.readthedocs.io/en/latest/api/nn/fsdp.html).
  
  To save memory while initializing a model, you should call [`.wrap_module()`](#wrap_module) on submodules
  as they're created.
  
  See the :class:`allennlp.modules.transformer.t5.T5` class for an example of how to use this.
class_docstr_tok: ["A",":","class",":","`","allennlp.nn.parallel.ddp_accelerator.DdpAccelerator","`","for","FairScale","'s","[","`","FullyShardedDataParallel","`","]","(","https",":","\/\/fairscale.readthedocs.io\/en\/latest\/api\/nn\/fsdp.html",")",".","To","save","memory","while","initializing","a","model",",","you","should","call","[","`",".wrap_module","(",")","`","]","(","#","wrap_module",")","on","submodules","as","they","'re","created",".","See","the",":","class",":","`","allennlp.modules.transformer.t5.T5","`","class","for","an","example","of","how","to","use","this","."]
code: def __init__(
      self,
      *,
      mixed_precision: bool = False,
      reshard_after_forward: bool = True,
      flatten_parameters: bool = True,
      local_rank: Optional[int] = None,
      world_size: Optional[int] = None,
      cuda_device: Union[torch.device, int] = -1,
  ) -> None:
      super().__init__(
          local_rank=local_rank,
          world_size=world_size,
          cuda_device=cuda_device,
      )
      self._fsdp_kwargs = {
          "compute_device": self.cuda_device,
          "mixed_precision": mixed_precision,
          "reshard_after_forward": reshard_after_forward,
          "flatten_parameters": flatten_parameters,
      }
      if mixed_precision:
          self._fsdp_kwargs["move_params_to_cpu"] = True
          self._fsdp_kwargs["clear_autocast_cache"] = True
  
imports: import os
  from typing import Tuple, Union, Optional, TYPE_CHECKING, List, Any, Dict, Sequence
  from fairscale.nn import FullyShardedDataParallel
  from fairscale.nn.wrap import enable_wrap, wrap
  from fairscale.nn.misc import FlattenParamsWrapper
  from fairscale.optim.grad_scaler import GradScaler
  import torch
  from torch.cuda import amp
  from allennlp.nn.parallel.sharded_module_mixin import ShardedModuleMixin
  from allennlp.nn.parallel.ddp_accelerator import DdpAccelerator, DdpWrappedModel, StateDictType, LoadStateDictReturnType
label_desc: Use image node_id 1 to create a new FairScaleFsdpAccelerator object from inherited base classes: DdpAccelerator with example: obj = FairScaleFsdpAccelerator()