docstr: Wraps an arbitrary :class:`ClassyOptimizer <classy_vision.optim.ClassyOptimizer>`
  optimizer and shards its state as described by ZeRO_.
  ::
      opt = OSS(params, optim=torch.optim.Adam, lr=0.01)
  
  .. _ZeRO: https://arxiv.org/abs/1910.02054
  
  This instance holds all of the parameters for the model (in the .param_groups attribute)
  but relies on a wrapped optimizer, which only process an original shard of the parameters.
  Every step all the parameters are synced across the replicas. The Fairscale library is used
  https://github.com/facebookresearch/fairscale
code: def __init__(self, base_optimizer: ClassyOptimizer):
      """Wraps an arbitrary :class:`ClassyOptimizer <classy_vision.optim.ClassyOptimizer>`
      optimizer and shards its state as described by ZeRO_.
      ::
          opt = OSS(params, optim=torch.optim.Adam, lr=0.01)
  
      .. _ZeRO: https://arxiv.org/abs/1910.02054
  
      This instance holds all of the parameters for the model (in the .param_groups attribute)
      but relies on a wrapped optimizer, which only process an original shard of the parameters.
      Every step all the parameters are synced across the replicas. The Fairscale library is used
      https://github.com/facebookresearch/fairscale
      """
  
      assert (
          fairscale_available
      ), "The Fairscale library needs to be installed to use this optimizer."
  
      super().__init__()
      self.base_optimizer = base_optimizer
  
file_path: ClassyVision/classy_vision/optim/zero.py
imports: import logging
  from typing import Any
  import torch.distributed
  from classy_vision.generic.distributed_util import get_primary_rank
  from classy_vision.optim import build_optimizer, ClassyOptimizer, register_optimizer
label_desc: Use image node_id 1 to create a new ZeRO object from inherited base classes: ClassyOptimizer with example: obj = ZeRO(base_optimizer)