docstr: ShardedOptimizer collects all tensors and local shard tensors of
  ShardedTensor, then use these tensors as ``params`` for optimizers
  
  Args:
      named_params (Dict[str, Union[Tensor, ShardedTensor]]) : a Dict
          of parameters, where key is the parameter key, value is either
          Tensor or ShardedTensor parameter.
      optimizer_class (torch.optim.Optimizer): the Optimizer to use
          locally, i.e. torch.optim.SGD, torch.optim.Adagrad, etc.
      *optimizer_args: the arguments to initialize the optimizer.
      **optimizer_kwargs: the key-word arguments to initialize the optimizer.
code: def __init__(
      self,
      named_params: Mapping[str, Union[Tensor, ShardedTensor]],
      optimizer_class,
      *optimizer_args,
      **optimizer_kwargs,
  ):
      """
      ShardedOptimizer collects all tensors and local shard tensors of
      ShardedTensor, then use these tensors as ``params`` for optimizers
  
      Args:
          named_params (Dict[str, Union[Tensor, ShardedTensor]]) : a Dict
              of parameters, where key is the parameter key, value is either
              Tensor or ShardedTensor parameter.
          optimizer_class (torch.optim.Optimizer): the Optimizer to use
              locally, i.e. torch.optim.SGD, torch.optim.Adagrad, etc.
          *optimizer_args: the arguments to initialize the optimizer.
          **optimizer_kwargs: the key-word arguments to initialize the optimizer.
  
      """
      tensors: List[Tensor] = []
      for value in named_params.values():
          if isinstance(value, ShardedTensor):
              for local_shard in value.local_shards():
                  tensors.append(local_shard.tensor)
          else:
              tensors.append(value)
  
      self.named_params = named_params
      self._optim = optimizer_class(
          tensors, *optimizer_args, **optimizer_kwargs
      )
      self.param_groups = self._optim.param_groups
      self.state = self._optim.state
  
file_path: pytorch/torch/distributed/_shard/sharded_optim/api.py
imports: from typing import List, Union, Mapping, Dict, Any
  import torch.optim
  from torch import Tensor
  from torch.distributed._shard.sharded_tensor import ShardedTensor
label_desc: Use image node_id 1 to create a new ShardedOptimizer object from inherited base classes: optim with example: obj = ShardedOptimizer(named_params, optimizer_class)