keep "1 / world_size" optimizer states. After parameters are
   updated locally, each rank will broadcast its parameters to all
   other peers to keep all model replicas in the same state.
   "ZeroRedundancyOptimizer" can be used in conjunction with
   "torch.nn.parallel.DistributedDataParallel" to reduce per-rank peak
   memory consumption.
"ZeroRedundancyOptimizer" uses a sorted-greedy algorithm to pack a
   number of parameters at each rank. Each parameter belongs to a
   single rank and is not divided among ranks. The partition is
   arbitrary and might not match the the parameter registration or
   usage order.
Parameters:
      params ("Iterable") -- an "Iterable" of "torch.Tensor" s or
      "dict" s giving all parameters, which will be sharded across
      ranks.
Keyword Arguments:
      * optimizer_class ("torch.nn.Optimizer") -- the class of the
        local optimizer.
  * **process_group** ("ProcessGroup", optional) --
