frozen layers can be made trainable and added to the "Optimizer"
      as training progresses.
  Parameters:
     **param_group** (*dict*) -- specifies the parameters to be
     optimized and group-specific optimization options.

  Warning:

    This method handles updating the shards on all partitions but
    needs to be called on all ranks. Calling this on a subset of
    the ranks will cause the training to hang because
    communication primitives are called depending on the managed
    parameters and expect all the ranks to participate on the same
    set of parameters.

consolidate_state_dict(to=0)
  Consolidate a list of "state_dict" s (one per rank) on the
  target rank.

  Parameters:
     **to** (*int*) -- the rank that receives the optimizer states
     (default: 0).

  Raises:
     **RuntimeError** -- if "overlap_with_ddp=True" and this
     method is     called before this "ZeroRedundancyOptimizer"
