the full dict in CPU memory, where rank 0 moves each shard to
        GPU memory (for NCCL) and communicates it to ranks
        appropriately. Hence, the former has higher aggregate CPU
        memory cost, while the latter has higher communication cost.
  Parameters:
     * **full_optim_state_dict** (*Optional**[**Dict**[**str**,
       **Any**]**]*) -- Optimizer state dict corresponding to the
       unflattened parameters and holding the full non-sharded
       optimizer state if on rank 0; the argument is ignored on
       nonzero ranks.

     * **model** (*torch.nn.Module*) -- Root module (which may or
       may not be a "FullyShardedDataParallel" instance) whose
       parameters correspond to the optimizer state in
       "full_optim_state_dict".

     * **optim_input**
       (*Optional**[**Union**[**List**[**Dict**[**str**,
       **Any**]**]**, **Iterable**[**torch.nn.Parameter**]**]**]*)
