the full dict in CPU memory, where rank 0 moves each shard to
        GPU memory (for NCCL) and communicates it to ranks
        appropriately. Hence, the former has higher aggregate CPU
        memory cost, while the latter has higher communication cost.
  Parameters:
     * **full_optim_state_dict** (*Dict**[**str**, **Any**]*) --
       Optimizer state dict corresponding to the unflattened
       parameters and holding the full non-sharded optimizer
       state.

     * **model** (*torch.nn.Module*) -- Root module (which may or
       may not be a "FullyShardedDataParallel" instance) whose
       parameters correspond to the optimizer state in
       "full_optim_state_dict".

     * **optim_input**
       (*Optional**[**Union**[**List**[**Dict**[**str**,
       **Any**]**]**, **Iterable**[**torch.nn.Parameter**]**]**]*)
       -- Input passed into the optimizer representing either a
