modules' parameters if needed. Note that if "device_id" is
        specified but "module" is already on a different CUDA device,
        an error will be thrown. (Default: "None")
  * **sync_module_states** (*bool*) -- If "True", each
    individually wrapped FSDP unit will broadcast module
    parameters from rank 0 to ensure they are the same across all
    ranks after initialization. This helps ensure model parameters
    are the same across ranks before starting training, but adds
    communication overhead to "__init__", as at least one
    broadcast is triggered per individually wrapped FSDP unit.
    This can also help load checkpoints taken by "state_dict" and
    to be loaded by "load_state_dict" in a memory efficient way.
    See documentation for "FullStateDictConfig" for an example of
    this. (Default: "False")

  * **forward_prefetch** (*bool*) -- If "True", then FSDP
    *explicitly* prefetches the next upcoming all-gather while
