

cpu_offload (Optional[CPUOffload]) -- This
        configures CPU offloading. If this is set to "None", then no
        CPU offloading happens. See "CPUOffload" for details.
        (Default: "None")


auto_wrap_policy
    (Optional[Union[Callable[[nn.Module,
    bool, int], bool], _FSDPPolicy]]) --
This is either "None", an "_FSDPPolicy", or a callable of a
fixed signature. If it is "None", then "module" is wrapped
with only a top-level FSDP instance without any nested
wrapping. If it is an "_FSDPPolicy", then the wrapping follows
the given policy. "ModuleWrapPolicy" in
"torch.distributed.fsdp.wrap.py" is an example. If it is a
callable, then it should take in three arguments "module:
nn.Module", "recurse: bool", and "nonwrapped_numel: int" and
should return a "bool" specifying whether the passed-in



