by FSDP, their gradients will not be synchronized as well.
        With this newly added argument, "ignored_modules" could be
        deprecated soon. For backward compatibility, both
        "ignored_parameters" and "ignored_modules" are kept for now,
        but FSDP only allows one of them to be specified as not
        "None".
apply(fn)
  Applies "fn" recursively to every submodule (as returned by
  ".children()") as well as self. Typical use includes
  initializing the parameters of a model (see also torch.nn.init).

  Compared to "torch.nn.Module.apply", this version additionally
  gathers the full parameters before applying "fn". It should not
  be called from within another "summon_full_params" context.

  Parameters:
     **fn** ("Module" -> None) -- function to be applied to each
     submodule

  Returns:
     self

  Return type:
     Module

clip_grad_norm_(max_norm, norm_type=2.0)