the current set of parameter's gradient computation. This
     improves backward pass throughput by overlapping communication
     (next all-gather) and computation (current gradient computation).

"BACKWARD_POST": This prefetches the next set of parameters after
     the current set of parameter's gradient computation. This may
     improve backward pass throughput by overlapping communication
     (current reduce-scatter) and computation (next gradient
     computation). Specifically, the next all-gather is reordered to
     be before the current reduce-scatter.

Note:
 If the increase in peak memory usage from prefetching is an
 issue, you may consider passing "limit_all_gathers=True" to the
 FSDP constructor, which may help reduce peak memory usage in some
 cases.

class torch.distributed.fsdp.ShardingStrategy(value)
This specifies the sharding strategy to be used for distributed
   training by "FullyShardedDataParallel".