Performance efficiency is accomplished by overlapping all_gather communication calls for upcoming FSDP units with computation of the current FSDP unit. Once the current FSDP unit has been processed, the non-locally owned parameters are dropped, freeing memory for the upcoming FSDP units.  This process achieves training efficiency by the overlap of computation and communication, while also reducing the peak memory needed by each GPU.