PowerSGD State
class torch.distributed.algorithms.ddp_comm_hooks.powerSGD_hook.PowerSGDState(process_group, matrix_approximation_rank=1, start_powerSGD_iter=1000, min_compression_rate=2, use_error_feedback=True, warm_start=True, orthogonalization_epsilon=0, random_seed=0, compression_stats_logging_frequency=10000, batch_tensors_with_same_shape=False)
Stores both the algorithm's hyperparameters and the internal state
   for all the gradients during the training. Particularly,
   "matrix_approximation_rank" and "start_powerSGD_iter" are the main
   hyperparameters that should be tuned by the user. For performance,
   we suggest to keep binary hyperparameters "use_error_feedback" and
   "warm_start" on.

"matrix_approximation_rank" controls the size of compressed low-
      rank tensors, which determines the compression rate. The lower
      the rank, the stronger the compression. 1.1. If "matrix_approximation_rank" is too low, the full


