however this may reduce the overlap between computation and
      communication, and increase the memory footprint due to stacking
      the tensors of the same shape. Set to "True" if the compression
      / decompression computation is a bottleneck.
Warning:
 If error feedback or warm-up is enabled, the minimum value of
 "start_powerSGD_iter" allowed in DDP is 2. This is because there
 is another internal optimization that rebuilds buckets at
 iteration 1 in DDP, and this can conflict with any tensor
 memorized before the rebuild process.

PowerSGD Hooks
Warning:
PowerSGD typically requires extra memory of the same size as the
  model's gradients to enable error feedback, which can compensate for
  biased compressed communication and improve accuracy.
Warning:
PowerSGD hooks may conflict with Apex automatic mixed precision
  package. Please use PyTorch native automatic mixed precision package
  instead.