contiguous memory back to the input tensor.

Handles the tensors that should be compressed by PowerSGD
      compression: 3.1. For each tensor M, creates two low-rank tensors P and Q
 for decomposing M, such that M = PQ^T, where Q is initialized
 from a standard normal distribution and orthogonalized;

 3.2. Computes each P in Ps, which is equal to MQ;

 3.3. Allreduces Ps as a batch;

 3.4. Orthogonalizes each P in Ps;

 3.5. Computes each Q in Qs, which is approximately equal to
 M^TP;

 3.6. Allreduces Qs as a batch;

 3.7. Computes each M among all the compressed tensors, which
 is approximately equal to PQ^T.



Note that this communication hook enforces vanilla allreduce for
   the first "state.start_powerSGD_iter" iterations. This not only
   gives the user more control over the tradeoff between speedup and
   accuracy, but also helps abstract away some complexity of the