relatively small "matrix_approximation_rank" is used. This is
      because that, the beginning of training phase is usually very
      sensitive to inaccurate gradients, and compressing gradients too
      early may make the training quickly take a suboptimal
      trajectory, which can result in an irrecoverable impact on the
      accuracy.
To tune "start_powerSGD_iter", we suggest to start with 10% of
   total training steps, and increase it until a satisfactory accuracy
   is reached. If there is a warm-up stage in the training,
   "start_powerSGD_iter" typically should be no less than the number
   of warm-up steps.

"min_compression_rate" is the minimum compression rate required
      when a layer is compressed. Due to the computation overheads
      incurred by the compression, a tensor is worth compressing only
      if there can be sufficient saving in bandwidth, where "(num_rows
num_cols) * matrix_approximation_rank * min_compression_rate <


