num_rows * num_cols". If the specified compression rate
      threshold cannot be satisfied, the tensor will be directly
      allreduced without compression.
Compression statistics are logged every
   "compression_stats_logging_frequency" iterations once PowerSGD
   compression starts.


"orthogonalization_epsilon" can be a very small value (e.g.,
      1e-8) added to every normalized matrix column in
      orthogonalization step, to prevent div-by-zero error if any
      column has all 0s. If this can already be prevented (e.g., by
      batch normalization), an epsilon of 0 is recommended for
      accuracy.


"batch_tensors_with_same_shape" controls whether to compress and
      decompress tensors with same shape in a batched operation to
      achieve higher parallelism. Note that you should also increase
      the bucket size (i.e., "bucket_cap_mb" arg in DDP constructor)
      to make more same-shaped tensors appear in the same bucket,

