class_docstr: A gradient averager that implements PowerSGD compression: https://arxiv.org/abs/1905.13727
  For basic properties and guaranties of gradient averagers, please refer to the base class docstring.
  Put simply, this method approximates large gradient tensors (m,n) with a product of two
  smaller matrices (m,r) by (r,n), where r is a parameter chosen by the user (see averager_rank).
  
  As a result, PowerSGD only needs to aggregate O((m + n) * r) tensors instead of O(m * n).
  High r, e.g. sqrt(max(m, n)) typically reduce communication by 2-8x without affecting convergence.
  Low r, e.g. 1-8, further accelerate communication, but may converge worse depending on the task.
  
  To maintain convergence with low r, this averager uses the error feedback strategy. Put simply,
  if some part of the gradient is "lost in compression", it will be added to the next iteration.
  This has two implications: (a) it needs more RAM in order to store the "feedback buffers"
  and (b) if devices stay alive only for one step, training with small rank may converge slower.
  This is because error feedback takes multiple steps to kick in.
  
  Since not all gradients are matrices, PowerSGD views 3d+ tensors via tensor.flatten(1, -1).
  If a tensor has less than 2 dimensions or does not compress efficiently, it will be aggregated
  normally, i.e. without powerSGD. See min_compression_ratio for details.
  
  :note: due to the above rule, PowerSGD is *not* shape-invariant. For instance, a
   matrix of shape (256, 256) be compressed differently if you .reshape it to (32, 32, 32).
  
  :param parameters: pytorch parameters for which to aggregate gradients
  :param averager_rank: rank of compressed gradients
  :param dht: a DHT instance connected to the rest of the swarm. See hivemind.DHT docs
  :param prefix: a unique DHT key used for matchmaking. E.g. this can be your experiment name with optional suffixes
  :param reuse_grad_buffers: if True, use model's .grad buffers for accumulating gradients over multiple steps.
    This is more memory efficient, but it requires that the user does *not* call zero_grad or clip_by_whatever at all
  :param accumulate_grads_on: if specified, accumulate gradients on this device. By default, this will use the same
    device as model parameters. One can specify a different device (e.g. 'cpu' vs 'cuda') to save device memory at
    the cost of extra time per step. If reuse_grad_buffers is True, this parameter has no effect.
  :param client_mode: if False, this averager will accept incoming requests from other peers.
    if True, the averager will only join existing groups where at least one peer has client_mode=False.
    By default, this flag is copied from DHTNode inside the ``dht`` instance.
  :param warn: if True, warn when the averager did not reset accumulators after use or did not use averaging results
  :param min_compression_ratio: apply PowerSGD to a tensor only if it reduces communication by at least this factor,
    otherwise aggregate tensors as is
  :param averaged_grads: if provided, it will be used as a set of averagable gradients
docstr: Register a given group for one or more all-reduce rounds
docstr_tok: ["Register","a","given","group","for","one","or","more","all-reduce","rounds"]
code: def _register_allreduce_group(self, group_info: GroupInfo):
      """Register a given group for one or more all-reduce rounds"""
      try:
          for phase in list(AllReducePhases):
              self._running_groups[
                  group_info.group_id + phase.name.encode()
              ] = asyncio.Future()
          self._pending_groups_registered.set()
          yield
      finally:
          for phase in list(AllReducePhases):
              maybe_future = self._running_groups.pop(
                  group_info.group_id + phase.name.encode(), None
              )
              if maybe_future and not maybe_future.done():
                  logger.warning(
                      f"All-reduce group {group_info.group_id + phase.name.encode()} did not finish."
                  )
          self._pending_groups_registered.set()
  
imports: import asyncio
  import contextlib
  from enum import Enum
  from typing import Any, Iterable, Optional, Sequence
  import torch
  from hivemind.averaging.allreduce import AveragingMode
  from hivemind.averaging.group_info import GroupInfo
  from hivemind.averaging.load_balancing import load_balance_peers
  from hivemind.averaging.matchmaking import MatchmakingException
  from hivemind.compression import CompressionInfo, TensorRole
  from hivemind.dht import DHT
  from hivemind.optim.grad_averager import GradientAverager
  from hivemind.utils import get_logger
  from hivemind.utils.asyncio import enter_asynchronously
  from hivemind.utils.math import get_flatten_greedy_dims, orthogonalize_
label_desc: Use image node_id 2 for calling the PowerSGDGradientAverager obj's underlying member method code with example usage: obj._register_allreduce_group(group_info) without return types