


with ddp.no_sync():
         >>>     for input in inputs:
         >>>         ddp(input).backward()  # no synchronization, accumulate grads
         >>> ddp(another_input).backward()  # synchronize grads



  Warning:

    The forward pass should be included inside the context
    manager, or else gradients will still be synchronized.

register_comm_hook(state, hook)
  Registers a communication hook which is an enhancement that
  provides a flexible hook to users where they can specify how DDP
  aggregates gradients across multiple workers.

  This hook would be very useful for researchers to try out new
  ideas. For example, this hook can be used to implement several
  algorithms like GossipGrad and gradient compression which
  involve different communication strategies for parameter syncs
  while running Distributed DataParallel training.

  Parameters:
     * **state** (*object*) --
