decomposed into a list of per-parameter tensors within this
      bucket.
torch.distributed.GradBucket.gradients(self: torch._C._distributed_c10d.GradBucket) -> List[torch.Tensor]
Returns:
      A list of "torch.Tensor". Each tensor in the list corresponds to
      a gradient.
torch.distributed.GradBucket.is_last(self: torch._C._distributed_c10d.GradBucket) -> bool
Returns:
      Whether this bucket is the last bucket to allreduce in an
      iteration. This also means that this bucket corresponds to the
      first few layers in the forward pass.
torch.distributed.GradBucket.set_buffer(self: torch._C._distributed_c10d.GradBucket, buffer: torch.Tensor) -> None
Replaces the tensor in the bucket with the input tensor buffer.
torch.distributed.GradBucket.parameters(self: torch._C._distributed_c10d.GradBucket) -> List[torch.Tensor]
Returns:
      A list of "torch.Tensor". Each tensor in the list corresponds to
      a model parameter.
Default Communication Hooks