communication and computation. Particularly,
"torch.distributed.GradBucket" represents a bucket of gradient tensors
to be allreduced.
class torch.distributed.GradBucket
This class mainly passes a flattened gradient tensor (returned by
   "buffer()") to DDP communication hook. This tensor can be further
   decomposed into a list of per-parameter tensors within this bucket
   (returned by "get_per_parameter_tensors()") to apply layer-wise
   operations.
torch.distributed.GradBucket.index(self: torch._C._distributed_c10d.GradBucket) -> int
Warning:
 Since the buckets are rebuilt after the first iteration, should
 not rely on the indices at the beginning of training.

Returns:
      The index of a bucket that stores gradients of a few contiguous
      layers. All the gradients are bucketized.
torch.distributed.GradBucket.buffer(self: torch._C._distributed_c10d.GradBucket) -> torch.Tensor
Returns:
      A flattened 1D "torch.Tensor" buffer, which can be further