class_docstr: A wrapper for the All-Gather function to support auto-differentiation.
class_docstr_tok: ["A","wrapper","for","the","All-Gather","function","to","support","auto-differentiation","."]
code: def backward(ctx, grad_out):
      rank, dim0 = ctx.args
      return grad_out[rank * dim0 : (rank + 1) * dim0], None, None, None
  
imports: import torch
  from torch.autograd import Function
  import fmoe_cuda
  from .utils import get_torch_default_comm
label_desc: Use image node_id 2 for calling the AllGather obj's underlying member method code with example usage: obj.backward(ctx, grad_out) and returns: grad_out,  None,  None,  None