class_docstr: A light-weight wrapper class for torch.distributed API.
  Only a subset of functions are wrapped. Once the init_process_group
  is initialized, standard torch.distributed.* can be used directly
  so no need to wrap all the functions. We can keep adding wrappers as
  needed.
class_docstr_tok: ["A","light-weight","wrapper","class","for","torch.distributed","API",".","Only","a","subset","of","functions","are","wrapped",".","Once","the","init_process_group","is","initialized",",","standard","torch.distributed",".","*","can","be","used","directly","so","no","need","to","wrap","all","the","functions",".","We","can","keep","adding","wrappers","as","needed","."]
docstr: proxy func to torch.distributed.all_reduce_coalesced,
  which is included in PyTorch 1.13 and above
docstr_tok: ["proxy","func","to","torch.distributed.all_reduce_coalesced",",","which","is","included","in","PyTorch","1.13","and","above"]
code: def all_reduce_coalesced(
      self,
      tensors,
      op=torch.distributed.ReduceOp.SUM,
      group=None,
      async_op=False,
  ):
      """proxy func to torch.distributed.all_reduce_coalesced,
      which is included in PyTorch 1.13 and above
      """
      if not self.has_all_reduce_coalesced:
          raise RuntimeError(
              f"Current torch version does not have all_reduce_coalesced "
              f"api (torch.__version__: {torch.__version__})"
          )
      op = self._reduce_op(op)
      return torch.distributed.all_reduce_coalesced(
          tensors=tensors, op=op, group=group, async_op=async_op
      )
  
imports: from deepspeed import utils
  from .utils import *
  from .backend import *
  from .comm import *
  import os
label_desc: Use image node_id 9 for calling the TorchBackend obj's underlying member method code with example usage: obj.all_reduce_coalesced(tensors, op, group, async_op) and returns: torch