code: def new_process_group(ranks, backend):
      """
      This function creates process groups.
  
      In addition to simply creating the process groups, it initializes NCCL
      for hybrid IB/Socket network like in the following diagram:
  
                              ____________
        [GPU Node 0]---TCP---|            |---TCP---[GPU Node 2]
           |                 |            |            |
           |                 |            |            |
          IB                 | IP Network |           IB
           |                 |            |            |
           |                 |            |            |
        [GPU Node 1]---TCP---|____________|---TCP---[GPU Node 3]
  
  
      If an environment variable NUM_GPUS_PER_IB_BLOCK is defined it looks up the ranks
      and determines whether the list of ranks belong to the same computational block where
      GPUs nodes are interconnected via IB type of connection or not.
      If all ranks are in the same block, the process group will use NCCL_NET=IB for
      communication, otherwise it will use NCCL_NET=Socket.
  
      If NCCL_NET=Socket is ever to be used, the user must set NCCL_SOCKET_IFNAME.
      Additionally, it is recommended to set NCCL_SOCKET_NTHREADS and
      NCCL_NSOCKS_PERTHREAD before running the job.
      See: https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/env.html
      for more info
  
      The core assumption for this functionality is that the ranks are evenly divided
      into IB blocks and all these IB blocks are of the same size.
      """
      if backend is None:
          backend = "nccl"
  
      compute_block_size = os.getenv("NUM_GPUS_PER_IB_BLOCK")
      if backend == "nccl" and compute_block_size is not None:
          compute_block_size = int(compute_block_size)
          blocks = [rank // compute_block_size for rank in ranks]
          use_ib = all(block == blocks[0] for block in blocks)
          if use_ib:
              return new_nccl_ib_group(ranks)
          else:
              return new_nccl_socket_group(ranks)
      else:
          return torch.distributed.new_group(ranks, backend=backend)
  
imports: from typing import Tuple, Optional
  import warnings
  import os
  import torch
  from apex.transformer.log_util import get_transformer_logger
  from apex.transformer._ucc_util import HAS_UCC
label_desc: Use image node_id 7 for calling a global function with example usage: new_process_group(ranks, backend) and returns: torch, new_nccl_ib_group, new_nccl_socket_group