class_docstr: An object that slices tensors into rows or columns as described in the Megatron LM paper
  
  Args:
      mp_group (torch.distributed.ProcessGroupNCCL): Distributed group for model parallelism
code: def __init__(self, mp_group) -> None:
      if dist.is_initialized() and mp_group is not None:
          self.gpu_index = dist.get_rank(group=mp_group)
          self.world_size = int(os.getenv("WORLD_SIZE"))
      else:
          self.gpu_index = 0
          self.world_size = 1
  
file_path: parallelformers/parallelformers/parallel/slicing.py
imports: import os
  from typing import Dict, Tuple
  import torch
  import torch.distributed
label_desc: Use image node_id 1 to create a new TensorSlicer object from inherited base classes: object with example: obj = TensorSlicer(mp_group)