class_docstr: Implementation that extends @pytest.fixture to allow for distributed execution.
  This is primarily meant to be used when a test requires executing two pieces of
  code with different world sizes.
  
  There are 2 parameters that can be modified:
      - world_size: int = 2 -- the number of processes to launch
      - backend: Literal['nccl','mpi','gloo'] = 'nccl' -- which backend to use
  
  Features:
      - able to call pytest.skip() inside fixture
      - can be reused by multiple tests
      - can accept other fixtures as input
  
  Limitations:
      - cannot use @pytest.mark.parametrize
      - world_size cannot be modified after definition and only one world_size value is accepted
      - any fixtures used must also be used in the test that uses this fixture (see example below)
      - return values cannot be returned. Passing values to a DistributedTest
        object can be achieved using class_tmpdir and writing to file (see example below)
  
  Usage:
      - must implement a run(self, ...) method
      - fixture can be used by making the class name input to a test function
  
  Example:
      @pytest.fixture(params=[10,20])
      def regular_pytest_fixture(request):
          return request.param
  
      class distributed_fixture_example(DistributedFixture):
          world_size = 4
  
          def run(self, regular_pytest_fixture, class_tmpdir):
              assert int(os.environ["WORLD_SIZE"]) == self.world_size
              local_rank = os.environ["LOCAL_RANK"]
              print(f"Rank {local_rank} with value {regular_pytest_fixture}")
              with open(os.path.join(class_tmpdir, f"{local_rank}.txt"), "w") as f:
                  f.write(f"{local_rank},{regular_pytest_fixture}")
  
      class TestExample(DistributedTest):
          world_size = 1
  
          def test(self, distributed_fixture_example, regular_pytest_fixture, class_tmpdir):
              assert int(os.environ["WORLD_SIZE"]) == self.world_size
              for rank in range(4):
                  with open(os.path.join(class_tmpdir, f"{rank}.txt"), "r") as f:
                      assert f.read() == f"{rank},{regular_pytest_fixture}"
code: def __init__(self):
      assert isinstance(
          self.world_size, int
      ), "Only one world size is allowed for distributed fixtures"
      self.__name__ = type(self).__name__
      _pytestfixturefunction = FixtureFunctionMarker(
          scope="function", params=None, name=self.__name__
      )
  
imports: import os
  import re
  import time
  import inspect
  import socket
  import subprocess
  from abc import ABC, abstractmethod
  from pathlib import Path
  import torch
  import torch.multiprocessing
  import deepspeed
  from deepspeed.accelerator import get_accelerator
  import deepspeed.comm
  import pytest
  from _pytest.outcomes import Skipped
  from _pytest.fixtures import FixtureLookupError, FixtureFunctionMarker
label_desc: Use image node_id 1 to create a new DistributedFixture object from inherited base classes: DistributedExec with example: obj = DistributedFixture()