class_docstr: Modules to be parallelized with pipeline parallelism.
  
  The key constraint that enables pipeline parallelism is the
  representation of the forward pass as a sequence of layers
  and the enforcement of a simple interface between them. The
  forward pass is implicitly defined by the module ``layers``. The key
  assumption is that the output of each layer can be directly fed as
  input to the next, like a ``torch.nn.Sequence``. The forward pass is
  implicitly:
  
  .. code-block:: python
  
      def forward(self, inputs):
          x = inputs
          for layer in self.layers:
              x = layer(x)
          return x
  
  .. note::
      Pipeline parallelism is not compatible with ZeRO-2 and ZeRO-3.
  
  Args:
      layers (Iterable): A sequence of layers defining pipeline structure. Can be a ``torch.nn.Sequential`` module.
      num_stages (int, optional): The degree of pipeline parallelism. If not specified, ``topology`` must be provided.
      topology (``deepspeed.runtime.pipe.ProcessTopology``, optional): Defines the axes of parallelism axes for training. Must be provided if ``num_stages`` is ``None``.
      loss_fn (callable, optional): Loss is computed ``loss = loss_fn(outputs, label)``
      seed_layers(bool, optional): Use a different seed for each layer. Defaults to False.
      seed_fn(type, optional): The custom seed generating function. Defaults to random seed generator.
      base_seed (int, optional): The starting seed. Defaults to 1234.
      partition_method (str, optional): The method upon which the layers are partitioned. Defaults to 'parameters'.
      activation_checkpoint_interval (int, optional): The granularity activation checkpointing in terms of number of layers. 0 disables activation checkpointing.
      activation_checkpoint_func (callable, optional): The function to use for activation checkpointing. Defaults to ``deepspeed.checkpointing.checkpoint``.
      checkpointable_layers(list, optional): Checkpointable layers may not be checkpointed. Defaults to None which does not additional filtering.
class_docstr_tok: ["Modules","to","be","parallelized","with","pipeline","parallelism",".","The","key","constraint","that","enables","pipeline","parallelism","is","the","representation","of","the","forward","pass","as","a","sequence","of","layers","and","the","enforcement","of","a","simple","interface","between","them",".","The","forward","pass","is","implicitly","defined","by","the","module","``","layers","``",".","The","key","assumption","is","that","the","output","of","each","layer","can","be","directly","fed","as","input","to","the","next",",","like","a","``","torch.nn.Sequence","``",".","The","forward","pass","is","implicitly",":","..","code-block",":",":","python","def","forward","(","self",",","inputs",")",":","x","=","inputs","for","layer","in","self.layers",":","x","=","layer","(","x",")","return","x","..","note",":",":","Pipeline","parallelism","is","not","compatible","with","ZeRO-2","and","ZeRO-3",".","Args",":","layers","(","Iterable",")",":","A","sequence","of","layers","defining","pipeline","structure",".","Can","be","a","``","torch.nn.Sequential","``","module",".","num_stages","(","int",",","optional",")",":","The","degree","of","pipeline","parallelism",".","If","not","specified",",","``","topology","``","must","be","provided",".","topology","(","``","deepspeed.runtime.pipe.ProcessTopology","``",",","optional",")",":","Defines","the","axes","of","parallelism","axes","for","training",".","Must","be","provided","if","``","num_stages","``","is","``","None","``",".","loss_fn","(","callable",",","optional",")",":","Loss","is","computed","``","loss","=","loss_fn","(","outputs",",","label",")","``","seed_layers","(","bool",",","optional",")",":","Use","a","different","seed","for","each","layer",".","Defaults","to","False",".","seed_fn","(","type",",","optional",")",":","The","custom","seed","generating","function",".","Defaults","to","random","seed","generator",".","base_seed","(","int",",","optional",")",":","The","starting","seed",".","Defaults","to","1234.","partition_method","(","str",",","optional",")",":","The","method","upon","which","the","layers","are","partitioned",".","Defaults","to","'parameters","'",".","activation_checkpoint_interval","(","int",",","optional",")",":","The","granularity","activation","checkpointing","in","terms","of","number","of","layers",".","0","disables","activation","checkpointing",".","activation_checkpoint_func","(","callable",",","optional",")",":","The","function","to","use","for","activation","checkpointing",".","Defaults","to","``","deepspeed.checkpointing.checkpoint","``",".","checkpointable_layers","(","list",",","optional",")",":","Checkpointable","layers","may","not","be","checkpointed",".","Defaults","to","None","which","does","not","additional","filtering","."]
docstr: Manually define the range of layers that will be built on this process.
  
  These boundaries are treated as list slices and so start is inclusive and stop is
  exclusive. The default of None for both results in all layers being built
  locally.
docstr_tok: ["Manually","define","the","range","of","layers","that","will","be","built","on","this","process",".","These","boundaries","are","treated","as","list","slices","and","so","start","is","inclusive","and","stop","is","exclusive",".","The","default","of","None","for","both","results","in","all","layers","being","built","locally","."]
code: def _set_bounds(self, start=None, stop=None):
      """Manually define the range of layers that will be built on this process.
  
      These boundaries are treated as list slices and so start is inclusive and stop is
      exclusive. The default of None for both results in all layers being built
      locally.
      """
      self._local_start = start
      self._local_stop = stop
  
imports: import os
  import glob
  import re
  from functools import partial
  import torch
  import torch.nn
  from deepspeed import comm
  from deepspeed.utils import logger
  from ..None import utils
  from ..activation_checkpointing import checkpointing
  from .topology import PipeDataParallelTopology, PipelineParallelGrid
  from deepspeed.runtime.state_dict_factory import SDLoaderFactory
  from deepspeed.accelerator import get_accelerator
  from deepspeed.checkpoint.utils import clone_tensors_for_torch_save
label_desc: Use image node_id 14 for calling the PipelineModule obj's underlying member method code with example usage: obj._set_bounds(start, stop) without return types