


os.environ['MASTER_ADDR'] = 'localhost'
      >>> os.environ['MASTER_PORT'] = '29500'
      >>> torch.distributed.rpc.init_rpc('worker', rank=0, world_size=1)
      >>>
      >>> # Build pipe.
      >>> fc1 = nn.Linear(16, 8).cuda(0)
      >>> fc2 = nn.Linear(8, 4).cuda(1)
      >>> model = nn.Sequential(fc1, fc2)
      >>> model = Pipe(model, chunks=8)
      >>> input = torch.rand(16, 16).cuda(0)
      >>> output_rref = model(input)



Note:
 You can wrap a "Pipe" model with
 "torch.nn.parallel.DistributedDataParallel" only when the
 checkpoint parameter of "Pipe" is "'never'".

Note:
 "Pipe" only supports intra-node pipelining currently, but will be
 expanded to support inter-node pipelining in the future. The
 forward function returns an "RRef" to allow for inter-node
 pipelining in the future, where the output might be on a remote
 host. For intra-node pipelinining you can use "local_value()" to
 retrieve the output locally.

Warning: