implication that the value will be transferred to the local worker in
the future. RRefs can be used in multi-machine training by holding
references to nn.Modules that exist on other workers, and calling the
appropriate functions to retrieve or modify their parameters during
training. See Remote Reference Protocol for more details.
class torch.distributed.rpc.RRef
More Information about RRef
^^^^^^^^^^^^^^^^^^^^^^^^^^^


Remote Reference Protocol


Background


Assumptions


RRef Lifetime


Design Reasoning


Implementation




Protocol Scenarios


User Share RRef with Owner as Return Value


User Share RRef with Owner as Argument


Owner Share RRef with User


User Share RRef with User




RemoteModule
Warning:
RemoteModule is not currently supported when using CUDA tensors
"RemoteModule" is an easy way to create an nn.Module remotely on a
different process. The actual module resides on a remote host, but the