(the caller of "remote()") or the owner of the object. The
   distributed optimizer, as we will discuss below, is one example of
   such use cases.


Distributed Autograd stitches together local autograd engines
   on all the workers involved in the forward pass, and automatically
   reach out to them during the backward pass to compute gradients.
   This is especially helpful if the forward pass needs to span
   multiple machines when conducting, e.g., distributed model parallel
   training, parameter-server training, etc. With this feature, user
   code no longer needs to worry about how to send gradients across
   RPC boundaries and in which order should the local autograd engines
   be launched, which can become quite complicated where there are
   nested and inter-dependent RPC calls in the forward pass.


Distributed Optimizer's constructor takes a "Optimizer()"
   (e.g., "SGD()", "Adagrad()", etc.) and a list of parameter RRefs,

