Distributed Optimizers
Warning:
Distributed optimizer is not currently supported when using CUDA
  tensors
"torch.distributed.optim" exposes DistributedOptimizer, which takes a
list of remote parameters ("RRef") and runs the optimizer locally on
the workers where the parameters live.  The distributed optimizer can
use any of the local optimizer Base class to apply the gradients on
each worker.
class torch.distributed.optim.DistributedOptimizer(optimizer_class, params_rref, args, *kwargs)
DistributedOptimizer takes remote references to parameters
   scattered across workers and applies the given optimizer locally
   for each parameter.
This class uses "get_gradients()" in order to retrieve the
   gradients for specific parameters.
Concurrent calls to "step()", either from the same or different
   clients, will be serialized on each worker -- as each worker's
   optimizer can only work on one set of gradients at a time. However,