there is no guarantee that the full forward-backward-optimizer
   sequence will execute for one client at a time. This means that the
   gradients being applied may not correspond to the latest forward
   pass executed on a given worker. Also, there is no guaranteed
   ordering across workers.
DistributedOptimizer creates the local optimizer with TorchScript
   enabled by default, so that optimizer updates are not blocked by
   the Python Global Interpreter Lock (GIL) in the case of
   multithreaded training (e.g. Distributed Model Parallel). This
   feature is currently enabled for most optimizers. You can also
   follow the recipe in PyTorch tutorials to enable TorchScript
   support for your own custom optimizers.
Parameters:
      * optimizer_class (optim.Optimizer) -- the class of
        optimizer to instantiate on each worker.
  * **params_rref** (*list**[**RRef**]*) -- list of RRefs to local
    or remote parameters to optimize.
