Distributed Training Releases Now in Stable
In 1.10, there are a number of features that are moving from beta to stable in the distributed package:
* (Stable) Remote Module: This feature allows users to operate a module on a remote worker like using a local module, where the RPCs are transparent to the user. Refer to this documentation for more details.
* (Stable) DDP Communication Hook: This feature allows users to override how DDP synchronizes gradients across processes. Refer to this documentation for more details.