the time it is created. No parameters should be added nor removed
     later. Same applies to buffers.
Warning:
 This module assumes all parameters are registered in the model of
 each distributed processes are in the same order. The module
 itself will conduct gradient "allreduce" following the reverse
 order of the registered parameters of the model. In other words,
 it is users' responsibility to ensure that each distributed
 process has the exact same model and thus the exact same
 parameter registration order.

Warning:
 This module allows parameters with non-rowmajor-contiguous
 strides. For example, your model may contain some parameters
 whose "torch.memory_format" is "torch.contiguous_format" and
 others whose format is "torch.channels_last".  However,
 corresponding parameters in different processes must have the
 same strides.

Warning:
 This module doesn't work with "torch.autograd.grad()" (i.e. it
