Generic Join Context Manager
The generic join context manager facilitates distributed training on
uneven inputs. This page outlines the API of the relevant classes:
"Join", "Joinable", and "JoinHook". For a tutorial, see Distributed
Training with Uneven Inputs Using the Join Context Manager.
class torch.distributed.algorithms.Join(joinables, enable=True, throw_on_early_termination=False, **kwargs)
This class defines the generic join context manager, which allows
   custom hooks to be called after a process joins. These hooks should
   shadow the collective communications of non-joined processes to
   prevent hanging and erroring and to ensure algorithmic correctness.
   Refer to "JoinHook" for details about the hook definition.
Warning:
 The context manager requires each participating "Joinable" to
 call the method "notify_join_context()" before its own per-
 iteration collective communications to ensure correctness.

Warning: