manager are forwarded the same value for "kwargs".
  Return type:
     *JoinHook*

abstract property join_process_group: Any
  Returns the process group for the collective communications
  needed by the join context manager itself.

class torch.distributed.algorithms.JoinHook
This defines a join hook, which provides two entry points in the
   join context manager: a main hook, which is called repeatedly while
   there exists a non-joined process, and a post-hook, which is called
   once all processes have joined.
To implement a join hook for the generic join context manager,
   define a class that inherits from "JoinHook" and override
   "main_hook()" and "post_hook()" as appropriate.
main_hook()
  This hook is called repeatedly while there exists a non-joined
  process to shadow collective communications in one training
  iteration (i.e. in one forward pass, backward pass, and
  optimizer step).

post_hook(is_last_joiner)