class_docstr: Similar to ZMQServer except with broadcast/gather semantics on exactly two ports.
  
  Using a constant number of ports allows the SubprocessReceiver to be configured without knowing
  any rank information (i.e., before user code is read and horovod can be initialized).
  
  ZMQBroadcastServer uses ZMQ PUB-SUB pattern to transmit messages to worker processes, and uses
  the PUSH-PULL pattern to collect responses from workers. The reason for this asymmetry is that
  PUSH-PULL connections block during send() if the remote end is dead. Therefore, PUSH-PULL
  cannot be used to transmitting from server to worker, because if all the workers die, the
  server would hang.
  
  Additionally, the server can't receive messages from workers via the PUB-SUB pattern, because
  the startup semantics of PUB-SUB connections in ZMQ are slightly odd; the SUB socket must
  connect to the PUB socket.  Normally this happens when you do sub_socket.connect(), but if the
  server creates a SUB socket and does sub_socket.bind(), then when the client creates a PUB
  socket and calls pub_socket.connect(), ZMQ still has to create a connection from the SUB to the
  PUB (since sub_socket used bind() instead of connect()) and the server's SUB socket will
  usually miss the first message sent by the client's PUB socket.
  
  See ZMQ documentation for a related discussion on PUB-SUB sockets:
  http://zguide.zeromq.org/page:all#Getting-the-Message-Out (look for "one more important thing")
  http://zguide.zeromq.org/page:all#Node-Coordination
  (link broke, use http://web.archive.org/web/20191011190012/http://zguide.zeromq.org/page:all)
class_docstr_tok: ["Similar","to","ZMQServer","except","with","broadcast\/gather","semantics","on","exactly","two","ports",".","Using","a","constant","number","of","ports","allows","the","SubprocessReceiver","to","be","configured","without","knowing","any","rank","information","(","i.e.",",","before","user","code","is","read","and","horovod","can","be","initialized",")",".","ZMQBroadcastServer","uses","ZMQ","PUB-SUB","pattern","to","transmit","messages","to","worker","processes",",","and","uses","the","PUSH-PULL","pattern","to","collect","responses","from","workers",".","The","reason","for","this","asymmetry","is","that","PUSH-PULL","connections","block","during","send","(",")","if","the","remote","end","is","dead",".","Therefore",",","PUSH-PULL","can","not","be","used","to","transmitting","from","server","to","worker",",","because","if","all","the","workers","die",",","the","server","would","hang",".","Additionally",",","the","server","ca","n't","receive","messages","from","workers","via","the","PUB-SUB","pattern",",","because","the","startup","semantics","of","PUB-SUB","connections","in","ZMQ","are","slightly","odd",";","the","SUB","socket","must","connect","to","the","PUB","socket",".","Normally","this","happens","when","you","do","sub_socket.connect","(",")",",","but","if","the","server","creates","a","SUB","socket","and","does","sub_socket.bind","(",")",",","then","when","the","client","creates","a","PUB","socket","and","calls","pub_socket.connect","(",")",",","ZMQ","still","has","to","create","a","connection","from","the","SUB","to","the","PUB","(","since","sub_socket","used","bind","(",")","instead","of","connect","(",")",")","and","the","server","'s","SUB","socket","will","usually","miss","the","first","message","sent","by","the","client","'s","PUB","socket",".","See","ZMQ","documentation","for","a","related","discussion","on","PUB-SUB","sockets",":","http",":","\/\/zguide.zeromq.org\/page",":","all","#","Getting-the-Message-Out","(","look","for","``","one","more","important","thing","''",")","http",":","\/\/zguide.zeromq.org\/page",":","all","#","Node-Coordination","(","link","broke",",","use","http",":","\/\/web.archive.org\/web\/20191011190012\/http",":","\/\/zguide.zeromq.org\/page",":","all",")"]
docstr: Broadcast Hello messages over and over until all clients respond with a Hello message.
  
  The reason for this is that the only way to be 100% confident that a subscriber has
  connected is for it to actually receive a message over the pub/sub connection.
  
  After each client sees its first Hello, it will send a single Hello message to the
  server.
  
  After all connections have been made, the server will broadcast a FinalHello.
docstr_tok: ["Broadcast","Hello","messages","over","and","over","until","all","clients","respond","with","a","Hello","message",".","The","reason","for","this","is","that","the","only","way","to","be","100","%","confident","that","a","subscriber","has","connected","is","for","it","to","actually","receive","a","message","over","the","pub\/sub","connection",".","After","each","client","sees","its","first","Hello",",","it","will","send","a","single","Hello","message","to","the","server",".","After","all","connections","have","been","made",",","the","server","will","broadcast","a","FinalHello","."]
code: def safe_start(self) -> None:
      """
      Broadcast Hello messages over and over until all clients respond with a Hello message.
  
      The reason for this is that the only way to be 100% confident that a subscriber has
      connected is for it to actually receive a message over the pub/sub connection.
  
      After each client sees its first Hello, it will send a single Hello message to the
      server.
  
      After all connections have been made, the server will broadcast a FinalHello.
      """
  
      connections_made = 0
      while connections_made < self._num_connections:
          # Send a Hello.
          self._pub_socket.send_pyobj(_HelloMessage())
  
          # Check for an incoming connection.
          if self._pull_socket.poll(50) == 0:
              continue
  
          obj = self._pull_socket.recv_pyobj()
          if not isinstance(obj, _HelloMessage):
              raise RuntimeError(
                  f"got non-_HelloMessage: {type(obj).__name__}"
              )
          connections_made += 1
  
      self._pub_socket.send_pyobj(_FinalHelloMessage())
  
imports: import logging
  import os
  import selectors
  import signal
  import socket
  import subprocess
  import time
  from typing import Any, Callable, Dict, List, Optional, Tuple, Union
  import determined
label_desc: Use image node_id 2 for calling the ZMQBroadcastServer obj's underlying member method code with example usage: obj.safe_start() without return types