title: Python multiprocessing module: join processes with timeout
body: <p>I'm doing an optimization of parameters of a complex simulation. I'm using the multiprocessing module for enhancing the performance of the optimization algorithm. The basics of multiprocessing I learned at <a href="http://pymotw.com/2/multiprocessing/basics.html" rel="nofollow noreferrer">http://pymotw.com/2/multiprocessing/basics.html</a>.
  The complex simulation lasts different times depending on the given parameters from the optimization algorithm, around 1 to 5 minutes. If the parameters are chosen very badly, the simulation can last 30 minutes or more and the results are not useful. So I was thinking about build in a timeout to the multiprocessing, that terminates all simulations that last more than a defined time. Here is an abstracted version of the problem:</p>
  <pre><code>import numpy as np
  import time
  import multiprocessing
  
  def worker(num):
      
      time.sleep(np.random.random()*20)
  
  def main():
      
      pnum = 10    
      
      procs = []
      for i in range(pnum):
          p = multiprocessing.Process(target=worker, args=(i,), name = ('process_' + str(i+1)))
          procs.append(p)
          p.start()
          print('starting', p.name)
          
      for p in procs:
          p.join(5)
          print('stopping', p.name)
       
  if __name__ == &quot;__main__&quot;:
      main()
  </code></pre>
  <p>The line <code>p.join(5)</code> defines the timeout of 5 seconds. Because of the for-loop <code>for p in procs:</code> the program waits 5 seconds until the first process is finished and then again 5 seconds until the second process is finished and so on, but i want the program to terminate all processes that last more than 5 seconds. Additionally, if none of the processes last longer than 5 seconds the program must not wait this 5 seconds.</p>