title: Where is the memory leak? How to timeout threads during multiprocessing in python?
body: <p>It is unclear how to properly timeout workers of joblib's <code>Parallel</code> in python. Others have had similar questions <a href="https://stackoverflow.com/questions/42702199/gracefull-python-joblib-kill">here</a>, <a href="https://github.com/joblib/joblib/issues/263" rel="noreferrer">here</a>, <a href="https://stackoverflow.com/questions/24537379/python-multiprocessing-script-freezes-seemingly-without-error">here</a> and <a href="https://stackoverflow.com/questions/43586223/joblibparallel-always-was-stuck-in-the-end">here</a>. </p>
  
  <p>In my example I am utilizing a pool of 50 <code>joblib</code> workers with  <code>threading</code> backend.</p>
  
  <p><strong>Parallel Call (threading):</strong></p>
  
  <pre><code>output = Parallel(n_jobs=50, backend  = 'threading')
      (delayed(get_output)(INPUT) 
          for INPUT in list)
  </code></pre>
  
  <p>Here, <code>Parallel</code> hangs without errors as soon as <code>len(list) &lt;= n_jobs</code> but only when <code>n_jobs =&gt; -1</code>.</p>
  
  <p>In order to circumvent this issue, people give  <a href="https://github.com/joblib/joblib/pull/366#issuecomment-267603530" rel="noreferrer">instructions</a> on how to create a timeout decorator to the <code>Parallel</code> function (<code>get_output(INPUT)</code>) in the above example) using <code>multiprocessing</code>:</p>
  
  <p><strong>Main function (decorated):</strong></p>
  
  <pre><code>@with_timeout(10)    # multiprocessing
  def get_output(INPUT):     # threading
      output = do_stuff(INPUT)
      return output
  </code></pre>
  
  <p><strong>Multiprocessing Decorator:</strong></p>
  
  <pre><code>def with_timeout(timeout):
      def decorator(decorated):
          @functools.wraps(decorated)
          def inner(*args, **kwargs):
              pool = multiprocessing.pool.ThreadPool(1)
              async_result = pool.apply_async(decorated, args, kwargs)
              try:
                  return async_result.get(timeout)
              except multiprocessing.TimeoutError:
                  return
          return inner
      return decorator
  </code></pre>
  
  <p>Adding the decorator to the otherwise working code results in a memory leak after ~2x the length of the timeout plus a crash of eclipse. </p>
  
  <p><strong>Where is this leak in the decorator?</strong></p>
  
  <p><strong>How to timeout threads during multiprocessing in python?</strong></p>