title: Python Multiprocessing - 'Queue' object has no attribute 'task_done' / 'join'
body: <p>I am rewriting a threaded process into a multiprocessing queue to attempt to speed up a large calculation. I have gotten it 95% of the way there, but I can't figure out how to signal when the <code>Queue</code> is empty using <code>multiprocessing</code>.</p>
  
  <hr>
  
  <p>My original code is something like this:</p>
  
  <pre><code>import Queue
  from threading import Thread
  
  num_fetch_threads = 4
  enclosure_queue = Queue()
  
  for i in range(num_fetch_threads):
    worker = Thread(target=run_experiment, args=(i, enclosure_queue))
    worker.setDaemon(True)
    worker.start()
  
  for experiment in experiment_collection:
    enclosure_queue.put((experiment, otherVar))
  
  enclosure_queue.join()
  </code></pre>
  
  <p>And the queue function like this:</p>
  
  <pre><code>def run_experiment(i, q):
    while True:
      ... do stuff ...
      q.task_done()
  </code></pre>
  
  <hr>
  
  <p>My new code is somethings like this:</p>
  
  <pre><code>from multiprocessing import Process, Queue
  
  num_fetch_threads = 4
  enclosure_queue = Queue()
  
  for i in range(num_fetch_threads):
    worker = Process(target=run_experiment, args=(i, enclosure_queue))
    worker.daemon = True
    worker.start()
  
  for experiment in experiment_collection:
    enclosure_queue.put((experiment, otherVar))
  
  worker.join() ## I only put this here bc enclosure_queue.join() is not available
  </code></pre>
  
  <p>And the new queue function:</p>
  
  <pre><code>def run_experiment(i, q):
    while True:
      ... do stuff ...
      ## not sure what should go here
  </code></pre>
  
  <hr>
  
  <p>I have been reading the docs and Google, but can't figure out what I am missing - I know that <code>task_done</code> / <code>join</code> are not part of the <code>multiprocessing</code> <code>Queue</code> class, but it's not clear what I am supposed to use.</p>
  
  <blockquote>
    <p>"They differ in that Queue lacks the task_done() and join() methods
    introduced into Python 2.5’s Queue.Queue class." <a href="https://docs.python.org/2/library/multiprocessing.html#pipes-and-queues" rel="nofollow">Source</a></p>
  </blockquote>
  
  <p>But without either of those, I'm not sure how the queue knows it is done, and how to continue on with the program.</p>