title: Python 3 Multiprocessing queue deadlock when calling join before the queue is empty
body: <p>I have a question understanding the queue in the <code>multiprocessing</code> module in python 3</p>
  
  <p>This is what they say in the <a href="https://docs.python.org/3/library/multiprocessing.html#all-start-methods" rel="noreferrer">programming guidelines</a>:</p>
  
  <blockquote>
  <p>Bear in mind that a process that has put items in a queue will wait before
  terminating until all the buffered items are fed by the &#8220;feeder&#8221; thread to
  the underlying pipe.  (The child process can call the
  Queue.cancel_join_thread
  method of the queue to avoid this behaviour.)</p>
  <p>This means that whenever you use a queue you need to make sure that all
  items which have been put on the queue will eventually be removed before the
  process is joined.  Otherwise you cannot be sure that processes which have
  put items on the queue will terminate.  Remember also that non-daemonic
  processes will be joined automatically.</p>
  <p>An example which will deadlock is the following:</p>
  <pre>from multiprocessing import Process, Queue
  
  def f(q):
      q.put(&#39;X&#39; * 1000000)
  
  if __name__ == &#39;__main__&#39;:
      queue = Queue()
      p = Process(target=f, args=(queue,))
      p.start()
      p.join()                    # this deadlocks
      obj = queue.get()
  </pre>
  
  <p>A fix here would be to swap the last two lines (or simply remove the
  p.join() line).</p>
  </blockquote>
  
  <p>So apparently, <code>queue.get()</code> should not be called after a <code>join()</code>.</p>
  
  <p>However there are examples of using queues where <code>get</code> is called after a <code>join</code> like:</p>
  
  <pre><code>import multiprocessing as mp
  import random
  import string
  
  # define a example function
  def rand_string(length, output):
      """ Generates a random string of numbers, lower- and uppercase chars. """
      rand_str = ''.join(random.choice(
                  string.ascii_lowercase
                  + string.ascii_uppercase
                  + string.digits)
      for i in range(length))
          output.put(rand_str)
  
   if __name__ == "__main__":
       # Define an output queue
       output = mp.Queue()
  
       # Setup a list of processes that we want to run
       processes = [mp.Process(target=rand_string, args=(5, output))
                      for x in range(2)]
  
       # Run processes
      for p in processes:
          p.start()
  
      # Exit the completed processes
      for p in processes:
          p.join()
  
      # Get process results from the output queue
      results = [output.get() for p in processes]
  
      print(results)
  </code></pre>
  
  <p>I've run this program and it works (also posted as a solution to the StackOverFlow question <a href="https://stackoverflow.com/questions/29904010/python-3-multiprocessing-queue-get-does-not-respond">Python 3 - Multiprocessing - Queue.get() does not respond</a>).</p>
  
  <p>Could someone help me understand what the rule for the deadlock is here?</p>