title: Maximum size for multiprocessing.Queue item?
body: <p>I'm working on a fairly large project in Python that requires one of the compute-intensive background tasks to be offloaded to another core, so that the main service isn't slowed down.  I've come across some apparently strange behaviour when using <code>multiprocessing.Queue</code> to communicate results from the worker process.  Using the same queue for both a <code>threading.Thread</code> and a <code>multiprocessing.Process</code> for comparison purposes, the thread works just fine but the process fails to join after putting a large item in the queue.  Observe:</p>
  
  <pre><code>import threading
  import multiprocessing
  
  class WorkerThread(threading.Thread):
      def __init__(self, queue, size):
          threading.Thread.__init__(self)
          self.queue = queue
          self.size = size
  
      def run(self):
          self.queue.put(range(size))
  
  
  class WorkerProcess(multiprocessing.Process):
      def __init__(self, queue, size):
          multiprocessing.Process.__init__(self)
          self.queue = queue
          self.size = size
  
      def run(self):
          self.queue.put(range(size))
  
  
  if __name__ == "__main__":
      size = 100000
      queue = multiprocessing.Queue()
  
      worker_t = WorkerThread(queue, size)
      worker_p = WorkerProcess(queue, size)
  
      worker_t.start()
      worker_t.join()
      print 'thread results length:', len(queue.get())
  
      worker_p.start()
      worker_p.join()
      print 'process results length:', len(queue.get())
  </code></pre>
  
  <p>I've seen that this works fine for <code>size = 10000</code>, but hangs at <code>worker_p.join()</code> for <code>size = 100000</code>.  Is there some inherent size limit to what <code>multiprocessing.Process</code> instances can put in a <code>multiprocessing.Queue</code>?  Or am I making some obvious, fundamental mistake here?</p>
  
  <p>For reference, I am using Python 2.6.5 on Ubuntu 10.04.</p>