title: Non-blocking read from multiple subprocesses (Python)
body: <p>I currently have the following code, inspired by the answer to <a href="https://stackoverflow.com/questions/375427/non-blocking-read-on-a-subprocess-pipe-in-python">Non-blocking read on a subprocess.PIPE in python</a>. 
  It seems to work correctly, outputting the lines to the screen, however it only does so for the first created process, all other processes (which are running) don't get any data printed.</p>
  
  <p>How do I make sure I can read data (in a non-blocking way) from multiple subprocesses?</p>
  
  <pre class="lang-py prettyprint-override"><code>#!/usr/bin/env python
  import sys
  import os
  import subprocess
  from threading import Thread
  from Queue import Queue, Empty
  
  STREAMER_URL = 'rtmp://127.0.0.1/app'
  RTMPDUMP_EXECUTEABLE = 'rtmpdump'
  
  def enqueue_output(out, queue):
      for line in iter(lambda: out.read(16), b''):
          queue.put(line)
      out.close()
  
  def download_rtmp(media, filename):
    # Create parameters
    args=[RTMPDUMP_EXECUTEABLE]
    args.extend(['-r',media[0],'-y',media[1]])
  
    # Set output file
    OUTPUT_FILE = filename
    args.extend(['-o',OUTPUT_FILE])
  
    # Send rtmpdump any extra arguments
    if len(sys.argv) &gt; 2:
      args.extend(sys.argv[2:])
  
    # Execute rtmpdump
    p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT )
    q = Queue()
    t = Thread(target=enqueue_output, args=(p.stdout, q))
    t.daemon = True # thread dies with the program
    t.start()
    return (p, q, t)
  
  def main():
  
    # actual data is from somewhere else on the internet
    for (name, playpath, filepath) in data:
      print 'Spawning %s download...' % name
      PROCESSES.append(download_rtmp((STREAMER_URL, playpath), filepath))
  
    BUFS = dict()
  
    # infinite loop checking if all processes have finished
    while True:
      done = True
      for (process, queue, thread) in PROCESSES:
        try:
          readdata = queue.get_nowait()
        except Empty:
          pass
        else:
          if process in BUFS:
            readdata = BUFS[process] + readdata
          lines = readdata.split('\n')
          if len(lines) &gt; 1:
            for line in lines[:-1]:
              print 'Line: %s' % line
          if '\r' in lines[-1]:
            lines = readdata.split('\r')
            for line in lines[:-1]:
              print 'Line2: %s' % line
          BUFS[process] = lines[-1]
  
        process.poll()
  
        if process.returncode is None:
          done = False
          break
      if done:
        break
  
    print "Done"
  
  if __name__ == "__main__":
      main()
  </code></pre>