title: Real-time intercepting of stdout from another process in Python
body: <p>I'd like to run a system process, intercept the output, and modify it real-time, line by line, in a Python script.</p>
  
  <p>My best attempt, which waits for the process to complete before printing, is:</p>
  
  <pre><code>#!/usr/bin/env python
  import subprocess
  
  cmd = "waitsome.py"
  proc = subprocess.Popen(cmd, shell=True, bufsize=256, stdout=subprocess.PIPE)
  for line in proc.stdout:
      print "&gt;&gt;&gt; " + line.rstrip()
  </code></pre>
  
  <p>The script <code>waitsome.py</code> simply prints a line every half a second:</p>
  
  <pre><code>#!/usr/bin/env python
  import time
  from sys import stdout
  
  print "Starting"
  for i in range(0,20):
      time.sleep(0.5)
      print "Hello, iteration", i
      stdout.flush()
  </code></pre>
  
  <p>Is there an easy solution to get <code>subprocess</code> to allow iterating over the output in real time? Do I have to use threads?</p>
  
  <p>Once upon a time, I scripted in Perl, and this was a piece of cake:</p>
  
  <pre><code>open(CMD, "waitsome.py |");
  while (&lt;CMD&gt;) {
      print "&gt;&gt;&gt; $_";
  }
  close(CMD);
  </code></pre>