title: Reading from open file handle that is being written to in Python
body: <p>I know this is a classic problem in stream processing, but I don't know how to handle it in Python.  I have a file handle that is being written to by an active process.  I want to consume content from that file handle on a line-by-line basis, but I don't want to deadlock waiting to read.  I will keep reading until EOF or 60 seconds of looped reading, whichever comes first.  Advice on how to do this would be appreciated.  My pseudo code description of this problem is below.</p>
  
  <pre><code>proc = genprocess("command")
  found_a = False
  found_b = False
  start = time.time()
  while True:
      line = proc.readline()
      while line:
          if not found_a and grep(pattern_a, line):
              found_a = True
              print "Found A, now looking for B"
          elif not found_b and grep(pattern_b, line):
              found_b = True
              print "Found B, all done"
              break
      if time.time() - start &gt; 60:
          break
      else:
          time.sleep(5)
  
  proc.kill()
  </code></pre>
  
  <p>The problem is that this only reads one line on each interval.  Instead I want the inside of the loop to iterate as many times as possible, but <em>not</em> to block waiting for new content to be written to the file. Once it has read as much as is available, it should sleep for 5 seconds to allow more content to accumulate.</p>