title: How to spawn another process and capture output in python?
body: <p>I'm just learning Python but have about 16 years experience with PERL and PHP. </p>
  
  <p>I'm trying to get the output of ngrep and write it to a log file using Python while also tailing the log file. I've seen some examples online but some seem old and outdated and others use shell=True which is discouraged.</p>
  
  <p>In perl I just use something similar to the following</p>
  
  <pre><code>#!/usr/bin/perl
  open(NGFH,"ngrep -iW byline $filter");
  while ($line = &lt;NGFH&gt;) {
      open(LOG,"&gt;&gt; /path/to/file.log")
      // highlighting, filtering, other sub routine calls
      print LOG $line
  }
  </code></pre>
  
  <p>I've gotten tail to work but ngrep doesn't. I'd like to be able to run this infinately and output the stream from ngrep to the log file after filtering. I couldn't get the output from ngrep to show in stdout so that's as far as I've gotten. I was expecting to be able to see the data file tail as the log file was updated and see the output from ngrep. For now i was just using bash to run the following. </p>
  
  <pre><code>echo "." &gt;&gt; /path/to/ngrep.log
  </code></pre>
  
  <p>Thanks!</p>
  
  <p>Here's what I got so far...</p>
  
  <p><strong>Updated</strong> 
  This seems to work now. I wouldn't know how to improve on it though. </p>
  
  <pre><code>import subprocess
  import select
  import re
  
  log = open('/path/to/ngrep.log','a+',0)
  print log.name
  
  n = subprocess.Popen(['ngrep', '-iW', 'byline'],\
      stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
  p = select.poll()
  p.register(n.stdout)
  
  f = subprocess.Popen(['tail','-F','-n','0','/path/to/tailme.log'],\
      stdout=subprocess.PIPE,stderr=subprocess.PIPE)
  p2 = select.poll()
  p2.register(f.stdout)
  
  def srtrepl(match):
      if match.group(0) == 'x.x.x.x':
          # do something
      if match.group(0) == 'x.x.y.y':
          # do something else
  
      return '\033[92m'+ match.group(0) + '\033[0m'
  
  while True:
      if p.poll(1):
          line = n.stdout.readline()
          s = re.compile(r'(8.8.(4.4|8.8)|192.168.[0-9]{1,3}.[0-9]{1,3})' )
          print s.sub( srtrepl, line )
          log.write(n.stdout.readline())
  
      if p2.poll(1):
          print f.stdout.readline().rstrip('\n')
  </code></pre>