Python subprocess get output as process finishes
Question: Currently I am getting MAC addresses from devices via Bluetooth and I pass
these mac addresses one at a time to a method that calls a subprocess and
assigns the output to a variable. With that variable I run some filtering
functions to get a value from the command called in the subprocess. I then
return that value if it finds it from the output.

What I would like to do is pass all mac addresses to the method at once and
run them all at one time. How do I capture the output of each process and run
my filtering script as the processes complete, and at the same time notify me
if the process fails or errors.

Here is the current method that handles one mac at a time. Lets assume now
that I am passing a list of mac addresses.

    
    
     def getchan(mac):
      a = subprocess.Popen(["sdptool", "search","--bdaddr", mac, "OPUSH"], stdout = subprocess.PIPE).communicate()[0].split()[2:]
      if a==[]:
          print "ERROR"
          return "ERROR"
      else:
          count = 0
          for item in a:
              if item == "Channel:":
                  return a[count + 1]
              count += 1
          return "Could not find the OPUSH channel"
    

It should look something like

    
    
    def getchan(macs):
    processes = set()
    for mac in macs:
      processes.add(subprocess.Popen(["sdptool", "search","--bdaddr", mac, "OPUSH"], stdout =      subprocess.PIPE).communicate()[0].split()[2:])
      #this is where I need the help
    

Thank you for taking a look. Any help or clarification of Subprocesses would
be much appreciated.


Answer: 
    import select
    import subprocess
    
    def in_parallel(processes):
        pipes = {p.stdout.fileno(): (i, p.stdout) for i, p in enumerate(processes)}
    
        poller = select.poll()
        for fd, pipe in pipes.iteritems():
            poller.register(fd, select.POLLIN)
    
        outputs = [''] * len(processes)
    
        while pipes:
            active = poller.poll()
            for fd, event in active:
                idx, pipe = pipes[fd]
                o = pipe.read()
                if o:
                    outputs[idx] += o
                else:
                    poller.unregister(fd)
                    pipe.close()
                    del pipes[fd]
    
        for p in processes:
            p.wait()
    
        return outputs
    
    args = ['a', 'b', 'c']
    processes = [subprocess.Popen(['sleep 5; echo ' + arg], stdout=subprocess.PIPE, shell=True) for arg in args]
    outputs = in_parallel(processes)
    print outputs
    

* * *
    
    
    $ time python test.py
    ['a\n', 'b\n', 'c\n']
    
    real    0m5.042s
    user    0m0.016s
    sys 0m0.016s
    

