python stream subprocess stdout and stderr zip doesnt work
Question: There are already a bunch of answers that solve how to do this general thing,
but my main question is Why doesnt this approach work?

I am trying to "live stream" the stdout and stderr from a subprocess. I can do
this by doing:

    
    
    import sys
    import subprocess
    
    def run_command(cmd):
        process = subprocess.Popen(cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
    
        for out in iter(process.stdout.readline, b''):
            print(out)
    
        for err in iter(process.stderr.readline, b''):
            print(err)
    
    run_command(['echo', 'hello', 'world']) # should print hello world
    run_command(['rm', 'blarg223'])  # Should print to stderr (file doesnt exist)
    

and this works giving me the result:

    
    
    b'hello world\n'
    b'rm: cannot remove \xe2\x80\x98blarg223\xe2\x80\x99: No such file or directory\n'
    

However this causes a problem as it really only live streams the stdout, then
prints all the errors as the end. So I thought I could fix this using:

    
    
    def run_command(cmd):
        process = subprocess.Popen(cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
    
        for out, err in zip(iter(process.stdout.readline, b''), iter(process.stderr.readline, b'')):
            print(out)
            print(b'Error: ' + err)
    

But, this doesnt produce any output. Why does using zip not work?


Answer: zip stops when one of the iterators is finished.

In each of the examples you gave, one stream(stdout/stderr) is empty. So zip
will produce nothing.

To fix this you should use itertools.zip_longest

