Python Popen: Write to stdout AND log file simultaneously
Question: I am using Popen to call a shell script that is continuously writing its
stdout and stderr to a log file. Is there any way to simultaneously output the
log file continuously (to the screen), or alternatively, make the shell script
write to both the log file and stdout at the same time?

I basically want to do something like this in Python:

    
    
    cat file 2>&1 | tee -a logfile #"cat file" will be replaced with some script
    

Again, this pipes stderr/stdout together to tee, which writes it both to
stdout and my logfile.

I know how to write stdout and stderr to a logfile in Python. Where I'm stuck
is how to duplicate these back to the screen:

    
    
    subprocess.Popen("cat file", shell=True, stdout=logfile, stderr=logfile)
    

Of course I could just do something like this, but is there any way to do this
without tee and shell file descriptor redirection?:

    
    
    subprocess.Popen("cat file 2>&1 | tee -a logfile", shell=True)
    


Answer: You can use a pipe to read the data from the program's stdout and write it to
all the places you want:

    
    
    import sys
    import subprocess
    
    logfile = open('logfile', 'w')
    proc=subprocess.Popen(['cat', 'file'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    for line in proc.stdout:
        sys.stdout.write(line)
        logfile.write(line)
    proc.wait()
    

_UPDATE_

In python 3, the `universal_newlines` parameter controls how pipes are used.
If `False`, pipe reads return `bytes` objects and may need to be decoded
(e.g., `line.decode('utf-8')`) to get a string. If `True`, python does the
decode for you

> Changed in version 3.3: When universal_newlines is True, the class uses the
> encoding locale.getpreferredencoding(False) instead of
> locale.getpreferredencoding(). See the io.TextIOWrapper class for more
> information on this change.

