Unable to process python subprocess stderr on windows
Question: I am trying to simple file copy operation using `dd` under Windows. The code
is simple. Just open the command using subprocess and read `stderr`. Then
process the `stderr` to show the progress of the operation. Here is the
sample/simple code which I an trying to accomplish:-

    
    
    import time
    import signal
    import subprocess
    import os
    
    def dd_win():
        windd = "dd.exe"
        in_file = os.path.join("E:", "test-in.iso")
        out_file = os.path.join("E:", "test-out.iso")
        parameter1 = "if=" + in_file
        parameter2 = "of=" + out_file
        parameter3 = "bs=1M"
        parameter4 = "--progress"
        command = [windd, parameter1, parameter2, parameter4]
        print command
        dd_process = subprocess.Popen(command, stderr=subprocess.PIPE, stdout=subprocess.PIPE, shell=True, bufsize=0)
    
        while dd_process.poll() is None:
            time.sleep(.1)
            line = dd_process.stderr.readline()
            if not line:
                break
            print ">>>>>", line.strip()
        if dd_process.poll() is not None:
            print "Process finished."
    
    dd_win()
    

Here is wht I got as an output. I could see the `stderr` but unable to process
the transfer bytes to convert in to mega bytes:-

    
    
    C:\Users\user\Documents>python test.py
    ['dd.exe', 'if=E:test-in.iso', 'of=E:test-out.iso', '--progress']
    >>>>> rawwrite dd for windows version 0.6beta3.
    >>>>> Written by John Newbigin <jn@it.swin.edu.au>
    >>>>> This program is covered by terms of the GPL Version 2.
    >>>>>
    175,882,240  # <--- Problem output line. Unable to assign to other variable for progress calculation
    Process finished.
    

I actually do not know where the numbers are coming from. Is it from `stderr`
or buffer? If it is from `stderr` then ">>>>>" should have be printed before
the numbers. I work on Windows 7 and using windows version of dd.

Any help is appreciated.


Answer: As discussed in chat, the numbers are coming from stderr. By printing the
ascii-indexes of each character in line, we discovered that the final line
returned by readline() is `\t75,881,728 \r175,882,240 \r\n`. It looks like the
`\r` embedded in the middle of this string (which DD outputs) is confusing
your code.

