How to separate warnings from errors found in stderr when using Popen.communicate?
Question: I used Python's
[`subprocess.Popen`](https://docs.python.org/3/library/subprocess.html#subprocess.Popen)
to execute a command and capture its output:

    
    
    p = Popen(cmd, stdout=PIPE, stderr=PIPE,shell=True)
    stdout, stderr = p.communicate()
    

I want to use `stderr` to tell users when there was an error and exit my
script:

    
    
    if stderr !='':
        return {'error':stderr}
    

But now I've found that `stderr` can contain warnings that could be safely
ignored, so my script should not exit, but continue to finish the job.

Is there a way to separate warnings from errors in `stderr`?


Answer: univerio is correct, in that there is no specific meaning for any bytes you
discover in `stderr`... think of it as "standard _not_ -output" instead of
"standard error". However, on most operating systems, you can use the
process's exit status (or "return code") to skip most of the progress bars and
other non-error output.

A `Popen` object has a field called
[`returncode`](https://docs.python.org/3/library/subprocess.html#subprocess.Popen.returncode),
which is used to store whatever value the subprocess returned when it exited.
This value is `None` until 1) the process terminates, and 2) you collect its
exit status with either the
[`poll`](https://docs.python.org/3/library/subprocess.html#subprocess.Popen.poll)
or
[`wait`](https://docs.python.org/3/library/subprocess.html#subprocess.Popen.wait)
methods (at least on Unix-like systems). Since you're using
[`communicate`](https://docs.python.org/3/library/subprocess.html#subprocess.Popen.communicate),
which _always_ does both 1 and 2, `p.returncode` should always be an integer
by the time you care about it.

As a general rule, a `0` exit status indicates success, while any other value
indicates failure. If you trust the programs you're calling to return proper
values, you can use this to skip most of the junk output on `stderr`:

    
    
    # ...same as before...
    stdout, stderr = p.communicate()
    if p.returncode and stderr:
        return {'error': stderr}
    

If the bytes found in `stderr` weren't important enough to produce a non-`0`
exit status, they're not important enough for you to report, either.

To test this, you can write a few tiny scripts that produce `stderr` output
and then [`exit`](https://docs.python.org/3/library/sys.html#sys.exit), either
successfully or not.

`warnings.py`:

    
    
    import sys
    print('This is spam on stderr.', file=sys.stderr)
    sys.exit(0)
    

`errors.py`:

    
    
    import sys
    print('This is a real error message.', file=sys.stderr)
    sys.exit(1)
    

This still leaves the task of separating spinning batons and other progress-
report spam from actual error messages, but you'll only have to do that for
processes that failed... and maybe not even then, since the "not dead yet!"
messages might actually be useful in those cases.

PS: In Python 3, `stdout` and `stderr` will be `bytes` objects, so you'll want
to [`decode`](https://docs.python.org/3/library/stdtypes.html#bytes.decode)
them before treating them like strings.

