subprocess.Popen() has inconsistent behavior between Eclipse/PyCharm and terminal execution
Question: The problem I'm having is with Eclipse/PyCharm interpreting the results of
subprocess's Popen() differently from a standard terminal. All are using
python2.6.1 on OSX.

Here's a simple example script:

    
    
    import subprocess
    
    args = ["/usr/bin/which", "git"]
    print "Will execute %s" % " ".join(args)
    try:
      p = subprocess.Popen(["/usr/bin/which", "git"], shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
      # tuple of StdOut, StdErr is the responses, so ..
      ret = p.communicate()
      if ret[0] == '' and ret[1] <> '':
        msg = "cmd %s failed: %s" % (fullcmd, ret[1])
        if fail_on_error:
          raise NameError(msg)
    except OSError, e:
      print >>sys.stderr, "Execution failed:", e
    

With a standard terminal, the line:

    
    
    ret = p.communicate()
    

gives me:

    
    
    (Pdb) print ret
    ('/usr/local/bin/git\n', '')
    

Eclipse and PyCharm give me an empty tuple:

    
    
    ret = {tuple} ('','')
    

Changing the shell= value does not solve the problem either. On the terminal,
setting shell=True, and passing the command in altogether (i.e.,
args=["/usr/bin/which git"]) gives me the same result: ret =
('/usr/local/bin/git\n', ''). And Eclipse/PyCharm both give me an empty tuple.

Any ideas on what I could be doing wrong?


Answer: Ok, found the problem, and it's an important thing to keep in mind when using
an IDE in a Unix-type environment. IDE's operate under a different environment
context than the terminal user (duh, right?!). I was not considering that the
subprocess was using a different environment than the context that I have for
my terminal (my terminal has bash_profile set to have more things in PATH).

This is easily verified by changing the script as follows:

    
    
    import subprocess
    args = ["/usr/bin/which", "git"]
    print "Current path is %s" % os.path.expandvars("$PATH")
    try:
      p = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
      # tuple of StdOut, StdErr is the responses, so ..
      out, err = p.communicate()
      if err:
        msg = "cmd %s failed: %s" % (fullcmd, err)
    except OSError, e:
      print >>sys.stderr, "Execution failed:", e
    

Under the terminal, the path includes /usr/local/bin. Under the IDE it does
not!

This is an important gotcha for me - always remember about environments!

