title: Python Catching Popen Output
body: <p>I'm experiencing some strange behavior with my Python script exiting without any error messages. Based on my debugging it is happening around a popen() call to scp a file to a server.</p>
  
  <p>The code was (I was not the original author):</p>
  
  <pre><code>logMessage(LEVEL_INFO, "copyto is " + copyto)
  pid = Popen(["scp", "-i", "/root/.ssh/id_rsa", "/usr/gpsw/gpslog" + self.node_addr, copyto], stdout=PIPE)
  __s = pid.communicate()[0]
  logMessage(LEVEL_INFO, "GPS log SCP complete")
  </code></pre>
  
  <p>In an attempt to debug I enhanced it to:</p>
  
  <pre><code>logMessage(LEVEL_INFO, "copyto is " + copyto)
  pid = Popen(["scp", "-i", "/root/.ssh/id_rsa", "/usr/gpsw/gpslog" + self.node_addr, copyto], stdout=PIPE, stderr=PIPE)
  out, err = pid.communicate()
  if out:
      print "[" + self.node_addr + "] stdout of pid: " + str(out)
  if err:
      print "[" + self.node_addr + "] stdout of pid: " + str(err)
  print "[" + self.node_addr + "] returncode of pid: " + str(pid.returncode)
  logMessage(LEVEL_INFO, "GPS log SCP complete")
  </code></pre>
  
  <p>Here is my console output: 
  (The pattern should be 5dda77, 5dd9fa, 5dda0d repeatedly. This is based on physical events)</p>
  
  <pre><code>[5dda77] returncode of pid: 0
  [5dd9fa] returncode of pid: 0
  [5dda0d] returncode of pid: 0
  [5dda77] returncode of pid: 0
  [5dd9fa] returncode of pid: 0
  [5dda0d] returncode of pid: 0
  # (the script exited and I'm back at the prompt)
  </code></pre>
  
  <p>And here is my log output:</p>
  
  <pre><code>INFO copyto is root@192.168.20.1:/usr/scu/datafiles/gpslog5dda77
  INFO GPS log SCP complete
  INFO copyto is root@192.168.20.1:/usr/scu/datafiles/gpslog5dd9fa
  INFO GPS log SCP complete
  INFO copyto is root@192.168.20.1:/usr/scu/datafiles/gpslog5dda0d
  INFO GPS log SCP complete
  INFO copyto is root@192.168.20.1:/usr/scu/datafiles/gpslog5dda77
  INFO GPS log SCP complete
  INFO copyto is root@192.168.20.1:/usr/scu/datafiles/gpslog5dd9fa
  INFO GPS log SCP complete
  INFO copyto is root@192.168.20.1:/usr/scu/datafiles/gpslog5dda0d
  INFO GPS log SCP complete
  INFO copyto is root@192.168.20.1:/usr/scu/datafiles/gpslog5dda77
  </code></pre>
  
  <p>So based on the log and output data. I believe something is going wrong during the SCP because the Python script crashes before it logs "GPS log SCP complete". What is interesting though is I see on the server end the file was copied entirely. So two questions:</p>
  
  <ol>
  <li>Am I using popen incorrectly?</li>
  <li>Why am I not seeing any error messages in stderr?</li>
  </ol>
  
  <p>Thanks </p>
  
  <p><strong>EDIT:</strong>
  This script should <em>never</em> exit. There is no "normal exit" of the process. It should run indefinitely servicing requests from the server to retrieve data from GPS nodes that are available. </p>
  
  <p>The main loop code is:</p>
  
  <pre><code>stop_flags = 0
  
  ... (API for server and GPS nodes to interact with)
  
  def main():
  
      ... (System initialization)
  
      while stop_flags == 0:
          # listen for messages
          xmlrpcserver.handle_request()
          comm.poll()
      if stop_flags == STOP_FLAG_RESTART:
          # suppress Pylint warning for reimport of Popen,PIPE
          # pylint: disable-msg=W0404
          from subprocess import Popen, PIPE
          # use this instead of call to suppress output
          pid = Popen(["/etc/init.d/S999snap",
                      "restart"],
                      stdout=PIPE)
          __s = pid.communicate()[0]
      if stop_flags == STOP_FLAG_REBOOT:
          # suppress Pylint warning for reimport of Popen,PIPE
          # pylint: disable-msg=W0404
          from subprocess import Popen, PIPE
          # use this instead of call to suppress output
          pid = Popen(["reboot"],
                      stdout=PIPE)
          __s = pid.communicate()[0]
  
  if __name__ == '__main__':
      main()
  </code></pre>