title: Python subprocess.Popen erroring with OSError: [Errno 12] Cannot allocate memory after period of time
body: <p><strong>Note</strong>: This question has been re-asked with a summary of all debugging attempts <a href="https://stackoverflow.com/questions/1367373/python-subprocess-popen-oserror-errno-12-cannot-allocate-memory">here</a>.</p>
  
  <hr>
  
  <p>I have a Python script that is running as a background process executing every 60 seconds. Part of that is a call to <a href="http://docs.python.org/library/subprocess.html" rel="nofollow noreferrer">subprocess.Popen</a> to get the output of <a href="http://linuxcommand.org/man_pages/ps1.html" rel="nofollow noreferrer">ps</a>. </p>
  
  <pre><code>ps = subprocess.Popen(['ps', 'aux'], stdout=subprocess.PIPE).communicate()[0]
  </code></pre>
  
  <p>After running for a few days, the call is erroring with:</p>
  
  <pre>
  File "/home/admin/sd-agent/checks.py", line 436, in getProcesses
  File "/usr/lib/python2.4/subprocess.py", line 533, in __init__
  File "/usr/lib/python2.4/subprocess.py", line 835, in _get_handles
  OSError: [Errno 12] Cannot allocate memory
  </pre>
  
  <p>However the output of <a href="http://www.linuxcommand.org/man_pages/free1.html" rel="nofollow noreferrer">free</a> on the server is:</p>
  
  <pre>
  $ free -m
                    total       used       free     shared     buffers    cached
  Mem:                894        345        549          0          0          0
  -/+ buffers/cache:  345        549
  Swap:                 0          0          0
  </pre>
  
  <p>I have searched around for the problem and found <a href="http://www.zenoss.com/community/wiki/common-error-messages-and-solutions/oserror-errno-12-cannot-allocate-memory-in-popen2.py/" rel="nofollow noreferrer">this article</a> which says:</p>
  
  <p>Solution is to add more swap space to your server. When the kernel is forking to start the modeler or discovery process, it first ensures there's enough space available on the swap store the new process if needed.</p>
  
  <p>I note that there is no available swap from the free output above. Is this likely to be the problem and/or what other solutions might there be?</p>
  
  <p><strong>Update 13th Aug 09</strong> The code above is called every 60 seconds as part of a series of monitoring functions. The process is daemonized and the check is scheduled using <a href="http://docs.python.org/library/sched.html" rel="nofollow noreferrer">sched</a>. The specific code for the above function is:</p>
  
  <pre><code>def getProcesses(self):
      self.checksLogger.debug('getProcesses: start')
  
      # Memory logging (case 27152)
      if self.agentConfig['debugMode'] and sys.platform == 'linux2':
          mem = subprocess.Popen(['free', '-m'], stdout=subprocess.PIPE).communicate()[0]
          self.checksLogger.debug('getProcesses: memory before Popen - ' + str(mem))
  
      # Get output from ps
      try:
          self.checksLogger.debug('getProcesses: attempting Popen')
  
          ps = subprocess.Popen(['ps', 'aux'], stdout=subprocess.PIPE).communicate()[0]
  
      except Exception, e:
          import traceback
          self.checksLogger.error('getProcesses: exception = ' + traceback.format_exc())
          return False
  
      self.checksLogger.debug('getProcesses: Popen success, parsing')
  
      # Memory logging (case 27152)
      if self.agentConfig['debugMode'] and sys.platform == 'linux2':
          mem = subprocess.Popen(['free', '-m'], stdout=subprocess.PIPE).communicate()[0]
          self.checksLogger.debug('getProcesses: memory after Popen - ' + str(mem))
  
      # Split out each process
      processLines = ps.split('\n')
  
      del processLines[0] # Removes the headers
      processLines.pop() # Removes a trailing empty line
  
      processes = []
  
      self.checksLogger.debug('getProcesses: Popen success, parsing, looping')
  
      for line in processLines:
          line = line.split(None, 10)
          processes.append(line)
  
      self.checksLogger.debug('getProcesses: completed, returning')
  
      return processes
  </code></pre>
  
  <p>This is part of a bigger class called checks which is initialised once when the daemon is started.</p>
  
  <p>The entire checks class can be found at <a href="http://github.com/dmytton/sd-agent/blob/82f5ff9203e54d2adeee8cfed704d09e3f00e8eb/checks.py" rel="nofollow noreferrer">http://github.com/dmytton/sd-agent/blob/82f5ff9203e54d2adeee8cfed704d09e3f00e8eb/checks.py</a> with the getProcesses function defined from line 442. This is called by doChecks() starting at line 520.</p>