title: How do I use `setrlimit` to limit memory usage? RLIMIT_AS kills too soon; RLIMIT_DATA, RLIMIT_RSS, RLIMIT_STACK kill not at all
body: <p>I'm trying to use <code>setrlimit</code> to limit my memory usage on a Linux system, in order to stop my process from crashing the machine (my code was crashing nodes on a high performance cluster, because a bug led to memory consumption in excess of 100 GiB).  I can't seem to find the correct resource to pass to <code>setrlimit</code>; I think it should be resident, which <a href="https://stackoverflow.com/questions/3043709/resident-set-size-rss-limit-has-no-effect">cannot be limited with setrlimit</a>, but I am confused by resident, heap, stack.  In the code below; if I uncomment only <code>RLIMIT_AS</code>, the code fails with <code>MemoryError</code> at <code>numpy.ones(shape=(1000, 1000, 10), dtype="f8")</code> even though that array should be only 80 MB.  If I uncomment only <code>RLIMIT_DATA</code>, <code>RLIMIT_RSS</code>, or <code>RLIMIT_STACK</code> both arrays get allocated successfully, even though the total memory usage is 2 GB, or twice the desired maximum.</p>
  
  <p>I would like to make make my program fail (no matter how) as soon as it tries to allocate too much RAM.  Why do none of <code>RLIMIT_DATA</code>, <code>RLIMIT_RSS</code>, <code>RLIMIT_STACK</code> and <code>RLIMIT_AS</code> do what I mean, and what is the correct resource to pass to <code>setrlimit</code>?</p>
  
  <pre><code>$ cat mwe.py 
  #!/usr/bin/env python3.5
  
  import resource
  import numpy
  
  #rsrc = resource.RLIMIT_AS
  #rsrc = resource.RLIMIT_DATA
  #rsrc = resource.RLIMIT_RSS
  #rsrc = resource.RLIMIT_STACK
  
  soft, hard = resource.getrlimit(rsrc)
  print("Limit starts as:", soft, hard)
  
  resource.setrlimit(rsrc, (1e9, 1e9))
  
  soft, hard = resource.getrlimit(rsrc)
  print("Limit is now:", soft, hard)
  print("Allocating 80 KB, should certainly work")
  M1 = numpy.arange(100*100, dtype="u8")
  
  print("Allocating 80 MB, should work")
  M2 = numpy.arange(1000*1000*10, dtype="u8")
  
  print("Allocating 2 GB, should fail")
  M3 = numpy.arange(1000*1000*250, dtype="u8")
  
  input("Still here…")
  </code></pre>
  
  <p>Output with the <code>RLIMIT_AS</code> line uncommented:</p>
  
  <pre><code>$ ./mwe.py 
  Limit starts as: -1 -1
  Limit is now: 1000000000 -1
  Allocating 80 KB, should certainly work
  Allocating 80 MB, should work
  Traceback (most recent call last):
    File "./mwe.py", line 22, in &lt;module&gt;
      M2 = numpy.arange(1000*1000*10, dtype="u8")
  MemoryError
  </code></pre>
  
  <p>Output when running with any of the other ones uncommented:</p>
  
  <pre><code>$ ./mwe.py 
  Limit starts as: -1 -1
  Limit is now: 1000000000 -1
  Allocating 80 KB, should certainly work
  Allocating 80 MB, should work
  Allocating 2 GB, should fail
  Still here…
  </code></pre>
  
  <p>At the final line, <code>top</code> reports that my process is using 379 GB VIRT, 2.0 GB RES.</p>
  
  <hr>
  
  <p>System details:</p>
  
  <pre><code>$ uname -a
  Linux host.somewhere.ac.uk 2.6.32-573.3.1.el6.x86_64 #1 SMP Mon Aug 10 09:44:54 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
  
  $ cat /etc/redhat-release 
  Red Hat Enterprise Linux Server release 6.7 (Santiago)
  
  $ free -h
               total       used       free     shared    buffers     cached
  Mem:          2.0T       1.9T        37G       1.6G       3.4G       1.8T
  -/+ buffers/cache:        88G       1.9T 
  Swap:         464G       4.8M       464G 
  
  $ python3.5 --version
  Python 3.5.0
  
  $ python3.5 -c "import numpy; print(numpy.__version__)"
  1.11.1
  </code></pre>