title: Why can't I mmap(MAP_FIXED) the highest virtual page in a 32-bit Linux process on a 64-bit kernel?
body: <p>While attempting to test <a href="https://stackoverflow.com/questions/47702410/is-it-allowed-to-access-memory-that-spans-the-zero-boundary-in-x86#comment82366009_47702410">Is it allowed to access memory that spans the zero boundary in x86?</a> in user-space on Linux, I wrote a 32-bit test program that tries to map the low and high pages of 32-bit virtual address space.</p>
  
  <p>After <code>echo 0 | sudo tee /proc/sys/vm/mmap_min_addr</code>, I can map the zero page, but I don't know why I can't map <code>-4096</code>, i.e. <code>(void*)0xfffff000</code>, the highest page.  <strong>Why does <code>mmap2((void*)-4096)</code> return <code>-ENOMEM</code>?</strong></p>
  
  <pre><code>strace ./a.out 
  execve("./a.out", ["./a.out"], 0x7ffe08827c10 /* 65 vars */) = 0
  strace: [ Process PID=1407 runs in 32 bit mode. ]
  ....
  mmap2(0xfffff000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
  mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0
  </code></pre>
  
  <p>Also, what check is rejecting it in <a href="https://github.com/torvalds/linux/blob/master/mm/mmap.c" rel="noreferrer"><code>linux/mm/mmap.c</code></a>, and why is it designed that way?  Is this part of making sure that creating a pointer to one-past-an-object doesn't <a href="https://stackoverflow.com/questions/47687805/should-pointer-comparisons-be-signed-or-unsigned-in-64-bit-x86">wrap around and break pointer comparisons</a>, because ISO C and C++ allow creating a pointer to one-past-the-end, but otherwise not outside of objects.</p>
  
  <hr>
  
  <p>I'm running under a 64-bit kernel (4.12.8-2-ARCH on Arch Linux), so 32-bit user space has the entire 4GiB available.  (Unlike 64-bit code on a 64-bit kernel, or with a 32-bit kernel where the 2:2 or 3:1 user/kernel split would make the high page a kernel address.)</p>
  
  <p>I haven't tried from a minimal static executable (no CRT startup or libc, just asm) because I don't think that would make a difference.  None of the CRT startup system calls look suspicious.</p>
  
  <hr>
  
  <p>While stopped at a breakpoint, I checked <code>/proc/PID/maps</code>.  The top page isn't already in use.  The stack includes the 2nd highest page, but the top page is unmapped.</p>
  
  <pre><code>00000000-00001000 rw-p 00000000 00:00 0             ### the mmap(0) result
  08048000-08049000 r-xp 00000000 00:15 3120510                 /home/peter/src/SO/a.out
  08049000-0804a000 r--p 00000000 00:15 3120510                 /home/peter/src/SO/a.out
  0804a000-0804b000 rw-p 00001000 00:15 3120510                 /home/peter/src/SO/a.out
  f7d81000-f7f3a000 r-xp 00000000 00:15 1511498                 /usr/lib32/libc-2.25.so
  f7f3a000-f7f3c000 r--p 001b8000 00:15 1511498                 /usr/lib32/libc-2.25.so
  f7f3c000-f7f3d000 rw-p 001ba000 00:15 1511498                 /usr/lib32/libc-2.25.so
  f7f3d000-f7f40000 rw-p 00000000 00:00 0 
  f7f7c000-f7f7e000 rw-p 00000000 00:00 0 
  f7f7e000-f7f81000 r--p 00000000 00:00 0                       [vvar]
  f7f81000-f7f83000 r-xp 00000000 00:00 0                       [vdso]
  f7f83000-f7fa6000 r-xp 00000000 00:15 1511499                 /usr/lib32/ld-2.25.so
  f7fa6000-f7fa7000 r--p 00022000 00:15 1511499                 /usr/lib32/ld-2.25.so
  f7fa7000-f7fa8000 rw-p 00023000 00:15 1511499                 /usr/lib32/ld-2.25.so
  fffdd000-ffffe000 rw-p 00000000 00:00 0                       [stack]
  </code></pre>
  
  <p>Are there VMA regions that don't show up in <code>maps</code> that still convince the kernel to reject the address?  I looked at the occurrences of <code>ENOMEM</code> in <code>linux/mm/mmapc.</code>, but it's a lot of code to read so maybe I missed something.  Something that reserves some range of high addresses, or because it's next to the stack?</p>
  
  <p>Making the system calls in the other order doesn't help (but PAGE_ALIGN and similar macros are written carefully to avoid wrapping around before masking, so that wasn't likely anyway.)</p>
  
  <hr>
  
  <p>Full source, compiled with <code>gcc -O3 -fno-pie -no-pie -m32 address-wrap.c</code>:</p>
  
  <pre class="lang-c prettyprint-override"><code>#include &lt;sys/mman.h&gt;
  
  //void *mmap(void *addr, size_t len, int prot, int flags,
  //           int fildes, off_t off);
  
  int main(void) {
      volatile unsigned *high =
          mmap((void*)-4096L, 4096, PROT_READ | PROT_WRITE,
               MAP_FIXED|MAP_PRIVATE|MAP_ANONYMOUS,
               -1, 0);
      volatile unsigned *zeropage =
          mmap((void*)0, 4096, PROT_READ | PROT_WRITE,
               MAP_FIXED|MAP_PRIVATE|MAP_ANONYMOUS,
               -1, 0);
  
  
      return (high == MAP_FAILED) ? 2 : *high;
  }
  </code></pre>
  
  <p>(I left out the part that tried to deref <code>(int*)-2</code> because it just segfaults when mmap fails.)</p>