title: Stack allocation for C++ green threads
body: <p>I'm doing some research in C++ green threads, mostly <code>boost::coroutine2</code> and similar POSIX functions like <code>makecontext()/swapcontext()</code>, and planning to implement a C++ green thread library on top of <code>boost::coroutine2</code>. Both require the user code to allocate a stack for every new function/coroutine. </p>
  
  <p>My target platform is x64/Linux. I want my green thread library to be suitable for general use, so the stacks should expand as required (a reasonable upper limit is fine, e.g. 10MB), it would be great if the stacks could shrink when too much memory is unused (not required). I haven't figured out an appropriate algorithm to allocate stacks.</p>
  
  <p>After some googling, I figured out a few options myself:</p>
  
  <ol>
  <li>use split stack implemented by the compiler (gcc -fsplit-stack), but split stack has performance overhead. Go has already moved away from split stack due to performance reasons.</li>
  <li>allocate a large chunk of memory with <code>mmap()</code> hope the kernel is smart enough to leave the physical memory unallocated and allocate only when the stacks are accessed. In this case, we are at the mercy of the kernel.</li>
  <li>reserve a large memory space with <code>mmap(PROT_NONE)</code> and setup a <code>SIGSEGV</code> signal handler. In the signal handler, when the <code>SIGSEGV</code> is caused by stack access (the accessed memory is inside the large memory space reserved), allocate needed memory with <code>mmap(PROT_READ | PROT_WRITE)</code>. Here is the problem for this approach: <code>mmap()</code> isn't asynchronous safe, cannot be called inside a signal handler. It still can be implemented, <em>very tricky</em> though: create another thread during program startup for memory allocation, and use <code>pipe() + read()/write()</code> to send memory allocation information from the signal handler to the thread. </li>
  </ol>
  
  <p>A few more questions about option 3:</p>
  
  <ol>
  <li>I'm not sure the performance overhead of this approach, how well/bad the kernel/CPU performs when the memory space is extremely fragmented due to thousands of <code>mmap()</code> call ?</li>
  <li>Is this approach correct if the unallocated memory is accessed in kernel space ? e.g. when <code>read()</code> is called ?</li>
  </ol>
  
  <p>Are there any other (better) options for stack allocation for green threads ? How are green thread stacks allocated in other implementations, e.g. Go/Java ?</p>