title: fork() leaking? Taking longer and longer to fork a simple process
body: <p>I have a system in which two identical processes are run (let's call them replicas). When signaled, a replica will duplicate itself by using the <code>fork()</code> call. A third process selects one of the processes to kill randomly, and then signals the other to create a replacement. Functionally, the system works well; it can kill / respawn replicas all day except for the performance issue.</p>
  
  <p>The <code>fork()</code> call is taking longer and longer. The following is the simplest setup that still displays the problem. The timing be is displayed in the graph below: <img src="https://i.stack.imgur.com/ggOzC.png" alt="fork timing"></p>
  
  <p>The replica's code is the following:</p>
  
  <pre><code>void restartHandler(int signo) {
  // fork
    timestamp_t last = generate_timestamp();
    pid_t currentPID = fork();
  
  
    if (currentPID &gt;= 0) { // Successful fork
      if (currentPID == 0) { // Child process
        timestamp_t current = generate_timestamp();
        printf("%lld\n", current - last);
  
        // unblock the signal
        sigset_t signal_set;
        sigemptyset(&amp;signal_set);
        sigaddset(&amp;signal_set, SIGUSR1);
        sigprocmask(SIG_UNBLOCK, &amp;signal_set, NULL);
  
        return;
      } else {   // Parent just returns
        waitpid(-1, NULL, WNOHANG);
        return;
      }
    } else {
      printf("Fork error!\n");
      return;
    }
  }
  
  int main(int argc, const char **argv) {
    if (signal(SIGUSR1, restartHandler) == SIG_ERR) {
      perror("Failed to register the restart handler");
      return -1;
    }
  
    while(1) {
      sleep(1);
    }
  
    return 0;
  }
  </code></pre>
  
  <p>The longer the system runs, the worse it gets.</p>
  
  <p>Sorry to lack a specific question, but does anyone have any idea / clues as to what is going on? It seems to me that there is a resource leak in the kernel (thus the linux-kernel tag), but I don't know where where to start looking.</p>
  
  <p>What I have tried:</p>
  
  <ul>
  <li>Tried <a href="https://www.kernel.org/doc/Documentation/kmemleak.txt" rel="noreferrer">kmemleak</a>, which did not catch anything. This implies that if there is some memory "leak" that it is still reachable.</li>
  <li><code>/proc/&lt;pid&gt;/maps</code> is not growing.</li>
  <li>Currently running the 3.14 kernel with RT patch (note this happens with non-rt and rt processes), and have also tried on 3.2.</li>
  <li>zombie processes are not an issue. I have tried a version in which I setup another process as a subreaper using <a href="http://man7.org/linux/man-pages/man2/prctl.2.html" rel="noreferrer">prctl</a></li>
  <li>I first noticed this slowdown in a system in which the timing measurements are being down outside of the restarted process; same behavior.</li>
  </ul>
  
  <p>Any hints? Anything I can provide to help? Thanks!</p>