title: Avoiding the production of zombie processes in C++
body: <p>Very strange bug, perhaps someone will see something I'm missing.</p>
  
  <p>I have a C++ program which forks off a bash shell, and then passes commands to it. </p>
  
  <p>Periodically, the commands will contain nonsense and the bash process will hang. I detect this using semtimedwait, and then run a little function like this:</p>
  
  <pre><code>if (kill(*bash_pid, SIGKILL)) {
      cerr &lt;&lt; "Error sending SIGKILL to the bash process!" &lt;&lt; endl;
      exit(1); 
  } else {
      // collect exit status
      long counter = 0;
      do {
          pid = waitpid(*bash_pid, &amp;status, WNOHANG);
          if (pid == 0) { // status not available yet
              sleep(1);
          }
          if(counter++ &gt; 5){
              cerr &lt;&lt; "ERROR: Bash child process ignored SIGKILL &gt;5 sec!" &lt;&lt; endl;
          }
      } while (pid != *bash_pid &amp;&amp; pid != -1);
      if(pid == -1){
          cerr &lt;&lt; "Failed to clean up zombie bash process!" &lt;&lt; endl;
          exit(1);
      }
  
      // re-initialized bash process
      *bash_pid = init_bash();
   }
  </code></pre>
  
  <p>Assuming I understand the workings of waitpid correctly, this should first send SIGKILL to the shell, and then essentially sit in a spinlock, trying to reap the resulting process. Eventually, it succeeds and then a new bash process is started with init_bash().</p>
  
  <p>At least, that's what should happen. Instead, the child process's exit status is never collected, and it continues to exist as a zombie process. In spite of this, the parent <em>does</em> exit the loop and manages to restart the bash process, and continues with normal execution. Eventually too many zombies are generated and the system runs out of pids. </p>
  
  <p>Additionally:</p>
  
  <ul>
  <li>Fork is called in exactly one place in the program, inside init_bash.</li>
  <li>Checks prevent init_bash from being called except once at the program's start and after a call to the function above.</li>
  </ul>
  
  <p>Thoughts? </p>