title: Avoiding a potential deadlock / memory leak in a multithreaded application
body: <p><strong>Short version</strong>:</p>
  
  <p>How to deal with the non-atomicity of spawning a group of threads, running some custom (unspecified at the time of implementation) callback? Several possible solutions are described below, it seems like using a thread pool is the only good solution. Is there a standard way to deal with it? No need to post complete C++ solutions, pseudocode or a brief description will suffice. Performance is an important aspect here.</p>
  
  <p>Although it may seem trivial, I believe the below fragment of code occurs in many of the existing applications, many (beginning, probably also some of the advanced) programmers may write similar constructs without even realizing the dangers. And the problems are the same for pthread / C++11 <code>std::thread</code> / WinAPI and probably many other low-level multithreading libraries. Therefore it is an important question.</p>
  
  <p><strong>Long version</strong>:</p>
  
  <p>I'm designing some multithreaded application and I decided to make an utility function, where several threads are spawned. This is perhaps a quite common code, it comes up in many of my applications (unless they are using OpenMP):</p>
  
  <pre><code>void ParallelCall(void (*function)(int, int), int numThreads)
  {
      Thread *threads = new Thread[numThreads - 1];
      for(int i = 1; i &lt; numThreads; ++ i) {
          if(threads[i - 1].start(&amp;function, i, numThreads)) // this might fail
              abort(); // what now?
      }
  
      (*function)(0, numThreads);
      // use the calling thread as thread 0
  
      for(int i = 1; i &lt; numThreads; ++ i)
          threads[i - 1].join();
      delete[] threads;
  }
  </code></pre>
  
  <p>This is more of a pseudo code for illustration of the problem. A bunch of threads are being created and spawned (the <code>Thread</code> object wraps a pthread thread). Then they do something and finally they are joined.</p>
  
  <p>Now the problem: what if, for whatever reason, some of the threads fail to start (may be resource exhaustion or a per-user limit)? I know how to detect that it happened, but I'm not sure how to handle it.</p>
  
  <p>I guess I should wait for the successfully started threads to finish and then throw an exception. However, if the code in <code>function</code> contains some synchronization (such as a barrier), this can easily result in a deadlock because the rest of the expected threads will never spawn.</p>
  
  <p>Alternately, I could just throw an exception right away, ignoring the running threads, but then I leave the wrapper objects allocated, causing a memory leak (and also never joining the spawned threads).</p>
  
  <p>Doing stuff like killing the running threads doesn't seem to be a good idea (I'm frankly not quite sure what is the result of forcibly killing a thread of a multithreaded application - seems like the memory will be left in undefined state, which is mostly kind of hard to handle - and could on itself lead to more memory leaks if the callback <code>function</code> allocates memory).</p>
  
  <p>Inserting a wait for all the threads to start before letting them enter the callback <code>function</code> seems unbearable performance-wise (although it would solve the issue easily). Another option would be to have a pool of spawned threads with associated FIFOs, waiting for tasks, but there is a problem with the number of threads (I would spawn as many threads as there are logical CPUs, but then what if <code>numThreads</code> is larger? I would be essentially reimplementing OS' scheduler in my code).</p>
  
  <p>How is this commonly solved? Is there a better way? If not, is a potential (depending on what's in the callback <code>function</code>) deadlock better than a memory leak?</p>