title: What is the maximum number of threads that std::async will create and execute asynchronously?
body: <p>I have a large number (>>100K) of tasks with very high latency (minutes) and very little resource consumption. Potentially they could all be executed in parallel and I was considering using <code>std::async</code> to generate one future for each task.</p>
  
  <p><strong>My question is:</strong> what is the maximum number of threads that std::async will create and execute asynchronously? (using g++ 6.x on Ubuntu 16-xx or CentOs 7.x - x86_64)</p>
  
  <p>It is important for me to get that number right because if I do not have enough tasks actually running (waiting) in parallel the cumulative cost of latency will be very high.</p>
  
  <p>To get to an answer, I started by checking the capabilities of the system:</p>
  
  <pre><code>bob@vb:~/programming/cxx/async$ ulimit -u
  43735
  bob@vb:~/programming/cxx/async$ cat /proc/sys/kernel/threads-max 
  87470
  </code></pre>
  
  <p>From these numbers, I was expecting to be able to get in the order of 43K threads running (mostly waiting) in parallel. To verify that, I wrote the program below to check the number of distinct thread ids and the time required to call 100K <code>std::async</code> with an empty task:</p>
  
  <pre><code>#include &lt;thread&gt;
  #include &lt;future&gt;
  #include &lt;iostream&gt;
  #include &lt;vector&gt;
  #include &lt;algorithm&gt;
  #include &lt;chrono&gt;
  #include &lt;string&gt;
  
  std::thread::id foo()
  {
      using namespace std::chrono_literals;
      //std::this_thread::sleep_for(2s);
      return std::this_thread::get_id();
  }
  
  int main(int argc, char **argv)
  {
      if (2 != argc) exit(1);
      const size_t COUNT = std::stoi(argv[1]);
      std::vector&lt;decltype(std::async(foo))&gt; futures;
      futures.reserve(COUNT);
      while (futures.capacity() != futures.size())
      { 
          futures.push_back(std::async(foo));
      } 
      std::vector&lt;std::thread::id&gt; ids;
      ids.reserve(futures.size());
      for (auto &amp;f: futures)
      { 
          ids.push_back(f.get());
      } 
      std::sort(ids.begin(), ids.end());
      const auto end = std::unique(ids.begin(), ids.end());
      ids.erase(end, ids.end());
      std:: cerr &lt;&lt; "COUNT: " &lt;&lt; COUNT &lt;&lt; ": ids.size(): " &lt;&lt; ids.size() &lt;&lt; std::endl;
  }
  </code></pre>
  
  <p>The time was fine but the number of distinct thread ids was much less than expected (32748 instead of 43735):</p>
  
  <pre><code>bob@vb:~/programming/cxx/async$ /usr/bin/time -f "%E" ./testAsync 100000
  COUNT: 100000: ids.size(): 32748
  0:03.29
  </code></pre>
  
  <p>Then I un-commented the sleep line in <code>foo</code> to add a 2s sleeping time. The resulting timings are consistent with 2s up to 10K tasks or so, but at some point beyond that, some tasks end-up sharing the same thread id and the elapsed time increases by 2s for each additional task:</p>
  
  <pre><code>bob@vb:~/programming/cxx/async$ /usr/bin/time -f "%E" ./testAsync 10056
  COUNT: 10056: ids.size(): 10056
  0:02.24
  bob@vb:~/programming/cxx/async$ /usr/bin/time -f "%E" ./testAsync 10057
  COUNT: 10057: ids.size(): 10057
  0:04.27
  bob@vb:~/programming/cxx/async$ /usr/bin/time -f "%E" ./testAsync 10058
  COUNT: 10058: ids.size(): 10057
  0:06.28
  bob@vb:~/programming/cxx/async$ ps -eT | wc -l
  277
  </code></pre>
  
  <p>So, it looks that for my problem, on this system, the limit is in the order of 10K. I checked on another system and the limit was in the order of 4K.</p>
  
  <p>I can't figure out:</p>
  
  <ul>
  <li>why these values are so small </li>
  <li>how to predict these values from the specs of the system</li>
  </ul>