title: Why does omp_set_dynamic(1) never adjust the number of threads (in Visual C++)?
body: <p>If we look at the <a href="https://docs.microsoft.com/en-us/cpp/parallel/openmp/3-run-time-library-functions?view=vs-2019#317-omp_set_dynamic-function" rel="nofollow noreferrer">Visual C++ documentation</a> of <code>omp_set_dynamic</code>, it is literally copy-pasted from the <a href="https://www.openmp.org/wp-content/uploads/cspec20.pdf" rel="nofollow noreferrer">OMP 2.0 standard</a> (section 3.1.7 on page 39):</p>
  
  <blockquote>
    <p>If [the function argument] evaluates to a nonzero value, the number of threads that are used for executing upcoming parallel regions may be adjusted automatically by the run-time environment to best use system resources. As a consequence, the number of threads specified by the user is the maximum thread count. The number of threads in the team executing a parallel region stays fixed for the duration of that parallel region and is reported by the <code>omp_get_num_threads</code> function.</p>
  </blockquote>
  
  <p>It seems clear that <code>omp_set_dynamic(1)</code> allows the implementation to use fewer than the current maximum number of threads for a parallel region (presumably to prevent oversubscription under high loads). Any reasonable reading of this paragraph would suggest that said reduction should be observable by querying <code>omp_get_num_threads</code> inside parallel regions.</p>
  
  <p><sup>(Both documentations also show the signature as <code>void omp_set_dynamic(int dynamic_threads);</code>. It appears that "the number of threads specified by the user" does not refer to <code>dynamic_threads</code> but instead means "whatever the user specified using the remaining OpenMP interface").</sup></p>
  
  <p>However, no matter how high I push my system load under <code>omp_set_dynamic(1)</code>, the return value of <code>omp_get_num_threads</code> (queried inside the parallel regions) never changes from the maximum in my test program. Yet I can still observe clear performance differences between <code>omp_set_dynamic(1)</code> and <code>omp_set_dynamic(0)</code>.</p>
  
  <p>Here is a sample program to reproduce the issue:</p>
  
  <pre><code>#include &lt;atomic&gt;
  #include &lt;chrono&gt;
  #include &lt;iostream&gt;
  #include &lt;thread&gt;
  #include &lt;cstdlib&gt;
  #include &lt;cmath&gt;
  
  #include &lt;omp.h&gt;
  
  #define UNDER_LOAD true
  
  const int SET_DYNAMIC_TO = 1;
  
  const int REPEATS = 3000;
  const unsigned MAXCOUNT = 1000000;
  
  std::size_t threadNumSum = 0;
  std::size_t threadNumCount = 0;
  
  void oneRegion(int i)
  {
    // Pesudo-randomize the number of iterations.
    unsigned ui = static_cast&lt;unsigned&gt;(i);
    int count = static_cast&lt;int&gt;(((MAXCOUNT + 37) * (ui + 7) * ui) % MAXCOUNT);
  
  #pragma omp parallel for schedule(guided, 512)
    for (int j = 0; j &lt; count; ++j)
    {
      if (j == 0)
      {
        threadNumSum += omp_get_num_threads();
        threadNumCount++;
      }
  
      if ((j + i + count) % 16 != 0)
        continue;
  
      // Do some floating point math.
      double a = j + i;
      for (int k = 0; k &lt; 10; ++k)
        a = std::sin(i * (std::cos(a) * j + std::log(std::abs(a + count) + 1)));
  
      volatile double out = a;
    }
  }
  
  
  int main()
  {
    omp_set_dynamic(SET_DYNAMIC_TO);
  
  
  #if UNDER_LOAD
    for (int i = 0; i &lt; 10; ++i)
    {
      std::thread([]()
      {
        unsigned x = 0;
        float y = static_cast&lt;float&gt;(std::sqrt(2));
        while (true)
        {
  //#pragma omp parallel for
          for (int i = 0; i &lt; 100000; ++i)
          {
            x = x * 7 + 13;
            y = 4 * y * (1 - y);
          }
          volatile unsigned xx = x;
          volatile float yy = y;
        }
      }).detach();
    }
  #endif
  
  
    std::chrono::high_resolution_clock clk;
    auto start = clk.now();
  
    for (int i = 0; i &lt; REPEATS; ++i)
      oneRegion(i);
  
    std::cout &lt;&lt; (clk.now() - start).count() / 1000ull / 1000ull &lt;&lt; " ms for " &lt;&lt; REPEATS &lt;&lt; " iterations" &lt;&lt; std::endl;
  
    double averageThreadNum = double(threadNumSum) / threadNumCount;
    std::cout &lt;&lt; "Entered " &lt;&lt; threadNumCount &lt;&lt; " parallel regions with " &lt;&lt; averageThreadNum &lt;&lt; " threads each on average." &lt;&lt; std::endl;
  
    std::getchar();
  
    return 0;
  }
  </code></pre>
  
  <p>Compiler version: Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27024.1 for x64 </p>
  
  <p>On e.g. gcc, this program will print a significantly lower <code>averageThreadNum</code> for <code>omp_set_dynamic(1)</code> than for <code>omp_set_dynamic(0)</code>. But on MSVC, the same value is shown in both cases, despite a 30% performance difference (170s vs 230s).</p>
  
  <p>How can this be explained?</p>