title: With OpenCV T-API, why pure GPU load causes very high CPU load?
body: <p>At first look, <a href="http://opencv.org/platforms/opencl.html" rel="nofollow">T-API</a> works great, you just have to replace the cv::Mat with cv::UMat, and your code runs many times faster.</p>
  
  <p>The problem is, while pure GPU code is running, the CPU is also at 100%.</p>
  
  <p>In this way, you can not fully utilize the CPU.</p>
  
  <p>Here is the code, tested with OpenCV 3.1.0, where there is no memory transfer, but pure GPU computations, and I expect this to cause 0% CPU load, but in practice it is 100% or more (tested on many machines).</p>
  
  <pre><code>void test_tapi_inf() {
      int sum = 0;
      cv::UMat mat(1000, 1000, CV_8UC1);
      mat.setTo(1);
  
      while (true) {
          int64 tick1 = cv::getTickCount();
          for (int i = 0; i &lt; 10000; i++) {
              sum = cv::sum(mat)[0];
          }
          int64 tick2 = cv::getTickCount();
          printf("time: %f, sum: %d\n", (tick2 - tick1) / cv::getTickFrequency(), sum);
      }
  }
  </code></pre>
  
  <p>This prints:</p>
  
  <pre><code>time: 0.961175, sum: 1000000
  time: 1.097721, sum: 1000000
  ...
  </code></pre>
  
  <p>It works 5 times faster than the cv::Mat version, loads the GPU, but, unfortunately, loads the CPU to 100% or more.</p>
  
  <p>I have noticed that a number of threads are also spawned with the cv::UMat version, having high CPU load as well.</p>
  
  <p>I suspect that the CPU is constantly polling for the GPU result, without, for example sleeping between the checks, or blocking until there is a result, causing something like a busy-loop for the CPU.</p>
  
  <p>Why is this happening? Is there a way to change this behavior in OpenCV, in order to free the CPU to perform useful work in parallel?</p>
  
  <p>Thanks!</p>