title: RenderScript speedup 10x when forcing default CPU implementation
body: <p>I have implemented a CNN in RenderScript, described in a <a href="https://stackoverflow.com/questions/37080673/how-to-do-correct-timing-of-android-renderscript-code-on-nvidia-shield">previous question</a> which spawned this one. Basically, when running</p>
  
  <pre><code>adb shell setprop debug.rs.default-CPU-driver 1
  </code></pre>
  
  <p>there is a 10x speedup on both Nvidia Shield and Nexus 7. The average computation time goes from around 50ms to 5ms, the test app goes from around 50fps to 130 or more. There are two convolution algorithms:    </p>
  
  <p>(1) moving kernel<br>
   (2) im2col and GEMM from RenderScriptIntrinsicsBLAS.    </p>
  
  <p>Both experience similar speedup. The question is: why is this happening and can this effect be instantiated from the code in a predictable way? And is detailed information about this available somewhere?  </p>
  
  <p>Edit:  </p>
  
  <p>As per suggestions below, I verified the use of finish() and copyTo(), here is a breakdown of the procedure. The speedup reported is AFTER the call to copyTo() but without finish(). Uncommenting finish() adds about 1ms to the time.</p>
  
  <pre><code>double forwardTime = 0;
  long t = System.currentTimeMillis();
  //double t = SystemClock.elapsedRealtime(); // makes no difference
  for (Layer a : layers) {
      blob = a.forward(blob);
  }
  mRS.finish();   // adds about 1ms to measured time 
  
  blob.copyTo(outbuf);
  forwardTime = System.currentTimeMillis() - t;​
  </code></pre>
  
  <p>Maybe this is unrelated, but on the NVIDIA Shield I get an error message at startup which disappears when running with adb shell setprop debug.rs.default-CPU-driver 1  </p>
  
  <pre><code>E/Renderscript: rsAssert failed: 0, in vendor/nvidia/tegra/compute/rs/driver/nv/rsdNvBcc.cpp
  </code></pre>
  
  <p>I'm setting compileSdkVersion, minSdkVersion and targetSdkVersion to 23 right now, with buildToolsVersion "23.0.2". The tablets are autoupdated to the very latest Android version. Not sure about the minimum target I need to set and still have ScriptIntrinsicsBLAS available.</p>
  
  <p>I'm using #pragma rs_fp_relaxed in all scripts. The Allocations all use default flags.<br>
  <a href="https://stackoverflow.com/questions/28750267/android-renderscript-performance-drop-in-sdk-21">This question</a> has a similar situation, but it turned out OP was creating new Script objects every computational round. I do nothing of the sort, all Scripts and Allocations are created at init time.</p>