title: CUDA Thrust slow when operating large vectors on my machine
body: <p>I'm a CUDA beginner and reading on some thrust tutorials.I write a simple but terribly organized code and try to figure out the acceleration of thrust.(is this idea correct?). I try to add two vectors (with 10000000 int) to another vector, by adding array on cpu and adding device_vector on gpu.</p>
  
  <p>Here is the thing:</p>
  
  <pre><code>#include &lt;iostream&gt;
  #include "cuda.h"
  #include "cuda_runtime.h"
  #include "device_launch_parameters.h"
  #include &lt;thrust/device_vector.h&gt;
  #include &lt;thrust/host_vector.h&gt;
  
  #define N 10000000
  int main(void)
  {
      float time_cpu;
      float time_gpu;
      int *a = new int[N];
      int *b = new int[N];
      int *c = new int[N];
      for(int i=0;i&lt;N;i++)
      {
          a[i]=i;
          b[i]=i*i;
      }
      clock_t start_cpu,stop_cpu;
      start_cpu=clock();
      for(int i=0;i&lt;N;i++)
      {
          c[i]=a[i]+b[i];
      }
      stop_cpu=clock();   
      time_cpu=(double)(stop_cpu-start_cpu)/CLOCKS_PER_SEC*1000;
      std::cout&lt;&lt;"Time to generate (CPU):"&lt;&lt;time_cpu&lt;&lt;std::endl;
      thrust::device_vector&lt;int&gt; X(N);
      thrust::device_vector&lt;int&gt; Y(N);
      thrust::device_vector&lt;int&gt; Z(N);
      for(int i=0;i&lt;N;i++)
      {
          X[i]=i;
          Y[i]=i*i;
      }
      cudaEvent_t start, stop;
      cudaEventCreate(&amp;start);
      cudaEventCreate(&amp;stop);
      cudaEventRecord(start,0);       
      thrust::transform(X.begin(), X.end(),
          Y.begin(),
          Z.begin(),
          thrust::plus&lt;int&gt;());
      cudaEventRecord(stop,0);
      cudaEventSynchronize(stop);
      float elapsedTime;
      cudaEventElapsedTime(&amp;elapsedTime,start,stop);
      std::cout&lt;&lt;"Time to generate (thrust):"&lt;&lt;elapsedTime&lt;&lt;std::endl;
      cudaEventDestroy(start);
      cudaEventDestroy(stop); 
      getchar();
      return 0;
  }
  </code></pre>
  
  <p>The CPU results appear really fast, But gpu runs REALLY slow on my machine(i5-2320,4G,GTX 560 Ti), CPU time is about 26，GPU time is around 30! Did I just do the thrust wrong with stupid errors in my code? or was there a deeper reason? </p>
  
  <p>As a C++ rookie, I checked my code over and over and still got a slower time on GPU with thrust, so I did some experiments to show the difference of calculating vectorAdd with five different approaches.
  I use windows API <code>QueryPerformanceFrequency()</code> as unified time measurement method. </p>
  
  <p>Each of the experiments looks like this:</p>
  
  <pre><code>f = large_interger.QuadPart;  
  QueryPerformanceCounter(&amp;large_interger);  
  c1 = large_interger.QuadPart; 
  
  for(int j=0;j&lt;10;j++)
  {
      for(int i=0;i&lt;N;i++)//CPU array adding
      {
          c[i]=a[i]+b[i];
      }
  }
  QueryPerformanceCounter(&amp;large_interger);  
  c2 = large_interger.QuadPart;  
  printf("Time to generate (CPU array adding) %lf ms\n", (c2 - c1) * 1000 / f);
  </code></pre>
  
  <p>and here is my simple <code>__global__</code> function for GPU array adding:</p>
  
  <pre><code>__global__ void add(int *a, int *b, int *c)
  {
      int tid=threadIdx.x+blockIdx.x*blockDim.x;
      while(tid&lt;N)
      {
          c[tid]=a[tid]+b[tid];
          tid+=blockDim.x*gridDim.x;
      }
  }
  </code></pre>
  
  <p>and the function is called as:</p>
  
  <pre><code>for(int j=0;j&lt;10;j++)
  {
      add&lt;&lt;&lt;(N+127)/128,128&gt;&gt;&gt;(dev_a,dev_b,dev_c);//GPU array adding
  }   
  </code></pre>
  
  <p>I add vector a[N] and b[N] to vector c[N] for a loop of 10 times by:</p>
  
  <ol>
  <li>add array on CPU</li>
  <li>add std::vector on CPU</li>
  <li>add thrust::host_vector on CPU </li>
  <li>add thrust::device_vector on GPU </li>
  <li>add array on GPU. and here is the result</li>
  </ol>
  
  <p>with N=10000000</p>
  
  <p>and I get results:</p>
  
  <ol>
  <li>CPU array adding 268.992968ms</li>
  <li>CPU std::vector adding 1908.013595ms</li>
  <li>CPU Thrust::host_vector adding 10776.456803ms</li>
  <li>GPU Thrust::device_vector adding 297.156610ms</li>
  <li>GPU array adding 5.210573ms</li>
  </ol>
  
  <p>And this confused me, I'm not familiar with the implementation of template library. Did the performance really differs so much between containers and raw data structures? </p>