title: cuda: different answer between cpu and gpu reduce
body: <p>I got this really weird error. I ran a sum over all elements in a matrix using thrust reduce. It ran well for most data, but it went wrong on one set.</p>
  
  <p>code:</p>
  
  <pre><code>  lbfgsfloatval_t sum(const DeviceVector&amp; A){
      thrust::device_ptr&lt;lbfgsfloatval_t&gt; ptr(A.getPtr());
      thrust::device_vector&lt;double&gt; A_p(ptr, ptr + A.rows()*A.cols());
      lbfgsfloatval_t sums = 0.0;
  
      // reduce on host
      for(int i = 0; i &lt; A.rows()*A.cols();i++)
          sums += A_p[i];
      // reduce on device
      lbfgsfloatval_t res = thrust::reduce(A_p.begin(), A_p.end());
      cout &lt;&lt; "cpu: " &lt;&lt; sums &lt;&lt; endl; 
      cout &lt;&lt; "gpu: " &lt;&lt; res  &lt;&lt; endl;  
      return res;
   }
  </code></pre>
  
  <p>Notice the second group went wrong.</p>
  
  <p>output:</p>
  
  <pre><code>cpu: -568.691
  gpu: -568.691
  
  cpu: 3.4972e-14
  gpu: 1.40998e-14
  
  cpu: 0.234375
  gpu: 0.234375
  </code></pre>
  
  <p>I also tried not building thrust::device_vector, but use a raw pointer instead. Same output. I also tried cublas dot product. Same output.</p>
  
  <p>I used matlab to confirm the cpu result above is correct.</p>
  
  <p>What happened? Was it an underflow on GPU? Thanks!</p>