title: timing the Backward_gpu function for a particular layer in caffe
body: <p>I have added altered the backward_gpu function and I want to measure the change in the time so I have,</p>
  
  <pre><code>template &lt;typename Dtype&gt;
  void InnerProductLayer&lt;Dtype&gt;::Backward_gpu(const vector&lt;Blob&lt;Dtype&gt;*&gt;&amp; top,
                                          const vector&lt;bool&gt;&amp; propagate_down,
                                          const vector&lt;Blob&lt;Dtype&gt;*&gt;&amp; bottom) 
  {
  
   struct timeval backward_start, backward_end;
   double backward_gpu_run_time;
   gettimeofday(&amp;backward_start, NULL);
  
   ...
  
   gettimeofday(&amp;backward_end, NULL);
   backward_gpu_run_time = (backward_end.tv_sec+(backward_end.tv_usec/1000000.0))-(backward_start.tv_sec+(backward_start.tv_usec/1000000.0));
   LOG(INFO) &lt;&lt;  K_ * N_&lt;&lt;"backward_gpu_run_time: "&lt;&lt;backward_gpu_run_time; 
  
  }
  </code></pre>
  
  <p>But this returns the same value ( it really shouldn't based on the amount of work I've reduced) as it did before.  Is it possible that caffe is not waiting until everything is done inside to compute the time? Meaning the work inside was never accurately reflected in the time?  When I do similar things to the Forward_gpu function the reduction in time is apparent.  </p>
  
  <p>How do I accurately measure the timing of the Backward_gpu function for a particular layer in caffe?</p>