title: simultaneous calling of same cuda functions on multiple GPUs using independent streams
body: <p>I am implementing my program on multiple GPUs...
  I have two GPUs in my PC and would like to generate two independent streams to handle both devices. I have generated two streams:</p>
  
  <pre><code>cudaStream_t *streams; 
  streams = (cudaStream_t*) malloc(nstreams * sizeof(cudaStream_t));
  for(int j = 0; j &lt; nstreams; j++){
      cudaStreamCreate(&amp;(streams[j]));
  }
  </code></pre>
  
  <p>Then I try to call the kernel with the device id as a parameter, per the SDK examples.
  According to the device id, cudaDevice is set (cudaSetDevice(dev_ID);) and kernel calls are passed with respective streams:</p>
  
  <pre><code>kernel1&lt;&lt;&lt; grid, block,0,streamNo &gt;&gt;&gt;(tex, size);................
  
  
  for (i = 0;i&lt;gpuN;i++){
      initialize(i);
      printf("initialization done with stream %i on device %i\n", (int)streams[i], i);
      runTest(i);
      printf("running tests done with  %i  stream \n", i);
      deini(i);
      printf("deinitialization is done\n");
  }
  </code></pre>
  
  <p>My program is working fine with two different streams, but it runs sequentially.
  I would like to run functions simultaneously using both independent streams using both devices. Can anyone help me to solve this problem?</p>
  
  <p>I am working with CUDA 4.2 using two Tesla 2070.</p>