title: cudaMemcpy() calls to streams
body: <p>Consider the two snippets of code. </p>
  
  <pre><code>Snippet1
  
  cudaStream_t stream1, stream2 ;
  cudaStreamCreate(&amp;stream1);
  cudaStreamCreate(&amp;stream2);
  cudaMemcpyAsync( dst, src, size, dir, stream1 );
  kernel&lt;&lt;&lt;grid, block, 0, stream2&gt;&gt;&gt;(...);
  
  
  
  Snippet2
  cudaStreamCreate(&amp;stream1);
  cudaStreamCreate(&amp;stream2);
  cudaMemcpy( dst, src, size, dir, stream1 );
  kernel&lt;&lt;&lt;grid, block, 0, stream2&gt;&gt;&gt;(...);
  </code></pre>
  
  <p>In both snippets I am issuing a memcpy call (snippet1 asynchronous and snippet2 synchronous)</p>
  
  <p>Since the commands have been issued to two different streams, from my understanding there can be potential overlap in both cases. </p>
  
  <p>But in Snippet2 cudaMemcpy call being synchronous (aka blocking) leads to me a paradoxical conclusion that cudaMemcpy and kernel call will be executed one after another. </p>
  
  <p>Which one is the correct conclusion ?  </p>
  
  <p>To rephrase more compactly: When we issue the cudaMemcpy call to a stream does it block the "entire code" or just block the stream it was issued to? </p>