title: On CUDA __constant__ memory and multi-GPUs?
body: <p>In a shared-memory programming model, any global variables are visible to every threads.</p>
  
  <p>In CUDA, <strong>constant</strong> memory declared in simliar ways like global variables in shared-memory systems, which make me a little bit worried:</p>
  
  <p>Considering the following code:</p>
  
  <pre><code>__constant__ int array[1024];
  
  void hostFunction(int DeviceID, cudaStream_t streamIdx)
  {
      cudaSetDevice(DeviceID);
      someKernel&lt;&lt;&lt;100,1024,0, streamIdx&gt;&gt;&gt;(...);
      //The function someKernel will use data stored in array[] on current device;
  };
  </code></pre>
  
  <p>Then, Is the contents of <code>array[]</code> local to each cuda context/devices, such that we can safely update each Devices's "private" <code>array[]</code> without worrying about changing the values of <code>array[]</code> allocated on other cuda devices?</p>
  
  <p>BTW: I searched the site, there are some related questions, however I cannot find any clear answer from any of these.</p>