title: Understanding CUDA indexing
body: <p>I inherited some CUDA code that I need to work on but some of the indexing done in it is confusing me.</p>
  
  <p>A simple example would be the normalisation of data. Say we have a shared array <code>A[2*N]</code> which is a matrix of shape 2xN which has been unrolled to an array. Then we have the normalisation means and standard deviation: <code>norm_means[2]</code> and <code>norm_stds[2]</code>. The goal is to normalise the data in A in parallel. A minimal example would be:</p>
  
  <pre><code>__global__ void normalise(float *data, float *norm, float *std) {
  int tdy = threadIdx.y;
  
  for (int i=tdy; i&lt;D; i+=blockDim.y)
    data[i] = data[i] * norm[i] + std[i];
  }
  
  int main(int argc, char **argv) {
  
  // generate data
  int N=100;
  int D=2;
  MatrixXd A = MatrixXd::Random(N*D,1);
  MatrixXd norm_means = MatrixXd::Random(D,1);
  MatrixXd norm_stds = MatrixXd::Random(D,1);
  
  // transfer data to device
  float* A_d;
  float* norm_means_d;
  float* nrom_stds_d;
  cudaMalloc((void **)&amp;A_d, N * D * sizeof(float));
  cudaMalloc((void **)&amp;norm_means_d, D * sizeof(float));
  cudaMalloc((void **)&amp;norm_stds_d, D * sizeof(float));
  cudaMemcpy(A_d, A.data(), D * N * sizeof(float), cudaMemcpyHostToDevice);
  cudaMemcpy(norm_means_d, norm_means.data(), D * sizeof(float), cudaMemcpyHostToDevice);
  cudaMemcpy(norm_stds_d, norm_stds.data(), D * sizeof(float), cudaMemcpyHostToDevice);
  
  // Setup execution
  const int BLOCKSIZE_X = 8;
  const int BLOCKSIZE_Y = 16;
  const int GRIDSIZE_X = (N-1)/BLOCKSIZE_X + 1;
  dim3 dimBlock(BLOCKSIZE_X, BLOCKSIZE_Y, 1);
  dim3 dimGrid(GRIDSIZE_X, 1, 1);
  
  normalise&lt;&lt;dimGrid, dimBlock, 0&gt;&gt;&gt;(A_d, norm_means_d, norm_stds_d);
  
  }
  </code></pre>
  
  <p>Note that I am using Eigen for the matrix generation. I have omitted the includes for brevity.</p>
  
  <p>This code above through some magic works and achieves the desired results. However, the CUDA kernel function does not make any sense to me because the for loop should stop after one execution as <code>i&gt;D</code> after the first iteration .. but it doesn't?</p>
  
  <p>If I change the kernel that makes more sense to me eg.</p>
  
  <pre><code>__global__ void normalise(float *data, float *norm, float *std) {
  int tdy = threadIdx.y;
  for (int i=0; i&lt;D; i++)
    data[tdy + i*blockDim.y] = data[tdy + i*blockDim.y] * norm[i] + std[i];
  }
  </code></pre>
  
  <p>the program stops working and just outputs gibberish data.</p>
  
  <p>Can somebody explain why I get this behaviour?</p>
  
  <p>PS. I am very new to CUDA</p>