   Beginning with CUDA
  I am beginning with CUDA.I was trying to do some RMS kinda calculations, just to get the grasp.I want to get the square root of each element divided by the number of elements, then sum all of them. I can do the sqrt in the CPU, as it's just a single operation for all the data set.So, my kernel looks like:__global__ void RMS (float* in, int N, float fN, float* out){int idx = blockIdx.x * blockDim.x + threadIdx.x;float aux;if (idx < N){aux = in[idx] * in[idx] / fN;*out += aux;}}Of course, this won't work because I'm not synchronizing the threads and they are all writing to out at the same time.I've seen there's a __threadsync() function or sometihng similar, but it just waits until all the threads in a block complete the calculations; i don't think that's really useful, because then again every thread would write to out at the same time.I've also had a look into nvidia's "reduction" sample but it's overwhelming, I don't really understand what's going on. As far as I can tell I guess I should synchronize in some way every thread of each block to get a partial calculation, and then add together each partial result.So, what would be the way to go?   
  Perhaps the beginner's forum is not the best place to ask this question, but I can't find a better one. Any advice?   ;  I believe what you want is to implement a parallel prefix sum (scan) type operation so that you can effectively do your summation in parallel.  Take a look at this PDF put out by NVIDIA for more info on doing it within CUDA.   ;Parallel reduction (pdf).   