Mean shift image processing algorithm for color segmentation
The following code is a first shot and it is still not vectorized. The major points here are the extraction of the creation of cLow and cUp (don't create arrays in loops, always 'preallocate' memory), the calculation of the tolerance levels can be done in one operation (under the assumption that broadcasting is possible at this point) and at last I removed the conditional case for copying the imgNew to img (I also doubt that you do not want to copy the last iteration back into img. If so you have to remove the copy line before the loop and move the copy at the beginning of the loop to its end.).

diff_height_gL = height - gL
diff_width_gL = width - gL
sum_gSmp_one = gSmp + 1

cLow, cUp = np.empty((gL, gL, 3)), np.empty((gL, gL, 3))

imgNew = img.copy()

for itr in xrange(itrs):

    img[:] = imgNew

    for x in xrange(gL, diff_height_gL):
        for y in xrange(gL, diff_width_gL):

            cGrid = img[x-gSmp:(x + sum_gSmp_one), y-gSmp:(y + sum_gSmp_one)]

            cLow[:] = img[x, y, :] - tol
            cUp[:] = img[x, y, :] + tol

            cBool = np.any(((cLow &lt; cGrid) &amp; (cUp &gt; cGrid)), axis=2)

            imgNew[x, y] =  np.sum(cGrid[cBool], axis=0) / cBool.sum()


This problems seems to be perfectly shaped to do multiprocessing. This could be an alternative/extension to vectorization. If I have time I will try the vectorization...

Kind regeards