Image registration using gradient descent
It doesn't feel wrong to me.

A rotation by a small amount, say one degree, changes the location of an object in the image by many pixels.  For instance, suppose you have a 1000x1000 image, and you rotate it by one degree around the upper-left corner.  Then an object near the upper-right corner moves upward/downward by about 17 pixels.  There is "leverage" proportional to the size of the image; rotate a long lever by a small rotation, and the end of the lever still moves a large way.

In contrast, a translation by a small amount, say one pixel, changes the location of an object in the image by only a little.  For instance, if you have that same image and you translate by one pixel, then the object moves by only 1 pixel.

So it is not surprising (to me) that the gradient in the first case might be much larger than the gradient in the second case.