Strong (0,0) pixel among an otherwise normal cross correlation peak
Figures that the same day I post a bounty I answer my own question. 

The answer to this question is dead/speckle like pixels. For a fixed number of dead pixels of a given brightness, the smaller the relative shift between the two images, the fewer dead pixels that are needed to cause the (0,0) pixel to spike. It can take fewer brighter dead pixels to have the same effect as more dimmer ones. Lastly, the threshold between a strong (0,0) pixel and normal one is extremely tight in the number of dead pixels. Literally a change in just 10 dead pixels can massively increase the probability that the (0,0) pixel becomes a problem.

This problem can be solved by a combination of dead pixel finding/interpolation and applying a light Gaussian blur to the images to minimize speckle.