What is Magnitude and Phase actually represents in Fast Fourier Transform?
This question is longstanding, and not fully resolved, IMO.
It hinges upon data dimensionality (1D, 2D, 3D), size (short, large), sampling (loose, tight), and mostly morphology (smooth vs. sharp).

Basically, in natural images with distinguishable objects of different contrast/texture at different scales, the data is far from stationary. Fourier applies, but sines can have very complicated relative locations, therefore phases. And to make sense of an object, sepia or luminance-distorted images still work. So, disregarding relative amplitude of components, what may matter is their relative position. Hence, phase can be primordial. 

But for more stationary 2D data, with less "objects", phase can have a minor effect.