Why phase bump causes signal distortion in source follower?
It really depends on what you mean by the term "distortion".

Yes, the bump in the phase response will cause the output waveform to be different from the input waveform. But the non-flat frequency response has that kind of effect, too. But both of these effects are linear and can easily be compensated for, restoring the original waveform.

A more useful definition of "distortion" is any nonlinear effect that cannot readily be compensated for, such a clipping, crossover distortion, slew rate limiting, etc. The effects you're talking about do not fall within this definition.