Why does reversing the oscilloscope probes cause ringing in the signal?
Your probe ground clip really is ground i.e. tied to safety in the plug.

Here is snip the Tektronix website:


  Traditional Oscilloscopes
  
  Most traditional oscilloscopes have the “signal reference” terminal connected to the protective grounding system, commonly referred to as “earth” ground or just “ground”. This is done so that all signals applied to, or supplied from, the oscilloscope have a common connection point.
  
  This common connection point is usually the oscilloscope chassis and is held at (or near) zero volts by virtue of the third- wire ground in the power cord for AC-powered equipment. This means each input channel reference is tied to a single ground reference.
  A traditional passive probe should not be used to directly make floating measurements on a ground referenced oscilloscope. Depending on the amount of current flowing through the reference lead, it can begin to get hot or, if the current is high enough, it will melt open similar to the way a fuse operates.


Don't do that.