Spark at the Probe's ground lead
Measure the AC voltage between PC ground, Scope ground and mains ground, using a multimeter.

Your PC or scope is probably floating, due to not being earthed properly to the mains, so you'll see half of the supply voltage from case to ground. This is inevitable, if there's no earth, stray capacitance and the EMC filter capacitors are equal between each line and earth. The maximum current is probably only a few mA, by law, but it hurts and it damages electronics.

Many a project has been destroyed by this sort of problem. I once saw it happen to a million dollar antenna's output amplifier, because of one cheap power plug adapter. So keep a bench ground cable and clip everything to it, before connecting up.