Oscilloscope measuring of PMT in photon counting mode
I'm not sure what your 13 kHz ripple is, but it seems more like power supply ripple than a ground loop. If your HV supply is a positive supply, then the PMT 'ground' is going to be ripple-y. The floating ground lead on your scope presumably has a lot of parasitic capacitance, since it goes to the chassis and everywhere inside. When the ground lead is connected to PMT ground, there's not much capacitance on the anode, but when the ground lead is on the anode, there's a big chunk of capacitance to charge and discharge with the ripple. That's my speculation.

In general, what you want when dealing with pulses is a "pulse discriminator", which lets you put conditions on the height, length, etc. of the pulses you're interested in. Since you're pulse counting, you probably don't need anything fancy, a simple comparator will do, but it will need to have sufficient negative input range. You can build a simple one using an op-amp, but you can get ones off the shelf that are faster and with logic level output. Alternatively you could use an inverting amplifier topology to flip the signal, but you will still need negative input range and sufficiently high bandwidth to pass pulse waveforms through.

Edit: One more thing, isn't the max voltage on the R1166 1250V?