High voltage DC power supplier
There is the possibly that such a high voltage is creating a signal strong enough that disrupts other instruments or electronics. 

I doubt the ground is affecting the other instruments, as that should be neutral. Try adding higher resistors along the 1K range to manage the power supply. At such a high voltage, there is more of possibility of interference occurring, because not enough of the signal is being filtered. 

Here is a link explaining Electromagnetic Interference occurring from high voltage:

http://home.zcu.cz/~tesarova/IP/Proceedings/Proc_2010/Files/025%20IP2010%20Kysela.pdf

In more detail you could be dealing with Corona,

"Corona discharges are a source of radio noise. Spectral components ranging from a few kHz to 10
MHz. In this frequency band, the disturbance spreads leadership. Leadership is an effective radiator,
since the wavelength is due to the large size wires."

From the posted link. A discharge may be creating signal nose.