How can I eliminate the loop in this diagram?
There is no simple way to break that loop completely without actual isolation, but if the induced interference is of high-enough frequency you might have an easy work-around. 

Get a ferrite core with considerably high-frequency losses, and loop one of your USB cables through it several times. That creates a common-mode lossy choke that would present a high-impedance to your aggressor signal while letting USB flow through unimpeded. You can do the same to the other USB cable. 

There are ferrites designed for this purpose, but these are intended for low-level emissions not for high-magnitude aggressors, so their impedance might not be enough.