Why does my computer have a ground fault only when a peripheral is attached?
You most likely have poor grounding to or from the breaker box in your home.

The first thing I would do is get myself one of these nifty devices.  I generally dislike the cheap tester lights as they don't tell you very much.  It is called a Voltimeter and it will be able to tell you if a current is passing and how much voltage exists to your ground.



It has many settings, for testing your typical 120v AC home outlet you can set it to test AC voltage at the ACV 200 setting.  Test hot to neutral first and make sure that you have 115-130 volts on the meter display.  Next test hot to ground and make sure that you get the same reading.

If you get a lower number than this or if it remains close to zero then your outlet has not been properly grounded.

Test the other outlets in the home and they register the same then it could be a problem with your breaker box not being properly grounded.  It could also be if your house is older that the original outlet boxes were never grounded to the breaker, and someone had just installed three pronged outlets throughout the house without bothering to ground the outlet boxes.