USB line noise in aircraft cockpit recordings
I get basically the exact same issue with my car stereo if I charge my phone/MP3 player while having it connected to the line-in of the stereo. The ground loop is allowing common-mode noise in (in this case, the ignition system of the car is producing huge amounts of noise, and it's getting in through common-mode coupling).

Since this is common-mode, galvanic isolation will not necessarily fix the issue, since the critical measurement is common-mode coupling capacitance, which can be fairly high even on brand-name hardware (basically: efficiency, cost, and common mode isolation. Choose any 2).

What you need is a "hum eliminator", e.g. an audio isolation transformer. 

They seem to run about $10 in general:


  



It may seem counterintuitive that the solution to your issue of noise when you connect the USB line is to put a filter in the audio line, but that is indeed the proper solution. 

The noise is a function of the loop created by the two cables. Basically, both the USB and the audio cable are grounded together in the gopro. As such, if there is a voltage differential across between the audio-connection in the plane, and the power connection, it will cause current to flow through the audio cable, through the gopro, and out the usb cable (or vice versa). 

As such, breaking the connection in either one will solve the problem. Audio isolation transformers are much cheaper and easily available, which is why I suggest it over a isolated power supply.