How to eliminate computer noises from audio on shared power supply (12v/5v buck converter)?
This is a common problem in audio amplifier connections especially where there is digital noise and switching regulators. There is no ideal way around this problem without using differential inputs.

Why do differential inputs work - the signal return back to the box on the left won't carry any power supply currents because it feeds a high input impedance differential amplifier (or a transformer). Obviously you have to wire it correctly.

Without a differential input (silicon or transformer) it's difficult to find a signal return path that won't pass digital currents.

This problem has nothing to do with dodgy earths in your home or needing to put a decoupling capacitor somewhere.