Headphone amplifier: negative charge pump vs. phantom ground
I believe the correct term here is the virtual ground. "Phantom ground" sounds like a mix of "phantom power" and "virtual ground".

Virtual ground does have downsides, see for instance Virtual Grounds &amp; 3 Channel Amps. It's fine for low current applications, which may include a headphone amplifier, depending on its peak output power and the virtual ground implementation. If the implementation is not good enough, it may cause crosstalk between channels.

Whether it is better or worse than a switching power supply really depends on concrete implementations of both.