Using an audio amplifier with differential outputs with headphones
Each channel output is a H bridge therefore you cannot cross connect speaker wires to form a common 0V. You could use a transformer to convert a balanced output (H bridge output) to a single ended output and you'd need one transformer per output.

You might be able to use the half-H bridge outputs but distortion and other problems may occur due to imbalanced loading. Read the data sheet to see what it says.