Coaxial cables vs zip cords for unbalanced signalling
The coax shield protects against incoming electromagnetic fields.

However, if a current flows in the shield, for example due to the parasitic capacitances of power supply transformers in equipment on both ends, this current creates a voltage by Ohm's law due to the shield not having zero impedance.

Since the shield is also used as a voltage reference for the signal, this noise voltage is added to the signal.

The shield protects against electromagnetic pickup, but not against ground loops.

Balanced (differential) transmission uses two wires. The idea is to transmit both the signal and its voltage reference, via two paths of  equal impedance (this is why it's called balanced). These paths are not connected to ground at the receiving end. Instead the receiver performs a substraction, and any noise picked up by both conductors will appear equal on both and thus be nulled.

A shield is usually added also, it both protects the conductors from picking up EMI, and provides a way to connect the grounds on both sides.