3 axis magnetometer issue in one axis
I'm not 100% sure what the issue was, but found a better firmware that works much better, and even on the breadboard.

It calibrates the compass before using it, that's the only significant difference I see. 

https://github.com/helscream/HMC5883L_Header_Arduino_Auto_calibration