Can I manually specify Sound Output device in the Mac (like in Windows/Linux)
I think the general consensus on this appears to be 'no' ... but have you tried something like Soundflower or Jack Audio, which allow non-standard routings. I can't test for headphones here, as I'm on a Mac Pro, but it might be worth 5 mins of your time to try it - http://cycling74.com/products/soundflower/ or http://jackaudio.org