Is it noticeable if audio is inverted?
No, there's absolutely no difference if the audio signal is inverted or not. 

You only get in trouble if you try to combine sources where one is inverted and the other is not, for example if you connect one speaker in reverse in a stereo system. Then the audio waves will cancel. This is usually immediately obvious.