What is wrong with this GFCI outlet?
A non-contact tester might give you a false-positive on the neutral in some cases.  Try testing the outlet with an outlet tester which will indicate if the hot/neutral are switched or open.  Alternatively you can use a multi-meter to test it.  If you are in the US, you should get ~120VAC between hot and neutral, but 0VAC between neutral and ground if the outlet is wired correctly.