PIC12F ADC conversion has an unwanted offset
Credit for this should go to Dan Laks (see comment on original question), but this question is still sitting in the 'unanswered' section, so I'm putting an answer to it.

Ensure that weak pull-ups in the OPTIONS_REG are not active.