Unexpected ADC behaviour - can anyone help diagnose?
Unbuffered SAR ADCs present a loading that is dependent on sample rate (to somewhere mid-scale typically, rather than to ground as is usually shown in simplified diagrams), and your high impedance source affects the acquisition time. It's easiest to have a simple op-amp buffer, but sometimes you can get away without it.
This brief TI &quot;technote&quot; describes some of the considerations in connecting a sensor directly without a driver. Your ADC and circuit may or may not have the flexibility to implement their suggestions.

In general, decrease your sample rate and sample clock frequency to the minimum acceptable to both reduce the average loading and increase the acquisition time for the internal ADC capacitor to charge to close to the input voltage.
Or use the I2C and avoid touching all the analog stuff entirely, as others have suggested.