Understanding differential ADCs and reference voltages
Any differential ADC (and the op-amps too) will have a common mode rejection ratio (CMRR).  So The common mode voltage (the average of the + and -) terminals will affect the output.

All op-amps also have an input offset voltage that essentially adds to your differential input voltage when computing the output.  The input offset may be just a few microvolts for very precise amplifiers, or it may be a few millivolts for most common op amps.  ADCs have a similar parameter which would be the zero offset, which is typically several LSB.

Put mathematically, one could model the output as...
Vout = G * (V+ - V- + Vos) + C * (V+ + V-)/2
Where...
V+ is the non inverting input.
V- is the inverting input.
C is a factor edpending on your CMRR.  It would be 1/100 for 40dB of CMRR, and 1/1000 for 60dB of CMRR, etc.

So for example if the common mode rejection ratio say 60dB then you would see a change in the output measurement that is equivalent to 1mV of change at the differential input voltage for every 1V of change in the common mode voltage.

If your input voltage range is just a few mV then input offset and CMRR need to be considered or your measurement will be washed out by component tolerances.