LT1019 voltage reference output 5V to 3.3V
The reference input of the ADS8320 draws varying current depending on the sample rate, like many ADC converters without reference buffers. 



At, say, 25uA, a source resistance of only 2 ohms will result in 1 lsb of change in reading due to the reference current. It's usually best to use a buffer that has low noise output. Sometimes this is surprisingly expensive to do right, although fortunately it's only 16 bits.