Does lowering arduino analog reference increase precision in measuring low voltage?
The former, i.e precision would now be 1/1023 but the accuracy of the AD converter is 4 / 1024 or 1 part in 256  i.e. 0.25%. 

Also be aware that the internal reference voltages themselves are not very accurate : 

DEFAULT: depends on your computer power supply USB2  Vbus = 4.4V - 5.25V, Vint 1 = 1.1V  actually 1.0 -- 1.2V, Vint 2 = 2.56V  actually 2.4 -- 2.8V