3.7V Lipo battery discharge
I suspect that you are connecting Aref to the regulator output.

While you have sufficient voltage, and the regulator provides the correct (i.e. expected) voltage, you get correct readings.

When the battery falls below a limit, the output at the regulator falls, and will always be the input V minus its drop. The ADC will always read the same voltage (since regulator output = Aref = Battery - Reg. Drop), which happens to be a higher ADC value than before this condition.

You need a better reference voltage at Aref. You can, for example, use a Zener with a drop less than 2.8V, and a voltage divider to feed the ADC.

Update: (Regarding your comment) You should add a better reference voltage to Aref. Since you observe the problem it is a good practice to fix it. Your arduino the way it works now cannot tell whether the battery is dead or fully charged, which is not a good idea.

Another point is that you risk damaging your arduino, as you are bringing Aref above Vcc. There is limit for this (I can't remember, check datasheet).

Finally is is also a bad practice to rely on something you observed (and it is not a guaranteed specification), since it may soon behave differently. Save yourself from headaches, and design following good practices.