Arduino Uno data logger shield recording nonsense time
Where are you brining the power onto the Arduino board? If you bring it in through the external power connector you need to account for the voltage needed to run the regulator. You might be running into problems with low voltage and some part of your system that is sensitive to it.

You could try bring the power in on the 5 V pin, but then you are taking responsibility for the quality of the power supply.