LDO regulator below minimum input voltage
MCP1703 datasheet figure 2-31 shows the output tracking the input down to well below 2.0V.



However this is at a load current of only 1mA. Figure 2-17 shows short circuit current vs input voltage. Here we see that it can only deliver a few milliamps at 2V.  



This poor low voltage performance is probably caused by the pass FET having a relatively high Gate threshold voltage, which may be an inherent result of its 16V rating. 

I would use the MCP1700, which is specified for 200mA output at 2.3V. It can only handle 6V maximum, but this is well above your battery voltage.