powering android phone from 3 AA battery directly (not USB!)
Four cells will put you over the nominal voltage of a single Li-Ion cell and it might be OK considering they will use a switch-down converter. However that is not guaranteed, it may damage the phone, it may have over-voltage protection that will prevent it starting up or worst case may have some form of clamps that will damage your batteries and/or cause a hazard.

Personally I'd go down the well documented 5V USB path (if that will still operate the phone). Considering the exact design of the phone's power control will be proprietary I think without test equipment the only other alternative would be to try it and and see what happens, accepting it could damage the phone and keep checking that the batteries don't get too hot due to excess current draw. 

However it would be preferable to purchase at least a cheap multimeter to check the current being drawn just in case it does have clamps of some form. And even better would be a current limited power supply if you have access to one to see what happens as the voltage is ramped up.

Edit: Another thought would be to say put a 1A fuse in series. That should prevent any prolonged over current condition likely to be a hazard regarding the batteries, although the phone could still be damaged.