Are 4xAA DIY Gadget Chargers Safe?
The use of 4 x AA Alkaline would usually be safe BUT does exceed the USB spec and damage may occur in some cases. I have seen IC's in this role with max operating voltages of 5.5V (which is ludicrous) - you'd hope designers had more sense, but it can't be guaranteed.

While some devices may use converters between charge input and battery, many don't (probably most).  A LiIon battery has max charging voltage of 4.2V so a 5V nominal USB input will usually meet this need with enough headroom for a  linear regulator.

An Alkaline cell can be nearly 1.6V when fully charged - about 1.55V is common or 6.2V for 4, and up to 6.4V may be seen. There is not much energy in this initial high voltage "tail" and voltage falls to 1.5V or below very quickly.  

So, you should be safe, but YMMV, alas.

A solution would be to use an LDO (low dropout voltage) regulator OR a clamp regulator which takes the peak energy out of the battery or a series diode to drop 0.4 to 0.8V (Schotky / Silicon). 


LDO is best solution  but you want as little drop as possible. 
Clamp to drain peak battery voltage is unusual but viable. A zener could be used but is too inexact. An eg TL431 clamp regulator in a TO92 or other largish package (to get OK dissipation capability) ould do. A TL431 plus a transistor would be safer. 
Series diode is cheap and easy but prevents full battery use. Say minimum usable battery voltage is 4.6V (may be higher). At 1.15V/cell there is still some battery capacity left. Adding a Schottky diode increases minimum battery voltage to 4.6 + 0.4 = 5V or 1.25 V/cell. Some capacity wasted. At the top end a 0.4V drop diode results in Vbattmax of say (1.55V x 4 - 0.4) = 5.8V or 1.45V/cell."Almost certainly safe".


Using NimH works but is more marginal at bottom end and safer at top end. At 4.6V, V per cell is 1.15V where NimH still has modest energy left. At top end Vmax = say 1.35V, maybe 1.4V for short periods at start. 4 x 1.4V = 5.6V. Very probably safe.