Should my NiMH battery powered circuit/device switch power sources when charging?
I think the answer to that question is "yes": e.g. Panasonic - Nickel Metal Hydride Batteries - Technical Handbook 2000 explicitly says that you can't trickle charge NiMH, except for a defined period of a few hours after rapid charging. With NiMH you need to carefully measure the voltage while charging to look for the "end of charge" condition. Having a circuit run directly off the battery will also make that harder.

You could however do:


on plugging in, start charge
stop charging as normal (voltage dip detection) and set a timer
after approx 24 hours, retrigger the charging cycle


Or select a different battery chemistry which can be float charged such as NiCad.