Step up converter (3.7V to 5V) that will also charge battery!
Low battery should be easy.  You already have a 5V supply and you're measuring a 3.7V battery, so a comparator between the battery and a voltage divider should do the trick.

As for the boost converter, battery charger, and auto-switch, I would be tempted to skip the auto-switch and always run from the battery, even when charging.  Sure, you're converting 5V to 4.2V (full-charge) and back to 5V again, but it allows you to design a charger and booster almost independently.  Just set the current limit of the charger to cover the application too, but not enough to blow up the battery if the application is off.

While these devices are everywhere, most of them are on the same PCB as the rest of their application, so you may have to roll your own or do some salvaging.  Even if there is one you could buy, asking for that route is outside the scope of this site.