How can I use an IC family or adapter to switch automatically from solar cell to batteries & reverse?
You probably want a battery charger/manager IC.

They come in all sorts of flavors, so you'll need to look for one that meets your needs.  They can be simple "smart" switches, or more complex, incorporating DC-DC converters and possibly even peak power tracking on the solar cell.  

I would start with a Digi-key search for "battery charger" (https://www.digikey.com/products/en/integrated-circuits-ics/pmic-battery-chargers/781), and I would also browse the power management ICs from Analog Devices (https://www.analog.com/en/products/power-management/battery-management.html).  I'm sure countless other IC manufacturers have good selections as well, but it will take some browsing to find the right part for your situation.

Here's an example of a part that might work for you: