switching between solar input and 5V supply
The BQ25504 datasheet says the maximum input voltage at VBAT might be 5.5V. Make sure you don't exceed that and just connect your charger circuitry to the battery in parallel. Battery chargers will have a high output impedance when not powered externally (because they'd otherwise drain the battery), so this will not be a problem.

It might look comfortable to put the 5V supply on the input, but don't: The peak power the BQ25504 will sustain is 400mW, and it's not recommend to charge with more than 300mW flowing through the device (*recommended operation conditions in the datasheet). So, your battery will charge really slow. When you attach 5V, you'll probably just want the battery charged as fast as possible; so invest a couple of cents in one of the hundreds of LiIon Charger ICs and attach that to your battery.