Can I provide a constant charge to a battery?
Most 'top-up' chargers charge to 4.2v, then stop charging. Once the voltage has fallen back to 4.1v, they charge back to 4.2v again.

I cannot see that providing enough charge over time to a cell to keep it at 4.1v can be any more harmful to its cycle life than providing enough charge over time to cycle it between 4.1v and 4.2v. You appear to think the same way.

However, I've not found any reputable (ie from a manufacturer, from a university) source that says that leaving a lithium chemistry battery on permanent float charge at lower than 4.2v is OK. They only say that a permanent float at 4.2v is not, as the continuous current will eventually plate out metallic lithium. Manufacturers have put a lot of research into maximising the capacity, charge rate and discharge rates. A float charge at less than full capacity is financial anathema to them, so perhaps it's not surprising to have no data.

There are all sorts of fire hazards associated with using lithiums. As long as you have precautions in place (metal containment, no fuels lying about nearby), there's no reason you should not experiment with a lower voltage float if you want to. I would. Keep notes. Test their capacity every few months, and be prepared to change them. Maybe in a few years we'll have evidence for how well other charging regimes work. But you won't find any published source, or me, saying it will work as you hope.

If you want a solution that does have official backing, then 2 x 2v lead acid cells floated at 2.25v to 2.3v per cell will give you a nominal 4v.