AMS1117 and 3S Li-Ion battery
Yes, exceeding the spec is not good. It will probably work for a while. Or even forever. But a) no way to say for sure and b) if you build many circuits like that, you can never tell how many will fail.
The dumbest and, yet in this case still acceptable option I see is 2 series diodes before 1117 that will drop 1.2-1.6V. You could get away with one, but I'm just making sure you're not at the spec boundary. Of course, if you have some charging circuit, it will have to connect to battery before diodes. I assume you don't have high currents there since it's 1117, but still estimate heat output on the diode (VdropI~0.7I) to make sure the selected diode can take it. Also, take a diode with some headroom in terms of heat dissipation, rectifier diodes will work nicely.
Also, don't ever go to Absolute Maximum ratings. There is a recommended maximum in the datasheet, always look at that. You don't want to push things to the maximum. There are so many parts to choose from that you can always find a way to make things work stable and well within the spec.