Can I replace rechargable batteries with (super)capacitors?
You could do this, but will it work? The charging circuit will be confused as it is built to charge/discharge at certain voltages. Because lithium ion batteries are considered dead below a certain threshold (like 3.0V) the UPS charging circuit may not switch on at all because a supercapacitor will have no voltage initially. 

As far as the capacitor goes, it doesn't matter if it is charged below a threshold, the main thing to worry about is the breakdown voltage (max voltage) and that you don't exceed that rating. It can hold any value between the max voltage and zero.


Source: https://learn.adafruit.com/li-ion-and-lipoly-batteries/voltages


Source: https://www.tecategroup.com/ultracapacitors-supercapacitors/ultracapacitor-FAQ.php