Simple circuit to protect 2.7V supercapacitor from exploding
Few underlying issues with your idea, 


Lead acid is actually one of the few battery technologies that can be almost completely recycled. (That's why most scrap metal places separate them - just melt down the lead and purify the acid and you have a new battery.)
Super capacitors hold usually less than 1% of the same charge of an equivalent size battery, assuming you could use a boost converter to drain every last bit of charge in your capacitor. It only holds 0.1 Watt hours of energy, so say your router draws 5W, it would only store about 1 minute worth of power.


On to the actual questions you would be after a regulator, e.g. an LM317 with the correct resistors on the output to keep the output set to something like 2.65V. I am not clear what modem/routers have an input voltage as low as around 2.7V, and suspect there is another issue.

With older TVs, you may be able to find a zener diode for something easier to find, a 2.7V one "may" be suitable, but it will need a resistor between it and the capacitor to limit the current, and will get quite hot. 

Edit: Based on your update of having 6 of the capacitors with a lower capacity, The bast configuration you could have would be 4 of them in series, with some balancing resistors in parallel with each capacitor, this would make your capacitor breakdown voltage 10.8V, you would still need a regulator or similar to make sure this is never exceeded, but would work better to your end goal,

So we have a 10.8V capacitor arrangement, assuming your solar panel + regulator can keep it held at say 10.6V, and we take a guess that the modem / router will not cut out until about 6.5V (assuming internal 5V rail) your left with about 21 Joules of energy, so about 35 seconds of power. It will get you something usable, but not much power to hold you over.

For a lead acid based approach you would use a panel that could charge the battery to say 13.2V, and use a regulator to drop that to the routers 9V, assuming a scooter battery which is 7Ah, and about 80 Watt hours, a 5 Watt load could be powered for at least half a day.