Powering 800mA router for approximately 1 min
One problem with capacitors for power backup applications is not so much if they can be used, it's also a matter of cost. Just using an on-line supercap discharge calculator as an example and plugging in a few numbers assuming a 12 -> 9V drop would still keep it running OK I came up with a 20F capacitor for a bit over a minute. 

Although a bit larger than required on Digikey at the moment the cheapest part that I found that met or exceeded those requirements was a 58F 16V Capacitor at a bit over $US140. That's not too over the top but if you were trying to connect across a normal modem power supply you'd also need to consider the inrush current as the capacitor charges so it might not be as simple as just placing one across the supply.

I'd expect that for that voltage / current assuming that the modem is normally powered from mains that a small UPS using an SLA battery would be a cheaper and better solution. While SLA batteries do degrade over time I'd expect a small UPS to be able to provide that small amount of backup time for quite a long period, maybe say five years until the batteries needed replacing.

As an addendum to your updated question when connecting an SLA battery in parallel you may experience similar problems with an inrush current. While I wouldn't expect the battery to draw much current after a few minutes of discharge (if fully charged at the start) for safety you should also consider what would happen if power was out for hours and the battery discharged completely.