5-10 second backup for wifi router during power cut
Something a little cheaper than a mains voltage UPS is like:-



which you can read about here. It does what you want without the inverter/step up components to produce mains level voltage.

The problem with your capacitor only solution is that the capacitors need to be pretty large, and commonly you'd use super capacitors.  That then leads to huge inrush current management. And since most super caps are 2.7V, some form of voltage equalisation when you stack them up to increase voltage. A practical, safe and reliable circuit quickly gets complicated.

If you want a project, there are many DIY circuits on line for this exact scenario. If you simply want a solution, buy one of the above like thingies.