Creating a Li-on backup for a 12v router
Step up converter

Assuming peak load, your router will be drawing \$12V*0.5A=6W\$ of power. At 80% efficiency, your step-up converter will draw \$\frac{6W}{0.8}=7.5W\$ at its input. This means your 2 lithium cells will need to be able to provide \$\approx1A\$ each (\$\frac{7.5W}{3.6V}=2.08A\$). You don't specify which 18650 cells you plan to use so I can't say whether you can safely discharge that amount of current from it.

Battery charging

Your TP4056 is only capable of delivering \$1A\$ of current on the output. Since the step-up converter draws \$\approx 2A\$ during peak load (as calculated above), this will mean that the batteries will actually be discharging during peak load. Whether this is a problem depends on the average power consumption of the router and how long the periods of peak power consumption are.

USB charger

Lastly, your USB charger is only capable of putting out \$5V*1.5A=7.5W\$. Assuming your battery charging circuitry is less than 100% efficient, this is not enough to power your router at peak load. However, in this case, your peak load tolerance is limited by the TP4056, rather than the USB charger. If you decide to upgrade to a beefier battery charging circuit, your USB charger is going to be a bottleneck.

Lithium-ion batteries

Li-ion batteries don't like to be fully discharged. Depending on the nature of the circuit and how the batteries are discharged, it can lead to the battery catching fire or exploding when it is recharged again. Your 18650 batteries likely do not come with over-discharge protection circuitry so you should also factor this in. Edit: It appears the TP4056 has built-in over-discharging protection if you use the OUT terminals.

You also don't mention how long you'd like the batteries to last during a blackout. Based on your comment, I'm assuming you just need runtimes measured in the order of minutes.

Assuming peak current draw and you want the battery to last at least 5 minutes, your total battery capacity needs to be larger than \$2A*\frac{5}{60}=167mAh\$ (keep in mind the C rating needs to be large enough to carry the amount of current needed by the step-up converter as calculated above).

Objectives

To answer your questions:


Will this power your router in normal use? As mentioned above, it depends on what 'normal use' is. You only specify peak current consumption which is not enough to answer this question.
If you size your battery correctly, this should be possible as mentioned above.
If you select high quality cells with decent protection circuits and follow best practices, lithium cells should be fine to use. Obviously, this being a home-made project, there's no guarantees.