Can I use a 24V, 1A fuse for a 5V application?
The 24 V printed on the fuse is only relevant after the fuse has blown. There is no need to calculate the power (\$P=I^2R\$).

Why is that so?

When the fuse is intact, it will have a low resistance so the dissipated power will be low.

When the fuse has blow, no current should flow so then the power is zero.

So the case where both current and voltage are high (like 1 A, 24 V) should never happen.

After the fuse has blown, it must be able to keep the circuit open (not connected). The fuse can do this up to a certain voltage. For a 1A / 24 v fuse, that voltage is 24 V.

So as long as there is no more than 24 V across the fuse when it is blown, there is no issue. Your 5 V application has a low enough voltage so this fuse is suitable.

You would not want to use a 1 A / 24 V fuse in a mains voltage ( 120 V or 240 V AC) as then the voltage is too high and the fuse might not be able to keep the circuit open after it has blown.