Power Suppy for Modem Router from 12V Car battery
Actually, you should be able to.
The battery supplies a couple of volts more than you need, but the regulators in the modem should be able to handle it, though it will get warmer since it has to regulate a higher voltage.  It may also run closer to the limits of some parts (capacitors and such in the power supply.)

If you'd rather be safe than sorry, you could get a small DC/DC switching power supply that puts out a regulated 12Volts from the battery voltage.  That way, your modem gets the 12V it likes, and any extra heat is generated outside of the modem housing.



In response to comment:

I would use a small DC to DC converter module.  I find them available at various places on the internet.

This is an example of what you need.  I am not recommending that you buy that one from that dealer.  That is an example only.  I expect you can find better examples with better prices or more reliable dealers or what have you.