Some basic questions about voltage and lithium power pack
I've got several lithium powerbanks that output at 5v 2.1A... My
  router is rated 9v at 0.6A. If this is legitimate can I just connect a
  5.5mm x 2.1 DC plug to the end of a USB cable to connect the power bank to the router?


If the router is designed for 9 V then 5 V may not be enough, particularly if it has some circuitry that needs 5 V (because its on-board regulator will have some voltage drop).


  I also read that a better way to accomplish this would be to use a 5V
  to 9V step-up adapter which will supposedly allow me to power slightly
  higher amperage devices which might be better as a backup for my modem
  (12V at 1A).


Yes, matching voltage to the device specification is more likely to work. But remember that as the voltage is stepped up so the input current must go up to provide the required power (watts = volts x amps), and the voltage booster is less than 100% efficient so even more input current is required. For example if the converter is 75% efficient then to get 12 V at 1 A it must draw (12 V x 1 A / 75%) / 5 V = 3.2 A. Your 5 V 2.1 A supply will not be sufficient.   


  My first question related to this device is why the unit is listed as
  15V and not 12V which is the type of battery it is designed to jump
  start?


The jump starter needs to overcome resistance of the jumper leads and connections. At several hundred amps it doesn't take much resistance to cause a large voltage drop. The battery in the jumper starter is probably a high current lithium type, which puts out higher voltage than a 12 V lead-acid battery. This is good because unlike jump starting from another vehicle you don't have the option of running the engine to increase voltage (typically 14-15 V from the alternator).


  Secondly, because of this 15V spec, would it be safe to still use this
  to power the modem?


Perhaps, perhaps not. If the original power supply is unregulated (ie. a simple transformer and rectifier) then it may normally put out quite a bit more than 12 V. On the other hand, if the power supply is a switch-mode regulated type then the modem may be expecting no more than 12 V. Older modems often used an AC supply (ie. just a transformer) with the rectifier built into the modem. Powering such a modem from DC may either not work (if it derives positive and negative voltages from the AC voltage) or could overheat the half of the rectifier that is passing the DC current.       

So again, if you don't know exactly what the device can handle then matching the original supply specification is the only way to guarantee success.