Li-ion Battery - Constant Charge
Investigate the ESR (series resistance) of different supercaps. Some can take high currents, some (designed for memory backup) can not.

A 1 Farad capacitor (small for a supercap!) with low ESR could supply the missing 100mA for 10 seconds (losing about a volt) which greatly exceeds your requirement. So with suitable circuitry to limit the charge current they certainly store enough energy.
 Whether you can simply parallel it with the USB supply or need some switching arrangement to supply the extra power may depend on your voltage tolerance needs.

Searching Farnell for 1F capacitors shows 27 available with a wide range of ESR values :
0.5 ohm price £1.60 (or under £1 at 10 off)
0.031 ohms at a ridiculous price, probably useful for starting big motors!
and many with ESR of 30 ohms or higher.

I would consider the former (0.5 ohm ESR) with a suitable series resistor to limit charging current, and a Schottky diode across it (forward biassed when discharging) and ask if something as basic as this meets your requirements. Failing that, a MOSFET across the resistor switched by a brownout detector can give better voltage regulation, but with more complexity and expense.

EDIT : APOLOGIES : the highlighted capacitor is only rated at 2.5V. Further searching shows a more expensive 5V rated one to maintain simplicity, alternatively a boost regulator could generate 5V from this capacitor when demanded by a brownout detector.