Charging two 18650 batteries in series with a 5V USB wall charger?
You'll need a module designed to charge your two batteries in series (commonly referred to as a 2S arrangement).  Fortunately, these modules are available at low cost.  Here is one example (ebay has many more): https://www.mouser.com/new/dfrobot/dfrobot-dfr0564-usb-charger/

Note that you will need very similar cells, with the same capacity and health (ideally from the same manufacturing batch), to make sure the cells share the voltage fairly evenly.  In addition, you might consider a battery management circuit (BMS) to ensure they stay balanced.  You'll also need a circuit to avoid over-discharge, which can lead to a dangerous condition (vent with flame) next time you try to charge it.  Most BMSs will include this functionality as well, or you can build it into your boombox.  

It's also worth mentioning that three batteries in series (3S) is a somewhat more commonly-used arrangement, and it's often easier to find chargers and protection circuits for this configuration.