Do I need a full BMS or just protection circuit?
Lithium batteries need both charge control and discharge control.  The charge controller will do half the job. But what is protecting your batteries from being overly discharged?  What if the speaker gets left in the on state with no signal, so it's silently discharging the batteries?