3 AA batteries vs. USB battery pack?
It will probably be as easy as you are hoping. 

Chances are, the red wire is positive and the black wire is negative. 

Can't say for certain without knowing more about the LED string, but chances are if it could be powered with 3 AA batteries, a USB will have more than enough capacity. Hopefully there is an inline current limiting resistor to the LED's, and therefore the additional 0.5 V will be dissipated over that. A little more wasted power, but nothing to worry about.