How to connect LED strip to be powered by charger and batteries
Looks mostly fine. I'm not overly familiar with this specific charging module so correct me if I'm wrong, but it looks like there is an input to charge the batteries, but no output pins, so you would have to directly draw current from the Li-ions. It should be fine for just a 2A draw, but any higher and I would make sure I have output current protection.

When your batteries drop below 4V, you wont be able to directly supply 12V any more. I would suggest a 12V regulator or some kind of LED driver to maintain constant and adequate power.