Switched outlet still has some power when off
IMHO, it could be one of three things:

1) A feature of the lights. Some lights might have a "nightlight" feature.

2) Electromagnetic interference (EMI). Depending on how much electricity the lights require, and how dim they are, this might be possible. Especially if it's a half switched receptacle that was wired with a single 14-3 wire and the constant hot is powering the other half of the plug and something is plugged into it and drawing power while the switch is off... but I guess it could feasibly happen if the wires were just run next to other wires that are constantly powering something else. If this is the case, there's nothing wrong - it's just annoying

3) There's a small arc fault. I don't know how to verify this for sure because it might look just like EMI. These can go undetected for years and have the potential to start a fire. I've opened up walls before and found that half the 2x4 was burned away. I suggest doing whatever is cheapest and easiest first just in case the fault is in something easily replaceable (like replacing the switch and/or receptacle, or checking the receptacle with a volt meter). Or you could start with this just to make sure it's wired correctly (although I don't see how that could be the problem). But if you can't find it easily right away with those things, I'd install an AFCI on that circuit... and if that continues to trip, you know you have a serious problem, and it's time to call a professional electrician with equipment that can trace the wire and/or find the location of the fault.