Led strip turning on dimly when touched
Is this caused by me working as an antenna or by me working as a capacitor for some low current AC signal on the output of the power supply?

Forget Antenna, indeed it is mostly capacitive coupling and leakage of your body to all the mains wiring and mains connected devices in your house.

Is this safe?

Yes, assuming a properly designed power supply, the current will be so small that it cannot harm a human. Even if you'd touch the LED strip and touch a grounded conductor with your other hand.

Is the power supply working properly?

Yes, there will always be some capacitive coupling to the mains lines through a power supply. It cannot be avoided. As long as it is not too much (meaning a harmful (to humans) current can flow) this is OK.

Is the remote control switch working properly?

Yes, switching off only one line is OK as there should be enough isolation in the powers supply to safely touch the supply's output even if it only is connected to the live wire.