LED's connected to a relay remain on?
What you have there are solid state relays.  They are designed to switch AC, not DC.  

They rely on the voltage flowing through the switched side falling to 0V (the "zero crossing" point of an AC waveform) to turn off.

They are not suitable for switching DC.