Are these LEDs glowing due to leakage current? How to prevent it?
Your problem is that you need something to drive the Anodes of the LEDs down to GND. I would try the following:



The key is R6, which should carry any possible leakage currents. I would use a value ranging in the order of 4K7 or so, that should be more than enough.