Is it possible to apply for Ph. D. program in Physics with a CS background?
I don't know if it's possible. It depends on the country. If you live in one of the members of the UE, it is- this is the so called Bologna Process.

But another question is whether you should do it and I strongly advise you not to. You need strong knowledge of Maths and Physics. I highly doubt you will be able to catch up with that. Even if you do, it will take up a major part of your PhD and you won't have time for real work.