What should I say to my project advisor who thinks I may not get a Ph.D position?
Well, he has a point. You are not guaranteed to get a position (no one is), and you should have a backup plan.

Some possible solutions for a physicist are working as a software developer, a quant in finance, or teacher. You may also take a temporary job, hone your skills, and try again the next year. Another pre-emptive option, and what he probably has in mind, is to change to a field you are stronger in, and probably have a better chance of getting a position.

This said, if you really like this field, go for it and give it a shot. Just convince your potential supervisor that you are not delusional, you know the risks, and you are not overcomitting to a wild goose chase.