How can getting a masters hurt PhD admissions?
If you graduated successfully from a good math program in the US, I doubt that you are under qualified for a typical doctoral program in the US. Things might be different in other places, but in the US, the normal path to a doctorate doesn't start with and MS. In fact, the early couple of years of such a program include a lot of coursework to give you the needed breadth to enable you to begin research. But programming knowledge along with algorithmics should be fine and your math background suggests you have an analytical way of thinking that isn't too different from that of most CS students. 

My suggestion would be to apply to a few doctoral programs and see what feedback you get. I doubt that what you learn in the first couple of years there will close off future options in other fields if you decide it isn't for you. But I also doubt that getting a masters will lessen your chances for later getting a doctorate at the same or a different field. 

There may be a couple of universities that favor their own students for admission to doctoral programs, but that is fairly rare. Perhaps the advice you got was from someone with experience at such a place. I don't know why Princeton comes to mind, and my information is very old, but I do associate Princeton with the idea of favoring their own. 

Doctoral study normally consists of two major hurdles. You need to pass some qualifying exams and you need to write an acceptable thesis. The coursework is mostly (not all) to get you ready for the exams. But an MS in field also does that part.