Is there a chance to get in math phd for a physics student?
There is a chance, but you may need some preparation first. And you may need to make an appeal for a special situation.
In the US, the undergraduate math program consists of approximately 10 - 16 semester-length courses covering the major areas of math and a few less mainstream topics. If you don't have something very close to that, then there are probably gaps in your education that will affect your chances.
Pick a couple of graduate programs you are interested in, and look at the undergraduate math programs at those institutions, especially the required courses. That will give you a sense of where you stand.
But the only way to get a valid answer to your question is to apply for admission to a program, listing what you know and having letters of recommendation that attest to your likelihood of success.
But you can probably also talk to someone at a university you are interested in and ask them for an informal assessment of any gaps you need to fill. Then, you might need to take a year or so seriously filling those gaps, perhaps by enrolling as a non-degree undergraduate somewhere.