Switch to mathematics
You don't need to have earned any honors or prizes in math.  You do need to show that you are a strong student in general, and that you have sufficient background in math to be able to hit the ground running.

Physics and math are very close and give you a great deal of fluidity, especially in the direction you want to go -- as long as you have taken enough math classes.  (Some people graduate in physics with the bare minimum of math classes -- with PDEs as the most advanced class taken -- and they would have trouble making this switch -- but you are not one of those people.)