Will my cancer ruin my chances of going to graduate school in math?
First of all, I’m happy you’re doing better! It would take a truly heartless committee to ignore the circumstances. In the US it’s illegal if I’m not mistaken, but from a recruiter’s perspective I would definitely see the fact that you overcame adversity and managed to graduate with a good GPA as a point in your favor.

Mention this in your statement of purpose. Don’t be melodramatic, just state that you had to take a break from your studies to fight cancer, and then got back into it once you were done. I’d see that as a sign of fortitude and perseverance!

Good luck!