How does the ranking of a university relate to a future career in mathematics?
First, let me answer an easy question.  It is not frowned on to get a masters at one school, then transfer elsewhere for a Phd.  In fact, this is often encouraged.  Next, let me confirm that yes, the ranking of your PhD institution really does matter for a future career as a research mathematician.

In some sense, Gerry is right when he says that the ranking of importance is (1) quality of dissertation, (2) reputation of adviser, and (3) ranking of department.
One big problem though, is that many of the people evaluating you won't know how to evaluate criteria (1) and (2).  Hiring committees often include many (all?) people who are not familiar with your research area.  So they will often rely on what they (at least think that they) know, which is quality of department.

I have written elsewhere about why you should go to the strongest program that you can get into, particularly if you want a career as a research mathematician.
I don't know all the details of your personal situation, but I strongly encourage you to at least take the GRE again and reapply.  It may be possible for you to wait to see whether you get accepted to a school that you prefer before you decide whether you'll leave with a terminal masters or not.