Does doing a PhD in a small/simple university make sense after getting a Master's from a very prestigious school?
I guess the first question is What do you want to do with your PhD?

If you want to stay in academia, you will have the same problem after graduating all over again (and, speaking from personal experience, as a young postdoc/faculty, one does not generally have the luxury of choosing where one goes next). Further, going to a (presumably) under-funded local university with weak research programme is going to severely reduce the probability of you doing world-class research. While Moriaty is correct in saying that "It's the quality of your individual work that counts.", said quality is definitely strongly correlated to the excellence of your advisors and fellow students and collaborators. Another problem may be funding (e.g., for going to conferences), which may not be plentiful in a small university.

What I am also wondering is whether you actually need to switch university, or should just live off-campus? I work in Zurich, and I have one student who used to live between Zurich and Bern (about a one-hour car drive away), in an area that was pretty much as rural as it gets. He just did a lot of home office, and took the train approximately 3 times a week. Maybe an arrangement like that helps you more than throwing out the baby with the bath water?