How much does rigor and time commitment of a math PhD correlate to prestige of the program?
I think a more productive way to factor your considerations for your spouse and your family life into your choice of program would be by considering things like:


your spouse's preferred climate (temperatures, rain, snow, etc.)
how isolated vs. how easily connected your spouse would be with others -- there are two aspects here -- both people of your spouse's nationality, if your spouse is originally from a different country than the country where you hope to study, and also people not of your spouse's nationality (in the U.S., international trailing spouses sometimes find it rewarding to form friendships with other non-U.S. citizens, who are not necessarily of the same nationality)
how easy and quick it would be for you to pop home for dinner before getting back to school for some evening work, or for your spouse to stop by your department to spend some time with you in a study break
possible employment opportunities if that is something your spouse is interested in