Pursuing a PhD in applied math with a large amount of student debt
First, taking on a large amount of debt at this point could be a big problem. However, several candidates for president are currently discussing forgiving student debt, a good thing to do in my view. It will be seriously discussed over the next 15 months, at least. It may be that your debt problem will get erased. Whoopie. 

Second, spending your life doing something you aren't happy with is a terrible existence, but you say you are relatively happy now. There is no guarantee that you will land something better, but having already gone to ivy league places that seems like less of a problem. 

Third, ignore any issues about age. You will (hopefully) reach 45 with or without your degree in any case. Graduating after 50 is an accomplishment, just as is graduating at 30. 

My suggestion, worth about two cents, of course, is to continue on applying and see what happens. For a degree program in applied math you probably don't need to make an unbreakable commitment and can keep flexible as you see what happens, both in the general political/educational scene and in your life personally. 

Of course your future spouse has a say in how she wants to live her life, so don't forget to factor that in as well.

But none of your impediments seem to be actual blocks. Be your best.