Is it crazy for a PhD student to pay open access fees out of pocket?
I don't think you're crazy at all. From a purely economic point of view, although $500 sounds like a lot of money, it is quite insignificant (in a US context) compared both to your future earnings once you graduate, and to the present potential earnings you are foregoing by attending graduate school in the first place instead of working at a "real job". Thus, if you truly think that paying the fee will help you get on with your other research projects and graduate faster and/or with a more impressive publication record, then, assuming you can afford to pay the fee without sacrificing other important or crucial things at least, it seems to me that paying the fee is a reasonable decision.

With that said, I think a note of caution is also in order. My main concern is that a good proportion of the journals that charge open access publication fees are predatory journals, and given that you are a graduate student and new to publishing, there is a risk that the journal you are considering publishing in could be one of those. I would therefore suggest not paying any fee until you have made absolutely certain, by consulting your advisor or other experienced researchers, that the journal in question is a reputable one and that publishing in it will actually further your career and goals. Good luck!