How to leave a PhD without angering my academic references?
Note that I have exhausted all the logical possibilities


It doesn't sound like you have, though. You need to explain, in very clear terms, that you are unhappy enough to leave the program and apply somewhere else.*

Don't hold back, but I would avoid using vulgar language (you know, like in your titular question), and would otherwise strive to be professional about it.

Now, most academics are sane, understanding people. If your advisor comes up with some potential solutions that, after reflecting on them, you decide are not satisfactory, then you should make your decision to leave the program and ask your advisor for a strong letter of recommendation. (I would encourage you to be patient when working through the potential solutions with your advisor, though, as there may need to be some iteration.)

The key here is to really, really, really make every attempt to find some common ground with your advisor that works for both parties before deciding to "jump ship." I don't see how a sane, understanding advisor could possibly hold against you wanting to put yourself in a better situation, so long as you make every possible attempt to make things work out where you currently are first.



* I don't mean to be rude, but from the comments below your question, you seem to have an answer for everything, so I want to minimize that type of interaction here. My impression is that you are beating around the bush with your advisor, and not really giving them the "full scoop." Just basically tell them what you have told us here, but in a professional manner.