Dealing with poor management, lack of supervision, and the desire to move into a different field
It almost goes without saying but still: it's really hard for internet strangers to correctly judge exactly how difficult this situation is for you. I'm answering just based on what you wrote and it's entirely possible I am missing the mark here. In that case, feel free to disregard my answer.

I'd be a bit wary to apply to a different school for a PhD now. It will indeed look bad on your application, you will need to get letters of recommendation from other people than your current supervisor, and you will essentially lose all the time you put into your PhD until now.

Your supervisor's absence and lack of local leadership is really demotivating. And it is easy to lose interest in your own research in such circumstances. Then again, research can at times be very demotivating and future research employers will want to see a certain tenacity in you dealing with demotivation. Giving up three years of research to go somewhere else does not show that tenacity.

Are there other options to explore? Can you move overseas with your supervisor? Can you start collaborating with a local professor, or find local trustworthy people to help you regain direction and interest? Is there any way you can involve your current supervisor more in your project?