What are the challenges in changing a PhD supervisor after 2/3 of PhD is done
I have seen a lot of people switching advisers during my PhD. There are a few challenges I think you will face:


Your adviser, knowing his personality, won't let you go easily. He might tell other faculty there will be consequences if they take you as a student. It depends on his relationship to his colleagues and position within the department if he can make good of his threats. You should choose your future adviser from the faculty he cannot influence. If that is not possible, be prepared to transfer to a different graduate school. You may need to take a qualifier exam there again. They could also be happy to admit you if you have taken it already at your current school.
If you need to switch to a very different field, you should expect another 2-3 years as a beginner. You would graduate later. 
You seem to be already burnt out. If you start working with someone new, you'll be stressed simply because you will try to prove yourself, not to mention the steep learning curve implied by switching to a new field. It would be a good idea to take a vacation before.
Some departments have rules that limit the availability of funding from teaching assistantships once you have been long enough in the program. Make sure your new adviser can support you.
If you have family, you have to make sure they understand that you were forced to switch advisers and you might finish the PhD later. 


Given what you wrote, I believe switching advisers would be a good move. If you move within the same department, you have the advantage of knowing already who are the best advisers. If you don't, talk to their students and postdocs before switching.