Is it rude to change universities/supervisors after already indicating research interest with a current supervisor?
If Y is what you really want to do then go for it. There is no point in going down a career path just to please someone else.

Talk in person to your current supervisor once you have a written offer from the university that does Y. Explain to him that it is your dream to do Y and you hope to keep in contact with him even though you are leaving the university.

A good supervisor will understand - if not then he was not a good supervisor anyway. It's a win-win for you.