Careless uninterested potential advisor
Professors are busy and are always juggling multiple commitments.

This does not mean that you are unimportant to them but maybe some judicious memory jogging is required for them to remember what X has promised you.

Another excellent strategy which I have adopted is to show up to my committee members' open office hours with issues when I need them solved instead of bothering them over email. Usually I need 5-10 minutes of their time to explain my issue and solicit future directions. That works really well. I am not a big fan of long meetings.

In response to your specific question, if you want to not work with X just tell X that you do not feel that you can work with them in the future. It might be helpful to have someone else lined up before you do so though.