I failed to respond to a potential advisor
If it is a coursework masters, then you would have minimal or no interaction with the previous professor, unless the professor is teaching a subject in the Masters. 
If it is a research masters and you are in the same department, then you would be working or at least bumping into the prof at departmental meetings and events. 

Seems unlikely that there would be great animosity. Your circumstances seems very reasonable and most people would not react adversely or take it too personally. I am sure if you are in the professor's department, then you can have a casual chat and smooth over any frustrations. The prof might be able to collaborate or support you in some way as they were previously interested.