Advisor is unhappy with M.Sc. research results and is showing signs of preventing the student from graduating. What should be done?
This is half of an answer.

Has anybody ever heard of anything similar?

Yes. Conflicts like this unfortunately is not uncommon, judging from what I have observed in only two universities.
Indeed, it appears to be more and more common.

Is there an appeal process that I could take, if I am not allowed to graduate?

Formal appeal process?  Yes. The process varies from department to department. You should be able to find it in your students' handbook.
However, this should be considered as a last resort ---
it is quite unlikely going to work if your goal is to graduate.
Looking at the bigger problem though,
it seems the most straightforward solution is just to work hard and get results that will be sufficient to meet the requirement (publishable). This will certainly solve your problem, and it is likely the most efficient way.
If that's not the kind of solution you are looking for, then first, you need an unbiased assessment of your work to determine if you actually did work that is sufficient for graduation. While you may feel you have done enough, you need to consider the possibility that you are not an expert yet, and your assessment may not be correct.  If you can find another expert who thinks your work is sufficient for graduation, you may have a case, otherwise, I'm afraid there is not much you can do (if your goal is to actually graduate).
I disagree with the first comment, I think it is quite dangerous to start with graduate program director or some kind of ombuds. This would indeed be the official channel, but if you do that, your chance working out a reasonable solution with your advisor will be greatly reduced.