How to protect scientific open research from being patented?
Legally speaking, any disclosure of an invention prevents it from being patented: it counts as prior art. Disclosure includes not only publication but also public demonstrations, as well as sales (except possibly when subject to a non-disclosure agreement). This is part of the novelty requirement of patentability.

There is an exception: in some countries, the inventor has a grace period after disclosure during which they can still file a patent. This only applies to the inventor, a third party isn't allowed to file a patent for something they didn't invent, or for something that somebody else has already published. The longest grace period around the world is 1 year, in the US. Some jurisdictions such as the EU have no grace period.

Thus if you publish your research, this protects from anyone subsequently filing a patent on it. Even if they had invented it previously, your publication would invalidate a future application. After publication, the only remaining risk is that you, or a fellow inventor who worked with you, take advantage of the grace period to file a patent in countries where this is possible. I don't know of a way for you to legally guarantee to others that you won't file a patent.

Note that you need to publish (or at least publicize — a private disclosure is sufficient to invalidate a patent unless it falls under the scope of a confidentiality agreement). If you invent something and don't publicize it, that allows someone else (presumably working independently, since you didn't tell them anything) to file a patent on their own invention. Being the first to invent doesn't bring you an advantage (it used to in some countries, but since the US switched in 2013, all countries have a first-to-file system). Fortunately, publishing open research shouldn't be an onerous requirement.

That's the law. In practice, patent examiners don't always find prior publications. The ubiquitous Internet has made it easier than it used to be, but conversely the increasing volume of available knowledge has made it more difficult to find the needle in the haystack. If a patent has been accepted by a patent office, it can still be contested due to the existence of prior art, but this requires a court decision, which can be costly.

Some research institutions (public or private) have a formal practice of defensive publications: publish potential inventions that they don't intent to patent as soon as possible, in order to block anyone else from patenting them. Technically, any publication is a public disclosure, including an arXiv preprint, a blog post, or even a research seminar if it's legally open to external visitors. However, since it's easier to fight a patent before it's granted, it is advantageous to make it easy for patent examiners to find the defensive publication.

If you're concerned about someone filing a patent on something you discovered, or for that matter anything that you know about, you can watch patent applications. Patent applications are published for a period of at least a few months, during which time anyone can point the patent examiner to something that they consider to be prior art. Stack Exchange participates in this process through their Patents site where people can coordinate prior art searches.