Of course! Language models are designed specifically around understanding, analysing, and generating text. So it is possible to train these models on patents downloaded from a website.

Even though language model could understand the raw HTML that is download from the website (if you decide to do so), it is advised that you clean up and structure the data in a way that is more appropriate to what you are trying to achieve.
spam fails_task lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor helpfulness creativity violence