After training was completed, the surrogate models were evaluated on the evaluation split of the dataset using the ROUGE and BLEU scores. Both are commonly used metrics that range from 0 to 1 (higher is better) to assess the quality of text generated by NLP models, e.g., in machine translations or text summarization. The action graphs generated by Llama-3.1-8B-Instruct for the validation split were used as ground truth labels. The results are shown in Table . As already stated for the "Chemical reactions from US patents (1976-Sep2016)" dataset, duplicate reactions are frequent in the dataset due to the same or highly similar text occurring in multiple patents, and many reactions from patent applications also appear later in granted patents. This means that no strict separation of training and validation data can be guaranteed, and the metrics reported here for the validation dataset might be artificially inflated. This is however hard to avoid in practice, and the metrics are only used here for a relative comparison of the models trained in this work without attempting a comparison with the scores reported by other authors. It should also be noted that the ChemicalTagger dataset contained 5 tags less than the Llama dataset, including the very frequently used tag <MIX>. Since the evaluation split of the Llama dataset was used as ground truth for calculating the scores, the maximum achievable score for the surrogate models trained on the ChemicalTagger dataset is less than 1.0. Hence, for a better comparison, the scores the ground truth labels from the validation split of the ChemicalTagger dataset achieved when compared to the ground truth labels of the Llama dataset are also given (rows 2 and 3 in Table ). Interestingly, the surrogate models trained on the ChemicalTagger datasets achieved very similar (and in some cases even slightly higher) scores than the validation split of the ChemicalTagger dataset itself, indicating the ability of the models to leverage the context-sensitive token embeddings they learned during the pre-training phase to match (or even outperform) rule-based annotations. Lastly, these data also reveal that the raw output of these specialized surrogate models comes close to the raw output of the more general and much larger LLM (Llama_raw) while using only a fraction of the time and computational resources and not requiring any ICL prompt engineering, post-processing, or iterative prompting to remove unwanted action tags. Example outputs for experimental procedures from the domains of materials science, organic chemistry, inorganic chemistry, and a patent that were not part of the training or evaluation dataset are given in the supporting information (Table ), demonstrating the ability of the models to also adapt to procedures from other domains. In general, the models trained on the Llama dataset produced better outputs and struggled less with inconsistent uses of spaces or other characters in the example procedures. For example, in the part of an experimental procedure that read "A mixture of […] and K 2 CO 3 (138 g, 1.0 mol) in DMF (500 mL) was heated at 95° C" (Example 5 in Table ), both models trained on the ChemicalTagger dataset failed to annotate the addition of K2CO3 (as well as the subsequent ), since there is no <CENTRIFUGE> tag in the training data annotated with ChemicalTagger. Interestingly, the models trained on the Llama datasets, that included the <CENTRIFUGE> tag, did not annotate this step at all. Lastly, it should be highlighted that especially the BigBirdPegasus model trained on the Llama dataset was capable of some remarkable generalizations that were most likely never presented to the model during the fine-tuning phase.