a BERT-like encoder and a GPT-like decoder but handle the attention mechanism differently to circumvent the quadratic dependency on sequence length, i.e., by using block-sparse attention or local attention to tokens within a certain window-size, respectively. Pre-trained versions of both models and tokenizers were used for the fine-tuning approach (see the Methods section for details), but the action tokens were added to the respective pre-trained tokenizers. After tokenization of the datasets, the mean length of the input sequences was 187 tokens when using the BigBirdPegasus tokenizer and 221 tokens when using the LED tokenizer (see Figure .c). Given the length distributions of the tokenized datasets, a maximum sequence length of 512 for training the BigBirdPegasus model was used (longer sequences were truncated). Hence, the original full attention instead of BigBird's block-sparse attention is used since there is no benefit in using block-sparse attention for sequence lengths less than 1024 tokens. For the LED model, a maximum sequence length of 1024 is used, since the window size of the pre-trained model was set to 1024, and the sequences have to be padded to a multiple of the window size. To keep the total training time similar for both models despite the longer sequence length used for the LED model, the "base" version of the LED model is used, as compared to the "large" version of the BigBirdPegasus model. The models were then trained individually on the training split of the datasets (see Methods section for training hyperparameters and details) and evaluated on the evaluation split.