Sequence-to-sequence fine-tuning is analogous to pre-training; inputs are passed to the encoder, right-shifted outputs along with the memory embeddings from the encoder are applied to the decoder, and the decoder output embeddings are passed through a fully-connected layer to produce a distribution over the model's vocabulary. A cross-entropy loss function is used to train the model.