The While cross-entropy is the most common task for such prediction tasks, it did not work well for the amino acid sequence prediction task, as discussed below. A loss function of pairwise Kullback-Leibler (KL) divergence was used to measure the probability distribution of the predicted and the ground-truth amino acid sequences, as shown in Eq. 7, where 𝑃(𝑥) is the predicted probability distribtuion, and 𝑄(𝑥) is the ground-truth probablity distribution as a reference. To get these two prbability idstributions, a softmax activation function is applied to convert the logits produced from the model for each position in the sequence. The groundtruth probability distribution came from the one-hot encoding of each amino acid, followed by label smoothing and nomalisation. In the scope of the tensor of the same shape with true output language encoding 𝑦 (+56 and computed one 𝑦 &+64 , the pointwise KL divergence is determined in Eq. 8. , ESI.