To improve the calibration of the model, we can use other types of confidence scores rather than the logprob, such as self-evaluation . That is, at the end when the model outputs the <end> token, itself or another side model outputs a confidence score directly. This could be a better metric than logprob for final selection (logprob is still used in beam search).