Similarly the non-exponential-averaged (or non-EMA) version of the checkpoint should be used when continuing fine-tuning of the model checkpoint, but should not be used when using the checkpoint for inference. 