1) Sampling with Temperature: Toggling the intensity of probabilities by perturbing logits with temperature (sensitivity) hyperparameter δ > 0. For the penultimate layer output of i th sequence at time step t z = [z 1 , . . . , z |V| ] T , softmax with temperature is defined as: