Unnormalized Log Probability - RNN
You are right in a sense that it is better to be called log of unnormalized probability. This way, the quantity could be positive or negative. For example, $\text{log}(0.5) &lt; 0$ and $\text{log}(12) &gt; 0$ are both valid log of unnormalized probabilities. Here, in more detail:


Probability: $P(i) = e^{o_i}/\sum_{k=1}^{K}e^{o_k}$ (using softmax as mentioned in Figure 10.3 caption, and assuming $\mathbf{o}=(o_1,..,o_K)$ is the output of layer before softmax),
Unnormalized probability: $\tilde{P}(i) = e^{o_i}$, which can be larger than 1,
Log of unnormalized probability: $\text{log}\tilde{P}(i) = o_i$, which can be positive or negative.