Pointwise mutual information vs. Mutual information?
PMI can clearly be negative, since if $p(x,y)&lt;p(x)p(y)$, then taking $\log \frac{p(x,y)}{p(x)p(y)}$ gives a negative number.

But MI is always non-negative, as it can be written as $d_{KL}(p(x,y)||p(x),p(y))$, where $d_{KL}$ is the Kullback–Leibler divergence which is non-negative (the proof of the latter is not completely trivial).

Intuitively, this happens because if there is a point such that $p(x,y)&lt;p(x)p(y)$, then the probability must "compensate" for this later, with a pair $p(x',y')&gt;p(x')p(y')$.