PMI between lemma vs surface
The result that you are calling awkward is indeed what we expect from the definition of PMI. Here are some identities involving PMI:

$\operatorname{pmi}(x;y) \equiv \log\frac{p(x,y)}{p(x)p(y)} = \log\frac{p(x|y)}{p(x)} = \log\frac{p(y|x)}{p(y)}$

Let $x=$'went' and $y=$'to go' and use the last expression. $p("to go"|"went") = 1$, and voilà, only $p("to go")$ remains in the denominator of the expression.