To address this, we sample from an alternative distribution informed by the policy, but defined over a set of latent codes such that (i) they mutually attain a high log likelihood under the policy and (ii) they are sufficiently spread out to encourage coverage of distinct modes of the policy .