Valid divergences are non-negative and attain a value of zero if and only if π η = r τ , but otherwise vary in the manner by which they penalize differences between the source and target distributions . So-called mass-covering divergences, such as the forward KL divergence D KL (r τ ∥π η ) = E rτ (z) [log r τ (z) -log π η (z|τ )], prefer policies where r τ (z) > 0 =⇒ π η (z|τ ) > 0, thereby steering the policy iterates towards solutions that cover the bulk of the essential support of the target r τ . On the other hand, mode-seeking divergences, such as the reverse KL divergence D KL (π η ∥r τ ) = E πη(z|τ ) [log π η (z|τ ) -log r τ (z)], prefer policies where r τ (z) = 0 =⇒ π η (z|τ ) = 0, which can often have the pathological effect of steering the policy iterates towards the most salient and prominent nearby mode(s), leading to training instabilities that fail to consider the bulk of the target distribution (i.e., mode collapse). Policies optimized according to mass-covering (cf. mode-seeking) divergences show bias in favor of having a high recall (cf. precision) with respect to its coverage of the essential support of the target distribution.