A natural choice is to use normalizing flows , a class of generative models in which noise is sampled from some base distribution p(ϵ) that admits straightforward sampling and log density evaluations (e.g., an isotropic standard Gaussian) and transformed through a series of neural network layers that are specially constructed so as to be invertible functions with efficient evaluation of the log absolute determinant of the Jacobian. Because the entire network is invertible, log densities for the output distribution can be computed via application of the change of variables formula. Let g η : R d × R + → R d be a conditional normalizing flow with parameters η, such that g η satisfies g η (g -1 η (z, τ ), τ ) = z for all τ by construction. Then the conditional log density for z under the policy can be evaluated via log π η (z|τ ) = log p(g -1 η (z, τ )) + log det