Variational Inference using Implicit Distributions
Ferenc Huszár
 

[
Variational Inference using Implicit Distributions

equal*


Ferenc Huszártwtr

    
twtrTwitter, London, U.K

Ferenc Huszárfhuszar@twitter.com

variational inference, adversarial training, density ratio estimation

0.3in
]



 
Generative adversarial networks (GANs) have given us a great tool to fit implicit generative models to data. Implicit distributions are ones we can sample from easily, and take derivatives of samples with respect to model parameters. These models are highly expressive and we argue they can prove just as useful for variational inference (VI) as they are for generative modelling. Several papers have proposed GAN-like algorithms for inference, however, connections to the theory of VI are not always well understood. This paper provides a unifying review of existing algorithms establishing connections between variational autoencoders, adversarially learned inference, operator VI, GAN-based image reconstruction, and more. Secondly, the paper provides a framework for building new algorithms: depending on the way the variational bound is expressed we introduce prior-contrastive and joint-contrastive methods, and show practical inference algorithms based on either density ratio estimation or denoising.




§ INTRODUCTION



Implicit distributions are probability models whose probability density function may be intractable, but there is a way to


  * sample from them exactly and/or calculate and approximate expectations under them, and

  * calculate or estimate gradients of such expectations with respect to model parameters.


A popular example of implicit models are stochastic generative networks: samples from a simple distribution - such as uniform or Gaussian - are transformed nonlinearly and non-invertably by a deep neural network. Such networks can flexibly parametrise a wide range of probability distributions, including even degenerate ones which may not even have a continuous density.

Implicit models have been successfully applied to generative modelling in generative adversarial networks <cit.> and subsequent work <cit.>. They work particularly well for visual data, partly because they can exploit the inductive biases of convolutional neural networks, and partly because they can flexibly model potentially degenerate, manifold-like distributions which natural images are assumed to follow.

This note is about using implicit distributions in another important probabilistic machine learning problem: approximate inference in latent variable models. Unlike in the first applications of GANs to generative modelling where an implicit model directly models the distribution of observed data, in approximate inference we are interested in modelling the posterior distribution of latent variables given observed data. Direct generative modelling and approximate inference are very different problems indeed: in the former we are provided with samples x_i from the distribution to be modelled, in the latter we are given a joint distribution of latents z and observations x, and a set of observed samples x_i, but no samples from the posterior itself.

In this note we focus on variational inference (VI) which works by minimising a divergence between the approximate and real posteriors. More precisely we follow the usual KL-divergence formulation, but other, more general variational methods exist <cit.>. VI also provides a lower bound to the marginal likelihood or model evidence - the evidence lower bound or ELBO - which can be maximised with respect to parameters of the latent variable model to approximate maximum likelihood learning. It's important to keep in mind that despite several algorithms in this note look and feel like adversarial training procedures, the way the model is fit to observed data is more akin to variational auto-encoders <cit.> than to GANs.

There are several reasons to explore implicit distributions in the context of variational inference. Firstly, explicit VI  is often limited to exponential family distributions or other distributions with tractable densities <cit.> which may not be expressive enough to capture non-trivial dependencies that the real posterior exhibits. A flexible, implicit distribution may provide a better approximation to the posterior and a sharper lower bound. Secondly, It may be desirable to use an implicit likelihood as the resulting latent variable model may fit the data better. For example, the likelihood or forward model might be described as a probabilistic program <cit.> whose density is intractable or unknown. Finally, sometimes we may want to use implicit priors over latent variables. For example in a deep hierarchical latent variable model the prior for a layer may be a complicated probabilistic model with an intractable density. Or, when solving inference problems in computational photography, the prior may be the empirical distribution of natural images as in <cit.>. In summary, any or all of the prior, the likelihood and the approximate posterior may have to be modelled implicitly, and we need VI  procedures that are ready to tackle these situations.

In this note we present two sets of tools to handle implicit distributions in variational inference: GAN-like adversarial algorithms which rely on density ratio estimation, and denoising-based algorithms which build a representation of the gradients of each implicit distribution's log-density and use these gradient estimates directly in a stochastic gradient descent (SGD) algorithm. We further classify algorithms as prior-contrastive and joint-contrastive depending on which form of the variational bound they use. Prior-contrastive methods only deal with implicit distributions over latent variables (the prior or approximate posterior), while joint-contrastive methods can handle fully implicit models where none of the distributions involved has a tractable density. This classification gives rise to a range of algorithms listed in Table <ref>, alongside related algorithms from prior work. All of the algorithms presented here can perform variational approximate inference, which is the main focus of this note, but not all of them can perform learning unless the likelihood is explicitly defined.


 




 §.§ Overview of prior work


Several of the algorithms proposed here have been discovered in some form before. However, their connections to variational inference is rarely made explicit. In this section we review algorithms for inference and feature learning which use implicit distributions or adversarial training. As we will see, several of these admit a variational interpretation or can be rather straightforwardly modified to fit the variational framework.

GANs have been used rather successfully to solve inverse problems in computer vision. These inverse problems can be cast as a special case of approximate inference. <cit.> used GANs to reconstruct and generate images from non-linear feature representations. As pointed out later by <cit.>, this method, DeePSiM, can be interpreted as a special case of amortised maximum a posteriori (MAP) or variational inference with a Gaussian observation model. GANs have also been used for inference in image super-resolution <cit.>. Connections between GANs and VI  in this context were first pointed out in <cit.>. <cit.> also introduced a modified objective function for the GAN generator which ensures that the algorithm minimises the Kullback-Leibler divergence as opposed to the Jensen-Shannon divergence, an essential step to using GANs for VI. The AffGAN algorithm presented there is highly application specific, thus, it does not solve VI in general.

In more recent, parallel work, <cit.> proposes adversarial variational Bayes(AVB), perhaps the best description of the use of GANs for variational inference. AVB is a general algorithm that allows for implicit variational distributions and is in fact equivalent to the prior-contrastive adversarial algorithm (, Algorithm <ref>) described in Section <ref>. Operator variational inference <cit.> formulates a general class of variational lower bounds based on operator divergences, resulting in a practical algorithm for training implicit inference networks without a tractable density. As is shown in the paper, the KL-divergence-based variational bound used here and in <cit.> is a special case of OPVI. Adversarial autoencoders <cit.> are similar to variational autoencoders where the KL-divergence term is replaced by an adversarial objective. However, AAEs do not use the KL-divergence formulation of the adversarial loss and their discriminator is independent of the encoder's input, thus they are not a true variational method. Finally, <cit.> proposed variational message passing, in which adversaries are employed to minimise local Jensen-Shannon divergences in an algorithm more akin to expectation propagation <cit.> than to variational inference.

Another line of research extends GANs to latent variable models by training the discriminator on the joint distribution of latent and observed variables. This technique has been independently discovered as bi-directional GAN <cit.> and adversarially learned inference <cit.>. These algorithms are closely related to the joint-contrastive adversarial algorithm (, Algorithm <ref>). ALI and BiGAN use the Jensen-Shannon formulation of GANs rather than the KL-divergence ones used here. On the one hand, this means that the Jensen-Shannon variants aren't technically VI  algorithms. On the other hand, the symmetry of the Jensen-Shannon divergence makes ALI and BiGAN completely symmetric, enabling not only approximate inference but also learning in the same algorithm. Unfortunately, this is no longer true when KL divergences are used:  is an algorithm for inference only.

The algorithms mentioned so far are examples of adversarial techniques which rely on density ratio estimation as the primary tool for dealing with implicit distributions <cit.>. <cit.> and <cit.> demonstrated an alternative or complementary technique based on denoising autoencoders. As shown by <cit.> the optimal denoising function learns to represent gradients of the log data density - which in turn can be used in an inference method. <cit.> used this insight to build a denoiser-based inference algorithm for image super-resolution and connected it to amortised maximum a posteriori (MAP) inference. The extension from MAP to variational inference is straightforward and this method is closely related to the prior-contrastive denoising VI (, Algorithm <ref>) algorithm presented here.







§ VARIATIONAL INFERENCE: TWO FORMS


In this section, we give a lightweight overview of amortised variational inference (VI) in a latent variable model, in a model similar to variational autoencoders <cit.>. We observe an sequence of N observed data 𝒟 = {x_n, n=1… N}. For each data point there exists an associated latent variable z_n, n=1… N. We specify a prior p_θ(z) over latent variables and a forward model p_θ(x| y) which describes how the observations are related to latents. In such model we are interested in maximum likelihood learning, which maximises the marginal likelihood or model evidence ∑_n=1^Nlog p_θ(x_n) with respect to parameters θ, and inference which involves calculating the posterior p_θ(z| x). We assume that neither the marginal likelihood or the posterior are tractable.

In amortized VI we introduce an auxiliary probability distribution q(z| x_n; ψ), known as the recognition model, inference network or approximate posterior. Using q_ψ we define the evidence lower bound (ELBO) as follows:


    ℒ(θ, ψ) = ∑_n=1^N{log p_θ(x_n) - q_ψ(z| x_n)p_θ(z| x_n)}


As the name suggests, ELBO is a lower bound to the model evidence p_θ(𝒟) and it is exact when q_ψ matches the true posterior p_θ(z| x) exactly. Maximising ELBO with respect to ψ is known as variational inference. This minimises the KL divergence [q_ψ| p_θ(z| x)] thus moving the q_ψ closer to the posterior. Conversely, maximising ELBO with respect to θ is known as variational learning which approximates maximum likelihood learning.

The ELBO can be calculated exactly for many combinations of p_θ and q_ψ, whose densities are tractable. VAEs use a re-parametrisation trick to construct a low variance estimator to ELBO, but still require tractable densities for both the model p_θ and recognition model q_ψ. If p_θ and/or q_ψ are implicit the ELBO needs to be approximated differently. As we will see in the next sections, it is useful to formulate ELBO in terms of density ratios.

There are two main forms considered here. Firstly, the prior-contrastive form used also by VAEs <cit.>:


    ℒ = ∑_n=1^Nz∼ q_ψ(z| x_n)[ log p_θ(x_n| z) - _θ, ψ(z,x_n) ]
       = ∑_n=1^N[z∼ q_ψ(z| x_n)log p_θ(x_n| z) - q_ψ(z| x_n)p_θ(z)],


where we introduced notation for the logarithmic density ratio _θ, ψ = logq_ψ(z| x_n)/p_θ(z).

We call Eqn. (<ref>) the prior-contrastive expression as the KL term contrasts the approximate posterior q_ψ with the prior p_θ(z).Alternatively, we can write ELBO in a joint-contrastive form as follows:


    ℒ(θ, ψ) = -N·(q_ψ(z| x)(x)p_θ(x, z) - ℍ[])
       = - ∑_n=1^Nz∼ q_ψ(z| x_n)_θ, ψ(x_n, z) - N·ℍ[],











where we introduced notation  to denote the real data distribution and ℍ[] denotes its entropy[In practice,  is an empirical distribution of samples, so technically it does not have a continuous density or differential entropy ℍ[]. We still use this notation liberally to avoid unnecessarily complicating the derivations.]. ℍ[] can be ignored as it is constant with respect to both θ and ψ. We also introduced notation _θ, ψ to denote the logarithmic density ratio _θ, ψ(x_n, z) = logq_ψ(z| x_n)(x)/p_θ(x_n, z). Note that while _θ, ψ was a log-ratio between densities over z, _θ, ψ is the ratio of joint densities over the tuple (x,z). As this form contrasts joint distributions, we call Eqn. (<ref>) the joint-contrastive expression.

When using implicit models the density ratios _θ, ψ and _θ, ψ cannot be computed analytically. Indeed, even if all distributions involved are explicitly defined,  is only available as an empirical distribution, thus _θ, ψ cannot be calculated even if the densities of other distributions are tractable. In this note we rely on techniques for estimating _θ, ψ or _θ, ψ, or their gradients, directly from samples. For this to work we need to deal with a final difficulty: that _θ, ψ or _θ, ψ themselves implicitly depend on the parameter ψ which we would like to optimise.



 §.§ Dependence of _θ, ψ and _θ, ψ on ψ


The KL-divergences in equations  and  depend on ψ in two ways: first, an expectation is taken with respect to q_ψ - this is fine as we assumed expectations under implicit distributions and their gradients can be approximated easily. Secondly, the ratios _θ, ψ and _θ, ψ themselves depend on ψ, which may cause difficulties. If one optimised ELBO na ively via gradient descent, one should back-propagate through both of these dependencies. Fortunately, the second dependence can be ignored: 


    ∂/∂ψ.z ∼ q_ψ_θ, ψ(z) |_ψ = ψ_0 = . ∂/∂ψz ∼ q_ψ_θ, ψ_0(z) |_ψ = ψ_0


The only difference between the LHS and RHS of the equation is in the subscripts _θ, ψ v. s. _θ, ψ_0. As _θ, ψ_0 is a constant with respect to ψ, Eqn. (<ref>) reduces to the gradient of an expectation with respect to q_ψ, which we assumed we can approximate if q_ψ is an implicit distribution. The detailed proof of Eqn. (<ref>) is in Appendix <ref>, the key idea of which is the observation that for any ψ_0


    z ∼ q_ψ_θ, ψ(z) = x, z ∼ q_ψ_0_θ, ψ(x, z) + q_ψq_ψ_0


A similar equation analogously holds for _θ, ψ in Eqn. (<ref>), or indeed, any other KL divergence as well.



 §.§ Approximate SGD Algorithms for VI


In the following sections we outline algorithms for VI which allow for implicit distributions. These algorithms can generally described as two nested loops of the following nature:



  * the outer loop performs stochastic gradient descent (SGD) on an approximation to ELBO with respect to ϕ, using gradient estimates obtained by the inner loop

  * in each iteration of outer loop, with ψ=ψ_t fixed, the inner loop constructs an estimate to _θ, ψ_t, θ, ψ_t, or more generally to the gradient in Eqn. (<ref>)


As long as the gradient estimates provided by the inner loop has no memory between subsequent iterations of the outer loop, and the gradient estimates provided by the inner loop on average constitute a conservative vector field, the algorithms can be seen as instances of SGD, and as such, should have the convergence properties of SGD.







§ DIRECT DENSITY RATIO ESTIMATION


Direct density ratio estimation, also known as direct importance estimation, is the task of estimating the ratio between the densities of two probability distribution given only i. i. d. samples from each of the distributions <cit.>. This task is relevant in many machine learning applications, such as dealing with covariate shift or domain adaptation. A range of methods have been introduced to learn density ratios from samples, here we focus on adversarial techniques which employ a discriminator trained via logistic regression. We note that other methods such as KLIEP <cit.> or LSIF <cit.> could be used just as well.



 §.§ Adversarial approach using discriminators


<cit.> proposed estimating density ratios by training a logistic regression classifier between samples from the two distributions. Assuming the classifier is close to a unique Bayes-optimum, it can then be used directly to provide an estimate of the logarithmic density ratio. This approach has found great application in generative adversarial networks <cit.>, which work particularly well for generative modelling of images <cit.>.

Let us use this to construct an approximation _ϕ to the logarithmic density ratio _ψ,θ from Eqn. (<ref>). We can do this by minimising the following objective function, typically via SGD:


    ℓ^_ψ,θ(ϕ)    = ∑_n=1^Nz∼ p_θ(_ϕ(x_n,z)) 
       - ∑_n=1^Nz∼ q_ψ(z| x_n)(_ϕ(x_n,z)),


where (t) = log(1+exp^t) and (t) = t - (t) are the softplus and softminus functions, respectively. Once the approximate log ratio _ϕ is found, we can use it to take a gradient descent step along the approximate negative ELBO:


    -ℒ_ϕ(ψ, θ)    = ∑_n=1^Nϵ∼𝒩(0,I)_ϕ(x_n,g_ψ(x_n,ϵ))
       - ∑_n=1^Nϵ∼𝒩(0,I)log p_θ(x_n| g_ψ(x_n,ϵ)),


where we re-parametrised sampling from q_ψ in terms of a generator function g_ψ and noise ϵ. When q_ψ is explicitly defined, this re-parametrisation is the same as the re-parametrisation in VAEs. When q_ψ is an implicit distribution, it often already defined in terms of a non-linear function g_ψ and a noise variable ϵ which it transforms.

Equations  and  are analogous to the discriminator and generator losses in generative adversarial networks, with 1/(1+exp(-_ϕ)) taking the role of the discriminator. Optimising the two losses in tandem gives rise to the prior-contrastive adversarial algorithm (, Algorithm <ref>) for variational inference. This algorithm is equivalent to the independently developed adversarial variational Bayes <cit.>.

As the likelihood p_θ(x| z) appears in Eqn. (<ref>), in Algorithm <ref> the forward model has to be explicitly defined, but the prior p_θ(z) and approximate posterior q_ψ can be implicit. Algorithm <ref> only describes variational inference - finding ψ given θ - but the approximate ELBO in Eqn.  can be used for variational learning of θ as well, with the exception for parameters of the prior p_θ(z).

Learning prior parameters involves minimising the KL-divergence ∑_n=1^Nq_ψ(z| x_n)p_θ(z) which is akin to fitting p_θ to samples from the aggregate posterior ∑_n=1^Nq_ψ(z| x_n) via maximum likelihood. If the prior has a tractable density, this may be an easy task to do. A more interesting case is though when the prior p_θ(z) itself is a latent variable model, in which case we can lower bound the said KL divergence with another ELBO, thereby stacking multiple models on top of each other in a hierarchical fashion <cit.>.

A similar adversarial algorithm (, Algorithm <ref>) can be constructed to target _θ,ψ in the joint-contrastive formulation of ELBO (Eqn. <ref>).  is very similar to ALI <cit.> and BiGAN <cit.> in that it learns to discriminate between the joint distributions (x)q_ψ(z| x) and p_θ(x,z). Unlike these methods, however,  uses the correct loss functions so it maximises an approximation to the ELBO. Unlike in , which required a tractable likelihood p_θ(x| z),  also works with completely implicitly defined models. As a downside,  provides no direct way for variational learning of θ. ALI and BiGAN exploit the symmetry of the Jensen-Shannon divergence to optimise for θ, but as  uses the asymmetric KL-divergence, this is not an option. Section <ref> explores an idea for fixing this drawback of .

















§ DENOISER-GUIDED LEARNING


Although most versions of GAN use an adversarial discriminator based on logistic regression, there are other ways one can tackle learning and inference with implicit distributions. One interesting tool that has emerged in recent papers <cit.> is the use of denoising autoencoders <cit.> or reconstruction-contractive autoencoders <cit.>.

The key observation for using denoising is that the Bayes-optimal denoiser function captures gradients of the log density of the data generating distribution:


    v^*(x,z)    = _v x,z∼ q(x,z), η∼𝒩_σ_nv(x,z+η) - z^2 
       ≈ z + σ^2_n ∂log q(z| x)/∂ z as σ_n→ 0


This allows us to construct an estimator to the score of a distribution by fitting a DAE to samples. We note that it is possible to obtain a more precise analytical expression for for the optimal denoising function <cit.>.

Let's see how one can use this in the prior-contrastive scenario to deal with an implicit q_θ. First, we fit a denoising function _ϕ by minimising the following loss:


    ℓ^_ψ, σ_n(ϕ) = ∑_n=1^N z∼ q_θ(z| x_n), η∼𝒩_σ_n v_ϕ(z + η,x_n) - z ^2,


which can then be used to approximate the gradient of ELBO (Eqn. (<ref>)) with respect to ψ as follows:


    ∂ℒ(θ,ψ)/∂ψ≈∑_n=1^Nϵ∼𝒩∂ g_ψ(x_n,ϵ))/∂ψ.∂log p_θ(x_n, z)/∂ z|_z=g_ψ(x_n,ϵ)
       + ∑_n=1^Nϵ∼𝒩∂ g_ψ(x_n,ϵ))/∂ψg_ψ(x_n,ϵ) - _ϕ(g_ψ(x_n,ϵ))/σ^2_n.



Several SGD methods only require gradients of the objective function as input, this gradient estimate can be readily used to optimise an approximate ELBO. The resulting iterative algorithm, prior-contrastive denoising VI (, Algorithm <ref>) updates the denoiser and the variational distribution in tandem. Following similar derivation one can construct several other variants of the algorithm. The denoiser approach is more flexible than the adversarial approach as one can pick and choose which individual distributions are modelled explitly, and which ones are implicit. For example, when the prior p_θ is implicit, we can train a denoiser to represent its score function. Or, one can start from the joint-contrastive formulation of the ELBO and train a DAE over joint distribution of x and z, giving rise to the joint-contrastive denoising VI (). In the interest of space the detailed description of these variants is omitted here.

As the denoising criterion estimates the gradients of ELBO but not the value itself, the denoising approach does not provide a direct way to learn model parameters θ. The denoising method may be best utilised in conjunction with an adversarial algorithm such as a combination of  and . The denoising method works better early on in the training when q_ψ and p_θ are very different, and therefore the discrimination task is too easy. Conversely, as q_ψ approaches p_θ, the discriminator can focus its efforts on modelling the residual differences between them rather than trying to model everything about q_θ in isolation as the denoiser in Algorithm <ref> does. <cit.> already used such combination of adversarial training with a denoising criterion for generative modelling. However, the additional nonlinear transformation before denoising introduced in that work breaks the mathematical connections to KL divergence minimisation.




§ SUMMARY


To summarise, we have presented two main ways to formulate ELBO in terms of logarithmic density ratios  and . We called these prior-contrastive (PC) and joint-contrastive (JC). We have then described two techniques by which these density ratios, or their gradients, can be estimated if the distributions involved are implicit: adversarial methods ( and , Algorithms <ref>&<ref>) directly estimate density ratios via logistic regression, denoising methods ( and , Algorithm <ref>) estimate gradients of the log densities via denoising autoencoders. We have mentioned that these methods can be combined, and that such combination may improve convergence.

While all of these algorithms can perform variational inference - fitting the variational parameters ψ - not all of them can perform full variational learning of model parameters θ if the model itself is implicitly defined. In Section <ref> we outline an idea based on reverse mode differentiation (RMD) idea by <cit.> which, giving rise to an algorithm we refer to as , which can in theory perform fully variational inference and learning in a model where all distributions involved are implicit.

The capabilities of algorithms presented here and in related work are summarised in Table <ref>. The adversarial variational Bayes <cit.> is equivalent to PC-Adv, while ALI <cit.> and BiGAN <cit.> are closely related to JC-Adv. <cit.> and <cit.> are closely related to PC-Adv, although the former solves a limited special case and the latter uses the Jensen-Shannon formulation and hence is not fully variational.






§ EXPERIMENTS


Several related papers have already demonstrated the success of methods surveyed here on real world datasets, see for example <cit.> for , <cit.> for  and <cit.> for denoiser-based techniques. Experiments in these papers typically focus on the models' ability to learn θ, and the quality of samples from the learnt generative model p_θ(x).

As the focus here is on inference rather than learning, the goal of this section is to validate the algorithms' ability to perform inference. To this end, we have devised a simple toy problem loosely based on the “sprinkle” example which exhibits explaining away <cit.>. In our “continuous sprinkler” model, two independent scalar hidden variables z_1 and z_2 are combined nonlinearly to produce a univariate observation x:


    (z_1,z_2)    ∼𝒩(0, σ^2 I_2×2)
    
    x    ∼EXP(3 + max(0,z_1)^3 + max(0,z_2)^3)


Although z_1 and z_2 are a priori independent, the likelihood introduces dependence between the two variables once conditioned on data: either latent variable taking a large value can explain a large observed x. This is an example of explaining away which is an important phenomenon in latent variable models that is known to be hard to model with simple, unimodal distributions.

Column A in Figure <ref> illustrates the joint posterior density of z_1 and z_2 for various values of x. The subsequent columns show the posterior approximations by ,  and , respectively. q_θ is implemented as a stochastic generative network where the observation x and Gaussian noise variables ϵ are fed as input to a multilayer perceptron g_ψ. The discriminators and denoisers were implemented as multilayer perceptrons as well. Columns C and D illustrate the limiting behaviour of the discriminator in the  algorithm: as q_ψ converges to the true posterior, _ψ is expected resemble the likelihood p_θ(x| z) up to an additive constant. In  the discriminator eventually converges to the flat _ϕ=0 solution.




§ DISCUSSION AND FUTURE WORK


Are adversaries really needed? When using adversarial techniques for VI, we model the distribution of latent variables rather than observations. The distributions we encounter in VI are usually thought of as simpler than the distribution of observed data, so the question arises whether the flexibility of the adversarial framework is really needed.

Is the prior-contrastive too much like noise-contrastive? In the  algorithm, the discriminator compares samples from the approximate posterior to the prior, and the prior is often high-dimensional Gaussian noise. Even at convergence, the two distributions the discriminator sees never overlap, and this may slow down training. This can be remedied by observing that as q_ψ converges to the true posterior, the discriminator will converge to the log-likelihood plus constant _ϕ(x,z)≈log p(x| y) + c. Hence, the task of the discriminator can be made easier by forming an ensemble between a neural network and the actual log-likelihood.

Aren't denoising methods imprecise? The main criticism of denoiser-based methods is that the gradient estimates are imprecise. As <cit.> pointed out, the optimal denoising function represents the gradients of the noise-corrupted distribution rather than the original, and in practical cases the noise level σ_n may not be small enough for this effect to be negligible. <cit.> observed that denoiser-based methods can not produce results as sharp as adversarial counterparts. Finally, for the outer loop SGD to work consistently, the gradient estimates provided by the inner loop have to form a conservative vector field. While the Bayes-optimal denoiser function satisfies this, it is unclear to what degree this property is preserved when using suboptimal denoisers <cit.>. We believe that an alternative approach based on score matching <cit.> - a task intimately related to denoising <cit.> - might overcome both of these issues.

How to learn θ? The focus of this note is on variational inference, which is finding ψ. However, it is equally important to think about learning θ. Unfortunately, none of the algorithms presented here allow for fully variational learning of model parameters θ when p_θ(x| z) is implicit. ALI and BiGAN do provide an algorithm, but as we mentioned, they are not fully variational. We close by highlighting one possible avenue for future work to enable this: differentiating the inner loop of the  algorithm via reverse mode differentiation <cit.>. To learn θ via SGD, one only needs an estimate of the gradient ∂_ψ,θ/∂θ. We can't compute _ψ,θ, only an approximation _ϕ which is reached via SGD. Each step of SGD depends implicitly on θ. Following <cit.> we can algorithmically differentiate the SGD algorithm in a memory-efficient way to obtain an estimate of the gradient we need for learning θ. We have not validated this approach experimentally, but included it as  in Table <ref>.



icml2017



§ IGNORING IMPLICIT DEPENDENCE ON Ψ


Proof of Eqn. (<ref>):


    . ∂/∂ψq_ψp_θ|_ψ = ψ_0 = ∂/∂ψ.z ∼ q_ψ_θ, ψ(z) |_ψ = ψ_0
       = . ∂/∂ψz ∼ q_ψlogq_ψ_0(z)/p(z)|_ψ = ψ_0 + . ∂/∂ψq_ψq_ψ_0|_ψ = ψ_0
       = . ∂/∂ψz ∼ q_ψlogq_ψ_0(z)/p(z)|_ψ = ψ_0
       = . ∂/∂ψz ∼ q_ψ_θ, ψ_0(z) |_ψ = ψ_0,


where the third line is obtained by noting that ψ_0 is a local minimum of q_ψq_ψ_0, hence the second term in the second line is 0.

































