Title: Unbiased Markov chain Monte Carlo for intractable target distributions,
Abstract: Performing numerical integration when the integrand itself cannot be
evaluated point-wise is a challenging task that arises in statistical analysis,
notably in Bayesian inference for models with intractable likelihood functions.
Markov chain Monte Carlo (MCMC) algorithms have been proposed for this setting,
such as the pseudo-marginal method for latent variable models and the exchange
algorithm for a class of undirected graphical models. As with any MCMC
algorithm, the resulting estimators are justified asymptotically in the limit
of the number of iterations, but exhibit a bias for any fixed number of
iterations due to the Markov chains starting outside of stationarity. This
"burn-in" bias is known to complicate the use of parallel processors for MCMC
computations. We show how to use coupling techniques to generate unbiased
estimators in finite time, building on recent advances for generic MCMC
algorithms. We establish the theoretical validity of some of these procedures
by extending existing results to cover the case of polynomially ergodic Markov
chains. The efficiency of the proposed estimators is compared with that of
standard MCMC estimators, with theoretical arguments and numerical experiments
including state space models and Ising models.