Skip to main content
placeholder image

Accelerating pseudo-marginal MCMC using Gaussian processes

Journal Article


Abstract


  • The grouped independence Metropolis–Hastings (GIMH) and Markov chain within Metropolis (MCWM) algorithms are pseudo-marginal methods used to perform Bayesian inference in latent variable models. These methods replace intractable likelihood calculations with unbiased estimates within Markov chain Monte Carlo algorithms. The GIMH method has the posterior of interest as its limiting distribution, but suffers from poor mixing if it is too computationally intensive to obtain high-precision likelihood estimates. The MCWM algorithm has better mixing properties, but tends to give conservative approximations of the posterior and is still expensive. A new method is developed to accelerate the GIMH method by using a Gaussian process (GP) approximation to the log-likelihood and train this GP using a short pilot run of the MCWM algorithm. This new method called GP-GIMH is illustrated on simulated data from a stochastic volatility and a gene network model. The new approach produces reasonable posterior approximations in these examples with at least an order of magnitude improvement in computing time. Code to implement the method for the gene network example can be found at http://www.runmycode.org/companion/view/2663.

Authors


  •   Drovandi, Christopher C. (external author)
  •   Moores, Matt T.
  •   Boys, Richard J. (external author)

Publication Date


  • 2018

Citation


  • Drovandi, C. C., Moores, M. T. & Boys, R. J. (2018). Accelerating pseudo-marginal MCMC using Gaussian processes. Computational Statistics and Data Analysis, 118 1-17.

Scopus Eid


  • 2-s2.0-85029819485

Ro Metadata Url


  • http://ro.uow.edu.au/eispapers1/1677

Number Of Pages


  • 16

Start Page


  • 1

End Page


  • 17

Volume


  • 118

Place Of Publication


  • Netherlands

Abstract


  • The grouped independence Metropolis–Hastings (GIMH) and Markov chain within Metropolis (MCWM) algorithms are pseudo-marginal methods used to perform Bayesian inference in latent variable models. These methods replace intractable likelihood calculations with unbiased estimates within Markov chain Monte Carlo algorithms. The GIMH method has the posterior of interest as its limiting distribution, but suffers from poor mixing if it is too computationally intensive to obtain high-precision likelihood estimates. The MCWM algorithm has better mixing properties, but tends to give conservative approximations of the posterior and is still expensive. A new method is developed to accelerate the GIMH method by using a Gaussian process (GP) approximation to the log-likelihood and train this GP using a short pilot run of the MCWM algorithm. This new method called GP-GIMH is illustrated on simulated data from a stochastic volatility and a gene network model. The new approach produces reasonable posterior approximations in these examples with at least an order of magnitude improvement in computing time. Code to implement the method for the gene network example can be found at http://www.runmycode.org/companion/view/2663.

Authors


  •   Drovandi, Christopher C. (external author)
  •   Moores, Matt T.
  •   Boys, Richard J. (external author)

Publication Date


  • 2018

Citation


  • Drovandi, C. C., Moores, M. T. & Boys, R. J. (2018). Accelerating pseudo-marginal MCMC using Gaussian processes. Computational Statistics and Data Analysis, 118 1-17.

Scopus Eid


  • 2-s2.0-85029819485

Ro Metadata Url


  • http://ro.uow.edu.au/eispapers1/1677

Number Of Pages


  • 16

Start Page


  • 1

End Page


  • 17

Volume


  • 118

Place Of Publication


  • Netherlands