A Hybrid Approximation to the Marginal Likelihood

Eric Chuu, Debdeep Pati, Anirban Bhattacharya
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:3214-3222, 2021.

Abstract

Computing the marginal likelihood or evidence is one of the core challenges in Bayesian analysis. While there are many established methods for estimating this quantity, they predominantly rely on using a large number of posterior samples obtained from a Markov Chain Monte Carlo (MCMC) algorithm. As the dimension of the parameter space increases, however, many of these methods become prohibitively slow and potentially inaccurate. In this paper, we propose a novel method in which we use the MCMC samples to learn a high probability partition of the parameter space and then form a deterministic approximation over each of these partition sets. This two-step procedure, which constitutes both a probabilistic and a deterministic component, is termed a Hybrid approximation to the marginal likelihood. We demonstrate its versatility in a plethora of examples with varying dimension and sample size, and we also highlight the Hybrid approximation’s effectiveness in situations where there is either a limited number or only approximate MCMC samples available.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-chuu21a, title = { A Hybrid Approximation to the Marginal Likelihood }, author = {Chuu, Eric and Pati, Debdeep and Bhattacharya, Anirban}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {3214--3222}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/chuu21a/chuu21a.pdf}, url = {https://proceedings.mlr.press/v130/chuu21a.html}, abstract = { Computing the marginal likelihood or evidence is one of the core challenges in Bayesian analysis. While there are many established methods for estimating this quantity, they predominantly rely on using a large number of posterior samples obtained from a Markov Chain Monte Carlo (MCMC) algorithm. As the dimension of the parameter space increases, however, many of these methods become prohibitively slow and potentially inaccurate. In this paper, we propose a novel method in which we use the MCMC samples to learn a high probability partition of the parameter space and then form a deterministic approximation over each of these partition sets. This two-step procedure, which constitutes both a probabilistic and a deterministic component, is termed a Hybrid approximation to the marginal likelihood. We demonstrate its versatility in a plethora of examples with varying dimension and sample size, and we also highlight the Hybrid approximation’s effectiveness in situations where there is either a limited number or only approximate MCMC samples available. } }
Endnote
%0 Conference Paper %T A Hybrid Approximation to the Marginal Likelihood %A Eric Chuu %A Debdeep Pati %A Anirban Bhattacharya %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-chuu21a %I PMLR %P 3214--3222 %U https://proceedings.mlr.press/v130/chuu21a.html %V 130 %X Computing the marginal likelihood or evidence is one of the core challenges in Bayesian analysis. While there are many established methods for estimating this quantity, they predominantly rely on using a large number of posterior samples obtained from a Markov Chain Monte Carlo (MCMC) algorithm. As the dimension of the parameter space increases, however, many of these methods become prohibitively slow and potentially inaccurate. In this paper, we propose a novel method in which we use the MCMC samples to learn a high probability partition of the parameter space and then form a deterministic approximation over each of these partition sets. This two-step procedure, which constitutes both a probabilistic and a deterministic component, is termed a Hybrid approximation to the marginal likelihood. We demonstrate its versatility in a plethora of examples with varying dimension and sample size, and we also highlight the Hybrid approximation’s effectiveness in situations where there is either a limited number or only approximate MCMC samples available.
APA
Chuu, E., Pati, D. & Bhattacharya, A.. (2021). A Hybrid Approximation to the Marginal Likelihood . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:3214-3222 Available from https://proceedings.mlr.press/v130/chuu21a.html.

Related Material