Quasi-Bayesian nonparametric density estimation via autoregressive predictive updates

Sahra Ghalebikesabi, Chris C. Holmes, Edwin Fong, Brieuc Lehmann
Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, PMLR 216:658-668, 2023.

Abstract

Bayesian methods are a popular choice for statistical inference in small-data regimes due to the regularization effect induced by the prior. %, which serves to counteract overfitting. In the context of density estimation, the standard nonparametric Bayesian approach is to target the posterior predictive of the Dirichlet process mixture model. In general, direct estimation of the posterior predictive is intractable and so methods typically resort to approximating the posterior distribution as an intermediate step. The recent development of quasi-Bayesian predictive copula updates, however, has made it possible to perform tractable predictive density estimation without the need for posterior approximation. Although these estimators are computationally appealing, they struggle on non-smooth data distributions. This is due to the comparatively restrictive form of the likelihood models from which the proposed copula updates were derived. To address this shortcoming, we consider a Bayesian nonparametric model with an autoregressive likelihood decomposition and a Gaussian process prior. While the predictive update of such a model is typically intractable, we derive a quasi-Bayesian update that achieves state-of-the-art results in small-data regimes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v216-ghalebikesabi23a, title = {Quasi-{B}ayesian nonparametric density estimation via autoregressive predictive updates}, author = {Ghalebikesabi, Sahra and Holmes, Chris C. and Fong, Edwin and Lehmann, Brieuc}, booktitle = {Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence}, pages = {658--668}, year = {2023}, editor = {Evans, Robin J. and Shpitser, Ilya}, volume = {216}, series = {Proceedings of Machine Learning Research}, month = {31 Jul--04 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v216/ghalebikesabi23a/ghalebikesabi23a.pdf}, url = {https://proceedings.mlr.press/v216/ghalebikesabi23a.html}, abstract = {Bayesian methods are a popular choice for statistical inference in small-data regimes due to the regularization effect induced by the prior. %, which serves to counteract overfitting. In the context of density estimation, the standard nonparametric Bayesian approach is to target the posterior predictive of the Dirichlet process mixture model. In general, direct estimation of the posterior predictive is intractable and so methods typically resort to approximating the posterior distribution as an intermediate step. The recent development of quasi-Bayesian predictive copula updates, however, has made it possible to perform tractable predictive density estimation without the need for posterior approximation. Although these estimators are computationally appealing, they struggle on non-smooth data distributions. This is due to the comparatively restrictive form of the likelihood models from which the proposed copula updates were derived. To address this shortcoming, we consider a Bayesian nonparametric model with an autoregressive likelihood decomposition and a Gaussian process prior. While the predictive update of such a model is typically intractable, we derive a quasi-Bayesian update that achieves state-of-the-art results in small-data regimes.} }
Endnote
%0 Conference Paper %T Quasi-Bayesian nonparametric density estimation via autoregressive predictive updates %A Sahra Ghalebikesabi %A Chris C. Holmes %A Edwin Fong %A Brieuc Lehmann %B Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2023 %E Robin J. Evans %E Ilya Shpitser %F pmlr-v216-ghalebikesabi23a %I PMLR %P 658--668 %U https://proceedings.mlr.press/v216/ghalebikesabi23a.html %V 216 %X Bayesian methods are a popular choice for statistical inference in small-data regimes due to the regularization effect induced by the prior. %, which serves to counteract overfitting. In the context of density estimation, the standard nonparametric Bayesian approach is to target the posterior predictive of the Dirichlet process mixture model. In general, direct estimation of the posterior predictive is intractable and so methods typically resort to approximating the posterior distribution as an intermediate step. The recent development of quasi-Bayesian predictive copula updates, however, has made it possible to perform tractable predictive density estimation without the need for posterior approximation. Although these estimators are computationally appealing, they struggle on non-smooth data distributions. This is due to the comparatively restrictive form of the likelihood models from which the proposed copula updates were derived. To address this shortcoming, we consider a Bayesian nonparametric model with an autoregressive likelihood decomposition and a Gaussian process prior. While the predictive update of such a model is typically intractable, we derive a quasi-Bayesian update that achieves state-of-the-art results in small-data regimes.
APA
Ghalebikesabi, S., Holmes, C.C., Fong, E. & Lehmann, B.. (2023). Quasi-Bayesian nonparametric density estimation via autoregressive predictive updates. Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 216:658-668 Available from https://proceedings.mlr.press/v216/ghalebikesabi23a.html.

Related Material