PACm-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime

Warren R. Morningstar, Alex Alemi, Joshua V. Dillon
Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, PMLR 151:8270-8298, 2022.

Abstract

The Bayesian posterior minimizes the "inferential risk" which itself bounds the "predictive risk." This bound is tight when the likelihood and prior are well-specified. How-ever since misspecification induces a gap,the Bayesian posterior predictive distribution may have poor generalization performance. This work develops a multi-sample loss (PAC$^m$) which can close the gap by spanning a trade-off between the two risks. The loss is computationally favorable and offers PAC generalization guarantees. Empirical study demonstrates improvement to the predictive distribution

Cite this Paper

BibTeX
@InProceedings{pmlr-v151-morningstar22a, title = { PACm-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime }, author = {Morningstar, Warren R. and Alemi, Alex and Dillon, Joshua V.}, booktitle = {Proceedings of The 25th International Conference on Artificial Intelligence and Statistics}, pages = {8270--8298}, year = {2022}, editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel}, volume = {151}, series = {Proceedings of Machine Learning Research}, month = {28--30 Mar}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v151/morningstar22a/morningstar22a.pdf}, url = {https://proceedings.mlr.press/v151/morningstar22a.html}, abstract = { The Bayesian posterior minimizes the "inferential risk" which itself bounds the "predictive risk." This bound is tight when the likelihood and prior are well-specified. How-ever since misspecification induces a gap,the Bayesian posterior predictive distribution may have poor generalization performance. This work develops a multi-sample loss (PAC$^m$) which can close the gap by spanning a trade-off between the two risks. The loss is computationally favorable and offers PAC generalization guarantees. Empirical study demonstrates improvement to the predictive distribution } }
Endnote
%0 Conference Paper %T PACm-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime %A Warren R. Morningstar %A Alex Alemi %A Joshua V. Dillon %B Proceedings of The 25th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2022 %E Gustau Camps-Valls %E Francisco J. R. Ruiz %E Isabel Valera %F pmlr-v151-morningstar22a %I PMLR %P 8270--8298 %U https://proceedings.mlr.press/v151/morningstar22a.html %V 151 %X The Bayesian posterior minimizes the "inferential risk" which itself bounds the "predictive risk." This bound is tight when the likelihood and prior are well-specified. How-ever since misspecification induces a gap,the Bayesian posterior predictive distribution may have poor generalization performance. This work develops a multi-sample loss (PAC$^m$) which can close the gap by spanning a trade-off between the two risks. The loss is computationally favorable and offers PAC generalization guarantees. Empirical study demonstrates improvement to the predictive distribution
APA
Morningstar, W.R., Alemi, A. & Dillon, J.V.. (2022). PACm-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime . Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 151:8270-8298 Available from https://proceedings.mlr.press/v151/morningstar22a.html.

Related Material