Predictive variational inference: Learn the predictively optimal posterior distribution

Jinlin Lai, Antonio Ricardo Linero, Yuling Yao
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:62230-62254, 2026.

Abstract

Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density such that the resulting posterior predictive distribution is as close to the true data generating process as possible, while this closeness is measured by multiple scoring rules. By optimizing the objective, the predictive variational inference is generally not the same as, or even attempting to approximate, the Bayesian posterior, even asymptotically. Rather, we interpret it as implicit hierarchical expansion. Further, the learned posterior uncertainty detects heterogeneity of parameters among the population, enabling automatic model diagnosis. This framework applies to both likelihood-exact and likelihood-free models. We demonstrate its application in real data examples.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-lai26c, title = {Predictive variational inference: Learn the predictively optimal posterior distribution}, author = {Lai, Jinlin and Linero, Antonio Ricardo and Yao, Yuling}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {62230--62254}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/lai26c/lai26c.pdf}, url = {https://proceedings.mlr.press/v306/lai26c.html}, abstract = {Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density such that the resulting posterior predictive distribution is as close to the true data generating process as possible, while this closeness is measured by multiple scoring rules. By optimizing the objective, the predictive variational inference is generally not the same as, or even attempting to approximate, the Bayesian posterior, even asymptotically. Rather, we interpret it as implicit hierarchical expansion. Further, the learned posterior uncertainty detects heterogeneity of parameters among the population, enabling automatic model diagnosis. This framework applies to both likelihood-exact and likelihood-free models. We demonstrate its application in real data examples.} }
Endnote
%0 Conference Paper %T Predictive variational inference: Learn the predictively optimal posterior distribution %A Jinlin Lai %A Antonio Ricardo Linero %A Yuling Yao %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-lai26c %I PMLR %P 62230--62254 %U https://proceedings.mlr.press/v306/lai26c.html %V 306 %X Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density such that the resulting posterior predictive distribution is as close to the true data generating process as possible, while this closeness is measured by multiple scoring rules. By optimizing the objective, the predictive variational inference is generally not the same as, or even attempting to approximate, the Bayesian posterior, even asymptotically. Rather, we interpret it as implicit hierarchical expansion. Further, the learned posterior uncertainty detects heterogeneity of parameters among the population, enabling automatic model diagnosis. This framework applies to both likelihood-exact and likelihood-free models. We demonstrate its application in real data examples.
APA
Lai, J., Linero, A.R. & Yao, Y.. (2026). Predictive variational inference: Learn the predictively optimal posterior distribution. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:62230-62254 Available from https://proceedings.mlr.press/v306/lai26c.html.

Related Material