Learning Approximately Objective Priors

Eric Nalisnick, Padhraic Smyth
Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, PMLR R15:491-500, 2017.

Abstract

Informative Bayesian priors are often difficult to elicit, and when this is the case, modelers usually turn to noninformative or objective priors. However, objective priors such as the Jeffreys and reference priors are not tractable to derive for many models of interest. We address this issue by proposing techniques for learning reference prior approximations: we select a parametric family and optimize a black-box lower bound on the reference prior objective to find the member of the family that serves as a good approximation. We experimentally demonstrate the method’s effectiveness by recovering Jeffreys priors and learning the Variational Autoencoder’s reference prior.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR15-nalisnick17a, title = {Learning Approximately Objective Priors}, author = {Nalisnick, Eric and Smyth, Padhraic}, booktitle = {Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence}, pages = {491--500}, year = {2017}, editor = {Elidan, Gal and Kersting, Kristian}, volume = {R15}, series = {Proceedings of Machine Learning Research}, month = {11--15 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r15/main/assets/nalisnick17a/nalisnick17a.pdf}, url = {https://proceedings.mlr.press/r15/nalisnick17a.html}, abstract = {Informative Bayesian priors are often difficult to elicit, and when this is the case, modelers usually turn to noninformative or objective priors. However, objective priors such as the Jeffreys and reference priors are not tractable to derive for many models of interest. We address this issue by proposing techniques for learning reference prior approximations: we select a parametric family and optimize a black-box lower bound on the reference prior objective to find the member of the family that serves as a good approximation. We experimentally demonstrate the method’s effectiveness by recovering Jeffreys priors and learning the Variational Autoencoder’s reference prior.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Learning Approximately Objective Priors %A Eric Nalisnick %A Padhraic Smyth %B Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2017 %E Gal Elidan %E Kristian Kersting %F pmlr-vR15-nalisnick17a %I PMLR %P 491--500 %U https://proceedings.mlr.press/r15/nalisnick17a.html %V R15 %X Informative Bayesian priors are often difficult to elicit, and when this is the case, modelers usually turn to noninformative or objective priors. However, objective priors such as the Jeffreys and reference priors are not tractable to derive for many models of interest. We address this issue by proposing techniques for learning reference prior approximations: we select a parametric family and optimize a black-box lower bound on the reference prior objective to find the member of the family that serves as a good approximation. We experimentally demonstrate the method’s effectiveness by recovering Jeffreys priors and learning the Variational Autoencoder’s reference prior. %Z Reissued by PMLR on 04 October 2026.
APA
Nalisnick, E. & Smyth, P.. (2017). Learning Approximately Objective Priors. Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R15:491-500 Available from https://proceedings.mlr.press/r15/nalisnick17a.html. Reissued by PMLR on 04 October 2026.

Related Material