Flexible Martingale Priors for Deep Hierarchies

Jacob Steinhardt, Zoubin Ghahramani
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:1108-1116, 2012.

Abstract

When building priors over trees for Bayesian hierarchical models, there is a tension between maintaining desirable theoretical properties such as infinite exchangeability and important practical properties such as the ability to increase the depth of the tree to accommodate new data. We resolve this tension by presenting a family of infinitely exchangeable priors over discrete tree structures that allows the depth of the tree to grow with the data, and then showing that our family contains all hierarchical models with certain mild symmetry properties. We also show that deep hierarchical models are in general intimately tied to a process called a martingale, and use Doob’s martingale convergence theorem to demonstrate some unexpected properties of deep hierarchies.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-steinhardt12, title = {Flexible Martingale Priors for Deep Hierarchies}, author = {Steinhardt, Jacob and Ghahramani, Zoubin}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {1108--1116}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/steinhardt12/steinhardt12.pdf}, url = {https://proceedings.mlr.press/v22/steinhardt12.html}, abstract = {When building priors over trees for Bayesian hierarchical models, there is a tension between maintaining desirable theoretical properties such as infinite exchangeability and important practical properties such as the ability to increase the depth of the tree to accommodate new data. We resolve this tension by presenting a family of infinitely exchangeable priors over discrete tree structures that allows the depth of the tree to grow with the data, and then showing that our family contains all hierarchical models with certain mild symmetry properties. We also show that deep hierarchical models are in general intimately tied to a process called a martingale, and use Doob’s martingale convergence theorem to demonstrate some unexpected properties of deep hierarchies.} }
Endnote
%0 Conference Paper %T Flexible Martingale Priors for Deep Hierarchies %A Jacob Steinhardt %A Zoubin Ghahramani %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-steinhardt12 %I PMLR %P 1108--1116 %U https://proceedings.mlr.press/v22/steinhardt12.html %V 22 %X When building priors over trees for Bayesian hierarchical models, there is a tension between maintaining desirable theoretical properties such as infinite exchangeability and important practical properties such as the ability to increase the depth of the tree to accommodate new data. We resolve this tension by presenting a family of infinitely exchangeable priors over discrete tree structures that allows the depth of the tree to grow with the data, and then showing that our family contains all hierarchical models with certain mild symmetry properties. We also show that deep hierarchical models are in general intimately tied to a process called a martingale, and use Doob’s martingale convergence theorem to demonstrate some unexpected properties of deep hierarchies.
RIS
TY - CPAPER TI - Flexible Martingale Priors for Deep Hierarchies AU - Jacob Steinhardt AU - Zoubin Ghahramani BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-steinhardt12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 1108 EP - 1116 L1 - http://proceedings.mlr.press/v22/steinhardt12/steinhardt12.pdf UR - https://proceedings.mlr.press/v22/steinhardt12.html AB - When building priors over trees for Bayesian hierarchical models, there is a tension between maintaining desirable theoretical properties such as infinite exchangeability and important practical properties such as the ability to increase the depth of the tree to accommodate new data. We resolve this tension by presenting a family of infinitely exchangeable priors over discrete tree structures that allows the depth of the tree to grow with the data, and then showing that our family contains all hierarchical models with certain mild symmetry properties. We also show that deep hierarchical models are in general intimately tied to a process called a martingale, and use Doob’s martingale convergence theorem to demonstrate some unexpected properties of deep hierarchies. ER -
APA
Steinhardt, J. & Ghahramani, Z.. (2012). Flexible Martingale Priors for Deep Hierarchies. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:1108-1116 Available from https://proceedings.mlr.press/v22/steinhardt12.html.

Related Material