Flexible Martingale Priors for Deep Hierarchies

Jacob Steinhardt, Zoubin Ghahramani
; Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:1108-1116, 2012.

Abstract

When building priors over trees for Bayesian hierarchical models, there is a tension between maintaining desirable theoretical properties such as infinite exchangeability and important practical properties such as the ability to increase the depth of the tree to accommodate new data. We resolve this tension by presenting a family of infinitely exchangeable priors over discrete tree structures that allows the depth of the tree to grow with the data, and then showing that our family contains all hierarchical models with certain mild symmetry properties. We also show that deep hierarchical models are in general intimately tied to a process called a martingale, and use Doob’s martingale convergence theorem to demonstrate some unexpected properties of deep hierarchies.

Cite this Paper


BibTeX
@InProceedings{pmlr-v22-steinhardt12, title = {Flexible Martingale Priors for Deep Hierarchies}, author = {Jacob Steinhardt and Zoubin Ghahramani}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {1108--1116}, year = {2012}, editor = {Neil D. Lawrence and Mark Girolami}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/steinhardt12/steinhardt12.pdf}, url = {http://proceedings.mlr.press/v22/steinhardt12.html}, abstract = {When building priors over trees for Bayesian hierarchical models, there is a tension between maintaining desirable theoretical properties such as infinite exchangeability and important practical properties such as the ability to increase the depth of the tree to accommodate new data. We resolve this tension by presenting a family of infinitely exchangeable priors over discrete tree structures that allows the depth of the tree to grow with the data, and then showing that our family contains all hierarchical models with certain mild symmetry properties. We also show that deep hierarchical models are in general intimately tied to a process called a martingale, and use Doob’s martingale convergence theorem to demonstrate some unexpected properties of deep hierarchies.} }
Endnote
%0 Conference Paper %T Flexible Martingale Priors for Deep Hierarchies %A Jacob Steinhardt %A Zoubin Ghahramani %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-steinhardt12 %I PMLR %J Proceedings of Machine Learning Research %P 1108--1116 %U http://proceedings.mlr.press %V 22 %W PMLR %X When building priors over trees for Bayesian hierarchical models, there is a tension between maintaining desirable theoretical properties such as infinite exchangeability and important practical properties such as the ability to increase the depth of the tree to accommodate new data. We resolve this tension by presenting a family of infinitely exchangeable priors over discrete tree structures that allows the depth of the tree to grow with the data, and then showing that our family contains all hierarchical models with certain mild symmetry properties. We also show that deep hierarchical models are in general intimately tied to a process called a martingale, and use Doob’s martingale convergence theorem to demonstrate some unexpected properties of deep hierarchies.
RIS
TY - CPAPER TI - Flexible Martingale Priors for Deep Hierarchies AU - Jacob Steinhardt AU - Zoubin Ghahramani BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics PY - 2012/03/21 DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-steinhardt12 PB - PMLR SP - 1108 DP - PMLR EP - 1116 L1 - http://proceedings.mlr.press/v22/steinhardt12/steinhardt12.pdf UR - http://proceedings.mlr.press/v22/steinhardt12.html AB - When building priors over trees for Bayesian hierarchical models, there is a tension between maintaining desirable theoretical properties such as infinite exchangeability and important practical properties such as the ability to increase the depth of the tree to accommodate new data. We resolve this tension by presenting a family of infinitely exchangeable priors over discrete tree structures that allows the depth of the tree to grow with the data, and then showing that our family contains all hierarchical models with certain mild symmetry properties. We also show that deep hierarchical models are in general intimately tied to a process called a martingale, and use Doob’s martingale convergence theorem to demonstrate some unexpected properties of deep hierarchies. ER -
APA
Steinhardt, J. & Ghahramani, Z.. (2012). Flexible Martingale Priors for Deep Hierarchies. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in PMLR 22:1108-1116

Related Material