Entropic Graph-based Posterior Regularization

Maxwell Libbrecht, Michael Hoffman, Jeff Bilmes, William Noble
Proceedings of the 32nd International Conference on Machine Learning, PMLR 37:1992-2001, 2015.

Abstract

Graph smoothness objectives have achieved great success in semi-supervised learning but have not yet been applied extensively to unsupervised generative models. We define a new class of entropic graph-based posterior regularizers that augment a probabilistic model by encouraging pairs of nearby variables in a regularization graph to have similar posterior distributions. We present a three-way alternating optimization algorithm with closed-form updates for performing inference on this joint model and learning its parameters. This method admits updates linear in the degree of the regularization graph, exhibits monotone convergence and is easily parallelizable. We are motivated by applications in computational biology in which temporal models such as hidden Markov models are used to learn a human-interpretable representation of genomic data. On a synthetic problem, we show that our method outperforms existing methods for graph-based regularization and a comparable strategy for incorporating long-range interactions using existing methods for approximate inference. Using genome-scale functional genomics data, we integrate genome 3D interaction data into existing models for genome annotation and demonstrate significant improvements in predicting genomic activity.

Cite this Paper

BibTeX
@InProceedings{pmlr-v37-libbrecht15, title = {Entropic Graph-based Posterior Regularization}, author = {Libbrecht, Maxwell and Hoffman, Michael and Bilmes, Jeff and Noble, William}, booktitle = {Proceedings of the 32nd International Conference on Machine Learning}, pages = {1992--2001}, year = {2015}, editor = {Bach, Francis and Blei, David}, volume = {37}, series = {Proceedings of Machine Learning Research}, address = {Lille, France}, month = {07--09 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v37/libbrecht15.pdf}, url = {https://proceedings.mlr.press/v37/libbrecht15.html}, abstract = {Graph smoothness objectives have achieved great success in semi-supervised learning but have not yet been applied extensively to unsupervised generative models. We define a new class of entropic graph-based posterior regularizers that augment a probabilistic model by encouraging pairs of nearby variables in a regularization graph to have similar posterior distributions. We present a three-way alternating optimization algorithm with closed-form updates for performing inference on this joint model and learning its parameters. This method admits updates linear in the degree of the regularization graph, exhibits monotone convergence and is easily parallelizable. We are motivated by applications in computational biology in which temporal models such as hidden Markov models are used to learn a human-interpretable representation of genomic data. On a synthetic problem, we show that our method outperforms existing methods for graph-based regularization and a comparable strategy for incorporating long-range interactions using existing methods for approximate inference. Using genome-scale functional genomics data, we integrate genome 3D interaction data into existing models for genome annotation and demonstrate significant improvements in predicting genomic activity.} }
Endnote
%0 Conference Paper %T Entropic Graph-based Posterior Regularization %A Maxwell Libbrecht %A Michael Hoffman %A Jeff Bilmes %A William Noble %B Proceedings of the 32nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2015 %E Francis Bach %E David Blei %F pmlr-v37-libbrecht15 %I PMLR %P 1992--2001 %U https://proceedings.mlr.press/v37/libbrecht15.html %V 37 %X Graph smoothness objectives have achieved great success in semi-supervised learning but have not yet been applied extensively to unsupervised generative models. We define a new class of entropic graph-based posterior regularizers that augment a probabilistic model by encouraging pairs of nearby variables in a regularization graph to have similar posterior distributions. We present a three-way alternating optimization algorithm with closed-form updates for performing inference on this joint model and learning its parameters. This method admits updates linear in the degree of the regularization graph, exhibits monotone convergence and is easily parallelizable. We are motivated by applications in computational biology in which temporal models such as hidden Markov models are used to learn a human-interpretable representation of genomic data. On a synthetic problem, we show that our method outperforms existing methods for graph-based regularization and a comparable strategy for incorporating long-range interactions using existing methods for approximate inference. Using genome-scale functional genomics data, we integrate genome 3D interaction data into existing models for genome annotation and demonstrate significant improvements in predicting genomic activity.
RIS
TY - CPAPER TI - Entropic Graph-based Posterior Regularization AU - Maxwell Libbrecht AU - Michael Hoffman AU - Jeff Bilmes AU - William Noble BT - Proceedings of the 32nd International Conference on Machine Learning DA - 2015/06/01 ED - Francis Bach ED - David Blei ID - pmlr-v37-libbrecht15 PB - PMLR DP - Proceedings of Machine Learning Research VL - 37 SP - 1992 EP - 2001 L1 - http://proceedings.mlr.press/v37/libbrecht15.pdf UR - https://proceedings.mlr.press/v37/libbrecht15.html AB - Graph smoothness objectives have achieved great success in semi-supervised learning but have not yet been applied extensively to unsupervised generative models. We define a new class of entropic graph-based posterior regularizers that augment a probabilistic model by encouraging pairs of nearby variables in a regularization graph to have similar posterior distributions. We present a three-way alternating optimization algorithm with closed-form updates for performing inference on this joint model and learning its parameters. This method admits updates linear in the degree of the regularization graph, exhibits monotone convergence and is easily parallelizable. We are motivated by applications in computational biology in which temporal models such as hidden Markov models are used to learn a human-interpretable representation of genomic data. On a synthetic problem, we show that our method outperforms existing methods for graph-based regularization and a comparable strategy for incorporating long-range interactions using existing methods for approximate inference. Using genome-scale functional genomics data, we integrate genome 3D interaction data into existing models for genome annotation and demonstrate significant improvements in predicting genomic activity. ER -
APA
Libbrecht, M., Hoffman, M., Bilmes, J. & Noble, W.. (2015). Entropic Graph-based Posterior Regularization. Proceedings of the 32nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 37:1992-2001 Available from https://proceedings.mlr.press/v37/libbrecht15.html.

Related Material