Inferring latent structures via information inequalities

Rafael Chaves, Lukas Luft, Thiago Maciel Federal University of Minas Gerais, David Gross, Dominik Janzing, Bernhard Schölkopf
Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, PMLR R12:96-105, 2014.

Abstract

One of the goals of probabilistic inference is to decide whether an empirically observed distribution is compatible with a candidate Bayesian network. However, Bayesian net- works with hidden variables give rise to highly non-trivial constraints on the ob- served distribution. Here, we propose an information-theoretic approach, based on the insight that conditions on entropies of Bayesian networks take the form of simple linear inequalities. We describe an algorithm for deriving entropic tests for latent struc- tures. The well-known conditional indepen- dence tests appear as a special case. While the approach applies for generic Bayesian networks, we presently adopt the causal view, and show the versatility of the framework by treating several relevant problems from that domain: detecting common ancestors, quan- tifying the strength of causal influence, and inferring the direction of causation from two- variable marginals.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR12-chaves14a, title = {Inferring latent structures via information inequalities}, author = {Chaves, Rafael and Luft, Lukas and Gerais, Thiago Maciel Federal University of Minas and Gross, David and Janzing, Dominik and Sch{\"o}lkopf, Bernhard}, booktitle = {Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence}, pages = {96--105}, year = {2014}, editor = {Zhang, Nevin L. and Tian, Jin}, volume = {R12}, series = {Proceedings of Machine Learning Research}, month = {23--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r12/main/assets/chaves14a/chaves14a.pdf}, url = {https://proceedings.mlr.press/r12/chaves14a.html}, abstract = {One of the goals of probabilistic inference is to decide whether an empirically observed distribution is compatible with a candidate Bayesian network. However, Bayesian net- works with hidden variables give rise to highly non-trivial constraints on the ob- served distribution. Here, we propose an information-theoretic approach, based on the insight that conditions on entropies of Bayesian networks take the form of simple linear inequalities. We describe an algorithm for deriving entropic tests for latent struc- tures. The well-known conditional indepen- dence tests appear as a special case. While the approach applies for generic Bayesian networks, we presently adopt the causal view, and show the versatility of the framework by treating several relevant problems from that domain: detecting common ancestors, quan- tifying the strength of causal influence, and inferring the direction of causation from two- variable marginals.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Inferring latent structures via information inequalities %A Rafael Chaves %A Lukas Luft %A Thiago Maciel Federal University of Minas Gerais %A David Gross %A Dominik Janzing %A Bernhard Schölkopf %B Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2014 %E Nevin L. Zhang %E Jin Tian %F pmlr-vR12-chaves14a %I PMLR %P 96--105 %U https://proceedings.mlr.press/r12/chaves14a.html %V R12 %X One of the goals of probabilistic inference is to decide whether an empirically observed distribution is compatible with a candidate Bayesian network. However, Bayesian net- works with hidden variables give rise to highly non-trivial constraints on the ob- served distribution. Here, we propose an information-theoretic approach, based on the insight that conditions on entropies of Bayesian networks take the form of simple linear inequalities. We describe an algorithm for deriving entropic tests for latent struc- tures. The well-known conditional indepen- dence tests appear as a special case. While the approach applies for generic Bayesian networks, we presently adopt the causal view, and show the versatility of the framework by treating several relevant problems from that domain: detecting common ancestors, quan- tifying the strength of causal influence, and inferring the direction of causation from two- variable marginals. %Z Reissued by PMLR on 04 October 2026.
APA
Chaves, R., Luft, L., Gerais, T.M.F.U.o.M., Gross, D., Janzing, D. & Schölkopf, B.. (2014). Inferring latent structures via information inequalities. Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R12:96-105 Available from https://proceedings.mlr.press/r12/chaves14a.html. Reissued by PMLR on 04 October 2026.

Related Material