Nonparametric Estimation of Conditional Information and Divergences

Barnabas Poczos, Jeff Schneider
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:914-923, 2012.

Abstract

In this paper we propose new nonparametric estimators for a family of conditional mutual information and divergences. Our estimators are easy to compute; they only use simple k nearest neighbor based statistics. We prove that the proposed conditional information and divergence estimators are consistent under certain conditions, and demonstrate their consistency and applicability by numerical experiments on simulated and on real data as well.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-poczos12, title = {Nonparametric Estimation of Conditional Information and Divergences}, author = {Poczos, Barnabas and Schneider, Jeff}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {914--923}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/poczos12/poczos12.pdf}, url = {https://proceedings.mlr.press/v22/poczos12.html}, abstract = {In this paper we propose new nonparametric estimators for a family of conditional mutual information and divergences. Our estimators are easy to compute; they only use simple k nearest neighbor based statistics. We prove that the proposed conditional information and divergence estimators are consistent under certain conditions, and demonstrate their consistency and applicability by numerical experiments on simulated and on real data as well.} }
Endnote
%0 Conference Paper %T Nonparametric Estimation of Conditional Information and Divergences %A Barnabas Poczos %A Jeff Schneider %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-poczos12 %I PMLR %P 914--923 %U https://proceedings.mlr.press/v22/poczos12.html %V 22 %X In this paper we propose new nonparametric estimators for a family of conditional mutual information and divergences. Our estimators are easy to compute; they only use simple k nearest neighbor based statistics. We prove that the proposed conditional information and divergence estimators are consistent under certain conditions, and demonstrate their consistency and applicability by numerical experiments on simulated and on real data as well.
RIS
TY - CPAPER TI - Nonparametric Estimation of Conditional Information and Divergences AU - Barnabas Poczos AU - Jeff Schneider BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-poczos12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 914 EP - 923 L1 - http://proceedings.mlr.press/v22/poczos12/poczos12.pdf UR - https://proceedings.mlr.press/v22/poczos12.html AB - In this paper we propose new nonparametric estimators for a family of conditional mutual information and divergences. Our estimators are easy to compute; they only use simple k nearest neighbor based statistics. We prove that the proposed conditional information and divergence estimators are consistent under certain conditions, and demonstrate their consistency and applicability by numerical experiments on simulated and on real data as well. ER -
APA
Poczos, B. & Schneider, J.. (2012). Nonparametric Estimation of Conditional Information and Divergences. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:914-923 Available from https://proceedings.mlr.press/v22/poczos12.html.

Related Material