REGO: Rank-based Estimation of Renyi Information using Euclidean Graph Optimization

Barnabas Poczos, Sergey Kirshner, Csaba Szepesvári
Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR 9:605-612, 2010.

Abstract

We propose a new method for a non-parametric estimation of Renyi and Shannon information for a multivariate distribution using a corresponding copula, a multivariate distribution over normalized ranks of the data. As the information of the distribution is the same as the negative entropy of its copula, our method estimates this information by solving a Euclidean graph optimization problem on the empirical estimate of the distribution’s copula. Owing to the properties of the copula, we show that the resulting estimator of Renyi information is strongly consistent and robust. Further, we demonstrate its applicability in the image registration in addition to simulated experiments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v9-poczos10a, title = {REGO: Rank-based Estimation of Renyi Information using Euclidean Graph Optimization}, author = {Poczos, Barnabas and Kirshner, Sergey and Szepesvári, Csaba}, booktitle = {Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics}, pages = {605--612}, year = {2010}, editor = {Teh, Yee Whye and Titterington, Mike}, volume = {9}, series = {Proceedings of Machine Learning Research}, address = {Chia Laguna Resort, Sardinia, Italy}, month = {13--15 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v9/poczos10a/poczos10a.pdf}, url = {https://proceedings.mlr.press/v9/poczos10a.html}, abstract = {We propose a new method for a non-parametric estimation of Renyi and Shannon information for a multivariate distribution using a corresponding copula, a multivariate distribution over normalized ranks of the data. As the information of the distribution is the same as the negative entropy of its copula, our method estimates this information by solving a Euclidean graph optimization problem on the empirical estimate of the distribution’s copula. Owing to the properties of the copula, we show that the resulting estimator of Renyi information is strongly consistent and robust. Further, we demonstrate its applicability in the image registration in addition to simulated experiments.} }
Endnote
%0 Conference Paper %T REGO: Rank-based Estimation of Renyi Information using Euclidean Graph Optimization %A Barnabas Poczos %A Sergey Kirshner %A Csaba Szepesvári %B Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2010 %E Yee Whye Teh %E Mike Titterington %F pmlr-v9-poczos10a %I PMLR %P 605--612 %U https://proceedings.mlr.press/v9/poczos10a.html %V 9 %X We propose a new method for a non-parametric estimation of Renyi and Shannon information for a multivariate distribution using a corresponding copula, a multivariate distribution over normalized ranks of the data. As the information of the distribution is the same as the negative entropy of its copula, our method estimates this information by solving a Euclidean graph optimization problem on the empirical estimate of the distribution’s copula. Owing to the properties of the copula, we show that the resulting estimator of Renyi information is strongly consistent and robust. Further, we demonstrate its applicability in the image registration in addition to simulated experiments.
RIS
TY - CPAPER TI - REGO: Rank-based Estimation of Renyi Information using Euclidean Graph Optimization AU - Barnabas Poczos AU - Sergey Kirshner AU - Csaba Szepesvári BT - Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics DA - 2010/03/31 ED - Yee Whye Teh ED - Mike Titterington ID - pmlr-v9-poczos10a PB - PMLR DP - Proceedings of Machine Learning Research VL - 9 SP - 605 EP - 612 L1 - http://proceedings.mlr.press/v9/poczos10a/poczos10a.pdf UR - https://proceedings.mlr.press/v9/poczos10a.html AB - We propose a new method for a non-parametric estimation of Renyi and Shannon information for a multivariate distribution using a corresponding copula, a multivariate distribution over normalized ranks of the data. As the information of the distribution is the same as the negative entropy of its copula, our method estimates this information by solving a Euclidean graph optimization problem on the empirical estimate of the distribution’s copula. Owing to the properties of the copula, we show that the resulting estimator of Renyi information is strongly consistent and robust. Further, we demonstrate its applicability in the image registration in addition to simulated experiments. ER -
APA
Poczos, B., Kirshner, S. & Szepesvári, C.. (2010). REGO: Rank-based Estimation of Renyi Information using Euclidean Graph Optimization. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 9:605-612 Available from https://proceedings.mlr.press/v9/poczos10a.html.

Related Material