REGO: Rank-based Estimation of Renyi Information using Euclidean Graph Optimization


Barnabas Poczos, Sergey Kirshner, Csaba Szepesvári ;
Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR 9:605-612, 2010.


We propose a new method for a non-parametric estimation of Renyi and Shannon information for a multivariate distribution using a corresponding copula, a multivariate distribution over normalized ranks of the data. As the information of the distribution is the same as the negative entropy of its copula, our method estimates this information by solving a Euclidean graph optimization problem on the empirical estimate of the distribution’s copula. Owing to the properties of the copula, we show that the resulting estimator of Renyi information is strongly consistent and robust. Further, we demonstrate its applicability in the image registration in addition to simulated experiments.

Related Material