Generalized Exponential Concentration Inequality for Renyi Divergence Estimation

Shashank Singh, Barnabas Poczos
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(1):333-341, 2014.

Abstract

Estimating divergences between probability distributions in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main contribution of our work is to provide such a bound for an estimator of Renyi divergence for a smooth Holder class of densities on the d-dimensional unit cube. We also illustrate our theoretical results with a numerical experiment.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-singh14, title = {Generalized Exponential Concentration Inequality for Renyi Divergence Estimation}, author = {Singh, Shashank and Poczos, Barnabas}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {333--341}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {1}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/singh14.pdf}, url = {https://proceedings.mlr.press/v32/singh14.html}, abstract = {Estimating divergences between probability distributions in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main contribution of our work is to provide such a bound for an estimator of Renyi divergence for a smooth Holder class of densities on the d-dimensional unit cube. We also illustrate our theoretical results with a numerical experiment.} }
Endnote
%0 Conference Paper %T Generalized Exponential Concentration Inequality for Renyi Divergence Estimation %A Shashank Singh %A Barnabas Poczos %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-singh14 %I PMLR %P 333--341 %U https://proceedings.mlr.press/v32/singh14.html %V 32 %N 1 %X Estimating divergences between probability distributions in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main contribution of our work is to provide such a bound for an estimator of Renyi divergence for a smooth Holder class of densities on the d-dimensional unit cube. We also illustrate our theoretical results with a numerical experiment.
RIS
TY - CPAPER TI - Generalized Exponential Concentration Inequality for Renyi Divergence Estimation AU - Shashank Singh AU - Barnabas Poczos BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/01/27 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-singh14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 1 SP - 333 EP - 341 L1 - http://proceedings.mlr.press/v32/singh14.pdf UR - https://proceedings.mlr.press/v32/singh14.html AB - Estimating divergences between probability distributions in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main contribution of our work is to provide such a bound for an estimator of Renyi divergence for a smooth Holder class of densities on the d-dimensional unit cube. We also illustrate our theoretical results with a numerical experiment. ER -
APA
Singh, S. & Poczos, B.. (2014). Generalized Exponential Concentration Inequality for Renyi Divergence Estimation. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(1):333-341 Available from https://proceedings.mlr.press/v32/singh14.html.

Related Material