Novel Change of Measure Inequalities with Applications to PAC-Bayesian Bounds and Monte Carlo Estimation

Yuki Ohnishi, Jean Honorio
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:1711-1719, 2021.

Abstract

We introduce several novel change of measure inequalities for two families of divergences: $f$-divergences and $\alpha$-divergences. We show how the variational representation for $f$-divergences leads to novel change of measure inequalities. We also present a multiplicative change of measure inequality for $\alpha$-divergences and a generalized version of Hammersley-Chapman-Robbins inequality. Finally, we present several applications of our change of measure inequalities, including PAC-Bayesian bounds for various classes of losses and non-asymptotic intervals for Monte Carlo estimates.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-ohnishi21a, title = { Novel Change of Measure Inequalities with Applications to PAC-Bayesian Bounds and Monte Carlo Estimation }, author = {Ohnishi, Yuki and Honorio, Jean}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {1711--1719}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/ohnishi21a/ohnishi21a.pdf}, url = {https://proceedings.mlr.press/v130/ohnishi21a.html}, abstract = { We introduce several novel change of measure inequalities for two families of divergences: $f$-divergences and $\alpha$-divergences. We show how the variational representation for $f$-divergences leads to novel change of measure inequalities. We also present a multiplicative change of measure inequality for $\alpha$-divergences and a generalized version of Hammersley-Chapman-Robbins inequality. Finally, we present several applications of our change of measure inequalities, including PAC-Bayesian bounds for various classes of losses and non-asymptotic intervals for Monte Carlo estimates. } }
Endnote
%0 Conference Paper %T Novel Change of Measure Inequalities with Applications to PAC-Bayesian Bounds and Monte Carlo Estimation %A Yuki Ohnishi %A Jean Honorio %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-ohnishi21a %I PMLR %P 1711--1719 %U https://proceedings.mlr.press/v130/ohnishi21a.html %V 130 %X We introduce several novel change of measure inequalities for two families of divergences: $f$-divergences and $\alpha$-divergences. We show how the variational representation for $f$-divergences leads to novel change of measure inequalities. We also present a multiplicative change of measure inequality for $\alpha$-divergences and a generalized version of Hammersley-Chapman-Robbins inequality. Finally, we present several applications of our change of measure inequalities, including PAC-Bayesian bounds for various classes of losses and non-asymptotic intervals for Monte Carlo estimates.
APA
Ohnishi, Y. & Honorio, J.. (2021). Novel Change of Measure Inequalities with Applications to PAC-Bayesian Bounds and Monte Carlo Estimation . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:1711-1719 Available from https://proceedings.mlr.press/v130/ohnishi21a.html.

Related Material