PAC-Bayesian Generalization Bounds for Adversarial Generative Models

Sokhna Diarra Mbacke, Florence Clerc, Pascal Germain
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:24271-24290, 2023.

Abstract

We extend PAC-Bayesian theory to generative models and develop generalization bounds for models based on the Wasserstein distance and the total variation distance. Our first result on the Wasserstein distance assumes the instance space is bounded, while our second result takes advantage of dimensionality reduction. Our results naturally apply to Wasserstein GANs and Energy-Based GANs, and our bounds provide new training objectives for these two. Although our work is mainly theoretical, we perform numerical experiments showing non-vacuous generalization bounds for Wasserstein GANs on synthetic datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-mbacke23a, title = {{PAC}-{B}ayesian Generalization Bounds for Adversarial Generative Models}, author = {Mbacke, Sokhna Diarra and Clerc, Florence and Germain, Pascal}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {24271--24290}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/mbacke23a/mbacke23a.pdf}, url = {https://proceedings.mlr.press/v202/mbacke23a.html}, abstract = {We extend PAC-Bayesian theory to generative models and develop generalization bounds for models based on the Wasserstein distance and the total variation distance. Our first result on the Wasserstein distance assumes the instance space is bounded, while our second result takes advantage of dimensionality reduction. Our results naturally apply to Wasserstein GANs and Energy-Based GANs, and our bounds provide new training objectives for these two. Although our work is mainly theoretical, we perform numerical experiments showing non-vacuous generalization bounds for Wasserstein GANs on synthetic datasets.} }
Endnote
%0 Conference Paper %T PAC-Bayesian Generalization Bounds for Adversarial Generative Models %A Sokhna Diarra Mbacke %A Florence Clerc %A Pascal Germain %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-mbacke23a %I PMLR %P 24271--24290 %U https://proceedings.mlr.press/v202/mbacke23a.html %V 202 %X We extend PAC-Bayesian theory to generative models and develop generalization bounds for models based on the Wasserstein distance and the total variation distance. Our first result on the Wasserstein distance assumes the instance space is bounded, while our second result takes advantage of dimensionality reduction. Our results naturally apply to Wasserstein GANs and Energy-Based GANs, and our bounds provide new training objectives for these two. Although our work is mainly theoretical, we perform numerical experiments showing non-vacuous generalization bounds for Wasserstein GANs on synthetic datasets.
APA
Mbacke, S.D., Clerc, F. & Germain, P.. (2023). PAC-Bayesian Generalization Bounds for Adversarial Generative Models. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:24271-24290 Available from https://proceedings.mlr.press/v202/mbacke23a.html.

Related Material