Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distances

Ruben Ohana, Kimia Nadjahi, Alain Rakotomamonjy, Liva Ralaivola
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:26451-26473, 2023.

Abstract

The Sliced-Wasserstein distance (SW) is a computationally efficient and theoretically grounded alternative to the Wasserstein distance. Yet, the literature on its statistical properties – or, more accurately, its generalization properties – with respect to the distribution of slices, beyond the uniform measure, is scarce. To bring new contributions to this line of research, we leverage the PAC-Bayesian theory and a central observation that SW may be interpreted as an average risk, the quantity PAC-Bayesian bounds have been designed to characterize. We provide three types of results: i) PAC-Bayesian generalization bounds that hold on what we refer as adaptive Sliced-Wasserstein distances, i.e. SW defined with respect to arbitrary distributions of slices (among which data-dependent distributions), ii) a principled procedure to learn the distribution of slices that yields maximally discriminative SW, by optimizing our theoretical bounds, and iii) empirical illustrations of our theoretical findings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-ohana23a, title = {Shedding a {PAC}-{B}ayesian Light on Adaptive Sliced-{W}asserstein Distances}, author = {Ohana, Ruben and Nadjahi, Kimia and Rakotomamonjy, Alain and Ralaivola, Liva}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {26451--26473}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/ohana23a/ohana23a.pdf}, url = {https://proceedings.mlr.press/v202/ohana23a.html}, abstract = {The Sliced-Wasserstein distance (SW) is a computationally efficient and theoretically grounded alternative to the Wasserstein distance. Yet, the literature on its statistical properties – or, more accurately, its generalization properties – with respect to the distribution of slices, beyond the uniform measure, is scarce. To bring new contributions to this line of research, we leverage the PAC-Bayesian theory and a central observation that SW may be interpreted as an average risk, the quantity PAC-Bayesian bounds have been designed to characterize. We provide three types of results: i) PAC-Bayesian generalization bounds that hold on what we refer as adaptive Sliced-Wasserstein distances, i.e. SW defined with respect to arbitrary distributions of slices (among which data-dependent distributions), ii) a principled procedure to learn the distribution of slices that yields maximally discriminative SW, by optimizing our theoretical bounds, and iii) empirical illustrations of our theoretical findings.} }
Endnote
%0 Conference Paper %T Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distances %A Ruben Ohana %A Kimia Nadjahi %A Alain Rakotomamonjy %A Liva Ralaivola %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-ohana23a %I PMLR %P 26451--26473 %U https://proceedings.mlr.press/v202/ohana23a.html %V 202 %X The Sliced-Wasserstein distance (SW) is a computationally efficient and theoretically grounded alternative to the Wasserstein distance. Yet, the literature on its statistical properties – or, more accurately, its generalization properties – with respect to the distribution of slices, beyond the uniform measure, is scarce. To bring new contributions to this line of research, we leverage the PAC-Bayesian theory and a central observation that SW may be interpreted as an average risk, the quantity PAC-Bayesian bounds have been designed to characterize. We provide three types of results: i) PAC-Bayesian generalization bounds that hold on what we refer as adaptive Sliced-Wasserstein distances, i.e. SW defined with respect to arbitrary distributions of slices (among which data-dependent distributions), ii) a principled procedure to learn the distribution of slices that yields maximally discriminative SW, by optimizing our theoretical bounds, and iii) empirical illustrations of our theoretical findings.
APA
Ohana, R., Nadjahi, K., Rakotomamonjy, A. & Ralaivola, L.. (2023). Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distances. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:26451-26473 Available from https://proceedings.mlr.press/v202/ohana23a.html.

Related Material