On the Convergence of Projected Bures-Wasserstein Gradient Descent under Euclidean Strong Convexity

Junyi Fan, Yuxuan Han, Zijian Liu, Jian-Feng Cai, Yang Wang, Zhengyuan Zhou
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:12832-12857, 2024.

Abstract

The Bures-Wasserstein (BW) gradient descent method has gained considerable attention in various domains, including Gaussian barycenter, matrix recovery and variational inference problems, due to its alignment with the Wasserstein geometry of normal distributions. Despite its popularity, existing convergence analysis are often contingent upon specific loss functions, and the exploration of constrained settings within this framework remains limited. In this work, we make an attempt to bridge this gap by providing a general convergence rate guarantee for BW gradient descent when the Euclidean strong convexity of the loss and the constraints is assumed. In an effort to advance practical implementations, we also derive a closed-form solution for the projection onto BW distance-constrained sets, which enables the fast implementation of projected BW gradient descent for problems that arise in the constrained barycenter and distributionally robust optimization literature. Experimental results demonstrate significant improvements in computational efficiency and convergence speed, underscoring the efficacy of our method in practical scenarios.

Cite this Paper


BibTeX
@InProceedings{pmlr-v235-fan24b, title = {On the Convergence of Projected Bures-{W}asserstein Gradient Descent under {E}uclidean Strong Convexity}, author = {Fan, Junyi and Han, Yuxuan and Liu, Zijian and Cai, Jian-Feng and Wang, Yang and Zhou, Zhengyuan}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {12832--12857}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/fan24b/fan24b.pdf}, url = {https://proceedings.mlr.press/v235/fan24b.html}, abstract = {The Bures-Wasserstein (BW) gradient descent method has gained considerable attention in various domains, including Gaussian barycenter, matrix recovery and variational inference problems, due to its alignment with the Wasserstein geometry of normal distributions. Despite its popularity, existing convergence analysis are often contingent upon specific loss functions, and the exploration of constrained settings within this framework remains limited. In this work, we make an attempt to bridge this gap by providing a general convergence rate guarantee for BW gradient descent when the Euclidean strong convexity of the loss and the constraints is assumed. In an effort to advance practical implementations, we also derive a closed-form solution for the projection onto BW distance-constrained sets, which enables the fast implementation of projected BW gradient descent for problems that arise in the constrained barycenter and distributionally robust optimization literature. Experimental results demonstrate significant improvements in computational efficiency and convergence speed, underscoring the efficacy of our method in practical scenarios.} }
Endnote
%0 Conference Paper %T On the Convergence of Projected Bures-Wasserstein Gradient Descent under Euclidean Strong Convexity %A Junyi Fan %A Yuxuan Han %A Zijian Liu %A Jian-Feng Cai %A Yang Wang %A Zhengyuan Zhou %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-fan24b %I PMLR %P 12832--12857 %U https://proceedings.mlr.press/v235/fan24b.html %V 235 %X The Bures-Wasserstein (BW) gradient descent method has gained considerable attention in various domains, including Gaussian barycenter, matrix recovery and variational inference problems, due to its alignment with the Wasserstein geometry of normal distributions. Despite its popularity, existing convergence analysis are often contingent upon specific loss functions, and the exploration of constrained settings within this framework remains limited. In this work, we make an attempt to bridge this gap by providing a general convergence rate guarantee for BW gradient descent when the Euclidean strong convexity of the loss and the constraints is assumed. In an effort to advance practical implementations, we also derive a closed-form solution for the projection onto BW distance-constrained sets, which enables the fast implementation of projected BW gradient descent for problems that arise in the constrained barycenter and distributionally robust optimization literature. Experimental results demonstrate significant improvements in computational efficiency and convergence speed, underscoring the efficacy of our method in practical scenarios.
APA
Fan, J., Han, Y., Liu, Z., Cai, J., Wang, Y. & Zhou, Z.. (2024). On the Convergence of Projected Bures-Wasserstein Gradient Descent under Euclidean Strong Convexity. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:12832-12857 Available from https://proceedings.mlr.press/v235/fan24b.html.

Related Material