A Simulated Annealing Based Inexact Oracle for Wasserstein Loss Minimization

Jianbo Ye, James Z. Wang, Jia Li
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:3940-3948, 2017.

Abstract

Learning under a Wasserstein loss, a.k.a. Wasserstein loss minimization (WLM), is an emerging research topic for gaining insights from a large set of structured objects. Despite being conceptually simple, WLM problems are computationally challenging because they involve minimizing over functions of quantities (i.e. Wasserstein distances) that themselves require numerical algorithms to compute. In this paper, we introduce a stochastic approach based on simulated annealing for solving WLMs. Particularly, we have developed a Gibbs sampler to approximate effectively and efficiently the partial gradients of a sequence of Wasserstein losses. Our new approach has the advantages of numerical stability and readiness for warm starts. These characteristics are valuable for WLM problems that often require multiple levels of iterations in which the oracle for computing the value and gradient of a loss function is embedded. We applied the method to optimal transport with Coulomb cost and the Wasserstein non-negative matrix factorization problem, and made comparisons with the existing method of entropy regularization.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-ye17b, title = {A Simulated Annealing Based Inexact Oracle for {W}asserstein Loss Minimization}, author = {Jianbo Ye and James Z. Wang and Jia Li}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {3940--3948}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/ye17b/ye17b.pdf}, url = {https://proceedings.mlr.press/v70/ye17b.html}, abstract = {Learning under a Wasserstein loss, a.k.a. Wasserstein loss minimization (WLM), is an emerging research topic for gaining insights from a large set of structured objects. Despite being conceptually simple, WLM problems are computationally challenging because they involve minimizing over functions of quantities (i.e. Wasserstein distances) that themselves require numerical algorithms to compute. In this paper, we introduce a stochastic approach based on simulated annealing for solving WLMs. Particularly, we have developed a Gibbs sampler to approximate effectively and efficiently the partial gradients of a sequence of Wasserstein losses. Our new approach has the advantages of numerical stability and readiness for warm starts. These characteristics are valuable for WLM problems that often require multiple levels of iterations in which the oracle for computing the value and gradient of a loss function is embedded. We applied the method to optimal transport with Coulomb cost and the Wasserstein non-negative matrix factorization problem, and made comparisons with the existing method of entropy regularization.} }
Endnote
%0 Conference Paper %T A Simulated Annealing Based Inexact Oracle for Wasserstein Loss Minimization %A Jianbo Ye %A James Z. Wang %A Jia Li %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-ye17b %I PMLR %P 3940--3948 %U https://proceedings.mlr.press/v70/ye17b.html %V 70 %X Learning under a Wasserstein loss, a.k.a. Wasserstein loss minimization (WLM), is an emerging research topic for gaining insights from a large set of structured objects. Despite being conceptually simple, WLM problems are computationally challenging because they involve minimizing over functions of quantities (i.e. Wasserstein distances) that themselves require numerical algorithms to compute. In this paper, we introduce a stochastic approach based on simulated annealing for solving WLMs. Particularly, we have developed a Gibbs sampler to approximate effectively and efficiently the partial gradients of a sequence of Wasserstein losses. Our new approach has the advantages of numerical stability and readiness for warm starts. These characteristics are valuable for WLM problems that often require multiple levels of iterations in which the oracle for computing the value and gradient of a loss function is embedded. We applied the method to optimal transport with Coulomb cost and the Wasserstein non-negative matrix factorization problem, and made comparisons with the existing method of entropy regularization.
APA
Ye, J., Wang, J.Z. & Li, J.. (2017). A Simulated Annealing Based Inexact Oracle for Wasserstein Loss Minimization. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:3940-3948 Available from https://proceedings.mlr.press/v70/ye17b.html.

Related Material