Robust computation of optimal transport by $β$-potential regularization

Shintaro Nakamura, Han Bao, Masashi Sugiyama
Proceedings of The 14th Asian Conference on Machine Learning, PMLR 189:770-785, 2023.

Abstract

Optimal transport (OT) has become a widely used tool in the machine learning field to measure the discrepancy between probability distributions. For instance, OT is a popular loss function that quantifies the discrepancy between an empirical distribution and a parametric model. Recently, an entropic penalty term and the celebrated Sinkhorn algorithm have been commonly used to approximate the original OT in a computationally efficient way. However, since the Sinkhorn algorithm runs a projection associated with the Kullback-Leibler divergence, it is often vulnerable to outliers. To overcome this problem, we propose regularizing OT with the $\beta$-potential term associated with the so-called $\beta$-divergence, which was developed in robust statistics. Our theoretical analysis reveals that the $\beta$-potential can prevent the mass from being transported to outliers. We experimentally demonstrate that the transport matrix computed with our algorithm helps estimate a probability distribution robustly even in the presence of outliers. In addition, our proposed method can successfully detect outliers from a contaminated dataset.

Cite this Paper

BibTeX
@InProceedings{pmlr-v189-nakamura23a, title = {Robust computation of optimal transport by $β$-potential regularization}, author = {Nakamura, Shintaro and Bao, Han and Sugiyama, Masashi}, booktitle = {Proceedings of The 14th Asian Conference on Machine Learning}, pages = {770--785}, year = {2023}, editor = {Khan, Emtiyaz and Gonen, Mehmet}, volume = {189}, series = {Proceedings of Machine Learning Research}, month = {12--14 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v189/nakamura23a/nakamura23a.pdf}, url = {https://proceedings.mlr.press/v189/nakamura23a.html}, abstract = {Optimal transport (OT) has become a widely used tool in the machine learning field to measure the discrepancy between probability distributions. For instance, OT is a popular loss function that quantifies the discrepancy between an empirical distribution and a parametric model. Recently, an entropic penalty term and the celebrated Sinkhorn algorithm have been commonly used to approximate the original OT in a computationally efficient way. However, since the Sinkhorn algorithm runs a projection associated with the Kullback-Leibler divergence, it is often vulnerable to outliers. To overcome this problem, we propose regularizing OT with the $\beta$-potential term associated with the so-called $\beta$-divergence, which was developed in robust statistics. Our theoretical analysis reveals that the $\beta$-potential can prevent the mass from being transported to outliers. We experimentally demonstrate that the transport matrix computed with our algorithm helps estimate a probability distribution robustly even in the presence of outliers. In addition, our proposed method can successfully detect outliers from a contaminated dataset.} }
Endnote
%0 Conference Paper %T Robust computation of optimal transport by $β$-potential regularization %A Shintaro Nakamura %A Han Bao %A Masashi Sugiyama %B Proceedings of The 14th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Emtiyaz Khan %E Mehmet Gonen %F pmlr-v189-nakamura23a %I PMLR %P 770--785 %U https://proceedings.mlr.press/v189/nakamura23a.html %V 189 %X Optimal transport (OT) has become a widely used tool in the machine learning field to measure the discrepancy between probability distributions. For instance, OT is a popular loss function that quantifies the discrepancy between an empirical distribution and a parametric model. Recently, an entropic penalty term and the celebrated Sinkhorn algorithm have been commonly used to approximate the original OT in a computationally efficient way. However, since the Sinkhorn algorithm runs a projection associated with the Kullback-Leibler divergence, it is often vulnerable to outliers. To overcome this problem, we propose regularizing OT with the $\beta$-potential term associated with the so-called $\beta$-divergence, which was developed in robust statistics. Our theoretical analysis reveals that the $\beta$-potential can prevent the mass from being transported to outliers. We experimentally demonstrate that the transport matrix computed with our algorithm helps estimate a probability distribution robustly even in the presence of outliers. In addition, our proposed method can successfully detect outliers from a contaminated dataset.
APA
Nakamura, S., Bao, H. & Sugiyama, M.. (2023). Robust computation of optimal transport by $β$-potential regularization. Proceedings of The 14th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 189:770-785 Available from https://proceedings.mlr.press/v189/nakamura23a.html.

Related Material