On Excess Mass Behavior in Gaussian Mixture Models with Orlicz-Wasserstein Distances

Aritra Guha, Nhat Ho, Xuanlong Nguyen
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:11847-11870, 2023.

Abstract

Dirichlet Process mixture models (DPMM) in combination with Gaussian kernels have been an important modeling tool for numerous data domains arising from biological, physical, and social sciences. However, this versatility in applications does not extend to strong theoretical guarantees for the underlying parameter estimates, for which only a logarithmic rate is achieved. In this work, we (re)introduce and investigate a metric, named Orlicz-Wasserstein distance, in the study of the Bayesian contraction behavior for the parameters. We show that despite the overall slow convergence guarantees for all the parameters, posterior contraction for parameters happens at almost polynomial rates in outlier regions of the parameter space. Our theoretical results provide new insight in understanding the convergence behavior of parameters arising from various settings of hierarchical Bayesian nonparametric models. In addition, we provide an algorithm to compute the metric by leveraging Sinkhorn divergences and validate our findings through a simulation study.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-guha23a, title = {On Excess Mass Behavior in {G}aussian Mixture Models with Orlicz-{W}asserstein Distances}, author = {Guha, Aritra and Ho, Nhat and Nguyen, Xuanlong}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {11847--11870}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/guha23a/guha23a.pdf}, url = {https://proceedings.mlr.press/v202/guha23a.html}, abstract = {Dirichlet Process mixture models (DPMM) in combination with Gaussian kernels have been an important modeling tool for numerous data domains arising from biological, physical, and social sciences. However, this versatility in applications does not extend to strong theoretical guarantees for the underlying parameter estimates, for which only a logarithmic rate is achieved. In this work, we (re)introduce and investigate a metric, named Orlicz-Wasserstein distance, in the study of the Bayesian contraction behavior for the parameters. We show that despite the overall slow convergence guarantees for all the parameters, posterior contraction for parameters happens at almost polynomial rates in outlier regions of the parameter space. Our theoretical results provide new insight in understanding the convergence behavior of parameters arising from various settings of hierarchical Bayesian nonparametric models. In addition, we provide an algorithm to compute the metric by leveraging Sinkhorn divergences and validate our findings through a simulation study.} }
Endnote
%0 Conference Paper %T On Excess Mass Behavior in Gaussian Mixture Models with Orlicz-Wasserstein Distances %A Aritra Guha %A Nhat Ho %A Xuanlong Nguyen %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-guha23a %I PMLR %P 11847--11870 %U https://proceedings.mlr.press/v202/guha23a.html %V 202 %X Dirichlet Process mixture models (DPMM) in combination with Gaussian kernels have been an important modeling tool for numerous data domains arising from biological, physical, and social sciences. However, this versatility in applications does not extend to strong theoretical guarantees for the underlying parameter estimates, for which only a logarithmic rate is achieved. In this work, we (re)introduce and investigate a metric, named Orlicz-Wasserstein distance, in the study of the Bayesian contraction behavior for the parameters. We show that despite the overall slow convergence guarantees for all the parameters, posterior contraction for parameters happens at almost polynomial rates in outlier regions of the parameter space. Our theoretical results provide new insight in understanding the convergence behavior of parameters arising from various settings of hierarchical Bayesian nonparametric models. In addition, we provide an algorithm to compute the metric by leveraging Sinkhorn divergences and validate our findings through a simulation study.
APA
Guha, A., Ho, N. & Nguyen, X.. (2023). On Excess Mass Behavior in Gaussian Mixture Models with Orlicz-Wasserstein Distances. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:11847-11870 Available from https://proceedings.mlr.press/v202/guha23a.html.

Related Material