Clustering above Exponential Families with Tempered Exponential Measures

Ehsan Amid, Richard Nock, Manfred K. Warmuth
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:2994-3017, 2023.

Abstract

The link with exponential families has allowed k-means clustering to be generalized to a wide variety of data-generating distributions in exponential families and clustering distortions among Bregman divergences. Getting the framework to go beyond exponential families is important to lift roadblocks like the lack of robustness of some population minimizers, which is carved into their axiomatization. Current generalizations of exponential families like the q-exponential families or even the deformed exponential families fail at achieving the goal. In this paper, we provide a new attempt at getting a complete framework, grounded in a new generalization of exponential families that we introduce, called tempered exponential measures (TEMs). TEMs keep the maximum entropy axiomatization framework of q-exponential families, but instead of normalizing the measure, normalize a dual called a co-distribution. Numerous interesting properties arise for clustering, such as improved and controllable robustness for population minimizers, that keep a simple analytic form.

Cite this Paper


BibTeX
@InProceedings{pmlr-v206-amid23a, title = {Clustering above Exponential Families with Tempered Exponential Measures}, author = {Amid, Ehsan and Nock, Richard and Warmuth, Manfred K.}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {2994--3017}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/amid23a/amid23a.pdf}, url = {https://proceedings.mlr.press/v206/amid23a.html}, abstract = {The link with exponential families has allowed k-means clustering to be generalized to a wide variety of data-generating distributions in exponential families and clustering distortions among Bregman divergences. Getting the framework to go beyond exponential families is important to lift roadblocks like the lack of robustness of some population minimizers, which is carved into their axiomatization. Current generalizations of exponential families like the q-exponential families or even the deformed exponential families fail at achieving the goal. In this paper, we provide a new attempt at getting a complete framework, grounded in a new generalization of exponential families that we introduce, called tempered exponential measures (TEMs). TEMs keep the maximum entropy axiomatization framework of q-exponential families, but instead of normalizing the measure, normalize a dual called a co-distribution. Numerous interesting properties arise for clustering, such as improved and controllable robustness for population minimizers, that keep a simple analytic form.} }
Endnote
%0 Conference Paper %T Clustering above Exponential Families with Tempered Exponential Measures %A Ehsan Amid %A Richard Nock %A Manfred K. Warmuth %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-amid23a %I PMLR %P 2994--3017 %U https://proceedings.mlr.press/v206/amid23a.html %V 206 %X The link with exponential families has allowed k-means clustering to be generalized to a wide variety of data-generating distributions in exponential families and clustering distortions among Bregman divergences. Getting the framework to go beyond exponential families is important to lift roadblocks like the lack of robustness of some population minimizers, which is carved into their axiomatization. Current generalizations of exponential families like the q-exponential families or even the deformed exponential families fail at achieving the goal. In this paper, we provide a new attempt at getting a complete framework, grounded in a new generalization of exponential families that we introduce, called tempered exponential measures (TEMs). TEMs keep the maximum entropy axiomatization framework of q-exponential families, but instead of normalizing the measure, normalize a dual called a co-distribution. Numerous interesting properties arise for clustering, such as improved and controllable robustness for population minimizers, that keep a simple analytic form.
APA
Amid, E., Nock, R. & Warmuth, M.K.. (2023). Clustering above Exponential Families with Tempered Exponential Measures. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:2994-3017 Available from https://proceedings.mlr.press/v206/amid23a.html.

Related Material