Winner-takes-all learners are geometry-aware conditional density estimators

Victor Letzelter, David Perera, Cédric Rommel, Mathieu Fontaine, Slim Essid, Gaël Richard, Patrick Perez
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:27254-27287, 2024.

Abstract

Winner-takes-all training is a simple learning paradigm, which handles ambiguous tasks by predicting a set of plausible hypotheses. Recently, a connection was established between Winner-takes-all training and centroidal Voronoi tessellations, showing that, once trained, hypotheses should quantize optimally the shape of the conditional distribution to predict. However, the best use of these hypotheses for uncertainty quantification is still an open question. In this work, we show how to leverage the appealing geometric properties of the Winner-takes-all learners for conditional density estimation, without modifying its original training scheme. We theoretically establish the advantages of our novel estimator both in terms of quantization and density estimation, and we demonstrate its competitiveness on synthetic and real-world datasets, including audio data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-letzelter24a, title = {Winner-takes-all learners are geometry-aware conditional density estimators}, author = {Letzelter, Victor and Perera, David and Rommel, C\'{e}dric and Fontaine, Mathieu and Essid, Slim and Richard, Ga\"{e}l and Perez, Patrick}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {27254--27287}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/letzelter24a/letzelter24a.pdf}, url = {https://proceedings.mlr.press/v235/letzelter24a.html}, abstract = {Winner-takes-all training is a simple learning paradigm, which handles ambiguous tasks by predicting a set of plausible hypotheses. Recently, a connection was established between Winner-takes-all training and centroidal Voronoi tessellations, showing that, once trained, hypotheses should quantize optimally the shape of the conditional distribution to predict. However, the best use of these hypotheses for uncertainty quantification is still an open question. In this work, we show how to leverage the appealing geometric properties of the Winner-takes-all learners for conditional density estimation, without modifying its original training scheme. We theoretically establish the advantages of our novel estimator both in terms of quantization and density estimation, and we demonstrate its competitiveness on synthetic and real-world datasets, including audio data.} }
Endnote
%0 Conference Paper %T Winner-takes-all learners are geometry-aware conditional density estimators %A Victor Letzelter %A David Perera %A Cédric Rommel %A Mathieu Fontaine %A Slim Essid %A Gaël Richard %A Patrick Perez %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-letzelter24a %I PMLR %P 27254--27287 %U https://proceedings.mlr.press/v235/letzelter24a.html %V 235 %X Winner-takes-all training is a simple learning paradigm, which handles ambiguous tasks by predicting a set of plausible hypotheses. Recently, a connection was established between Winner-takes-all training and centroidal Voronoi tessellations, showing that, once trained, hypotheses should quantize optimally the shape of the conditional distribution to predict. However, the best use of these hypotheses for uncertainty quantification is still an open question. In this work, we show how to leverage the appealing geometric properties of the Winner-takes-all learners for conditional density estimation, without modifying its original training scheme. We theoretically establish the advantages of our novel estimator both in terms of quantization and density estimation, and we demonstrate its competitiveness on synthetic and real-world datasets, including audio data.
APA
Letzelter, V., Perera, D., Rommel, C., Fontaine, M., Essid, S., Richard, G. & Perez, P.. (2024). Winner-takes-all learners are geometry-aware conditional density estimators. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:27254-27287 Available from https://proceedings.mlr.press/v235/letzelter24a.html.

Related Material