A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation

Jongha Jon Ryu, Abhin Shah, Gregory W. Wornell
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:52444-52474, 2025.

Abstract

This paper studies a family of estimators based on noise-contrastive estimation (NCE) for learning unnormalized distributions. The main contribution of this work is to provide a unified perspective on various methods for learning unnormalized distributions, which have been independently proposed and studied in separate research communities, through the lens of NCE. This unified view offers new insights into existing estimators. Specifically, for exponential families, we establish the finite-sample convergence rates of the proposed estimators under a set of regularity assumptions, most of which are new.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-ryu25a, title = {A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation}, author = {Ryu, Jongha Jon and Shah, Abhin and Wornell, Gregory W.}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {52444--52474}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/ryu25a/ryu25a.pdf}, url = {https://proceedings.mlr.press/v267/ryu25a.html}, abstract = {This paper studies a family of estimators based on noise-contrastive estimation (NCE) for learning unnormalized distributions. The main contribution of this work is to provide a unified perspective on various methods for learning unnormalized distributions, which have been independently proposed and studied in separate research communities, through the lens of NCE. This unified view offers new insights into existing estimators. Specifically, for exponential families, we establish the finite-sample convergence rates of the proposed estimators under a set of regularity assumptions, most of which are new.} }
Endnote
%0 Conference Paper %T A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation %A Jongha Jon Ryu %A Abhin Shah %A Gregory W. Wornell %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-ryu25a %I PMLR %P 52444--52474 %U https://proceedings.mlr.press/v267/ryu25a.html %V 267 %X This paper studies a family of estimators based on noise-contrastive estimation (NCE) for learning unnormalized distributions. The main contribution of this work is to provide a unified perspective on various methods for learning unnormalized distributions, which have been independently proposed and studied in separate research communities, through the lens of NCE. This unified view offers new insights into existing estimators. Specifically, for exponential families, we establish the finite-sample convergence rates of the proposed estimators under a set of regularity assumptions, most of which are new.
APA
Ryu, J.J., Shah, A. & Wornell, G.W.. (2025). A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:52444-52474 Available from https://proceedings.mlr.press/v267/ryu25a.html.

Related Material