Conjugate Energy-Based Models

Hao Wu, Babak Esmaeili, Michael Wick, Jean-Baptiste Tristan, Jan-Willem Van De Meent
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:11228-11239, 2021.

Abstract

In this paper, we propose conjugate energy-based models (CEBMs), a new class of energy-based models that define a joint density over data and latent variables. The joint density of a CEBM decomposes into an intractable distribution over data and a tractable posterior over latent variables. CEBMs have similar use cases as variational autoencoders, in the sense that they learn an unsupervised mapping from data to latent variables. However, these models omit a generator network, which allows them to learn more flexible notions of similarity between data points. Our experiments demonstrate that conjugate EBMs achieve competitive results in terms of image modelling, predictive power of latent space, and out-of-domain detection on a variety of datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-wu21a, title = {Conjugate Energy-Based Models}, author = {Wu, Hao and Esmaeili, Babak and Wick, Michael and Tristan, Jean-Baptiste and Van De Meent, Jan-Willem}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {11228--11239}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/wu21a/wu21a.pdf}, url = {https://proceedings.mlr.press/v139/wu21a.html}, abstract = {In this paper, we propose conjugate energy-based models (CEBMs), a new class of energy-based models that define a joint density over data and latent variables. The joint density of a CEBM decomposes into an intractable distribution over data and a tractable posterior over latent variables. CEBMs have similar use cases as variational autoencoders, in the sense that they learn an unsupervised mapping from data to latent variables. However, these models omit a generator network, which allows them to learn more flexible notions of similarity between data points. Our experiments demonstrate that conjugate EBMs achieve competitive results in terms of image modelling, predictive power of latent space, and out-of-domain detection on a variety of datasets.} }
Endnote
%0 Conference Paper %T Conjugate Energy-Based Models %A Hao Wu %A Babak Esmaeili %A Michael Wick %A Jean-Baptiste Tristan %A Jan-Willem Van De Meent %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-wu21a %I PMLR %P 11228--11239 %U https://proceedings.mlr.press/v139/wu21a.html %V 139 %X In this paper, we propose conjugate energy-based models (CEBMs), a new class of energy-based models that define a joint density over data and latent variables. The joint density of a CEBM decomposes into an intractable distribution over data and a tractable posterior over latent variables. CEBMs have similar use cases as variational autoencoders, in the sense that they learn an unsupervised mapping from data to latent variables. However, these models omit a generator network, which allows them to learn more flexible notions of similarity between data points. Our experiments demonstrate that conjugate EBMs achieve competitive results in terms of image modelling, predictive power of latent space, and out-of-domain detection on a variety of datasets.
APA
Wu, H., Esmaeili, B., Wick, M., Tristan, J. & Van De Meent, J.. (2021). Conjugate Energy-Based Models. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:11228-11239 Available from https://proceedings.mlr.press/v139/wu21a.html.

Related Material