Mean-Field Microcanonical Gradient Descent

Marcus Häggbom, Morten Karlsmark, Joakim Andén
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:5185-5193, 2025.

Abstract

Microcanonical gradient descent is a sampling procedure for energy-based models allowing for efficient sampling of distributions in high dimension. It works by transporting samples from a high-entropy distribution, such as Gaussian white noise, to a low-energy region using gradient descent. We put this model in the framework of normalizing flows, showing how it can often overfit by losing an unnecessary amount of entropy in the descent. As a remedy, we propose a mean-field microcanonical gradient descent that samples several weakly coupled data points simultaneously, allowing for better control of the entropy loss while paying little in terms of likelihood fit. We study these models in the context of stationary time series and 2D textures.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-haggbom25a, title = {Mean-Field Microcanonical Gradient Descent}, author = {H{\"a}ggbom, Marcus and Karlsmark, Morten and And{\'e}n, Joakim}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {5185--5193}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/haggbom25a/haggbom25a.pdf}, url = {https://proceedings.mlr.press/v258/haggbom25a.html}, abstract = {Microcanonical gradient descent is a sampling procedure for energy-based models allowing for efficient sampling of distributions in high dimension. It works by transporting samples from a high-entropy distribution, such as Gaussian white noise, to a low-energy region using gradient descent. We put this model in the framework of normalizing flows, showing how it can often overfit by losing an unnecessary amount of entropy in the descent. As a remedy, we propose a mean-field microcanonical gradient descent that samples several weakly coupled data points simultaneously, allowing for better control of the entropy loss while paying little in terms of likelihood fit. We study these models in the context of stationary time series and 2D textures.} }
Endnote
%0 Conference Paper %T Mean-Field Microcanonical Gradient Descent %A Marcus Häggbom %A Morten Karlsmark %A Joakim Andén %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-haggbom25a %I PMLR %P 5185--5193 %U https://proceedings.mlr.press/v258/haggbom25a.html %V 258 %X Microcanonical gradient descent is a sampling procedure for energy-based models allowing for efficient sampling of distributions in high dimension. It works by transporting samples from a high-entropy distribution, such as Gaussian white noise, to a low-energy region using gradient descent. We put this model in the framework of normalizing flows, showing how it can often overfit by losing an unnecessary amount of entropy in the descent. As a remedy, we propose a mean-field microcanonical gradient descent that samples several weakly coupled data points simultaneously, allowing for better control of the entropy loss while paying little in terms of likelihood fit. We study these models in the context of stationary time series and 2D textures.
APA
Häggbom, M., Karlsmark, M. & Andén, J.. (2025). Mean-Field Microcanonical Gradient Descent. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:5185-5193 Available from https://proceedings.mlr.press/v258/haggbom25a.html.

Related Material