Mean-field Underdamped Langevin Dynamics and its Spacetime Discretization

Qiang Fu, Ashia Camage Wilson
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:14175-14206, 2024.

Abstract

We propose a new method called the N-particle underdamped Langevin algorithm for optimizing a special class of non-linear functionals defined over the space of probability measures. Examples of problems with this formulation include training mean-field neural networks, maximum mean discrepancy minimization and kernel Stein discrepancy minimization. Our algorithm is based on a novel spacetime discretization of the mean-field underdamped Langevin dynamics, for which we provide a new, fast mixing guarantee. In addition, we demonstrate that our algorithm converges globally in total variation distance, bridging the theoretical gap between the dynamics and its practical implementation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-fu24g, title = {Mean-field Underdamped {L}angevin Dynamics and its Spacetime Discretization}, author = {Fu, Qiang and Wilson, Ashia Camage}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {14175--14206}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/fu24g/fu24g.pdf}, url = {https://proceedings.mlr.press/v235/fu24g.html}, abstract = {We propose a new method called the N-particle underdamped Langevin algorithm for optimizing a special class of non-linear functionals defined over the space of probability measures. Examples of problems with this formulation include training mean-field neural networks, maximum mean discrepancy minimization and kernel Stein discrepancy minimization. Our algorithm is based on a novel spacetime discretization of the mean-field underdamped Langevin dynamics, for which we provide a new, fast mixing guarantee. In addition, we demonstrate that our algorithm converges globally in total variation distance, bridging the theoretical gap between the dynamics and its practical implementation.} }
Endnote
%0 Conference Paper %T Mean-field Underdamped Langevin Dynamics and its Spacetime Discretization %A Qiang Fu %A Ashia Camage Wilson %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-fu24g %I PMLR %P 14175--14206 %U https://proceedings.mlr.press/v235/fu24g.html %V 235 %X We propose a new method called the N-particle underdamped Langevin algorithm for optimizing a special class of non-linear functionals defined over the space of probability measures. Examples of problems with this formulation include training mean-field neural networks, maximum mean discrepancy minimization and kernel Stein discrepancy minimization. Our algorithm is based on a novel spacetime discretization of the mean-field underdamped Langevin dynamics, for which we provide a new, fast mixing guarantee. In addition, we demonstrate that our algorithm converges globally in total variation distance, bridging the theoretical gap between the dynamics and its practical implementation.
APA
Fu, Q. & Wilson, A.C.. (2024). Mean-field Underdamped Langevin Dynamics and its Spacetime Discretization. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:14175-14206 Available from https://proceedings.mlr.press/v235/fu24g.html.

Related Material