Rigorous Asymptotics for First-Order Algorithms Through the Dynamical Cavity Method

Yatin Dandi, David Gamarnik, Francisco Pernice, Lenka Zdeborová
Proceedings of Thirty Ninth Conference on Learning Theory, PMLR 336:1611-1646, 2026.

Abstract

Dynamical Mean Field Theory (DMFT) provides an asymptotic description of the dynamics of macroscopic observables in certain disordered systems. Originally pioneered in the context of spin glasses, it has since been used to derive asymptotic dynamical equations for a wide range of models in physics, high-dimensional statistics and machine learning. One of the main tools used by physicists to obtain these equations is the dynamical cavity method, which has remained largely non-rigorous. In contrast, existing mathematical formalizations have relied on alternative approaches, including Gaussian conditioning, large deviations over paths, or Fourier analysis. In this work, we formalize the dynamical cavity method and use it to give a new proof of the DMFT equations for General First Order Methods, a broad class of dynamics encompassing algorithms such as Gradient Descent and Approximate Message Passing.

Cite this Paper

BibTeX
@InProceedings{pmlr-v336-dandi26a, title = {Rigorous Asymptotics for First-Order Algorithms Through the Dynamical Cavity Method}, author = {Dandi, Yatin and Gamarnik, David and Pernice, Francisco and Zdeborov{\'a}, Lenka}, booktitle = {Proceedings of Thirty Ninth Conference on Learning Theory}, pages = {1611--1646}, year = {2026}, editor = {Hanneke, Steve and Lattimore, Tor}, volume = {336}, series = {Proceedings of Machine Learning Research}, month = {29 Jun--03 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v336/main/assets/dandi26a/dandi26a.pdf}, url = {https://proceedings.mlr.press/v336/dandi26a.html}, abstract = {Dynamical Mean Field Theory (DMFT) provides an asymptotic description of the dynamics of macroscopic observables in certain disordered systems. Originally pioneered in the context of spin glasses, it has since been used to derive asymptotic dynamical equations for a wide range of models in physics, high-dimensional statistics and machine learning. One of the main tools used by physicists to obtain these equations is the dynamical cavity method, which has remained largely non-rigorous. In contrast, existing mathematical formalizations have relied on alternative approaches, including Gaussian conditioning, large deviations over paths, or Fourier analysis. In this work, we formalize the dynamical cavity method and use it to give a new proof of the DMFT equations for General First Order Methods, a broad class of dynamics encompassing algorithms such as Gradient Descent and Approximate Message Passing.} }
Endnote
%0 Conference Paper %T Rigorous Asymptotics for First-Order Algorithms Through the Dynamical Cavity Method %A Yatin Dandi %A David Gamarnik %A Francisco Pernice %A Lenka Zdeborová %B Proceedings of Thirty Ninth Conference on Learning Theory %C Proceedings of Machine Learning Research %D 2026 %E Steve Hanneke %E Tor Lattimore %F pmlr-v336-dandi26a %I PMLR %P 1611--1646 %U https://proceedings.mlr.press/v336/dandi26a.html %V 336 %X Dynamical Mean Field Theory (DMFT) provides an asymptotic description of the dynamics of macroscopic observables in certain disordered systems. Originally pioneered in the context of spin glasses, it has since been used to derive asymptotic dynamical equations for a wide range of models in physics, high-dimensional statistics and machine learning. One of the main tools used by physicists to obtain these equations is the dynamical cavity method, which has remained largely non-rigorous. In contrast, existing mathematical formalizations have relied on alternative approaches, including Gaussian conditioning, large deviations over paths, or Fourier analysis. In this work, we formalize the dynamical cavity method and use it to give a new proof of the DMFT equations for General First Order Methods, a broad class of dynamics encompassing algorithms such as Gradient Descent and Approximate Message Passing.
APA
Dandi, Y., Gamarnik, D., Pernice, F. & Zdeborová, L.. (2026). Rigorous Asymptotics for First-Order Algorithms Through the Dynamical Cavity Method. Proceedings of Thirty Ninth Conference on Learning Theory, in Proceedings of Machine Learning Research 336:1611-1646 Available from https://proceedings.mlr.press/v336/dandi26a.html.

Related Material