Gardner formula for Ising perceptron models at small densities

Erwin Bolthausen, Shuta Nakajima, Nike Sun, Changji Xu
Proceedings of Thirty Fifth Conference on Learning Theory, PMLR 178:1787-1911, 2022.

Abstract

We consider the Ising perceptron model with N spins and M = N*alpha patterns, with a general activation function U that is bounded above. For U bounded away from zero, or U a one-sided threshold function, it was shown by Talagrand (2000, 2011) that for small densities alpha, the free energy of the model converges in the large-N limit to the replica symmetric formula conjectured in the physics literature (Krauth–Mezard 1989, see also Gardner–Derrida 1988). We give a new proof of this result, which covers the more general class of all functions U that are bounded above and satisfy a certain variance bound. The proof uses the (first and second) moment method conditional on the approximate message passing iterates of the model. In order to deduce our main theorem, we also prove a new concentration result for the perceptron model in the case where U is not bounded away from zero.

Cite this Paper

BibTeX
@InProceedings{pmlr-v178-bolthausen22a, title = {Gardner formula for Ising perceptron models at small densities}, author = {Bolthausen, Erwin and Nakajima, Shuta and Sun, Nike and Xu, Changji}, booktitle = {Proceedings of Thirty Fifth Conference on Learning Theory}, pages = {1787--1911}, year = {2022}, editor = {Loh, Po-Ling and Raginsky, Maxim}, volume = {178}, series = {Proceedings of Machine Learning Research}, month = {02--05 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v178/bolthausen22a/bolthausen22a.pdf}, url = {https://proceedings.mlr.press/v178/bolthausen22a.html}, abstract = {We consider the Ising perceptron model with N spins and M = N*alpha patterns, with a general activation function U that is bounded above. For U bounded away from zero, or U a one-sided threshold function, it was shown by Talagrand (2000, 2011) that for small densities alpha, the free energy of the model converges in the large-N limit to the replica symmetric formula conjectured in the physics literature (Krauth–Mezard 1989, see also Gardner–Derrida 1988). We give a new proof of this result, which covers the more general class of all functions U that are bounded above and satisfy a certain variance bound. The proof uses the (first and second) moment method conditional on the approximate message passing iterates of the model. In order to deduce our main theorem, we also prove a new concentration result for the perceptron model in the case where U is not bounded away from zero.} }
Endnote
%0 Conference Paper %T Gardner formula for Ising perceptron models at small densities %A Erwin Bolthausen %A Shuta Nakajima %A Nike Sun %A Changji Xu %B Proceedings of Thirty Fifth Conference on Learning Theory %C Proceedings of Machine Learning Research %D 2022 %E Po-Ling Loh %E Maxim Raginsky %F pmlr-v178-bolthausen22a %I PMLR %P 1787--1911 %U https://proceedings.mlr.press/v178/bolthausen22a.html %V 178 %X We consider the Ising perceptron model with N spins and M = N*alpha patterns, with a general activation function U that is bounded above. For U bounded away from zero, or U a one-sided threshold function, it was shown by Talagrand (2000, 2011) that for small densities alpha, the free energy of the model converges in the large-N limit to the replica symmetric formula conjectured in the physics literature (Krauth–Mezard 1989, see also Gardner–Derrida 1988). We give a new proof of this result, which covers the more general class of all functions U that are bounded above and satisfy a certain variance bound. The proof uses the (first and second) moment method conditional on the approximate message passing iterates of the model. In order to deduce our main theorem, we also prove a new concentration result for the perceptron model in the case where U is not bounded away from zero.
APA
Bolthausen, E., Nakajima, S., Sun, N. & Xu, C.. (2022). Gardner formula for Ising perceptron models at small densities. Proceedings of Thirty Fifth Conference on Learning Theory, in Proceedings of Machine Learning Research 178:1787-1911 Available from https://proceedings.mlr.press/v178/bolthausen22a.html.

Related Material