Finite Sample Bounds for Learning with Score Matching

Devin Smedira, Abhijith Jayakumar, Sidhant Misra, Marc Vuffray, Andrey Y. Lokhov
Proceedings of Thirty Ninth Conference on Learning Theory, PMLR 336:5928-5949, 2026.

Abstract

Learning of continuous exponential family distributions with unbounded support remains an important area of research for both theory and applications in high-dimensional statistics. In recent years, score matching has become a widely used method for learning exponential families with continuous variables due to its computational ease when compared against maximum likelihood estimation. However, theoretical understanding of the statistical properties of score matching is still lacking. In this work, we provide a non-asymptotic sample complexity analysis for learning the structure of exponential families of polynomials with score matching. The derived sample bounds show a polynomial dependence on the model dimension. These bounds are the first of its kind, as all prior work has shown only asymptotic bounds on the sample complexity.

Cite this Paper

BibTeX
@InProceedings{pmlr-v336-smedira26a, title = {Finite Sample Bounds for Learning with Score Matching}, author = {Smedira, Devin and Jayakumar, Abhijith and Misra, Sidhant and Vuffray, Marc and Lokhov, Andrey Y.}, booktitle = {Proceedings of Thirty Ninth Conference on Learning Theory}, pages = {5928--5949}, year = {2026}, editor = {Hanneke, Steve and Lattimore, Tor}, volume = {336}, series = {Proceedings of Machine Learning Research}, month = {29 Jun--03 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v336/main/assets/smedira26a/smedira26a.pdf}, url = {https://proceedings.mlr.press/v336/smedira26a.html}, abstract = {Learning of continuous exponential family distributions with unbounded support remains an important area of research for both theory and applications in high-dimensional statistics. In recent years, score matching has become a widely used method for learning exponential families with continuous variables due to its computational ease when compared against maximum likelihood estimation. However, theoretical understanding of the statistical properties of score matching is still lacking. In this work, we provide a non-asymptotic sample complexity analysis for learning the structure of exponential families of polynomials with score matching. The derived sample bounds show a polynomial dependence on the model dimension. These bounds are the first of its kind, as all prior work has shown only asymptotic bounds on the sample complexity. } }
Endnote
%0 Conference Paper %T Finite Sample Bounds for Learning with Score Matching %A Devin Smedira %A Abhijith Jayakumar %A Sidhant Misra %A Marc Vuffray %A Andrey Y. Lokhov %B Proceedings of Thirty Ninth Conference on Learning Theory %C Proceedings of Machine Learning Research %D 2026 %E Steve Hanneke %E Tor Lattimore %F pmlr-v336-smedira26a %I PMLR %P 5928--5949 %U https://proceedings.mlr.press/v336/smedira26a.html %V 336 %X Learning of continuous exponential family distributions with unbounded support remains an important area of research for both theory and applications in high-dimensional statistics. In recent years, score matching has become a widely used method for learning exponential families with continuous variables due to its computational ease when compared against maximum likelihood estimation. However, theoretical understanding of the statistical properties of score matching is still lacking. In this work, we provide a non-asymptotic sample complexity analysis for learning the structure of exponential families of polynomials with score matching. The derived sample bounds show a polynomial dependence on the model dimension. These bounds are the first of its kind, as all prior work has shown only asymptotic bounds on the sample complexity.
APA
Smedira, D., Jayakumar, A., Misra, S., Vuffray, M. & Lokhov, A.Y.. (2026). Finite Sample Bounds for Learning with Score Matching. Proceedings of Thirty Ninth Conference on Learning Theory, in Proceedings of Machine Learning Research 336:5928-5949 Available from https://proceedings.mlr.press/v336/smedira26a.html.

Related Material