Spectral Phase Transition and Optimal PCA in Block-Structured Spiked Models

Pierre Mergny, Justin Ko, Florent Krzakala
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:35470-35491, 2024.

Abstract

We discuss the inhomogeneous Wigner spike model, a theoretical framework recently introduced to study structured noise in various learning scenarios, through the prism of random matrix theory, with a specific focus on its spectral properties. Our primary objective is to find an optimal spectral method, and to extend the celebrated (BBP) phase transition criterion —well-known in the homogeneous case— to our inhomogeneous, block-structured, Wigner model. We provide a thorough rigorous analysis of a transformed matrix and show that the transition for the appearance of 1) an outlier outside the bulk of the limiting spectral distribution and 2) a positive overlap between the associated eigenvector and the signal, occurs precisely at the optimal threshold, making the proposed spectral method optimal within the class of iterative methods for the inhomogeneous Wigner problem.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-mergny24a, title = {Spectral Phase Transition and Optimal {PCA} in Block-Structured Spiked Models}, author = {Mergny, Pierre and Ko, Justin and Krzakala, Florent}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {35470--35491}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/mergny24a/mergny24a.pdf}, url = {https://proceedings.mlr.press/v235/mergny24a.html}, abstract = {We discuss the inhomogeneous Wigner spike model, a theoretical framework recently introduced to study structured noise in various learning scenarios, through the prism of random matrix theory, with a specific focus on its spectral properties. Our primary objective is to find an optimal spectral method, and to extend the celebrated (BBP) phase transition criterion —well-known in the homogeneous case— to our inhomogeneous, block-structured, Wigner model. We provide a thorough rigorous analysis of a transformed matrix and show that the transition for the appearance of 1) an outlier outside the bulk of the limiting spectral distribution and 2) a positive overlap between the associated eigenvector and the signal, occurs precisely at the optimal threshold, making the proposed spectral method optimal within the class of iterative methods for the inhomogeneous Wigner problem.} }
Endnote
%0 Conference Paper %T Spectral Phase Transition and Optimal PCA in Block-Structured Spiked Models %A Pierre Mergny %A Justin Ko %A Florent Krzakala %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-mergny24a %I PMLR %P 35470--35491 %U https://proceedings.mlr.press/v235/mergny24a.html %V 235 %X We discuss the inhomogeneous Wigner spike model, a theoretical framework recently introduced to study structured noise in various learning scenarios, through the prism of random matrix theory, with a specific focus on its spectral properties. Our primary objective is to find an optimal spectral method, and to extend the celebrated (BBP) phase transition criterion —well-known in the homogeneous case— to our inhomogeneous, block-structured, Wigner model. We provide a thorough rigorous analysis of a transformed matrix and show that the transition for the appearance of 1) an outlier outside the bulk of the limiting spectral distribution and 2) a positive overlap between the associated eigenvector and the signal, occurs precisely at the optimal threshold, making the proposed spectral method optimal within the class of iterative methods for the inhomogeneous Wigner problem.
APA
Mergny, P., Ko, J. & Krzakala, F.. (2024). Spectral Phase Transition and Optimal PCA in Block-Structured Spiked Models. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:35470-35491 Available from https://proceedings.mlr.press/v235/mergny24a.html.

Related Material