On the Spectrum of Random Features Maps of High Dimensional Data

Zhenyu Liao, Romain Couillet
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:3063-3071, 2018.

Abstract

Random feature maps are ubiquitous in modern statistical machine learning, where they generalize random projections by means of powerful, yet often difficult to analyze nonlinear operators. In this paper we leverage the "concentration" phenomenon induced by random matrix theory to perform a spectral analysis on the Gram matrix of these random feature maps, here for Gaussian mixture models of simultaneously large dimension and size. Our results are instrumental to a deeper understanding on the interplay of the nonlinearity and the statistics of the data, thereby allowing for a better tuning of random feature-based techniques.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-liao18a, title = {On the Spectrum of Random Features Maps of High Dimensional Data}, author = {Liao, Zhenyu and Couillet, Romain}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {3063--3071}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/liao18a/liao18a.pdf}, url = {https://proceedings.mlr.press/v80/liao18a.html}, abstract = {Random feature maps are ubiquitous in modern statistical machine learning, where they generalize random projections by means of powerful, yet often difficult to analyze nonlinear operators. In this paper we leverage the "concentration" phenomenon induced by random matrix theory to perform a spectral analysis on the Gram matrix of these random feature maps, here for Gaussian mixture models of simultaneously large dimension and size. Our results are instrumental to a deeper understanding on the interplay of the nonlinearity and the statistics of the data, thereby allowing for a better tuning of random feature-based techniques.} }
Endnote
%0 Conference Paper %T On the Spectrum of Random Features Maps of High Dimensional Data %A Zhenyu Liao %A Romain Couillet %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-liao18a %I PMLR %P 3063--3071 %U https://proceedings.mlr.press/v80/liao18a.html %V 80 %X Random feature maps are ubiquitous in modern statistical machine learning, where they generalize random projections by means of powerful, yet often difficult to analyze nonlinear operators. In this paper we leverage the "concentration" phenomenon induced by random matrix theory to perform a spectral analysis on the Gram matrix of these random feature maps, here for Gaussian mixture models of simultaneously large dimension and size. Our results are instrumental to a deeper understanding on the interplay of the nonlinearity and the statistics of the data, thereby allowing for a better tuning of random feature-based techniques.
APA
Liao, Z. & Couillet, R.. (2018). On the Spectrum of Random Features Maps of High Dimensional Data. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:3063-3071 Available from https://proceedings.mlr.press/v80/liao18a.html.

Related Material