Randomized Nonlinear Component Analysis

David Lopez-Paz, Suvrit Sra, Alex Smola, Zoubin Ghahramani, Bernhard Schoelkopf
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):1359-1367, 2014.

Abstract

Classical methods such as Principal Component Analysis (PCA) and Canonical Correlation Analysis (CCA) are ubiquitous in statistics. However, these techniques are only able to reveal linear relationships in data. Although nonlinear variants of PCA and CCA have been proposed, these are computationally prohibitive in the large scale. In a separate strand of recent research, randomized methods have been proposed to construct features that help reveal nonlinear patterns in data. For basic tasks such as regression or classification, random features exhibit little or no loss in performance, while achieving drastic savings in computational requirements. In this paper we leverage randomness to design scalable new variants of nonlinear PCA and CCA; our ideas extend to key multivariate analysis tools such as spectral clustering or LDA. We demonstrate our algorithms through experiments on real-world data, on which we compare against the state-of-the-art. A simple R implementation of the presented algorithms is provided.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-lopez-paz14, title = {Randomized Nonlinear Component Analysis}, author = {Lopez-Paz, David and Sra, Suvrit and Smola, Alex and Ghahramani, Zoubin and Schoelkopf, Bernhard}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {1359--1367}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/lopez-paz14.pdf}, url = {https://proceedings.mlr.press/v32/lopez-paz14.html}, abstract = {Classical methods such as Principal Component Analysis (PCA) and Canonical Correlation Analysis (CCA) are ubiquitous in statistics. However, these techniques are only able to reveal linear relationships in data. Although nonlinear variants of PCA and CCA have been proposed, these are computationally prohibitive in the large scale. In a separate strand of recent research, randomized methods have been proposed to construct features that help reveal nonlinear patterns in data. For basic tasks such as regression or classification, random features exhibit little or no loss in performance, while achieving drastic savings in computational requirements. In this paper we leverage randomness to design scalable new variants of nonlinear PCA and CCA; our ideas extend to key multivariate analysis tools such as spectral clustering or LDA. We demonstrate our algorithms through experiments on real-world data, on which we compare against the state-of-the-art. A simple R implementation of the presented algorithms is provided.} }
Endnote
%0 Conference Paper %T Randomized Nonlinear Component Analysis %A David Lopez-Paz %A Suvrit Sra %A Alex Smola %A Zoubin Ghahramani %A Bernhard Schoelkopf %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-lopez-paz14 %I PMLR %P 1359--1367 %U https://proceedings.mlr.press/v32/lopez-paz14.html %V 32 %N 2 %X Classical methods such as Principal Component Analysis (PCA) and Canonical Correlation Analysis (CCA) are ubiquitous in statistics. However, these techniques are only able to reveal linear relationships in data. Although nonlinear variants of PCA and CCA have been proposed, these are computationally prohibitive in the large scale. In a separate strand of recent research, randomized methods have been proposed to construct features that help reveal nonlinear patterns in data. For basic tasks such as regression or classification, random features exhibit little or no loss in performance, while achieving drastic savings in computational requirements. In this paper we leverage randomness to design scalable new variants of nonlinear PCA and CCA; our ideas extend to key multivariate analysis tools such as spectral clustering or LDA. We demonstrate our algorithms through experiments on real-world data, on which we compare against the state-of-the-art. A simple R implementation of the presented algorithms is provided.
RIS
TY - CPAPER TI - Randomized Nonlinear Component Analysis AU - David Lopez-Paz AU - Suvrit Sra AU - Alex Smola AU - Zoubin Ghahramani AU - Bernhard Schoelkopf BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-lopez-paz14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 1359 EP - 1367 L1 - http://proceedings.mlr.press/v32/lopez-paz14.pdf UR - https://proceedings.mlr.press/v32/lopez-paz14.html AB - Classical methods such as Principal Component Analysis (PCA) and Canonical Correlation Analysis (CCA) are ubiquitous in statistics. However, these techniques are only able to reveal linear relationships in data. Although nonlinear variants of PCA and CCA have been proposed, these are computationally prohibitive in the large scale. In a separate strand of recent research, randomized methods have been proposed to construct features that help reveal nonlinear patterns in data. For basic tasks such as regression or classification, random features exhibit little or no loss in performance, while achieving drastic savings in computational requirements. In this paper we leverage randomness to design scalable new variants of nonlinear PCA and CCA; our ideas extend to key multivariate analysis tools such as spectral clustering or LDA. We demonstrate our algorithms through experiments on real-world data, on which we compare against the state-of-the-art. A simple R implementation of the presented algorithms is provided. ER -
APA
Lopez-Paz, D., Sra, S., Smola, A., Ghahramani, Z. & Schoelkopf, B.. (2014). Randomized Nonlinear Component Analysis. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):1359-1367 Available from https://proceedings.mlr.press/v32/lopez-paz14.html.

Related Material