Large-Scale Sparse Kernel Canonical Correlation Analysis

Viivi Uurtio, Sahely Bhadra, Juho Rousu
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:6383-6391, 2019.

Abstract

This paper presents gradKCCA, a large-scale sparse non-linear canonical correlation method. Like Kernel Canonical Correlation Analysis (KCCA), our method finds non-linear relations through kernel functions, but it does not rely on a kernel matrix, a known bottleneck for scaling up kernel methods. gradKCCA corresponds to solving KCCA with the additional constraint that the canonical projection directions in the kernel-induced feature space have preimages in the original data space. Firstly, this modification allows us to very efficiently maximize kernel canonical correlation through an alternating projected gradient algorithm working in the original data space. Secondly, we can control the sparsity of the projection directions by constraining the $\ell_1$ norm of the preimages of the projection directions, facilitating the interpretation of the discovered patterns, which is not available through KCCA. Our empirical experiments demonstrate that gradKCCA outperforms state-of-the-art CCA methods in terms of speed and robustness to noise both in simulated and real-world datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-uurtio19a, title = {Large-Scale Sparse Kernel Canonical Correlation Analysis}, author = {Uurtio, Viivi and Bhadra, Sahely and Rousu, Juho}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {6383--6391}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/uurtio19a/uurtio19a.pdf}, url = {https://proceedings.mlr.press/v97/uurtio19a.html}, abstract = {This paper presents gradKCCA, a large-scale sparse non-linear canonical correlation method. Like Kernel Canonical Correlation Analysis (KCCA), our method finds non-linear relations through kernel functions, but it does not rely on a kernel matrix, a known bottleneck for scaling up kernel methods. gradKCCA corresponds to solving KCCA with the additional constraint that the canonical projection directions in the kernel-induced feature space have preimages in the original data space. Firstly, this modification allows us to very efficiently maximize kernel canonical correlation through an alternating projected gradient algorithm working in the original data space. Secondly, we can control the sparsity of the projection directions by constraining the $\ell_1$ norm of the preimages of the projection directions, facilitating the interpretation of the discovered patterns, which is not available through KCCA. Our empirical experiments demonstrate that gradKCCA outperforms state-of-the-art CCA methods in terms of speed and robustness to noise both in simulated and real-world datasets.} }
Endnote
%0 Conference Paper %T Large-Scale Sparse Kernel Canonical Correlation Analysis %A Viivi Uurtio %A Sahely Bhadra %A Juho Rousu %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-uurtio19a %I PMLR %P 6383--6391 %U https://proceedings.mlr.press/v97/uurtio19a.html %V 97 %X This paper presents gradKCCA, a large-scale sparse non-linear canonical correlation method. Like Kernel Canonical Correlation Analysis (KCCA), our method finds non-linear relations through kernel functions, but it does not rely on a kernel matrix, a known bottleneck for scaling up kernel methods. gradKCCA corresponds to solving KCCA with the additional constraint that the canonical projection directions in the kernel-induced feature space have preimages in the original data space. Firstly, this modification allows us to very efficiently maximize kernel canonical correlation through an alternating projected gradient algorithm working in the original data space. Secondly, we can control the sparsity of the projection directions by constraining the $\ell_1$ norm of the preimages of the projection directions, facilitating the interpretation of the discovered patterns, which is not available through KCCA. Our empirical experiments demonstrate that gradKCCA outperforms state-of-the-art CCA methods in terms of speed and robustness to noise both in simulated and real-world datasets.
APA
Uurtio, V., Bhadra, S. & Rousu, J.. (2019). Large-Scale Sparse Kernel Canonical Correlation Analysis. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:6383-6391 Available from https://proceedings.mlr.press/v97/uurtio19a.html.

Related Material