Parameter-Free Spectral Kernel Learning

Qi Mao, Ivor W. Tsang
Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, PMLR R8:357-364, 2010.

Abstract

Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increas- ing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learn- ing method which can seamlessly combine man- ifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a new kernel. Inter- estingly, the new kernel matrix can be obtained analytically with the use of spectral decomposi- tion of graph Laplacian matrix. Hence, the pro- posed algorithm does not require any numerical optimization solvers. Moreover, by maximizing kernel target alignment on labeled data, we can also learn model parameters automatically with a closed-form solution. For a given graph Lapla- cian matrix, our proposed method does not need to tune any model parameter including the trade- off parameter in RLS and the balance parame- ter for unlabeled data. Extensive experiments on ten benchmark datasets show that our proposed two-stage parameter-free spectral kernel learning algorithm can obtain comparable performance with fine-tuned manifold regularization methods in transductive setting, and outperform multiple kernel learning in supervised setting.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR8-mao10a, title = {Parameter-Free Spectral Kernel Learning}, author = {Mao, Qi and Tsang, Ivor W.}, booktitle = {Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence}, pages = {357--364}, year = {2010}, editor = {Grünwald, Peter and Spirtes, Peter}, volume = {R8}, series = {Proceedings of Machine Learning Research}, month = {08--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r8/main/assets/mao10a/mao10a.pdf}, url = {https://proceedings.mlr.press/r8/mao10a.html}, abstract = {Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increas- ing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learn- ing method which can seamlessly combine man- ifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a new kernel. Inter- estingly, the new kernel matrix can be obtained analytically with the use of spectral decomposi- tion of graph Laplacian matrix. Hence, the pro- posed algorithm does not require any numerical optimization solvers. Moreover, by maximizing kernel target alignment on labeled data, we can also learn model parameters automatically with a closed-form solution. For a given graph Lapla- cian matrix, our proposed method does not need to tune any model parameter including the trade- off parameter in RLS and the balance parame- ter for unlabeled data. Extensive experiments on ten benchmark datasets show that our proposed two-stage parameter-free spectral kernel learning algorithm can obtain comparable performance with fine-tuned manifold regularization methods in transductive setting, and outperform multiple kernel learning in supervised setting.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Parameter-Free Spectral Kernel Learning %A Qi Mao %A Ivor W. Tsang %B Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2010 %E Peter Grünwald %E Peter Spirtes %F pmlr-vR8-mao10a %I PMLR %P 357--364 %U https://proceedings.mlr.press/r8/mao10a.html %V R8 %X Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increas- ing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learn- ing method which can seamlessly combine man- ifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a new kernel. Inter- estingly, the new kernel matrix can be obtained analytically with the use of spectral decomposi- tion of graph Laplacian matrix. Hence, the pro- posed algorithm does not require any numerical optimization solvers. Moreover, by maximizing kernel target alignment on labeled data, we can also learn model parameters automatically with a closed-form solution. For a given graph Lapla- cian matrix, our proposed method does not need to tune any model parameter including the trade- off parameter in RLS and the balance parame- ter for unlabeled data. Extensive experiments on ten benchmark datasets show that our proposed two-stage parameter-free spectral kernel learning algorithm can obtain comparable performance with fine-tuned manifold regularization methods in transductive setting, and outperform multiple kernel learning in supervised setting. %Z Reissued by PMLR on 04 October 2026.
APA
Mao, Q. & Tsang, I.W.. (2010). Parameter-Free Spectral Kernel Learning. Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R8:357-364 Available from https://proceedings.mlr.press/r8/mao10a.html. Reissued by PMLR on 04 October 2026.

Related Material