Theoretical Analyses on Ensemble and Multiple Kernel Regressors

Akira Tanaka, Ichigaku Takigawa, Hideyuki Imai, Mineichi Kudo
Proceedings of the Sixth Asian Conference on Machine Learning, PMLR 39:1-15, 2015.

Abstract

For the last few decades, a combination of different learning machines so-called ensemble learning, including learning with multiple kernels, has attracted much attention in the field of machine learning. Although its efficacy was revealed numerically in many works, its theoretical grounds are not investigated sufficiently. In this paper, we discuss regression problems with a class of kernels whose corresponding reproducing kernel Hilbert spaces have a common subspace with an invariant metric and prove that the ensemble kernel regressor (the mean of kernel regressors with those kernels) gives a better learning result than the multiple kernel regressor (the kernel regressor with the sum of those kernels) in terms of the generalization ability of a model space.

Cite this Paper

BibTeX
@InProceedings{pmlr-v39-tanaka14, title = {Theoretical Analyses on Ensemble and Multiple Kernel Regressors}, author = {Tanaka, Akira and Takigawa, Ichigaku and Imai, Hideyuki and Kudo, Mineichi}, booktitle = {Proceedings of the Sixth Asian Conference on Machine Learning}, pages = {1--15}, year = {2015}, editor = {Phung, Dinh and Li, Hang}, volume = {39}, series = {Proceedings of Machine Learning Research}, address = {Nha Trang City, Vietnam}, month = {26--28 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v39/tanaka14.pdf}, url = {https://proceedings.mlr.press/v39/tanaka14.html}, abstract = {For the last few decades, a combination of different learning machines so-called ensemble learning, including learning with multiple kernels, has attracted much attention in the field of machine learning. Although its efficacy was revealed numerically in many works, its theoretical grounds are not investigated sufficiently. In this paper, we discuss regression problems with a class of kernels whose corresponding reproducing kernel Hilbert spaces have a common subspace with an invariant metric and prove that the ensemble kernel regressor (the mean of kernel regressors with those kernels) gives a better learning result than the multiple kernel regressor (the kernel regressor with the sum of those kernels) in terms of the generalization ability of a model space.} }
Endnote
%0 Conference Paper %T Theoretical Analyses on Ensemble and Multiple Kernel Regressors %A Akira Tanaka %A Ichigaku Takigawa %A Hideyuki Imai %A Mineichi Kudo %B Proceedings of the Sixth Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2015 %E Dinh Phung %E Hang Li %F pmlr-v39-tanaka14 %I PMLR %P 1--15 %U https://proceedings.mlr.press/v39/tanaka14.html %V 39 %X For the last few decades, a combination of different learning machines so-called ensemble learning, including learning with multiple kernels, has attracted much attention in the field of machine learning. Although its efficacy was revealed numerically in many works, its theoretical grounds are not investigated sufficiently. In this paper, we discuss regression problems with a class of kernels whose corresponding reproducing kernel Hilbert spaces have a common subspace with an invariant metric and prove that the ensemble kernel regressor (the mean of kernel regressors with those kernels) gives a better learning result than the multiple kernel regressor (the kernel regressor with the sum of those kernels) in terms of the generalization ability of a model space.
RIS
TY - CPAPER TI - Theoretical Analyses on Ensemble and Multiple Kernel Regressors AU - Akira Tanaka AU - Ichigaku Takigawa AU - Hideyuki Imai AU - Mineichi Kudo BT - Proceedings of the Sixth Asian Conference on Machine Learning DA - 2015/02/16 ED - Dinh Phung ED - Hang Li ID - pmlr-v39-tanaka14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 39 SP - 1 EP - 15 L1 - http://proceedings.mlr.press/v39/tanaka14.pdf UR - https://proceedings.mlr.press/v39/tanaka14.html AB - For the last few decades, a combination of different learning machines so-called ensemble learning, including learning with multiple kernels, has attracted much attention in the field of machine learning. Although its efficacy was revealed numerically in many works, its theoretical grounds are not investigated sufficiently. In this paper, we discuss regression problems with a class of kernels whose corresponding reproducing kernel Hilbert spaces have a common subspace with an invariant metric and prove that the ensemble kernel regressor (the mean of kernel regressors with those kernels) gives a better learning result than the multiple kernel regressor (the kernel regressor with the sum of those kernels) in terms of the generalization ability of a model space. ER -
APA
Tanaka, A., Takigawa, I., Imai, H. & Kudo, M.. (2015). Theoretical Analyses on Ensemble and Multiple Kernel Regressors. Proceedings of the Sixth Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 39:1-15 Available from https://proceedings.mlr.press/v39/tanaka14.html.

Related Material