Theoretical Analyses on Ensemble and Multiple Kernel Regressors


Akira Tanaka, Ichigaku Takigawa, Hideyuki Imai, Mineichi Kudo ;
Proceedings of the Sixth Asian Conference on Machine Learning, PMLR 39:1-15, 2015.


For the last few decades, a combination of different learning machines so-called ensemble learning, including learning with multiple kernels, has attracted much attention in the field of machine learning. Although its efficacy was revealed numerically in many works, its theoretical grounds are not investigated sufficiently. In this paper, we discuss regression problems with a class of kernels whose corresponding reproducing kernel Hilbert spaces have a common subspace with an invariant metric and prove that the ensemble kernel regressor (the mean of kernel regressors with those kernels) gives a better learning result than the multiple kernel regressor (the kernel regressor with the sum of those kernels) in terms of the generalization ability of a model space.

Related Material