Fast Learning Rate of Multiple Kernel Learning: Trade-Off between Sparsity and Smoothness

Taiji Suzuki, Masashi Sugiyama
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:1152-1183, 2012.

Abstract

We investigate the learning rate of multiple kernel leaning (MKL) with L1 and elastic-net regularizations. The elastic-net regularization is a composition of an L1-regularizer for inducing the sparsity and an L2-regularizer for controlling the smoothness. We focus on a sparse setting where the total number of kernels is large but the number of non-zero components of the ground truth is relatively small, and show sharper convergence rates than the learning rates ever shown for both L1 and elastic-net regularizations. Our analysis shows there appears a trade-off between the sparsity and the smoothness when it comes to selecting which of L1 and elastic-net regularizations to use; if the ground truth is smooth, the elastic-net regularization is preferred, otherwise the L1 regularization is preferred.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-suzuki12, title = {Fast Learning Rate of Multiple Kernel Learning: Trade-Off between Sparsity and Smoothness}, author = {Suzuki, Taiji and Sugiyama, Masashi}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {1152--1183}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/suzuki12/suzuki12.pdf}, url = {https://proceedings.mlr.press/v22/suzuki12.html}, abstract = {We investigate the learning rate of multiple kernel leaning (MKL) with L1 and elastic-net regularizations. The elastic-net regularization is a composition of an L1-regularizer for inducing the sparsity and an L2-regularizer for controlling the smoothness. We focus on a sparse setting where the total number of kernels is large but the number of non-zero components of the ground truth is relatively small, and show sharper convergence rates than the learning rates ever shown for both L1 and elastic-net regularizations. Our analysis shows there appears a trade-off between the sparsity and the smoothness when it comes to selecting which of L1 and elastic-net regularizations to use; if the ground truth is smooth, the elastic-net regularization is preferred, otherwise the L1 regularization is preferred.} }
Endnote
%0 Conference Paper %T Fast Learning Rate of Multiple Kernel Learning: Trade-Off between Sparsity and Smoothness %A Taiji Suzuki %A Masashi Sugiyama %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-suzuki12 %I PMLR %P 1152--1183 %U https://proceedings.mlr.press/v22/suzuki12.html %V 22 %X We investigate the learning rate of multiple kernel leaning (MKL) with L1 and elastic-net regularizations. The elastic-net regularization is a composition of an L1-regularizer for inducing the sparsity and an L2-regularizer for controlling the smoothness. We focus on a sparse setting where the total number of kernels is large but the number of non-zero components of the ground truth is relatively small, and show sharper convergence rates than the learning rates ever shown for both L1 and elastic-net regularizations. Our analysis shows there appears a trade-off between the sparsity and the smoothness when it comes to selecting which of L1 and elastic-net regularizations to use; if the ground truth is smooth, the elastic-net regularization is preferred, otherwise the L1 regularization is preferred.
RIS
TY - CPAPER TI - Fast Learning Rate of Multiple Kernel Learning: Trade-Off between Sparsity and Smoothness AU - Taiji Suzuki AU - Masashi Sugiyama BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-suzuki12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 1152 EP - 1183 L1 - http://proceedings.mlr.press/v22/suzuki12/suzuki12.pdf UR - https://proceedings.mlr.press/v22/suzuki12.html AB - We investigate the learning rate of multiple kernel leaning (MKL) with L1 and elastic-net regularizations. The elastic-net regularization is a composition of an L1-regularizer for inducing the sparsity and an L2-regularizer for controlling the smoothness. We focus on a sparse setting where the total number of kernels is large but the number of non-zero components of the ground truth is relatively small, and show sharper convergence rates than the learning rates ever shown for both L1 and elastic-net regularizations. Our analysis shows there appears a trade-off between the sparsity and the smoothness when it comes to selecting which of L1 and elastic-net regularizations to use; if the ground truth is smooth, the elastic-net regularization is preferred, otherwise the L1 regularization is preferred. ER -
APA
Suzuki, T. & Sugiyama, M.. (2012). Fast Learning Rate of Multiple Kernel Learning: Trade-Off between Sparsity and Smoothness. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:1152-1183 Available from https://proceedings.mlr.press/v22/suzuki12.html.

Related Material