A Unifying View of Representer Theorems

Andreas Argyriou, Francesco Dinuzzo
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):748-756, 2014.

Abstract

It is known that the solution of regularization and interpolation problems with Hilbertian penalties can be expressed as a linear combination of the data. This very useful property, called the representer theorem, has been widely studied and applied to machine learning problems. Analogous optimality conditions have appeared in other contexts, notably in matrix regularization. In this paper we propose a unified view, which generalizes the concept of representer theorems and extends necessary and sufficient conditions for such theorems to hold. Our main result shows a close connection between representer theorems and certain classes of regularization penalties, which we call orthomonotone functions. This result not only subsumes previous representer theorems as special cases but also yields a new class of optimality conditions, which goes beyond the classical linear combination of the data. Moreover, orthomonotonicity provides a useful criterion for testing whether a representer theorem holds for a specific regularization problem.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-argyriou14, title = {A Unifying View of Representer Theorems}, author = {Argyriou, Andreas and Dinuzzo, Francesco}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {748--756}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/argyriou14.pdf}, url = {https://proceedings.mlr.press/v32/argyriou14.html}, abstract = {It is known that the solution of regularization and interpolation problems with Hilbertian penalties can be expressed as a linear combination of the data. This very useful property, called the representer theorem, has been widely studied and applied to machine learning problems. Analogous optimality conditions have appeared in other contexts, notably in matrix regularization. In this paper we propose a unified view, which generalizes the concept of representer theorems and extends necessary and sufficient conditions for such theorems to hold. Our main result shows a close connection between representer theorems and certain classes of regularization penalties, which we call orthomonotone functions. This result not only subsumes previous representer theorems as special cases but also yields a new class of optimality conditions, which goes beyond the classical linear combination of the data. Moreover, orthomonotonicity provides a useful criterion for testing whether a representer theorem holds for a specific regularization problem.} }
Endnote
%0 Conference Paper %T A Unifying View of Representer Theorems %A Andreas Argyriou %A Francesco Dinuzzo %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-argyriou14 %I PMLR %P 748--756 %U https://proceedings.mlr.press/v32/argyriou14.html %V 32 %N 2 %X It is known that the solution of regularization and interpolation problems with Hilbertian penalties can be expressed as a linear combination of the data. This very useful property, called the representer theorem, has been widely studied and applied to machine learning problems. Analogous optimality conditions have appeared in other contexts, notably in matrix regularization. In this paper we propose a unified view, which generalizes the concept of representer theorems and extends necessary and sufficient conditions for such theorems to hold. Our main result shows a close connection between representer theorems and certain classes of regularization penalties, which we call orthomonotone functions. This result not only subsumes previous representer theorems as special cases but also yields a new class of optimality conditions, which goes beyond the classical linear combination of the data. Moreover, orthomonotonicity provides a useful criterion for testing whether a representer theorem holds for a specific regularization problem.
RIS
TY - CPAPER TI - A Unifying View of Representer Theorems AU - Andreas Argyriou AU - Francesco Dinuzzo BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-argyriou14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 748 EP - 756 L1 - http://proceedings.mlr.press/v32/argyriou14.pdf UR - https://proceedings.mlr.press/v32/argyriou14.html AB - It is known that the solution of regularization and interpolation problems with Hilbertian penalties can be expressed as a linear combination of the data. This very useful property, called the representer theorem, has been widely studied and applied to machine learning problems. Analogous optimality conditions have appeared in other contexts, notably in matrix regularization. In this paper we propose a unified view, which generalizes the concept of representer theorems and extends necessary and sufficient conditions for such theorems to hold. Our main result shows a close connection between representer theorems and certain classes of regularization penalties, which we call orthomonotone functions. This result not only subsumes previous representer theorems as special cases but also yields a new class of optimality conditions, which goes beyond the classical linear combination of the data. Moreover, orthomonotonicity provides a useful criterion for testing whether a representer theorem holds for a specific regularization problem. ER -
APA
Argyriou, A. & Dinuzzo, F.. (2014). A Unifying View of Representer Theorems. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):748-756 Available from https://proceedings.mlr.press/v32/argyriou14.html.

Related Material