Learning with Square Loss: Localization through Offset Rademacher Complexity

Tengyuan Liang, Alexander Rakhlin, Karthik Sridharan
Proceedings of The 28th Conference on Learning Theory, PMLR 40:1260-1285, 2015.

Abstract

We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a two-step estimator is shown to be upper bounded by this offset complexity through a novel geometric inequality. In the convex case, the estimator reduces to an empirical risk minimizer. The method recovers the results of \citepRakSriTsy15 for the bounded case while also providing guarantees without the boundedness assumption.

Cite this Paper

BibTeX
@InProceedings{pmlr-v40-Liang15, title = {Learning with Square Loss: Localization through Offset Rademacher Complexity}, author = {Liang, Tengyuan and Rakhlin, Alexander and Sridharan, Karthik}, booktitle = {Proceedings of The 28th Conference on Learning Theory}, pages = {1260--1285}, year = {2015}, editor = {Grünwald, Peter and Hazan, Elad and Kale, Satyen}, volume = {40}, series = {Proceedings of Machine Learning Research}, address = {Paris, France}, month = {03--06 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v40/Liang15.pdf}, url = {https://proceedings.mlr.press/v40/Liang15.html}, abstract = {We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a two-step estimator is shown to be upper bounded by this offset complexity through a novel geometric inequality. In the convex case, the estimator reduces to an empirical risk minimizer. The method recovers the results of \citepRakSriTsy15 for the bounded case while also providing guarantees without the boundedness assumption.} }
Endnote
%0 Conference Paper %T Learning with Square Loss: Localization through Offset Rademacher Complexity %A Tengyuan Liang %A Alexander Rakhlin %A Karthik Sridharan %B Proceedings of The 28th Conference on Learning Theory %C Proceedings of Machine Learning Research %D 2015 %E Peter Grünwald %E Elad Hazan %E Satyen Kale %F pmlr-v40-Liang15 %I PMLR %P 1260--1285 %U https://proceedings.mlr.press/v40/Liang15.html %V 40 %X We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a two-step estimator is shown to be upper bounded by this offset complexity through a novel geometric inequality. In the convex case, the estimator reduces to an empirical risk minimizer. The method recovers the results of \citepRakSriTsy15 for the bounded case while also providing guarantees without the boundedness assumption.
RIS
TY - CPAPER TI - Learning with Square Loss: Localization through Offset Rademacher Complexity AU - Tengyuan Liang AU - Alexander Rakhlin AU - Karthik Sridharan BT - Proceedings of The 28th Conference on Learning Theory DA - 2015/06/26 ED - Peter Grünwald ED - Elad Hazan ED - Satyen Kale ID - pmlr-v40-Liang15 PB - PMLR DP - Proceedings of Machine Learning Research VL - 40 SP - 1260 EP - 1285 L1 - http://proceedings.mlr.press/v40/Liang15.pdf UR - https://proceedings.mlr.press/v40/Liang15.html AB - We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a two-step estimator is shown to be upper bounded by this offset complexity through a novel geometric inequality. In the convex case, the estimator reduces to an empirical risk minimizer. The method recovers the results of \citepRakSriTsy15 for the bounded case while also providing guarantees without the boundedness assumption. ER -
APA
Liang, T., Rakhlin, A. & Sridharan, K.. (2015). Learning with Square Loss: Localization through Offset Rademacher Complexity. Proceedings of The 28th Conference on Learning Theory, in Proceedings of Machine Learning Research 40:1260-1285 Available from https://proceedings.mlr.press/v40/Liang15.html.

Related Material