Learning with Square Loss: Localization through Offset Rademacher Complexity

Tengyuan Liang, Alexander Rakhlin, Karthik Sridharan
; Proceedings of The 28th Conference on Learning Theory, PMLR 40:1260-1285, 2015.

Abstract

We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a two-step estimator is shown to be upper bounded by this offset complexity through a novel geometric inequality. In the convex case, the estimator reduces to an empirical risk minimizer. The method recovers the results of \citepRakSriTsy15 for the bounded case while also providing guarantees without the boundedness assumption.

Cite this Paper


BibTeX
@InProceedings{pmlr-v40-Liang15, title = {Learning with Square Loss: Localization through Offset Rademacher Complexity}, author = {Tengyuan Liang and Alexander Rakhlin and Karthik Sridharan}, booktitle = {Proceedings of The 28th Conference on Learning Theory}, pages = {1260--1285}, year = {2015}, editor = {Peter Grünwald and Elad Hazan and Satyen Kale}, volume = {40}, series = {Proceedings of Machine Learning Research}, address = {Paris, France}, month = {03--06 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v40/Liang15.pdf}, url = {http://proceedings.mlr.press/v40/Liang15.html}, abstract = {We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a two-step estimator is shown to be upper bounded by this offset complexity through a novel geometric inequality. In the convex case, the estimator reduces to an empirical risk minimizer. The method recovers the results of \citepRakSriTsy15 for the bounded case while also providing guarantees without the boundedness assumption.} }
Endnote
%0 Conference Paper %T Learning with Square Loss: Localization through Offset Rademacher Complexity %A Tengyuan Liang %A Alexander Rakhlin %A Karthik Sridharan %B Proceedings of The 28th Conference on Learning Theory %C Proceedings of Machine Learning Research %D 2015 %E Peter Grünwald %E Elad Hazan %E Satyen Kale %F pmlr-v40-Liang15 %I PMLR %J Proceedings of Machine Learning Research %P 1260--1285 %U http://proceedings.mlr.press %V 40 %W PMLR %X We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a two-step estimator is shown to be upper bounded by this offset complexity through a novel geometric inequality. In the convex case, the estimator reduces to an empirical risk minimizer. The method recovers the results of \citepRakSriTsy15 for the bounded case while also providing guarantees without the boundedness assumption.
RIS
TY - CPAPER TI - Learning with Square Loss: Localization through Offset Rademacher Complexity AU - Tengyuan Liang AU - Alexander Rakhlin AU - Karthik Sridharan BT - Proceedings of The 28th Conference on Learning Theory PY - 2015/06/26 DA - 2015/06/26 ED - Peter Grünwald ED - Elad Hazan ED - Satyen Kale ID - pmlr-v40-Liang15 PB - PMLR SP - 1260 DP - PMLR EP - 1285 L1 - http://proceedings.mlr.press/v40/Liang15.pdf UR - http://proceedings.mlr.press/v40/Liang15.html AB - We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a two-step estimator is shown to be upper bounded by this offset complexity through a novel geometric inequality. In the convex case, the estimator reduces to an empirical risk minimizer. The method recovers the results of \citepRakSriTsy15 for the bounded case while also providing guarantees without the boundedness assumption. ER -
APA
Liang, T., Rakhlin, A. & Sridharan, K.. (2015). Learning with Square Loss: Localization through Offset Rademacher Complexity. Proceedings of The 28th Conference on Learning Theory, in PMLR 40:1260-1285

Related Material