Marginal Singularity, and the Benefits of Labels in Covariate-Shift

Samory Kpotufe, Guillaume Martinet
Proceedings of the 31st Conference On Learning Theory, PMLR 75:1882-1886, 2018.

Abstract

We present new minimax results that concisely capture the relative benefits of source and target labeled data, under {covariate-shift}. Namely, we show that, in general classification settings, the benefits of target labels are controlled by a \emph{transfer-exponent} $\gamma$ that encodes how \emph{singular} $Q$ is locally w.r.t. $P$, and interestingly allows situations where transfer did not seem possible under previous insights. In fact, our new minimax analysis – in terms of $\gamma$ – reveals a \emph{continuum of regimes} ranging from situations where target labels have little benefit, to regimes where target labels dramatically improve classification. We then show that a recently proposed semi-supervised procedure can be extended to adapt to unknown $\gamma$, and therefore requests target labels only when beneficial, while achieving nearly minimax transfer rates.

Cite this Paper

BibTeX
@InProceedings{pmlr-v75-kpotufe18a, title = {Marginal Singularity, and the Benefits of Labels in Covariate-Shift}, author = {Kpotufe, Samory and Martinet, Guillaume}, booktitle = {Proceedings of the 31st Conference On Learning Theory}, pages = {1882--1886}, year = {2018}, editor = {Bubeck, Sébastien and Perchet, Vianney and Rigollet, Philippe}, volume = {75}, series = {Proceedings of Machine Learning Research}, month = {06--09 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v75/kpotufe18a/kpotufe18a.pdf}, url = {https://proceedings.mlr.press/v75/kpotufe18a.html}, abstract = {We present new minimax results that concisely capture the relative benefits of source and target labeled data, under {covariate-shift}. Namely, we show that, in general classification settings, the benefits of target labels are controlled by a \emph{transfer-exponent} $\gamma$ that encodes how \emph{singular} $Q$ is locally w.r.t. $P$, and interestingly allows situations where transfer did not seem possible under previous insights. In fact, our new minimax analysis – in terms of $\gamma$ – reveals a \emph{continuum of regimes} ranging from situations where target labels have little benefit, to regimes where target labels dramatically improve classification. We then show that a recently proposed semi-supervised procedure can be extended to adapt to unknown $\gamma$, and therefore requests target labels only when beneficial, while achieving nearly minimax transfer rates. } }
Endnote
%0 Conference Paper %T Marginal Singularity, and the Benefits of Labels in Covariate-Shift %A Samory Kpotufe %A Guillaume Martinet %B Proceedings of the 31st Conference On Learning Theory %C Proceedings of Machine Learning Research %D 2018 %E Sébastien Bubeck %E Vianney Perchet %E Philippe Rigollet %F pmlr-v75-kpotufe18a %I PMLR %P 1882--1886 %U https://proceedings.mlr.press/v75/kpotufe18a.html %V 75 %X We present new minimax results that concisely capture the relative benefits of source and target labeled data, under {covariate-shift}. Namely, we show that, in general classification settings, the benefits of target labels are controlled by a \emph{transfer-exponent} $\gamma$ that encodes how \emph{singular} $Q$ is locally w.r.t. $P$, and interestingly allows situations where transfer did not seem possible under previous insights. In fact, our new minimax analysis – in terms of $\gamma$ – reveals a \emph{continuum of regimes} ranging from situations where target labels have little benefit, to regimes where target labels dramatically improve classification. We then show that a recently proposed semi-supervised procedure can be extended to adapt to unknown $\gamma$, and therefore requests target labels only when beneficial, while achieving nearly minimax transfer rates.
APA
Kpotufe, S. & Martinet, G.. (2018). Marginal Singularity, and the Benefits of Labels in Covariate-Shift. Proceedings of the 31st Conference On Learning Theory, in Proceedings of Machine Learning Research 75:1882-1886 Available from https://proceedings.mlr.press/v75/kpotufe18a.html.

Related Material