Active Learning under Label Shift

Eric Zhao, Anqi Liu, Animashree Anandkumar, Yisong Yue
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:3412-3420, 2021.

Abstract

We address the problem of active learning under label shift: when the class proportions of source and target domains differ. We introduce a "medial distribution" to incorporate a tradeoff between importance weighting and class-balanced sampling and propose their combined usage in active learning. Our method is known as Mediated Active Learning under Label Shift (MALLS). It balances the bias from class-balanced sampling and the variance from importance weighting. We prove sample complexity and generalization guarantees for MALLS which show active learning reduces asymptotic sample complexity even under arbitrary label shift. We empirically demonstrate MALLS scales to high-dimensional datasets and can reduce the sample complexity of active learning by 60% in deep active learning tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-zhao21b, title = { Active Learning under Label Shift }, author = {Zhao, Eric and Liu, Anqi and Anandkumar, Animashree and Yue, Yisong}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {3412--3420}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/zhao21b/zhao21b.pdf}, url = {https://proceedings.mlr.press/v130/zhao21b.html}, abstract = { We address the problem of active learning under label shift: when the class proportions of source and target domains differ. We introduce a "medial distribution" to incorporate a tradeoff between importance weighting and class-balanced sampling and propose their combined usage in active learning. Our method is known as Mediated Active Learning under Label Shift (MALLS). It balances the bias from class-balanced sampling and the variance from importance weighting. We prove sample complexity and generalization guarantees for MALLS which show active learning reduces asymptotic sample complexity even under arbitrary label shift. We empirically demonstrate MALLS scales to high-dimensional datasets and can reduce the sample complexity of active learning by 60% in deep active learning tasks. } }
Endnote
%0 Conference Paper %T Active Learning under Label Shift %A Eric Zhao %A Anqi Liu %A Animashree Anandkumar %A Yisong Yue %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-zhao21b %I PMLR %P 3412--3420 %U https://proceedings.mlr.press/v130/zhao21b.html %V 130 %X We address the problem of active learning under label shift: when the class proportions of source and target domains differ. We introduce a "medial distribution" to incorporate a tradeoff between importance weighting and class-balanced sampling and propose their combined usage in active learning. Our method is known as Mediated Active Learning under Label Shift (MALLS). It balances the bias from class-balanced sampling and the variance from importance weighting. We prove sample complexity and generalization guarantees for MALLS which show active learning reduces asymptotic sample complexity even under arbitrary label shift. We empirically demonstrate MALLS scales to high-dimensional datasets and can reduce the sample complexity of active learning by 60% in deep active learning tasks.
APA
Zhao, E., Liu, A., Anandkumar, A. & Yue, Y.. (2021). Active Learning under Label Shift . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:3412-3420 Available from https://proceedings.mlr.press/v130/zhao21b.html.

Related Material