Active Learning with Expert Advice

Peilin ZHAO, Steven Hoi
Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, PMLR R11:252-261, 2013.

Abstract

Conventional learning with expert advice methods assume a learner is always receiv- ing the outcome (e.g., class labels) of every incoming training instance at the end of each trial. In real applications, acquiring the out- come from oracle can be costly or time con- suming. In this paper, we address a new problem of active learning with expert ad- vice, where the outcome of an instance is dis- closed only when it is requested by the on- line learner. Our goal is to learn an accu- rate prediction model by asking the oracle the number of questions as small as possi- ble. To address this challenge, we propose a framework of active forecasters for online active learning with expert advice, which at- tempts to extend two regular forecasters, i.e., Exponentially Weighted Average Forecaster and Greedy Forecaster, to tackle the task of active learning with expert advice. We prove that the proposed algorithms satisfy the Han- nan consistency under some proper assump- tions, and validate the efficacy of our tech- nique by an extensive set of experiments.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR11-zhao13a, title = {Active Learning with Expert Advice}, author = {ZHAO, Peilin and Hoi, Steven}, booktitle = {Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence}, pages = {252--261}, year = {2013}, editor = {Nicholson, Ann and Smyth, Padhraic}, volume = {R11}, series = {Proceedings of Machine Learning Research}, month = {12--14 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r11/main/assets/zhao13a/zhao13a.pdf}, url = {https://proceedings.mlr.press/r11/zhao13a.html}, abstract = {Conventional learning with expert advice methods assume a learner is always receiv- ing the outcome (e.g., class labels) of every incoming training instance at the end of each trial. In real applications, acquiring the out- come from oracle can be costly or time con- suming. In this paper, we address a new problem of active learning with expert ad- vice, where the outcome of an instance is dis- closed only when it is requested by the on- line learner. Our goal is to learn an accu- rate prediction model by asking the oracle the number of questions as small as possi- ble. To address this challenge, we propose a framework of active forecasters for online active learning with expert advice, which at- tempts to extend two regular forecasters, i.e., Exponentially Weighted Average Forecaster and Greedy Forecaster, to tackle the task of active learning with expert advice. We prove that the proposed algorithms satisfy the Han- nan consistency under some proper assump- tions, and validate the efficacy of our tech- nique by an extensive set of experiments.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Active Learning with Expert Advice %A Peilin ZHAO %A Steven Hoi %B Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2013 %E Ann Nicholson %E Padhraic Smyth %F pmlr-vR11-zhao13a %I PMLR %P 252--261 %U https://proceedings.mlr.press/r11/zhao13a.html %V R11 %X Conventional learning with expert advice methods assume a learner is always receiv- ing the outcome (e.g., class labels) of every incoming training instance at the end of each trial. In real applications, acquiring the out- come from oracle can be costly or time con- suming. In this paper, we address a new problem of active learning with expert ad- vice, where the outcome of an instance is dis- closed only when it is requested by the on- line learner. Our goal is to learn an accu- rate prediction model by asking the oracle the number of questions as small as possi- ble. To address this challenge, we propose a framework of active forecasters for online active learning with expert advice, which at- tempts to extend two regular forecasters, i.e., Exponentially Weighted Average Forecaster and Greedy Forecaster, to tackle the task of active learning with expert advice. We prove that the proposed algorithms satisfy the Han- nan consistency under some proper assump- tions, and validate the efficacy of our tech- nique by an extensive set of experiments. %Z Reissued by PMLR on 04 October 2026.
APA
ZHAO, P. & Hoi, S.. (2013). Active Learning with Expert Advice. Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R11:252-261 Available from https://proceedings.mlr.press/r11/zhao13a.html. Reissued by PMLR on 04 October 2026.

Related Material