From Theories to Queries: Active Learning in Practice

Burr Settles
; Active Learning and Experimental Design workshop In conjunction with AISTATS 2010, JMLR Workshop and Conference Proceedings 16:1-18, 2011.

Abstract

This article surveys recent work in active learning aimed at making it more practical for real-world use. In general, active learning systems aim to make machine learning more economical, since they can participate in the acquisition of their own training data. An active learner might iteratively select informative query instances to be labeled by an oracle, for example. Work over the last two decades has shown that such approaches are effective at maintaining accuracy while reducing training set size in many machine learning applications. However, as we begin to deploy active learning in real ongoing learning systems and data annotation projects, we are encountering unexpected problems–due in part to practical realities that violate the basic assumptions of earlier foundational work. I review some of these issues, and discuss recent work being done to address the challenges.

Cite this Paper


BibTeX
@InProceedings{pmlr-v16-settles11a, title = {From Theories to Queries: Active Learning in Practice}, author = {Burr Settles}, booktitle = {Active Learning and Experimental Design workshop In conjunction with AISTATS 2010}, pages = {1--18}, year = {2011}, editor = {Isabelle Guyon and Gavin Cawley and Gideon Dror and Vincent Lemaire and Alexander Statnikov}, volume = {16}, series = {Proceedings of Machine Learning Research}, address = {Sardinia, Italy}, month = {16 May}, publisher = {JMLR Workshop and Conference Proceedings}, pdf = {http://proceedings.mlr.press/v16/settles11a/settles11a.pdf}, url = {http://proceedings.mlr.press/v16/settles11a.html}, abstract = {This article surveys recent work in active learning aimed at making it more practical for real-world use. In general, active learning systems aim to make machine learning more economical, since they can participate in the acquisition of their own training data. An active learner might iteratively select informative query instances to be labeled by an oracle, for example. Work over the last two decades has shown that such approaches are effective at maintaining accuracy while reducing training set size in many machine learning applications. However, as we begin to deploy active learning in real ongoing learning systems and data annotation projects, we are encountering unexpected problems–due in part to practical realities that violate the basic assumptions of earlier foundational work. I review some of these issues, and discuss recent work being done to address the challenges.} }
Endnote
%0 Conference Paper %T From Theories to Queries: Active Learning in Practice %A Burr Settles %B Active Learning and Experimental Design workshop In conjunction with AISTATS 2010 %C Proceedings of Machine Learning Research %D 2011 %E Isabelle Guyon %E Gavin Cawley %E Gideon Dror %E Vincent Lemaire %E Alexander Statnikov %F pmlr-v16-settles11a %I PMLR %J Proceedings of Machine Learning Research %P 1--18 %U http://proceedings.mlr.press %V 16 %W PMLR %X This article surveys recent work in active learning aimed at making it more practical for real-world use. In general, active learning systems aim to make machine learning more economical, since they can participate in the acquisition of their own training data. An active learner might iteratively select informative query instances to be labeled by an oracle, for example. Work over the last two decades has shown that such approaches are effective at maintaining accuracy while reducing training set size in many machine learning applications. However, as we begin to deploy active learning in real ongoing learning systems and data annotation projects, we are encountering unexpected problems–due in part to practical realities that violate the basic assumptions of earlier foundational work. I review some of these issues, and discuss recent work being done to address the challenges.
RIS
TY - CPAPER TI - From Theories to Queries: Active Learning in Practice AU - Burr Settles BT - Active Learning and Experimental Design workshop In conjunction with AISTATS 2010 PY - 2011/04/21 DA - 2011/04/21 ED - Isabelle Guyon ED - Gavin Cawley ED - Gideon Dror ED - Vincent Lemaire ED - Alexander Statnikov ID - pmlr-v16-settles11a PB - PMLR SP - 1 DP - PMLR EP - 18 L1 - http://proceedings.mlr.press/v16/settles11a/settles11a.pdf UR - http://proceedings.mlr.press/v16/settles11a.html AB - This article surveys recent work in active learning aimed at making it more practical for real-world use. In general, active learning systems aim to make machine learning more economical, since they can participate in the acquisition of their own training data. An active learner might iteratively select informative query instances to be labeled by an oracle, for example. Work over the last two decades has shown that such approaches are effective at maintaining accuracy while reducing training set size in many machine learning applications. However, as we begin to deploy active learning in real ongoing learning systems and data annotation projects, we are encountering unexpected problems–due in part to practical realities that violate the basic assumptions of earlier foundational work. I review some of these issues, and discuss recent work being done to address the challenges. ER -
APA
Settles, B.. (2011). From Theories to Queries: Active Learning in Practice. Active Learning and Experimental Design workshop In conjunction with AISTATS 2010, in PMLR 16:1-18

Related Material