Optimal sampling in unbiased active learning

Henrik Imberg, Johan Jonasson, Marina Axelson-Fisk
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR 108:559-569, 2020.

Abstract

A common belief in unbiased active learning is that, in order to capture the most informative instances, the sampling probabilities should be proportional to the uncertainty of the class labels. We argue that this produces suboptimal predictions and present sampling schemes for unbiased pool-based active learning that minimise the actual prediction error, and demonstrate a better predictive performance than competing methods on a number of benchmark datasets. In contrast, both probabilistic and deterministic uncertainty sampling performed worse than simple random sampling on some of the datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v108-imberg20a, title = {Optimal sampling in unbiased active learning}, author = {Imberg, Henrik and Jonasson, Johan and Axelson-Fisk, Marina}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {559--569}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/imberg20a/imberg20a.pdf}, url = {https://proceedings.mlr.press/v108/imberg20a.html}, abstract = {A common belief in unbiased active learning is that, in order to capture the most informative instances, the sampling probabilities should be proportional to the uncertainty of the class labels. We argue that this produces suboptimal predictions and present sampling schemes for unbiased pool-based active learning that minimise the actual prediction error, and demonstrate a better predictive performance than competing methods on a number of benchmark datasets. In contrast, both probabilistic and deterministic uncertainty sampling performed worse than simple random sampling on some of the datasets.} }
Endnote
%0 Conference Paper %T Optimal sampling in unbiased active learning %A Henrik Imberg %A Johan Jonasson %A Marina Axelson-Fisk %B Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2020 %E Silvia Chiappa %E Roberto Calandra %F pmlr-v108-imberg20a %I PMLR %P 559--569 %U https://proceedings.mlr.press/v108/imberg20a.html %V 108 %X A common belief in unbiased active learning is that, in order to capture the most informative instances, the sampling probabilities should be proportional to the uncertainty of the class labels. We argue that this produces suboptimal predictions and present sampling schemes for unbiased pool-based active learning that minimise the actual prediction error, and demonstrate a better predictive performance than competing methods on a number of benchmark datasets. In contrast, both probabilistic and deterministic uncertainty sampling performed worse than simple random sampling on some of the datasets.
APA
Imberg, H., Jonasson, J. & Axelson-Fisk, M.. (2020). Optimal sampling in unbiased active learning. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 108:559-569 Available from https://proceedings.mlr.press/v108/imberg20a.html.

Related Material