Selective sampling algorithms for cost-sensitive multiclass prediction


Alekh Agarwal ;
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):1220-1228, 2013.


In this paper, we study the problem of active learning for cost-sensitive multiclass classification. We propose selective sampling algorithms, which process the data in a streaming fashion, querying only a subset of the labels. For these algorithms, we analyze the regret and label complexity when the labels are generated according to a generalized linear model. We establish that the gains of active learning over passive learning can range from none to exponentially large, based on a natural notion of margin. We also present a safety guarantee to guard against model mismatch. Numerical simulations show that our algorithms indeed obtain a low regret with a small number of queries.

Related Material