On discriminative learning of prediction uncertainty

Vojtech Franc, Daniel Prusa
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:1963-1971, 2019.

Abstract

In classification with a reject option, the classifier is allowed in uncertain cases to abstain from prediction. The classical cost based model of an optimal classifier with a reject option requires the cost of rejection to be defined explicitly. An alternative bounded-improvement model, avoiding the notion of the reject cost, seeks for a classifier with a guaranteed selective risk and maximal cover. We prove that both models share the same class of optimal strategies, and we provide an explicit relation between the reject cost and the target risk being the parameters of the two models. An optimal rejection strategy for both models is based on thresholding the conditional risk defined by posterior probabilities which are usually unavailable. We propose a discriminative algorithm learning an uncertainty function which preserves ordering of the input space induced by the conditional risk, and hence can be used to construct optimal rejection strategies.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-franc19a, title = {On discriminative learning of prediction uncertainty}, author = {Franc, Vojtech and Prusa, Daniel}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {1963--1971}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/franc19a/franc19a.pdf}, url = {https://proceedings.mlr.press/v97/franc19a.html}, abstract = {In classification with a reject option, the classifier is allowed in uncertain cases to abstain from prediction. The classical cost based model of an optimal classifier with a reject option requires the cost of rejection to be defined explicitly. An alternative bounded-improvement model, avoiding the notion of the reject cost, seeks for a classifier with a guaranteed selective risk and maximal cover. We prove that both models share the same class of optimal strategies, and we provide an explicit relation between the reject cost and the target risk being the parameters of the two models. An optimal rejection strategy for both models is based on thresholding the conditional risk defined by posterior probabilities which are usually unavailable. We propose a discriminative algorithm learning an uncertainty function which preserves ordering of the input space induced by the conditional risk, and hence can be used to construct optimal rejection strategies.} }
Endnote
%0 Conference Paper %T On discriminative learning of prediction uncertainty %A Vojtech Franc %A Daniel Prusa %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-franc19a %I PMLR %P 1963--1971 %U https://proceedings.mlr.press/v97/franc19a.html %V 97 %X In classification with a reject option, the classifier is allowed in uncertain cases to abstain from prediction. The classical cost based model of an optimal classifier with a reject option requires the cost of rejection to be defined explicitly. An alternative bounded-improvement model, avoiding the notion of the reject cost, seeks for a classifier with a guaranteed selective risk and maximal cover. We prove that both models share the same class of optimal strategies, and we provide an explicit relation between the reject cost and the target risk being the parameters of the two models. An optimal rejection strategy for both models is based on thresholding the conditional risk defined by posterior probabilities which are usually unavailable. We propose a discriminative algorithm learning an uncertainty function which preserves ordering of the input space induced by the conditional risk, and hence can be used to construct optimal rejection strategies.
APA
Franc, V. & Prusa, D.. (2019). On discriminative learning of prediction uncertainty. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:1963-1971 Available from https://proceedings.mlr.press/v97/franc19a.html.

Related Material