Self Learning using Venn-Abers predictors

Come Rodriguez, Vitor Martin Bordini, Sebastien Destercke, Benjamin Quost
Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 204:234-250, 2023.

Abstract

In supervised learning problems, it is common to have a lot of unlabeled data, but little labeled data. It is then desirable to leverage the unlabeled data to improve the learning procedure. One way to do this is to have a model predict “pseudolabels” for the unlabeled data, so as to use them for learning. In self-learning, the pseudo-labels are provided by the very same model to which they are fed. As these pseudo-labels are by nature uncertain and only partially reliable, it is then natural to model this uncertainty and take it into account in the learning process, if only to robustify the self-learning procedure. This paper describes such an approach, where we use Venn-Abers Predictors to produce calibrated credal labels so as to quantify the pseudo-labeling uncertainty. These labels are then included in the learning process by optimizing an adapted loss. Experiments show that taking into account pseudo-label uncertainty both robustifies the self-learning procedure and allows it to converge faster in general.

Cite this Paper

BibTeX
@InProceedings{pmlr-v204-rodriguez23a, title = {Self Learning using Venn-Abers predictors}, author = {Rodriguez, Come and Martin Bordini, Vitor and Destercke, Sebastien and Quost, Benjamin}, booktitle = {Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {234--250}, year = {2023}, editor = {Papadopoulos, Harris and Nguyen, Khuong An and Boström, Henrik and Carlsson, Lars}, volume = {204}, series = {Proceedings of Machine Learning Research}, month = {13--15 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v204/rodriguez23a/rodriguez23a.pdf}, url = {https://proceedings.mlr.press/v204/rodriguez23a.html}, abstract = {In supervised learning problems, it is common to have a lot of unlabeled data, but little labeled data. It is then desirable to leverage the unlabeled data to improve the learning procedure. One way to do this is to have a model predict “pseudolabels” for the unlabeled data, so as to use them for learning. In self-learning, the pseudo-labels are provided by the very same model to which they are fed. As these pseudo-labels are by nature uncertain and only partially reliable, it is then natural to model this uncertainty and take it into account in the learning process, if only to robustify the self-learning procedure. This paper describes such an approach, where we use Venn-Abers Predictors to produce calibrated credal labels so as to quantify the pseudo-labeling uncertainty. These labels are then included in the learning process by optimizing an adapted loss. Experiments show that taking into account pseudo-label uncertainty both robustifies the self-learning procedure and allows it to converge faster in general.} }
Endnote
%0 Conference Paper %T Self Learning using Venn-Abers predictors %A Come Rodriguez %A Vitor Martin Bordini %A Sebastien Destercke %A Benjamin Quost %B Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2023 %E Harris Papadopoulos %E Khuong An Nguyen %E Henrik Boström %E Lars Carlsson %F pmlr-v204-rodriguez23a %I PMLR %P 234--250 %U https://proceedings.mlr.press/v204/rodriguez23a.html %V 204 %X In supervised learning problems, it is common to have a lot of unlabeled data, but little labeled data. It is then desirable to leverage the unlabeled data to improve the learning procedure. One way to do this is to have a model predict “pseudolabels” for the unlabeled data, so as to use them for learning. In self-learning, the pseudo-labels are provided by the very same model to which they are fed. As these pseudo-labels are by nature uncertain and only partially reliable, it is then natural to model this uncertainty and take it into account in the learning process, if only to robustify the self-learning procedure. This paper describes such an approach, where we use Venn-Abers Predictors to produce calibrated credal labels so as to quantify the pseudo-labeling uncertainty. These labels are then included in the learning process by optimizing an adapted loss. Experiments show that taking into account pseudo-label uncertainty both robustifies the self-learning procedure and allows it to converge faster in general.
APA
Rodriguez, C., Martin Bordini, V., Destercke, S. & Quost, B.. (2023). Self Learning using Venn-Abers predictors. Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 204:234-250 Available from https://proceedings.mlr.press/v204/rodriguez23a.html.

Related Material