Conformal Prediction with Partially Labeled Data

Alireza Javanmardi, Yusuf Sale, Paul Hofman, Eyke Hüllermeier
Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 204:251-266, 2023.

Abstract

While the predictions produced by conformal prediction are set-valued, the data used for training and calibration is supposed to be precise. In the setting of superset learning or learning from partial labels, a variant of weakly supervised learning, it is exactly the other way around: training data is possibly imprecise (set-valued), but the model induced from this data yields precise predictions. In this paper, we combine the two settings by making conformal prediction amenable to set-valued training data. We propose a generalization of the conformal prediction procedure that can be applied to set-valued training and calibration data. We prove the validity of the proposed method and present experimental studies in which it compares favorably to natural baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v204-javanmardi23a, title = {Conformal Prediction with Partially Labeled Data}, author = {Javanmardi, Alireza and Sale, Yusuf and Hofman, Paul and H\"ullermeier, Eyke}, booktitle = {Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {251--266}, year = {2023}, editor = {Papadopoulos, Harris and Nguyen, Khuong An and Boström, Henrik and Carlsson, Lars}, volume = {204}, series = {Proceedings of Machine Learning Research}, month = {13--15 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v204/javanmardi23a/javanmardi23a.pdf}, url = {https://proceedings.mlr.press/v204/javanmardi23a.html}, abstract = {While the predictions produced by conformal prediction are set-valued, the data used for training and calibration is supposed to be precise. In the setting of superset learning or learning from partial labels, a variant of weakly supervised learning, it is exactly the other way around: training data is possibly imprecise (set-valued), but the model induced from this data yields precise predictions. In this paper, we combine the two settings by making conformal prediction amenable to set-valued training data. We propose a generalization of the conformal prediction procedure that can be applied to set-valued training and calibration data. We prove the validity of the proposed method and present experimental studies in which it compares favorably to natural baselines.} }
Endnote
%0 Conference Paper %T Conformal Prediction with Partially Labeled Data %A Alireza Javanmardi %A Yusuf Sale %A Paul Hofman %A Eyke Hüllermeier %B Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2023 %E Harris Papadopoulos %E Khuong An Nguyen %E Henrik Boström %E Lars Carlsson %F pmlr-v204-javanmardi23a %I PMLR %P 251--266 %U https://proceedings.mlr.press/v204/javanmardi23a.html %V 204 %X While the predictions produced by conformal prediction are set-valued, the data used for training and calibration is supposed to be precise. In the setting of superset learning or learning from partial labels, a variant of weakly supervised learning, it is exactly the other way around: training data is possibly imprecise (set-valued), but the model induced from this data yields precise predictions. In this paper, we combine the two settings by making conformal prediction amenable to set-valued training data. We propose a generalization of the conformal prediction procedure that can be applied to set-valued training and calibration data. We prove the validity of the proposed method and present experimental studies in which it compares favorably to natural baselines.
APA
Javanmardi, A., Sale, Y., Hofman, P. & Hüllermeier, E.. (2023). Conformal Prediction with Partially Labeled Data. Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 204:251-266 Available from https://proceedings.mlr.press/v204/javanmardi23a.html.

Related Material