Conformal Predictive Systems Under Covariate Shift

Jef Jonkers, Glenn Van Wallendael, Luc Duchateau, Sofie Van Hoecke
Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 230:406-423, 2024.

Abstract

Conformal predictive systems (CPS) offer a versatile framework for constructing predictive distributions, allowing for calibrated inference and informative decision-making. However, their applicability has been limited to scenarios adhering to the independent and identically distributed (IID) model assumption. This paper extends CPS to accommodate scenarios characterized by covariate shifts. We therefore propose weighted CPS (WCPS), akin to weighted conformal prediction (WCP), leveraging likelihood ratios between training and testing covariate distributions. This extension enables the construction of nonparametric predictive distributions capable of handling covariate shifts. We present theoretical underpinnings and conjectures regarding the validity and efficacy of WCPS and demonstrate its utility through empirical evaluations on both synthetic and real-world datasets. Our simulation experiments indicate that WCPS are probabilistically calibrated under covariate shift.

Cite this Paper

BibTeX
@InProceedings{pmlr-v230-jonkers24a, title = {Conformal Predictive Systems Under Covariate Shift}, author = {Jonkers, Jef and Van Wallendael, Glenn and Duchateau, Luc and Van Hoecke, Sofie}, booktitle = {Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {406--423}, year = {2024}, editor = {Vantini, Simone and Fontana, Matteo and Solari, Aldo and Boström, Henrik and Carlsson, Lars}, volume = {230}, series = {Proceedings of Machine Learning Research}, month = {09--11 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v230/main/assets/jonkers24a/jonkers24a.pdf}, url = {https://proceedings.mlr.press/v230/jonkers24a.html}, abstract = {Conformal predictive systems (CPS) offer a versatile framework for constructing predictive distributions, allowing for calibrated inference and informative decision-making. However, their applicability has been limited to scenarios adhering to the independent and identically distributed (IID) model assumption. This paper extends CPS to accommodate scenarios characterized by covariate shifts. We therefore propose weighted CPS (WCPS), akin to weighted conformal prediction (WCP), leveraging likelihood ratios between training and testing covariate distributions. This extension enables the construction of nonparametric predictive distributions capable of handling covariate shifts. We present theoretical underpinnings and conjectures regarding the validity and efficacy of WCPS and demonstrate its utility through empirical evaluations on both synthetic and real-world datasets. Our simulation experiments indicate that WCPS are probabilistically calibrated under covariate shift.} }
Endnote
%0 Conference Paper %T Conformal Predictive Systems Under Covariate Shift %A Jef Jonkers %A Glenn Van Wallendael %A Luc Duchateau %A Sofie Van Hoecke %B Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2024 %E Simone Vantini %E Matteo Fontana %E Aldo Solari %E Henrik Boström %E Lars Carlsson %F pmlr-v230-jonkers24a %I PMLR %P 406--423 %U https://proceedings.mlr.press/v230/jonkers24a.html %V 230 %X Conformal predictive systems (CPS) offer a versatile framework for constructing predictive distributions, allowing for calibrated inference and informative decision-making. However, their applicability has been limited to scenarios adhering to the independent and identically distributed (IID) model assumption. This paper extends CPS to accommodate scenarios characterized by covariate shifts. We therefore propose weighted CPS (WCPS), akin to weighted conformal prediction (WCP), leveraging likelihood ratios between training and testing covariate distributions. This extension enables the construction of nonparametric predictive distributions capable of handling covariate shifts. We present theoretical underpinnings and conjectures regarding the validity and efficacy of WCPS and demonstrate its utility through empirical evaluations on both synthetic and real-world datasets. Our simulation experiments indicate that WCPS are probabilistically calibrated under covariate shift.
APA
Jonkers, J., Van Wallendael, G., Duchateau, L. & Van Hoecke, S.. (2024). Conformal Predictive Systems Under Covariate Shift. Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 230:406-423 Available from https://proceedings.mlr.press/v230/jonkers24a.html.

Related Material